|
2026-10-22
ยง
|
| 22:39 |
<btullis@cumin1004> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1019.eqiad.wmnet |
[production] |
| 22:39 |
<btullis@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1019.eqiad.wmnet |
[production] |
| 22:32 |
<btullis@cumin1004> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1019.eqiad.wmnet |
[production] |
| 22:32 |
<btullis@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1019.eqiad.wmnet |
[production] |
| 22:32 |
<btullis@cumin1004> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1018.eqiad.wmnet |
[production] |
| 22:32 |
<btullis@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1018.eqiad.wmnet |
[production] |
| 22:26 |
<btullis@cumin1004> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1018.eqiad.wmnet |
[production] |
| 22:25 |
<btullis@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1018.eqiad.wmnet |
[production] |
| 22:25 |
<btullis@cumin1004> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1017.eqiad.wmnet |
[production] |
| 22:25 |
<btullis@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1017.eqiad.wmnet |
[production] |
| 22:14 |
<btullis@cumin1004> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1017.eqiad.wmnet |
[production] |
| 22:13 |
<btullis@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1017.eqiad.wmnet |
[production] |
| 22:13 |
<btullis@cumin1004> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1016.eqiad.wmnet |
[production] |
| 22:13 |
<btullis@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1016.eqiad.wmnet |
[production] |
| 22:12 |
<bd808> |
Webservice restart to reconnect with front proxy |
[tools.toolviews] |
| 22:09 |
<bd808> |
Webservice restart to reconnect with front proxy |
[tools.phab-ban] |
| 22:07 |
<btullis@cumin1004> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1016.eqiad.wmnet |
[production] |
| 21:37 |
<btullis@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1016.eqiad.wmnet |
[production] |
| 21:37 |
<btullis@cumin1004> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1015.eqiad.wmnet |
[production] |
| 21:37 |
<btullis@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1015.eqiad.wmnet |
[production] |
| 21:31 |
<btullis@cumin1004> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1015.eqiad.wmnet |
[production] |
| 21:30 |
<bd808> |
Webservice restart to reconnect with front proxy |
[tools.etherpad-backup] |
| 21:26 |
<bd808> |
Webservice restart to reconnect with front proxy |
[tools.disabled-tools] |
| 21:24 |
<akhatun@deploy1003> |
helmfile [staging] DONE helmfile.d/services/lac-lambdas: sync |
[production] |
| 21:24 |
<akhatun@deploy1003> |
helmfile [staging] START helmfile.d/services/lac-lambdas: sync |
[production] |
| 21:05 |
<bd808> |
Hard stop/start to try and get out of CrashLoopbackOff |
[tools.k8s-status] |
| 21:03 |
<dancy@deploy1003> |
Locking from deployment [ALL REPOSITORIES]: Deployments locked for the week |
[production] |
| 21:03 |
<dancy@deploy1003> |
Finished scap sync-world: Backport for [[gerrit:1357836|wikisource-ocr-langs: Lower timeout, cache absence, set lockTSE+busyValue (T442064)]] (1.48.0-wmf.1) (duration: 08m 23s) |
[production] |
| 21:01 |
<btullis@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1015.eqiad.wmnet |
[production] |
| 21:01 |
<btullis@cumin1004> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1014.eqiad.wmnet |
[production] |
| 21:01 |
<btullis@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1014.eqiad.wmnet |
[production] |
| 20:57 |
<dancy@deploy1003> |
dancy: Continuing with deployment |
[production] |
| 20:57 |
<dancy@deploy1003> |
dancy: Backport for [[gerrit:1357836|wikisource-ocr-langs: Lower timeout, cache absence, set lockTSE+busyValue (T442064)]] (1.48.0-wmf.1) synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 20:55 |
<dancy@deploy1003> |
Started scap sync-world: Backport for [[gerrit:1357836|wikisource-ocr-langs: Lower timeout, cache absence, set lockTSE+busyValue (T442064)]] (1.48.0-wmf.1) |
[production] |
| 20:53 |
<dancy@deploy1003> |
Unlocked for deployment [ALL REPOSITORIES]: Deployments locked for the week (duration: 11m 08s) |
[production] |
| 20:53 |
<dancy@deploy1003> |
Forcefully removing global lock: Deployments locked for the week |
[production] |
| 20:45 |
<btullis@cumin1004> |
END (FAIL) - Cookbook sre.k8s.pool-depool-node (exit_code=99) depool for host dse-k8s-worker1014.eqiad.wmnet |
[production] |
| 20:42 |
<dancy@deploy1003> |
Locking from deployment [ALL REPOSITORIES]: Deployments locked for the week |
[production] |
| 20:42 |
<dancy@deploy1003> |
Finished scap sync-world: Backport for [[gerrit:1357790|wikisource-ocr-langs: Lower timeout, cache absence, set lockTSE+busyValue (T442064)]] (1.47.0-wmf.24) (duration: 08m 51s) |
[production] |
| 20:41 |
<btullis@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1014.eqiad.wmnet |
[production] |
| 20:40 |
<btullis@cumin1004> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1013.eqiad.wmnet |
[production] |
| 20:40 |
<btullis@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1013.eqiad.wmnet |
[production] |
| 20:36 |
<dancy@deploy1003> |
jforrester, dancy: Continuing with deployment |
[production] |
| 20:35 |
<dancy@deploy1003> |
jforrester, dancy: Backport for [[gerrit:1357790|wikisource-ocr-langs: Lower timeout, cache absence, set lockTSE+busyValue (T442064)]] (1.47.0-wmf.24) synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 20:34 |
<btullis@cumin1004> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1013.eqiad.wmnet |
[production] |
| 20:34 |
<btullis@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1013.eqiad.wmnet |
[production] |
| 20:34 |
<btullis@cumin1004> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1012.eqiad.wmnet |
[production] |
| 20:34 |
<btullis@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1012.eqiad.wmnet |
[production] |
| 20:33 |
<dancy@deploy1003> |
Started scap sync-world: Backport for [[gerrit:1357790|wikisource-ocr-langs: Lower timeout, cache absence, set lockTSE+busyValue (T442064)]] (1.47.0-wmf.24) |
[production] |
| 20:30 |
<thcipriani@deploy1003> |
Unlocked for deployment [ALL REPOSITORIES]: Deployments locked for the week (duration: 156m 22s) |
[production] |