|
2026-11-05
§
|
| 23:30 |
<mutante> |
codesearch2001 initial repo pull from github. codesearch1001 temp puppet disabled for careful rollout. |
[production] |
| 23:18 |
<maryum> |
Deployed security fix for T443424, T443467, T443036 |
[production] |
| 23:10 |
<ladsgroup@cumin1004> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2198.codfw.wmnet with reason: Maintenance |
[production] |
| 23:10 |
<ladsgroup@cumin1004> |
dbctl commit (dc=all): 'Repooling after maintenance db2195 (T441633)', diff saved to https://phabricator.wikimedia.org/P97417 and previous config saved to /var/cache/conftool/dbconfig/20261105-231002-ladsgroup.json |
[production] |
| 23:00 |
<ladsgroup@cumin1004> |
dbctl commit (dc=all): 'Repooling after maintenance db2195', diff saved to https://phabricator.wikimedia.org/P97416 and previous config saved to /var/cache/conftool/dbconfig/20261105-230000-ladsgroup.json |
[production] |
| 22:51 |
<sbassett> |
Deployed security fixes for T439878, T440323, T440657, T440656, T440655 |
[production] |
| 22:49 |
<ladsgroup@cumin1004> |
dbctl commit (dc=all): 'Repooling after maintenance db2195', diff saved to https://phabricator.wikimedia.org/P97415 and previous config saved to /var/cache/conftool/dbconfig/20261105-224958-ladsgroup.json |
[production] |
| 22:39 |
<ladsgroup@cumin1004> |
dbctl commit (dc=all): 'Repooling after maintenance db2195 (T441633)', diff saved to https://phabricator.wikimedia.org/P97414 and previous config saved to /var/cache/conftool/dbconfig/20261105-223956-ladsgroup.json |
[production] |
| 22:32 |
<ladsgroup@cumin1004> |
dbctl commit (dc=all): 'Depooling db2195 (T441633)', diff saved to https://phabricator.wikimedia.org/P97413 and previous config saved to /var/cache/conftool/dbconfig/20261105-223248-ladsgroup.json |
[production] |
| 22:32 |
<ladsgroup@cumin1004> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2195.codfw.wmnet with reason: Maintenance |
[production] |
| 22:32 |
<ladsgroup@cumin1004> |
dbctl commit (dc=all): 'Repooling after maintenance db2181 (T441633)', diff saved to https://phabricator.wikimedia.org/P97412 and previous config saved to /var/cache/conftool/dbconfig/20261105-223225-ladsgroup.json |
[production] |
| 22:30 |
<egardner@deploy2003> |
Finished scap sync-world: Backport for [[gerrit:1364473|ShareCard: Log client errors with the topic "error.sharecard" (T439163)]] (1.48.0-wmf.3), [[gerrit:1363474|ShareCard: Show animated GIFs as a still image (T442968)]] (1.48.0-wmf.3) (duration: 11m 46s) |
[production] |
| 22:24 |
<egardner@deploy2003> |
egardner: Continuing with deployment |
[production] |
| 22:23 |
<egardner@deploy2003> |
egardner: Backport for [[gerrit:1364473|ShareCard: Log client errors with the topic "error.sharecard" (T439163)]] (1.48.0-wmf.3), [[gerrit:1363474|ShareCard: Show animated GIFs as a still image (T442968)]] (1.48.0-wmf.3) synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 22:22 |
<ssastry@deploy2003> |
helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply |
[production] |
| 22:22 |
<ladsgroup@cumin1004> |
dbctl commit (dc=all): 'Repooling after maintenance db2181', diff saved to https://phabricator.wikimedia.org/P97411 and previous config saved to /var/cache/conftool/dbconfig/20261105-222223-ladsgroup.json |
[production] |
| 22:22 |
<ssastry@deploy2003> |
helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply |
[production] |
| 22:19 |
<egardner@deploy2003> |
Started scap sync-world: Backport for [[gerrit:1364473|ShareCard: Log client errors with the topic "error.sharecard" (T439163)]] (1.48.0-wmf.3), [[gerrit:1363474|ShareCard: Show animated GIFs as a still image (T442968)]] (1.48.0-wmf.3) |
[production] |
| 22:15 |
<egardner@deploy2003> |
Finished scap sync-world: Backport for [[gerrit:1358851|Add Creative Commons license icons for the share card (T439162)]] (duration: 14m 19s) |
[production] |
| 22:12 |
<ladsgroup@cumin1004> |
dbctl commit (dc=all): 'Repooling after maintenance db2181', diff saved to https://phabricator.wikimedia.org/P97410 and previous config saved to /var/cache/conftool/dbconfig/20261105-221221-ladsgroup.json |
[production] |
| 22:08 |
<egardner@deploy2003> |
egardner: Continuing with deployment |
[production] |
| 22:07 |
<egardner@deploy2003> |
egardner: Backport for [[gerrit:1358851|Add Creative Commons license icons for the share card (T439162)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 22:02 |
<ladsgroup@cumin1004> |
dbctl commit (dc=all): 'Repooling after maintenance db2181 (T441633)', diff saved to https://phabricator.wikimedia.org/P97409 and previous config saved to /var/cache/conftool/dbconfig/20261105-220220-ladsgroup.json |
[production] |
| 22:01 |
<egardner@deploy2003> |
Started scap sync-world: Backport for [[gerrit:1358851|Add Creative Commons license icons for the share card (T439162)]] |
[production] |
| 21:54 |
<ladsgroup@cumin1004> |
dbctl commit (dc=all): 'Depooling db2181 (T441633)', diff saved to https://phabricator.wikimedia.org/P97408 and previous config saved to /var/cache/conftool/dbconfig/20261105-215453-ladsgroup.json |
[production] |
| 21:54 |
<mutante> |
ci / contint1003/contint2003 - if "sudo docker info -f '{{.Driver}}'" says "overlay2" it's the classic way. after restart when it says "overlayfs2" that will be the NEW way! (T440668) |
[production] |
| 21:54 |
<ladsgroup@cumin1004> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2181.codfw.wmnet with reason: Maintenance |
[production] |
| 21:54 |
<ladsgroup@cumin1004> |
dbctl commit (dc=all): 'Repooling after maintenance db2167 (T441633)', diff saved to https://phabricator.wikimedia.org/P97407 and previous config saved to /var/cache/conftool/dbconfig/20261105-215432-ladsgroup.json |
[production] |
| 21:51 |
<mutante> |
ci / contint1003/contint2003 - docker will be reconfigured to use containerd-snapshotter rather than dockers legacy store, overlay2 (a thing docker has been moving towards for a while). images pulled under layer2 become invisible on next docker restart. releng will handle that and prune the old images. new images will have to be pulled down. needed for ST6.1.4 (T440668) |
[production] |
| 21:44 |
<ladsgroup@cumin1004> |
dbctl commit (dc=all): 'Repooling after maintenance db2167', diff saved to https://phabricator.wikimedia.org/P97406 and previous config saved to /var/cache/conftool/dbconfig/20261105-214430-ladsgroup.json |
[production] |
| 21:34 |
<ssastry@deploy2003> |
helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply |
[production] |
| 21:34 |
<ladsgroup@cumin1004> |
dbctl commit (dc=all): 'Repooling after maintenance db2167', diff saved to https://phabricator.wikimedia.org/P97405 and previous config saved to /var/cache/conftool/dbconfig/20261105-213428-ladsgroup.json |
[production] |
| 21:33 |
<ssastry@deploy2003> |
helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply |
[production] |
| 21:30 |
<topranks> |
re-enable BGP for pybal on lvs1019 to fail back from lvs1020 after vlan add T440859 |
[production] |
| 21:28 |
<kemayo@deploy2003> |
mwscript-k8s job started: namespaceDupes zh_min_nanwiki --fix # T430731 |
[production] |
| 21:26 |
<cmooney@cumin1004> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host lvs1019.eqiad.wmnet |
[production] |
| 21:24 |
<ladsgroup@cumin1004> |
dbctl commit (dc=all): 'Repooling after maintenance db2167 (T441633)', diff saved to https://phabricator.wikimedia.org/P97403 and previous config saved to /var/cache/conftool/dbconfig/20261105-212426-ladsgroup.json |
[production] |
| 21:24 |
<kemayo@deploy2003> |
Finished scap sync-world: Backport for [[gerrit:1364770|VisualEditor: Precompute edit suggestion counts on 25% of jawiki + 5% of arwiki (T442767)]], [[gerrit:1340815|Add new namespace for zh_min_nanwiki (T430731)]] (duration: 16m 22s) |
[production] |
| 21:19 |
<cmooney@cumin1004> |
START - Cookbook sre.hosts.reboot-single for host lvs1019.eqiad.wmnet |
[production] |
| 21:17 |
<ladsgroup@cumin1004> |
dbctl commit (dc=all): 'Depooling db2167 (T441633)', diff saved to https://phabricator.wikimedia.org/P97402 and previous config saved to /var/cache/conftool/dbconfig/20261105-211708-ladsgroup.json |
[production] |
| 21:17 |
<ladsgroup@cumin1004> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2167.codfw.wmnet with reason: Maintenance |
[production] |
| 21:16 |
<ladsgroup@cumin1004> |
dbctl commit (dc=all): 'Repooling after maintenance db2166 (T441633)', diff saved to https://phabricator.wikimedia.org/P97401 and previous config saved to /var/cache/conftool/dbconfig/20261105-211647-ladsgroup.json |
[production] |
| 21:15 |
<kemayo@deploy2003> |
pppery, kemayo: Continuing with deployment |
[production] |
| 21:13 |
<kemayo@deploy2003> |
pppery, kemayo: Backport for [[gerrit:1364770|VisualEditor: Precompute edit suggestion counts on 25% of jawiki + 5% of arwiki (T442767)]], [[gerrit:1340815|Add new namespace for zh_min_nanwiki (T430731)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 21:08 |
<kemayo@deploy2003> |
Started scap sync-world: Backport for [[gerrit:1364770|VisualEditor: Precompute edit suggestion counts on 25% of jawiki + 5% of arwiki (T442767)]], [[gerrit:1340815|Add new namespace for zh_min_nanwiki (T430731)]] |
[production] |
| 21:06 |
<ladsgroup@cumin1004> |
dbctl commit (dc=all): 'Repooling after maintenance db2166', diff saved to https://phabricator.wikimedia.org/P97400 and previous config saved to /var/cache/conftool/dbconfig/20261105-210645-ladsgroup.json |
[production] |