|
2025-09-12
§
|
| 21:25 |
<dzahn@cumin2002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3 days, 0:00:00 on zuul1002.eqiad.wmnet with reason: in setup |
[production] |
| 21:25 |
<dzahn@cumin2002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3 days, 0:00:00 on zuul2002.codfw.wmnet with reason: in setup |
[production] |
| 20:20 |
<ladsgroup@cumin1003> |
END (FAIL) - Cookbook sre.mysql.pool (exit_code=99) db1160* gradually with 4 steps - Work done |
[production] |
| 20:05 |
<ladsgroup@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on dbstore1009.eqiad.wmnet with reason: Maintenance |
[production] |
| 20:05 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1226 (T402763)', diff saved to https://phabricator.wikimedia.org/P83301 and previous config saved to /var/cache/conftool/dbconfig/20250912-200538-ladsgroup.json |
[production] |
| 19:50 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1226', diff saved to https://phabricator.wikimedia.org/P83300 and previous config saved to /var/cache/conftool/dbconfig/20250912-195030-ladsgroup.json |
[production] |
| 19:35 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1226', diff saved to https://phabricator.wikimedia.org/P83299 and previous config saved to /var/cache/conftool/dbconfig/20250912-193523-ladsgroup.json |
[production] |
| 19:20 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1226 (T402763)', diff saved to https://phabricator.wikimedia.org/P83298 and previous config saved to /var/cache/conftool/dbconfig/20250912-192015-ladsgroup.json |
[production] |
| 19:14 |
<James_F> |
Zuul: [mediawiki/extensions/DataTable2] Add Scribuntu phan dependency |
[releng] |
| 19:13 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Depooling db1226 (T402763)', diff saved to https://phabricator.wikimedia.org/P83297 and previous config saved to /var/cache/conftool/dbconfig/20250912-191338-ladsgroup.json |
[production] |
| 19:13 |
<ladsgroup@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on db1226.eqiad.wmnet with reason: Maintenance |
[production] |
| 19:13 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1214 (T402763)', diff saved to https://phabricator.wikimedia.org/P83296 and previous config saved to /var/cache/conftool/dbconfig/20250912-191314-ladsgroup.json |
[production] |
| 18:58 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1214', diff saved to https://phabricator.wikimedia.org/P83295 and previous config saved to /var/cache/conftool/dbconfig/20250912-185807-ladsgroup.json |
[production] |
| 18:43 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1214', diff saved to https://phabricator.wikimedia.org/P83294 and previous config saved to /var/cache/conftool/dbconfig/20250912-184259-ladsgroup.json |
[production] |
| 18:27 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1214 (T402763)', diff saved to https://phabricator.wikimedia.org/P83293 and previous config saved to /var/cache/conftool/dbconfig/20250912-182752-ladsgroup.json |
[production] |
| 18:21 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Depooling db1214 (T402763)', diff saved to https://phabricator.wikimedia.org/P83292 and previous config saved to /var/cache/conftool/dbconfig/20250912-182126-ladsgroup.json |
[production] |
| 18:21 |
<ladsgroup@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on db1214.eqiad.wmnet with reason: Maintenance |
[production] |
| 18:21 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1209 (T402763)', diff saved to https://phabricator.wikimedia.org/P83291 and previous config saved to /var/cache/conftool/dbconfig/20250912-182104-ladsgroup.json |
[production] |
| 18:14 |
<mutante> |
DNS - added new project language 'tok' (tok.wikipedia.org) (Toki Pona) https://en.wikipedia.org/wiki/Toki_Pona - T404457 |
[production] |
| 18:13 |
<dzahn@dns1004> |
END - running authdns-update |
[production] |
| 18:11 |
<dzahn@dns1004> |
START - running authdns-update |
[production] |
| 18:05 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1209', diff saved to https://phabricator.wikimedia.org/P83290 and previous config saved to /var/cache/conftool/dbconfig/20250912-180557-ladsgroup.json |
[production] |
| 17:50 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1209', diff saved to https://phabricator.wikimedia.org/P83289 and previous config saved to /var/cache/conftool/dbconfig/20250912-175049-ladsgroup.json |
[production] |
| 17:49 |
<bking@deploy1003> |
helmfile [dse-k8s-codfw] DONE helmfile.d/admin 'apply'. |
[production] |
| 17:49 |
<bking@deploy1003> |
helmfile [dse-k8s-codfw] START helmfile.d/admin 'apply'. |
[production] |
| 17:48 |
<bking@deploy1003> |
helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. |
[production] |
| 17:48 |
<bking@deploy1003> |
helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. |
[production] |
| 17:35 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1209 (T402763)', diff saved to https://phabricator.wikimedia.org/P83288 and previous config saved to /var/cache/conftool/dbconfig/20250912-173542-ladsgroup.json |
[production] |
| 17:29 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Depooling db1209 (T402763)', diff saved to https://phabricator.wikimedia.org/P83287 and previous config saved to /var/cache/conftool/dbconfig/20250912-172909-ladsgroup.json |
[production] |
| 17:29 |
<ladsgroup@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on db1209.eqiad.wmnet with reason: Maintenance |
[production] |
| 17:28 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1203 (T402763)', diff saved to https://phabricator.wikimedia.org/P83286 and previous config saved to /var/cache/conftool/dbconfig/20250912-172847-ladsgroup.json |
[production] |
| 17:13 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1203', diff saved to https://phabricator.wikimedia.org/P83285 and previous config saved to /var/cache/conftool/dbconfig/20250912-171339-ladsgroup.json |
[production] |
| 16:58 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1203', diff saved to https://phabricator.wikimedia.org/P83284 and previous config saved to /var/cache/conftool/dbconfig/20250912-165832-ladsgroup.json |
[production] |
| 16:43 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1203 (T402763)', diff saved to https://phabricator.wikimedia.org/P83283 and previous config saved to /var/cache/conftool/dbconfig/20250912-164324-ladsgroup.json |
[production] |
| 16:38 |
<brett> |
Manually running clean-stale-certs.service on acmechief2002 - T399419 |
[production] |
| 16:36 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Depooling db1203 (T402763)', diff saved to https://phabricator.wikimedia.org/P83281 and previous config saved to /var/cache/conftool/dbconfig/20250912-163605-ladsgroup.json |
[production] |
| 16:35 |
<ladsgroup@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on db1203.eqiad.wmnet with reason: Maintenance |
[production] |
| 16:35 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1192 (T402763)', diff saved to https://phabricator.wikimedia.org/P83280 and previous config saved to /var/cache/conftool/dbconfig/20250912-163541-ladsgroup.json |
[production] |
| 16:23 |
<Krinkle> |
Enable `/etc/trafficserver/lua/rb-mw-mangling-beta.lua` plugin on /api/rest_v1 (restbase05) and /w/api.api (mediawiki13) regex_map routes in the Beta Cluster. It was previously only enabled for the mediawiki14 route, which ironically does not cover restbase. https://gerrit.wikimedia.org/r/plugins/gitiles/cloud/instance-puppet/+/1310d67ac6dffe81e4e7a2e68ec33cbba1a7c7ea%5E%21/#F0 ref T404387 |
[releng] |
| 16:20 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P83279 and previous config saved to /var/cache/conftool/dbconfig/20250912-162033-ladsgroup.json |
[production] |
| 16:15 |
<btullis@cumin1003> |
END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host an-worker1236.eqiad.wmnet with OS bullseye |
[production] |
| 16:05 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P83278 and previous config saved to /var/cache/conftool/dbconfig/20250912-160526-ladsgroup.json |
[production] |
| 16:01 |
<herron@cumin1003> |
END (PASS) - Cookbook sre.kafka.roll-restart-reboot-brokers (exit_code=0) rolling restart_daemons on A:kafka-logging-codfw |
[production] |
| 15:50 |
<ladsgroup@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1192 (T402763)', diff saved to https://phabricator.wikimedia.org/P83277 and previous config saved to /var/cache/conftool/dbconfig/20250912-155018-ladsgroup.json |
[production] |