|
2025-05-27
§
|
| 07:33 |
<ayounsi@cumin1002> |
END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'email' for AS: 15932 |
[production] |
| 07:33 |
<ayounsi@cumin1002> |
START - Cookbook sre.network.peering with action 'email' for AS: 15932 |
[production] |
| 07:32 |
<ayounsi@cumin1002> |
END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'email' for AS: 10089 |
[production] |
| 07:32 |
<ayounsi@cumin1002> |
START - Cookbook sre.network.peering with action 'email' for AS: 10089 |
[production] |
| 07:32 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host mc-misc2002.codfw.wmnet |
[production] |
| 07:32 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.ganeti.changedisk (exit_code=0) for changing disk type of durum7002.magru.wmnet to plain |
[production] |
| 07:31 |
<jmm@cumin2002> |
START - Cookbook sre.ganeti.changedisk for changing disk type of durum7002.magru.wmnet to plain |
[production] |
| 07:30 |
<ayounsi@cumin1002> |
END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'email' for AS: 17806 |
[production] |
| 07:30 |
<ayounsi@cumin1002> |
START - Cookbook sre.network.peering with action 'email' for AS: 17806 |
[production] |
| 07:23 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host mc-misc2002.codfw.wmnet |
[production] |
| 07:19 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host mc-misc2002.codfw.wmnet |
[production] |
| 07:10 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host mc-misc2002.codfw.wmnet |
[production] |
| 07:07 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet |
[production] |
| 07:05 |
<ayounsi@cumin1002> |
END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'email' for AS: 15133 |
[production] |
| 07:05 |
<ayounsi@cumin1002> |
START - Cookbook sre.network.peering with action 'email' for AS: 15133 |
[production] |
| 07:01 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet |
[production] |
| 06:58 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet |
[production] |
| 06:38 |
<moritzm> |
failover Ganeti master in magru/B4 to ganeti7004 T394263 |
[production] |
| 06:31 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet |
[production] |
| 04:01 |
<mwpresync@deploy1003> |
Pruned MediaWiki: 1.44.0-wmf.28 (duration: 01m 50s) |
[production] |
| 03:49 |
<mwpresync@deploy1003> |
Finished scap sync-world: testwikis to 1.45.0-wmf.3 refs T392173 (duration: 46m 49s) |
[production] |
| 03:02 |
<mwpresync@deploy1003> |
Started scap sync-world: testwikis to 1.45.0-wmf.3 refs T392173 |
[production] |
| 00:20 |
<dani@deploy1003> |
helmfile [codfw] DONE helmfile.d/services/miscweb: apply |
[production] |
| 00:20 |
<dani@deploy1003> |
helmfile [codfw] START helmfile.d/services/miscweb: apply |
[production] |
| 00:20 |
<dani@deploy1003> |
helmfile [eqiad] DONE helmfile.d/services/miscweb: apply |
[production] |
| 00:20 |
<dani@deploy1003> |
helmfile [eqiad] START helmfile.d/services/miscweb: apply |
[production] |
| 00:20 |
<dani@deploy1003> |
helmfile [staging] DONE helmfile.d/services/miscweb: apply |
[production] |
| 00:20 |
<dani@deploy1003> |
helmfile [staging] START helmfile.d/services/miscweb: apply |
[production] |
|
2025-05-26
§
|
| 21:56 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2228 (T395241)', diff saved to https://phabricator.wikimedia.org/P76461 and previous config saved to /var/cache/conftool/dbconfig/20250526-215649-fceratto.json |
[production] |
| 21:41 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2228', diff saved to https://phabricator.wikimedia.org/P76460 and previous config saved to /var/cache/conftool/dbconfig/20250526-214142-fceratto.json |
[production] |
| 21:26 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2228', diff saved to https://phabricator.wikimedia.org/P76459 and previous config saved to /var/cache/conftool/dbconfig/20250526-212634-fceratto.json |
[production] |
| 21:11 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2228 (T395241)', diff saved to https://phabricator.wikimedia.org/P76458 and previous config saved to /var/cache/conftool/dbconfig/20250526-211127-fceratto.json |
[production] |
| 21:04 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Depooling db2228 (T395241)', diff saved to https://phabricator.wikimedia.org/P76457 and previous config saved to /var/cache/conftool/dbconfig/20250526-210445-fceratto.json |
[production] |
| 21:04 |
<fceratto@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on db2186.codfw.wmnet with reason: Maintenance |
[production] |
| 21:04 |
<fceratto@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2228.codfw.wmnet with reason: Maintenance |
[production] |
| 21:04 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2223 (T395241)', diff saved to https://phabricator.wikimedia.org/P76456 and previous config saved to /var/cache/conftool/dbconfig/20250526-210402-fceratto.json |
[production] |
| 20:48 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2223', diff saved to https://phabricator.wikimedia.org/P76455 and previous config saved to /var/cache/conftool/dbconfig/20250526-204855-fceratto.json |
[production] |
| 20:33 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2223', diff saved to https://phabricator.wikimedia.org/P76454 and previous config saved to /var/cache/conftool/dbconfig/20250526-203348-fceratto.json |
[production] |
| 20:18 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2223 (T395241)', diff saved to https://phabricator.wikimedia.org/P76453 and previous config saved to /var/cache/conftool/dbconfig/20250526-201840-fceratto.json |
[production] |
| 20:11 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Depooling db2223 (T395241)', diff saved to https://phabricator.wikimedia.org/P76452 and previous config saved to /var/cache/conftool/dbconfig/20250526-201150-fceratto.json |
[production] |
| 20:11 |
<fceratto@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2223.codfw.wmnet with reason: Maintenance |
[production] |
| 20:11 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2211 (T395241)', diff saved to https://phabricator.wikimedia.org/P76451 and previous config saved to /var/cache/conftool/dbconfig/20250526-201123-fceratto.json |
[production] |
| 19:56 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2211', diff saved to https://phabricator.wikimedia.org/P76450 and previous config saved to /var/cache/conftool/dbconfig/20250526-195614-fceratto.json |
[production] |
| 19:41 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2211', diff saved to https://phabricator.wikimedia.org/P76449 and previous config saved to /var/cache/conftool/dbconfig/20250526-194107-fceratto.json |
[production] |
| 19:33 |
<aokoth@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3 days, 0:00:00 on doc1004.eqiad.wmnet with reason: Bookworm |
[production] |
| 19:30 |
<denisse> |
Re-enable sync between grafana hosts - T395098 |
[production] |
| 19:29 |
<aokoth@deploy1003> |
helmfile [aux-k8s-codfw] DONE helmfile.d/services/miscweb: apply |
[production] |
| 19:28 |
<aokoth@deploy1003> |
helmfile [aux-k8s-codfw] START helmfile.d/services/miscweb: apply |
[production] |
| 19:26 |
<aokoth@deploy1003> |
helmfile [aux-k8s-codfw] DONE helmfile.d/admin 'apply'. |
[production] |
| 19:26 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2211 (T395241)', diff saved to https://phabricator.wikimedia.org/P76448 and previous config saved to /var/cache/conftool/dbconfig/20250526-192600-fceratto.json |
[production] |