|
2025-11-11
ยง
|
| 16:46 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1221', diff saved to https://phabricator.wikimedia.org/P85199 and previous config saved to /var/cache/conftool/dbconfig/20251111-164651-marostegui.json |
[production] |
| 16:38 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'db2246 (re)pooling @ 100%: After schema change', diff saved to https://phabricator.wikimedia.org/P85198 and previous config saved to /var/cache/conftool/dbconfig/20251111-163811-root.json |
[production] |
| 16:31 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1221', diff saved to https://phabricator.wikimedia.org/P85197 and previous config saved to /var/cache/conftool/dbconfig/20251111-163144-marostegui.json |
[production] |
| 16:24 |
<btullis@cumin1003> |
END (PASS) - Cookbook sre.k8s.reboot-nodes (exit_code=0) rolling reboot on P{dse-k8s-worker1004.eqiad.wmnet} and (A:dse-k8s-master-eqiad or A:dse-k8s-worker-eqiad) |
[production] |
| 16:23 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'db2246 (re)pooling @ 75%: After schema change', diff saved to https://phabricator.wikimedia.org/P85196 and previous config saved to /var/cache/conftool/dbconfig/20251111-162305-root.json |
[production] |
| 16:22 |
<marostegui> |
Drop afl_ip related triggers from s4 T408780 |
[production] |
| 16:21 |
<marostegui> |
Install MariaDB 10.11.15 on Debian Trixie es1033 T409533 |
[production] |
| 16:16 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db1221 (T407997)', diff saved to https://phabricator.wikimedia.org/P85195 and previous config saved to /var/cache/conftool/dbconfig/20251111-161636-marostegui.json |
[production] |
| 16:09 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Depooling db1221 (T407997)', diff saved to https://phabricator.wikimedia.org/P85194 and previous config saved to /var/cache/conftool/dbconfig/20251111-160950-marostegui.json |
[production] |
| 16:09 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1015,1019].eqiad.wmnet,db1155.eqiad.wmnet with reason: Maintenance |
[production] |
| 16:09 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1221.eqiad.wmnet with reason: Maintenance |
[production] |
| 16:08 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'db2246 (re)pooling @ 50%: After schema change', diff saved to https://phabricator.wikimedia.org/P85193 and previous config saved to /var/cache/conftool/dbconfig/20251111-160759-root.json |
[production] |
| 16:01 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2246', diff saved to https://phabricator.wikimedia.org/P85192 and previous config saved to /var/cache/conftool/dbconfig/20251111-160132-marostegui.json |
[production] |
| 15:53 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host puppetboard1003.eqiad.wmnet |
[production] |
| 15:49 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host puppetboard1003.eqiad.wmnet |
[production] |
| 15:46 |
<btullis@cumin1003> |
START - Cookbook sre.k8s.reboot-nodes rolling reboot on P{dse-k8s-worker1004.eqiad.wmnet} and (A:dse-k8s-master-eqiad or A:dse-k8s-worker-eqiad) |
[production] |
| 15:46 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2246 (T407997)', diff saved to https://phabricator.wikimedia.org/P85191 and previous config saved to /var/cache/conftool/dbconfig/20251111-154624-marostegui.json |
[production] |
| 15:42 |
<wm-bot2> |
Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/19270642865 (https://github.com/cluebotng/component-configs/commits/3fe913812986e82db75d4a6657cba3f697f5649c) |
[tools.cluebotng] |
| 15:41 |
<wm-bot2> |
Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/19270642940 (https://github.com/cluebotng/component-configs/commits/3fe913812986e82db75d4a6657cba3f697f5649c) |
[tools.cluebotng-monitoring] |
| 15:39 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Depooling db2246 (T407997)', diff saved to https://phabricator.wikimedia.org/P85189 and previous config saved to /var/cache/conftool/dbconfig/20251111-153942-marostegui.json |
[production] |
| 15:39 |
<wm-bot2> |
Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/19270642915 (https://github.com/cluebotng/component-configs/commits/3fe913812986e82db75d4a6657cba3f697f5649c) |
[tools.cluebotng-review] |
| 15:39 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2246.codfw.wmnet with reason: Maintenance |
[production] |
| 15:39 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2245 (T407997)', diff saved to https://phabricator.wikimedia.org/P85188 and previous config saved to /var/cache/conftool/dbconfig/20251111-153918-marostegui.json |
[production] |
| 15:38 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet |
[production] |
| 15:38 |
<wm-bot2> |
Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/19270642949 (https://github.com/cluebotng/component-configs/commits/3fe913812986e82db75d4a6657cba3f697f5649c) |
[tools.cluebotng-staging] |
| 15:37 |
<wm-bot2> |
Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/19270642882 (https://github.com/cluebotng/component-configs/commits/3fe913812986e82db75d4a6657cba3f697f5649c) |
[tools.cluebotng-trainer] |
| 15:37 |
<wm-bot2> |
Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/19270643026 (https://github.com/cluebotng/component-configs/commits/3fe913812986e82db75d4a6657cba3f697f5649c) |
[tools.toolforge-functional-runner] |
| 15:37 |
<wm-bot2> |
Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/19270642926 (https://github.com/cluebotng/component-configs/commits/3fe913812986e82db75d4a6657cba3f697f5649c) |
[tools.cluebot-syncer] |
| 15:37 |
<wm-bot2> |
Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/19270642903 (https://github.com/cluebotng/component-configs/commits/3fe913812986e82db75d4a6657cba3f697f5649c) |
[tools.cluebot3] |
| 15:37 |
<wm-bot2> |
Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/19270642885 (https://github.com/cluebotng/component-configs/commits/3fe913812986e82db75d4a6657cba3f697f5649c) |
[tools.cluebotng-editsets] |
| 15:36 |
<wm-bot2> |
Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/19270642865 (https://github.com/cluebotng/component-configs/commits/3fe913812986e82db75d4a6657cba3f697f5649c) |
[tools.cluebotng] |
| 15:36 |
<wm-bot2> |
Deployment failed: https://github.com/cluebotng/component-configs/actions/runs/19270642940 (https://github.com/cluebotng/component-configs/commits/3fe913812986e82db75d4a6657cba3f697f5649c) |
[tools.cluebotng-monitoring] |
| 15:32 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet |
[production] |
| 15:31 |
<wm-bot2> |
Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/19270294471 (https://github.com/cluebotng/component-configs/commits/df4e433c6a567df4484b7115ddf2c53fe1f9494f) |
[tools.cluebotng-monitoring] |
| 15:30 |
<wm-bot2> |
Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/19270301650 (https://github.com/cluebotng/component-configs/commits/f28dcaec8c5882b4a1b7d861fe7f5e400312a5b4) |
[tools.cluebotng] |
| 15:28 |
<volans@cloudcumin1001> |
END (PASS) - Cookbook wmcs.toolforge.k8s.logging.copy_images_to_registry (exit_code=0) for Loki 3.5.7 (T399313) |
[tools] |
| 15:28 |
<wm-bot2> |
Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/19270313845 (https://github.com/cluebotng/component-configs/commits/900a30b8983bc80a537330fa4345e7952f2081b4) |
[tools.cluebotng-trainer] |
| 15:28 |
<volans@cloudcumin1001> |
Updating container image docker-registry.svc.toolforge.org/grafana/loki:3.5.7 (T399313) |
[tools] |
| 15:28 |
<volans@cloudcumin1001> |
START - Cookbook wmcs.toolforge.k8s.logging.copy_images_to_registry for Loki 3.5.7 (T399313) |
[tools] |
| 15:27 |
<wm-bot2> |
Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/19270285003 (https://github.com/cluebotng/component-configs/commits/e103f6ac56b26a2d6e3c0705c81c75a4419287cc) |
[tools.cluebotng-monitoring] |
| 15:27 |
<wm-bot2> |
Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/19270301618 (https://github.com/cluebotng/component-configs/commits/f28dcaec8c5882b4a1b7d861fe7f5e400312a5b4) |
[tools.cluebotng-staging] |
| 15:27 |
<wm-bot2> |
Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/19270263370 (https://github.com/cluebotng/component-configs/commits/d1674e8f4f6cec3b48e848137ce42585278d4a67) |
[tools.cluebotng-review] |
| 15:26 |
<wm-bot2> |
Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/19270303362 (https://github.com/cluebotng/component-configs/commits/df94fd299dd1ed07436488c348bad61d859da592) |
[tools.cluebot-syncer] |
| 15:25 |
<wm-bot2> |
Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/19270264807 (https://github.com/cluebotng/component-configs/commits/531100511571d8262fc874f0f4cddfa2df3d6c2b) |
[tools.cluebotng-trainer] |
| 15:25 |
<wm-bot2> |
Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/19270264790 (https://github.com/cluebotng/component-configs/commits/531100511571d8262fc874f0f4cddfa2df3d6c2b) |
[tools.toolforge-functional-runner] |
| 15:25 |
<wm-bot2> |
Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/19270276189 (https://github.com/cluebotng/component-configs/commits/040b83debcd71b207ee872095a3d2a1166ccc618) |
[tools.cluebotng-editsets] |
| 15:24 |
<wm-bot2> |
Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/19270254406 (https://github.com/cluebotng/component-configs/commits/5dfa0eff0c51e0d878863a5ac689e59f79d59876) |
[tools.cluebot3] |
| 15:24 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2245', diff saved to https://phabricator.wikimedia.org/P85187 and previous config saved to /var/cache/conftool/dbconfig/20251111-152410-marostegui.json |
[production] |
| 15:09 |
<kostajh> |
UTC afternoon deploys done |
[production] |
| 15:09 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Repooling after maintenance db2245', diff saved to https://phabricator.wikimedia.org/P85186 and previous config saved to /var/cache/conftool/dbconfig/20251111-150902-marostegui.json |
[production] |