|
2026-10-22
ยง
|
| 20:30 |
<dancy@deploy1003> |
Forcefully removing global lock: Deployments locked for the week |
[production] |
| 20:29 |
<dancy@deploy1003> |
Forcefully removing global lock: Deployments locked for the week |
[production] |
| 20:28 |
<btullis@cumin1004> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1012.eqiad.wmnet |
[production] |
| 20:17 |
<rzl@cumin1004> |
END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for phab2003.codfw.wmnet,phab1005.eqiad.wmnet |
[production] |
| 20:17 |
<rzl@cumin1004> |
START - Cookbook sre.hosts.remove-downtime for phab2003.codfw.wmnet,phab1005.eqiad.wmnet |
[production] |
| 20:10 |
<thcipriani> |
phab restart |
[production] |
| 20:10 |
<rzl@cumin1004> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on phab2003.codfw.wmnet,phab1005.eqiad.wmnet with reason: deploying |
[production] |
| 19:58 |
<btullis@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1012.eqiad.wmnet |
[production] |
| 19:57 |
<btullis@cumin1004> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1011.eqiad.wmnet |
[production] |
| 19:57 |
<btullis@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1011.eqiad.wmnet |
[production] |
| 19:51 |
<btullis@cumin1004> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1011.eqiad.wmnet |
[production] |
| 19:39 |
<brett@cumin2003> |
END (PASS) - Cookbook sre.cdn.roll-upgrade-ats (exit_code=0) Rolling upgrade of ATS on P{cp5017.eqsin.wmnet} and A:cp - 9.2.16 upgrade (T441945) |
[production] |
| 19:39 |
<brett@cumin2003> |
END (PASS) - Cookbook sre.cdn.roll-upgrade-ats (exit_code=0) Rolling upgrade of ATS on P{cp5032.eqsin.wmnet} and A:cp - 9.2.16 upgrade (T441945) |
[production] |
| 19:33 |
<larssandergreen> |
civicrm upgraded from b7039ce3 to 5a51e07f |
[fundraising] |
| 19:33 |
<brett@cumin2003> |
START - Cookbook sre.cdn.roll-upgrade-ats Rolling upgrade of ATS on P{cp5032.eqsin.wmnet} and A:cp - 9.2.16 upgrade (T441945) |
[production] |
| 19:33 |
<brett@cumin2003> |
START - Cookbook sre.cdn.roll-upgrade-ats Rolling upgrade of ATS on P{cp5017.eqsin.wmnet} and A:cp - 9.2.16 upgrade (T441945) |
[production] |
| 19:27 |
<rzl@deploy1003> |
helmfile [codfw] DONE helmfile.d/services/mw-debug: apply |
[production] |
| 19:26 |
<rzl@deploy1003> |
helmfile [codfw] START helmfile.d/services/mw-debug: apply |
[production] |
| 19:26 |
<rzl@deploy1003> |
helmfile [eqiad] DONE helmfile.d/services/mw-debug: apply |
[production] |
| 19:26 |
<rzl@deploy1003> |
helmfile [eqiad] START helmfile.d/services/mw-debug: apply |
[production] |
| 19:21 |
<btullis@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1011.eqiad.wmnet |
[production] |
| 19:21 |
<btullis@cumin1004> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1010.eqiad.wmnet |
[production] |
| 19:21 |
<btullis@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1010.eqiad.wmnet |
[production] |
| 19:15 |
<btullis@cumin1004> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1010.eqiad.wmnet |
[production] |
| 19:10 |
<brennen> |
Updating development images on contint primary for https://gitlab.wikimedia.org/repos/releng/dev-images/-/merge_requests/129 (T441915) |
[releng] |
| 19:01 |
<rzl@deploy1003> |
helmfile [codfw] DONE helmfile.d/services/mw-debug: apply |
[production] |
| 18:59 |
<rzl@deploy1003> |
helmfile [codfw] START helmfile.d/services/mw-debug: apply |
[production] |
| 18:55 |
<rzl@deploy1003> |
helmfile [eqiad] DONE helmfile.d/services/mw-debug: apply |
[production] |
| 18:53 |
<rzl@deploy1003> |
helmfile [eqiad] START helmfile.d/services/mw-debug: apply |
[production] |
| 18:52 |
<chlod> |
deployment recreated without PHP component |
[tools.zoomiebot] |
| 18:45 |
<btullis@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1010.eqiad.wmnet |
[production] |
| 18:45 |
<btullis@cumin1004> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1009.eqiad.wmnet |
[production] |
| 18:45 |
<btullis@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1009.eqiad.wmnet |
[production] |
| 18:39 |
<btullis@cumin1004> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1009.eqiad.wmnet |
[production] |
| 18:30 |
<lucaswerkmeister> |
deployed a58bb7b: Upgrade dependencies |
[tools.song-quiz] |
| 18:27 |
<lucaswerkmeister> |
deployed 55e2057: Upgrade dependencies |
[tools.translate-link] |
| 18:23 |
<lucaswerkmeister> |
deployed 19c85e9: Upgrade dependencies |
[tools.ranker] |
| 18:23 |
<AntiComposite> |
webservice stop && webservice start # logrotate still needs to be fixed, but I don't see a config for it |
[tools.poty-stuff] |
| 18:17 |
<AntiComposite> |
toolforge jobs flush && toolforge jobs load jobs.yaml |
[tools.ww-monitor] |
| 18:15 |
<lucaswerkmeister> |
deployed 8cf4a07: Upgrade dependencies |
[tools.speedpatrolling] |
| 18:10 |
<lucaswerkmeister> |
deployed d69d109: Upgrade dependencies |
[tools.wd-image-positions] |
| 18:09 |
<AntiComposite> |
toolforge jobs delete flr-framework && toolforge jobs load toolforge_jobs.yaml |
[tools.yifeibot] |
| 18:09 |
<btullis@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1009.eqiad.wmnet |
[production] |
| 18:08 |
<btullis@cumin1004> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1008.eqiad.wmnet |
[production] |
| 18:08 |
<btullis@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1008.eqiad.wmnet |
[production] |
| 18:06 |
<AntiComposite> |
webservice python3.9 start |
[tools.bullseye] |
| 18:05 |
<AntiComposite> |
webservice stop -- could not restart, needs upgrade to python 3.13 |
[tools.bullseye] |
| 18:02 |
<AntiComposite> |
webservice stop && webservice start |
[tools.stewardbots] |
| 18:02 |
<AntiComposite> |
kubectl delete deployment.apps/sulwatcher && kubectl apply --validate=true -f SULWatcher/k8s-deployment.yaml |
[tools.stewardbots] |
| 18:01 |
<btullis@cumin1004> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1008.eqiad.wmnet |
[production] |