|
2026-11-05
ยง
|
| 08:01 |
<marostegui@cumin1004> |
START - Cookbook sre.mysql.decommission |
[production] |
| 07:59 |
<jmm@cumin2003> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast6003.wikimedia.org |
[production] |
| 07:56 |
<moritzm> |
installing expat security updates |
[production] |
| 07:53 |
<jmm@cumin2003> |
START - Cookbook sre.hosts.reboot-single for host bast6003.wikimedia.org |
[production] |
| 07:37 |
<jmm@cumin2003> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast2003.wikimedia.org |
[production] |
| 07:30 |
<jmm@cumin2003> |
START - Cookbook sre.hosts.reboot-single for host bast2003.wikimedia.org |
[production] |
| 07:26 |
<marostegui> |
Drop thread_reaction tables T442289 |
[production] |
| 07:23 |
<marostegui@cumin1004> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host dbprov2007.codfw.wmnet with OS trixie |
[production] |
| 07:07 |
<marostegui@cumin1004> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on dbprov2007.codfw.wmnet with reason: host reimage |
[production] |
| 07:03 |
<marostegui@cumin1004> |
START - Cookbook sre.hosts.downtime for 2:00:00 on dbprov2007.codfw.wmnet with reason: host reimage |
[production] |
| 07:01 |
<marostegui@cumin1004> |
END (PASS) - Cookbook sre.hosts.dhcp (exit_code=0) for host dbprov2007.codfw.wmnet (remove config only) |
[production] |
| 07:01 |
<marostegui@cumin1004> |
START - Cookbook sre.hosts.dhcp for host dbprov2007.codfw.wmnet (remove config only) |
[production] |
| 06:49 |
<marostegui@cumin1004> |
END (PASS) - Cookbook sre.hosts.dhcp (exit_code=0) for host dbprov2007.codfw.wmnet (push config only) |
[production] |
| 06:49 |
<marostegui@cumin1004> |
START - Cookbook sre.hosts.dhcp for host dbprov2007.codfw.wmnet (push config only) |
[production] |
| 06:48 |
<marostegui@cumin1004> |
START - Cookbook sre.hosts.reimage for host dbprov2007.codfw.wmnet with OS trixie |
[production] |
| 05:55 |
<btullis@cumin1004> |
END (PASS) - Cookbook sre.k8s.reboot-nodes (exit_code=0) rolling reboot on A:dse-k8s-worker-eqiad |
[production] |
| 05:55 |
<btullis@cumin1004> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1044.eqiad.wmnet |
[production] |
| 05:55 |
<btullis@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1044.eqiad.wmnet |
[production] |
| 05:48 |
<btullis@cumin1004> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1044.eqiad.wmnet |
[production] |
| 05:21 |
<denisse@cumin1004> |
DONE (FAIL) - Cookbook sre.idm.logout (exit_code=99) Logging Cfoster out of all services on: 4 hosts |
[production] |
| 05:18 |
<btullis@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1044.eqiad.wmnet |
[production] |
| 05:18 |
<btullis@cumin1004> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1043.eqiad.wmnet |
[production] |
| 05:18 |
<btullis@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1043.eqiad.wmnet |
[production] |
| 05:11 |
<btullis@cumin1004> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1043.eqiad.wmnet |
[production] |
| 05:08 |
<denisse@cumin1004> |
DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Cfoster out of all services on: 4 hosts |
[production] |
| 04:41 |
<btullis@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1043.eqiad.wmnet |
[production] |
| 04:41 |
<btullis@cumin1004> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1042.eqiad.wmnet |
[production] |
| 04:41 |
<btullis@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1042.eqiad.wmnet |
[production] |
| 04:33 |
<ryankemper@cumin2003> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) (T441682, copy the reloaded categories journal to wdqs-internal-main (fills the 10-15 to 10-28 Commons gap)) xfer categories from wdqs2008.codfw.wmnet -> wdqs2020.codfw.wmnet, repooling both afterwards |
[production] |
| 04:33 |
<btullis@cumin1004> |
END (FAIL) - Cookbook sre.k8s.pool-depool-node (exit_code=99) depool for host dse-k8s-worker1042.eqiad.wmnet |
[production] |
| 04:31 |
<btullis@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1042.eqiad.wmnet |
[production] |
| 04:31 |
<btullis@cumin1004> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1041.eqiad.wmnet |
[production] |
| 04:31 |
<btullis@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1041.eqiad.wmnet |
[production] |
| 04:27 |
<ryankemper@cumin2003> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) (T441682, copy the reloaded categories journal to wdqs-internal-main (fills the 10-15 to 10-28 Commons gap)) xfer categories from wdqs1011.eqiad.wmnet -> wdqs1026.eqiad.wmnet, repooling both afterwards |
[production] |
| 04:26 |
<ryankemper@cumin2003> |
START - Cookbook sre.wdqs.data-transfer (T441682, copy the reloaded categories journal to wdqs-internal-main (fills the 10-15 to 10-28 Commons gap)) xfer categories from wdqs2008.codfw.wmnet -> wdqs2020.codfw.wmnet, repooling both afterwards |
[production] |
| 04:26 |
<ryankemper@cumin2003> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) (T441682, copy the reloaded categories journal to wdqs-internal-main (fills the 10-15 to 10-28 Commons gap)) xfer categories from wdqs2008.codfw.wmnet -> wdqs2019.codfw.wmnet, repooling both afterwards |
[production] |
| 04:26 |
<btullis@cumin1004> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1041.eqiad.wmnet |
[production] |
| 04:19 |
<ryankemper@cumin2003> |
START - Cookbook sre.wdqs.data-transfer (T441682, copy the reloaded categories journal to wdqs-internal-main (fills the 10-15 to 10-28 Commons gap)) xfer categories from wdqs1011.eqiad.wmnet -> wdqs1026.eqiad.wmnet, repooling both afterwards |
[production] |
| 04:19 |
<ryankemper@cumin2003> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) (T441682, copy the reloaded categories journal to wdqs-internal-main (fills the 10-15 to 10-28 Commons gap)) xfer categories from wdqs1011.eqiad.wmnet -> wdqs1025.eqiad.wmnet, repooling both afterwards |
[production] |
| 04:18 |
<ryankemper@cumin2003> |
START - Cookbook sre.wdqs.data-transfer (T441682, copy the reloaded categories journal to wdqs-internal-main (fills the 10-15 to 10-28 Commons gap)) xfer categories from wdqs2008.codfw.wmnet -> wdqs2019.codfw.wmnet, repooling both afterwards |
[production] |
| 04:18 |
<ryankemper@cumin2003> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) (T441682, copy the reloaded categories journal to wdqs-internal-main (fills the 10-15 to 10-28 Commons gap)) xfer categories from wdqs2008.codfw.wmnet -> wdqs2018.codfw.wmnet, repooling both afterwards |
[production] |
| 04:11 |
<ryankemper@cumin2003> |
START - Cookbook sre.wdqs.data-transfer (T441682, copy the reloaded categories journal to wdqs-internal-main (fills the 10-15 to 10-28 Commons gap)) xfer categories from wdqs1011.eqiad.wmnet -> wdqs1025.eqiad.wmnet, repooling both afterwards |
[production] |
| 04:11 |
<ryankemper@cumin2003> |
START - Cookbook sre.wdqs.data-transfer (T441682, copy the reloaded categories journal to wdqs-internal-main (fills the 10-15 to 10-28 Commons gap)) xfer categories from wdqs2008.codfw.wmnet -> wdqs2018.codfw.wmnet, repooling both afterwards |
[production] |
| 03:56 |
<btullis@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1041.eqiad.wmnet |
[production] |
| 03:56 |
<btullis@cumin1004> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1040.eqiad.wmnet |
[production] |
| 03:56 |
<btullis@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1040.eqiad.wmnet |
[production] |
| 03:46 |
<btullis@cumin1004> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host dse-k8s-worker1040.eqiad.wmnet |
[production] |
| 03:16 |
<btullis@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node depool for host dse-k8s-worker1040.eqiad.wmnet |
[production] |
| 03:16 |
<btullis@cumin1004> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host dse-k8s-worker1039.eqiad.wmnet |
[production] |
| 03:16 |
<btullis@cumin1004> |
START - Cookbook sre.k8s.pool-depool-node pool for host dse-k8s-worker1039.eqiad.wmnet |
[production] |