|
2024-11-18
ยง
|
| 22:37 |
<jhancock@cumin2002> |
END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host es2042.codfw.wmnet with OS bookworm |
[production] |
| 22:37 |
<bking@deploy2002> |
Started deploy [wdqs/wdqs@9927a5a]: 0.3.150 |
[production] |
| 22:22 |
<jhathaway@cumin2002> |
END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host thanos-be2005.codfw.wmnet with OS bookworm |
[production] |
| 22:18 |
<urbanecm@deploy2002> |
Finished scap sync-world: Backport for [[gerrit:1092336|[GrowthExperiments] testwiki: Only enable Add Link for new accounts (T380204)]] (duration: 09m 14s) |
[production] |
| 22:13 |
<urbanecm@deploy2002> |
urbanecm: Continuing with sync |
[production] |
| 22:13 |
<urbanecm@deploy2002> |
urbanecm: Backport for [[gerrit:1092336|[GrowthExperiments] testwiki: Only enable Add Link for new accounts (T380204)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 22:09 |
<urbanecm@deploy2002> |
Started scap sync-world: Backport for [[gerrit:1092336|[GrowthExperiments] testwiki: Only enable Add Link for new accounts (T380204)]] |
[production] |
| 21:58 |
<urbanecm@deploy2002> |
Finished scap sync-world: Backport for [[gerrit:1092304|Use WAN cache for JsonConfig remote fetch cache (T374746)]], [[gerrit:1092300|Create no-link-recommendation variant (T377787 T380204)]], [[gerrit:1092295|[GrowthExperiments] testwiki: Enable no-link-recommendation experiment (T380204)]] (duration: 12m 10s) |
[production] |
| 21:54 |
<urbanecm@deploy2002> |
urbanecm, bvibber: Continuing with sync |
[production] |
| 21:52 |
<urbanecm@deploy2002> |
urbanecm, bvibber: Backport for [[gerrit:1092304|Use WAN cache for JsonConfig remote fetch cache (T374746)]], [[gerrit:1092300|Create no-link-recommendation variant (T377787 T380204)]], [[gerrit:1092295|[GrowthExperiments] testwiki: Enable no-link-recommendation experiment (T380204)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 21:48 |
<effie> |
upload prometheus-mcrouter-exporter_0.4.0+git20241118-1~wmf1 - T380212 |
[production] |
| 21:46 |
<urbanecm@deploy2002> |
Started scap sync-world: Backport for [[gerrit:1092304|Use WAN cache for JsonConfig remote fetch cache (T374746)]], [[gerrit:1092300|Create no-link-recommendation variant (T377787 T380204)]], [[gerrit:1092295|[GrowthExperiments] testwiki: Enable no-link-recommendation experiment (T380204)]] |
[production] |
| 21:42 |
<pt1979@cumin2002> |
START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - pt1979@cumin2002" |
[production] |
| 21:36 |
<urbanecm@deploy2002> |
Finished scap sync-world: Backport for [[gerrit:1091839|Rename everything referring to "SSO domain" to use "shared domain" (T379811)]], [[gerrit:1091841|Rename shared domain sso.wikimedia.org to auth.wikimedia.org (T379811)]], [[gerrit:1091842|Use DB name rather than server name in shared domain path prefix (T379811)]] (duration: 10m 54s) |
[production] |
| 21:31 |
<urbanecm@deploy2002> |
matmarex, urbanecm: Continuing with sync |
[production] |
| 21:30 |
<urbanecm@deploy2002> |
matmarex, urbanecm: Backport for [[gerrit:1091839|Rename everything referring to "SSO domain" to use "shared domain" (T379811)]], [[gerrit:1091841|Rename shared domain sso.wikimedia.org to auth.wikimedia.org (T379811)]], [[gerrit:1091842|Use DB name rather than server name in shared domain path prefix (T379811)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 21:29 |
<urbanecm> |
Add bvibber to wmf-deployment Gerrit group (existing deployer) |
[production] |
| 21:26 |
<urbanecm@deploy2002> |
Started scap sync-world: Backport for [[gerrit:1091839|Rename everything referring to "SSO domain" to use "shared domain" (T379811)]], [[gerrit:1091841|Rename shared domain sso.wikimedia.org to auth.wikimedia.org (T379811)]], [[gerrit:1091842|Use DB name rather than server name in shared domain path prefix (T379811)]] |
[production] |
| 21:21 |
<pt1979@cumin2002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on maps-test2001.codfw.wmnet with reason: host reimage |
[production] |
| 21:18 |
<pt1979@cumin2002> |
START - Cookbook sre.hosts.downtime for 2:00:00 on maps-test2001.codfw.wmnet with reason: host reimage |
[production] |
| 21:17 |
<jhancock@cumin2002> |
START - Cookbook sre.hosts.reimage for host es2046.codfw.wmnet with OS bookworm |
[production] |
| 21:17 |
<jhancock@cumin2002> |
START - Cookbook sre.hosts.reimage for host es2045.codfw.wmnet with OS bookworm |
[production] |
| 21:17 |
<jhancock@cumin2002> |
START - Cookbook sre.hosts.reimage for host es2044.codfw.wmnet with OS bookworm |
[production] |
| 21:17 |
<jhancock@cumin2002> |
START - Cookbook sre.hosts.reimage for host es2043.codfw.wmnet with OS bookworm |
[production] |
| 21:17 |
<jhancock@cumin2002> |
START - Cookbook sre.hosts.reimage for host es2042.codfw.wmnet with OS bookworm |
[production] |
| 21:17 |
<jhancock@cumin2002> |
START - Cookbook sre.hosts.reimage for host es2041.codfw.wmnet with OS bookworm |
[production] |
| 21:16 |
<pt1979@cumin2002> |
START - Cookbook sre.hosts.reimage for host maps-test2002.codfw.wmnet with OS bookworm |
[production] |
| 21:15 |
<jhancock@cumin2002> |
END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['es2042'] |
[production] |
| 21:15 |
<jhancock@cumin2002> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['es2042'] |
[production] |
| 21:15 |
<jhancock@cumin2002> |
END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['es2041'] |
[production] |
| 21:15 |
<jhancock@cumin2002> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['es2041'] |
[production] |
| 21:14 |
<jhancock@cumin2002> |
END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host es2042.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED |
[production] |
| 21:11 |
<jhancock@cumin2002> |
END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host es2044.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED |
[production] |
| 21:11 |
<jhancock@cumin2002> |
END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host es2045.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED |
[production] |
| 21:10 |
<jhancock@cumin2002> |
END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host es2043.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED |
[production] |
| 21:10 |
<jhancock@cumin2002> |
END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host es2041.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED |
[production] |
| 21:03 |
<jhathaway@cumin2002> |
START - Cookbook sre.hosts.reimage for host thanos-be2005.codfw.wmnet with OS bookworm |
[production] |
| 21:01 |
<jhathaway@cumin2002> |
END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host thanos-be2005.codfw.wmnet with OS bookworm |
[production] |
| 21:01 |
<jhancock@cumin2002> |
END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host es2046.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED |
[production] |
| 20:52 |
<jhathaway@cumin2002> |
START - Cookbook sre.hosts.reimage for host thanos-be2005.codfw.wmnet with OS bookworm |
[production] |
| 20:51 |
<jhathaway@cumin2002> |
END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host thanos-be2005.codfw.wmnet with OS bullseye |
[production] |
| 20:51 |
<jhathaway@cumin2002> |
START - Cookbook sre.hosts.reimage for host thanos-be2005.codfw.wmnet with OS bullseye |
[production] |
| 20:49 |
<jhathaway> |
disabling auto-reboot on re-imaging for debugging |
[production] |
| 20:49 |
<pt1979@cumin2002> |
START - Cookbook sre.hosts.reimage for host maps-test2001.codfw.wmnet with OS bookworm |
[production] |
| 20:39 |
<jhancock@cumin2002> |
START - Cookbook sre.hosts.provision for host es2046.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED |
[production] |
| 20:39 |
<jhancock@cumin2002> |
START - Cookbook sre.hosts.provision for host es2045.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED |
[production] |
| 20:39 |
<jhancock@cumin2002> |
START - Cookbook sre.hosts.provision for host es2044.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED |
[production] |
| 20:39 |
<jhancock@cumin2002> |
START - Cookbook sre.hosts.provision for host es2043.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED |
[production] |
| 20:39 |
<jhancock@cumin2002> |
START - Cookbook sre.hosts.provision for host es2042.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED |
[production] |
| 20:39 |
<jhancock@cumin2002> |
START - Cookbook sre.hosts.provision for host es2041.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED |
[production] |