|
2024-11-18
§
|
| 23:15 |
<jhathaway@cumin2002> |
START - Cookbook sre.hosts.downtime for 2:00:00 on thanos-be2005.codfw.wmnet with reason: host reimage |
[production] |
| 23:12 |
<tzatziki> |
removing 2 files for legal compliance |
[production] |
| 23:09 |
<eevans@cumin1002> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 23:09 |
<eevans@cumin1002> |
END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Additional IPs for Cassandra — restbase2036 - eevans@cumin1002" |
[production] |
| 23:09 |
<eevans@cumin1002> |
START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Additional IPs for Cassandra — restbase2036 - eevans@cumin1002" |
[production] |
| 23:08 |
<pt1979@cumin2002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on maps-test2002.codfw.wmnet with reason: host reimage |
[production] |
| 23:06 |
<eevans@cumin1002> |
START - Cookbook sre.dns.netbox |
[production] |
| 23:05 |
<pt1979@cumin2002> |
START - Cookbook sre.hosts.downtime for 2:00:00 on maps-test2002.codfw.wmnet with reason: host reimage |
[production] |
| 23:04 |
<eevans@cumin1002> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 23:04 |
<eevans@cumin1002> |
END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Additional IPs for Cassandra — restbase2036 - eevans@cumin1002" |
[production] |
| 23:04 |
<pt1979@cumin2002> |
START - Cookbook sre.hosts.reimage for host maps-test2003.codfw.wmnet with OS bookworm |
[production] |
| 23:04 |
<eevans@cumin1002> |
START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Additional IPs for Cassandra — restbase2036 - eevans@cumin1002" |
[production] |
| 23:03 |
<jhathaway@cumin2002> |
START - Cookbook sre.hosts.reimage for host thanos-be2005.codfw.wmnet with OS bookworm |
[production] |
| 23:01 |
<jhathaway@cumin2002> |
END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host thanos-be2005.codfw.wmnet with OS bookworm |
[production] |
| 23:00 |
<jclark@cumin1002> |
START - Cookbook sre.hosts.reimage for host kafka-jumbo1018.eqiad.wmnet with OS bullseye |
[production] |
| 23:00 |
<jclark@cumin1002> |
START - Cookbook sre.hosts.reimage for host kafka-jumbo1017.eqiad.wmnet with OS bullseye |
[production] |
| 23:00 |
<eevans@cumin1002> |
START - Cookbook sre.dns.netbox |
[production] |
| 22:59 |
<jclark@cumin1002> |
START - Cookbook sre.hosts.reimage for host kafka-jumbo1016.eqiad.wmnet with OS bullseye |
[production] |
| 22:57 |
<jhathaway@cumin2002> |
START - Cookbook sre.hosts.reimage for host thanos-be2005.codfw.wmnet with OS bookworm |
[production] |
| 22:55 |
<jhancock@cumin2002> |
END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host es2045.codfw.wmnet with OS bookworm |
[production] |
| 22:55 |
<jhathaway@cumin2002> |
END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host thanos-be2005.codfw.wmnet with OS bookworm |
[production] |
| 22:55 |
<jhancock@cumin2002> |
END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host es2044.codfw.wmnet with OS bookworm |
[production] |
| 22:54 |
<jhancock@cumin2002> |
END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host es2046.codfw.wmnet with OS bookworm |
[production] |
| 22:54 |
<jhancock@cumin2002> |
END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host es2043.codfw.wmnet with OS bookworm |
[production] |
| 22:54 |
<jhancock@cumin2002> |
END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host es2041.codfw.wmnet with OS bookworm |
[production] |
| 22:52 |
<tzatziki> |
removing 10 files for legal compliance |
[production] |
| 22:50 |
<pt1979@cumin2002> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host maps-test2001.codfw.wmnet with OS bookworm |
[production] |
| 22:50 |
<pt1979@cumin2002> |
END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - pt1979@cumin2002" |
[production] |
| 22:49 |
<bking@deploy2002> |
Finished deploy [wdqs/wdqs@9927a5a]: 0.3.150 (duration: 11m 59s) |
[production] |
| 22:47 |
<jhathaway@cumin2002> |
START - Cookbook sre.hosts.reimage for host thanos-be2005.codfw.wmnet with OS bookworm |
[production] |
| 22:37 |
<jhancock@cumin2002> |
END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host es2042.codfw.wmnet with OS bookworm |
[production] |
| 22:37 |
<bking@deploy2002> |
Started deploy [wdqs/wdqs@9927a5a]: 0.3.150 |
[production] |
| 22:22 |
<jhathaway@cumin2002> |
END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host thanos-be2005.codfw.wmnet with OS bookworm |
[production] |
| 22:18 |
<urbanecm@deploy2002> |
Finished scap sync-world: Backport for [[gerrit:1092336|[GrowthExperiments] testwiki: Only enable Add Link for new accounts (T380204)]] (duration: 09m 14s) |
[production] |
| 22:13 |
<urbanecm@deploy2002> |
urbanecm: Continuing with sync |
[production] |
| 22:13 |
<urbanecm@deploy2002> |
urbanecm: Backport for [[gerrit:1092336|[GrowthExperiments] testwiki: Only enable Add Link for new accounts (T380204)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 22:09 |
<urbanecm@deploy2002> |
Started scap sync-world: Backport for [[gerrit:1092336|[GrowthExperiments] testwiki: Only enable Add Link for new accounts (T380204)]] |
[production] |
| 21:58 |
<urbanecm@deploy2002> |
Finished scap sync-world: Backport for [[gerrit:1092304|Use WAN cache for JsonConfig remote fetch cache (T374746)]], [[gerrit:1092300|Create no-link-recommendation variant (T377787 T380204)]], [[gerrit:1092295|[GrowthExperiments] testwiki: Enable no-link-recommendation experiment (T380204)]] (duration: 12m 10s) |
[production] |
| 21:54 |
<urbanecm@deploy2002> |
urbanecm, bvibber: Continuing with sync |
[production] |
| 21:52 |
<urbanecm@deploy2002> |
urbanecm, bvibber: Backport for [[gerrit:1092304|Use WAN cache for JsonConfig remote fetch cache (T374746)]], [[gerrit:1092300|Create no-link-recommendation variant (T377787 T380204)]], [[gerrit:1092295|[GrowthExperiments] testwiki: Enable no-link-recommendation experiment (T380204)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 21:48 |
<effie> |
upload prometheus-mcrouter-exporter_0.4.0+git20241118-1~wmf1 - T380212 |
[production] |
| 21:46 |
<urbanecm@deploy2002> |
Started scap sync-world: Backport for [[gerrit:1092304|Use WAN cache for JsonConfig remote fetch cache (T374746)]], [[gerrit:1092300|Create no-link-recommendation variant (T377787 T380204)]], [[gerrit:1092295|[GrowthExperiments] testwiki: Enable no-link-recommendation experiment (T380204)]] |
[production] |
| 21:42 |
<pt1979@cumin2002> |
START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - pt1979@cumin2002" |
[production] |
| 21:36 |
<urbanecm@deploy2002> |
Finished scap sync-world: Backport for [[gerrit:1091839|Rename everything referring to "SSO domain" to use "shared domain" (T379811)]], [[gerrit:1091841|Rename shared domain sso.wikimedia.org to auth.wikimedia.org (T379811)]], [[gerrit:1091842|Use DB name rather than server name in shared domain path prefix (T379811)]] (duration: 10m 54s) |
[production] |
| 21:31 |
<urbanecm@deploy2002> |
matmarex, urbanecm: Continuing with sync |
[production] |
| 21:30 |
<urbanecm@deploy2002> |
matmarex, urbanecm: Backport for [[gerrit:1091839|Rename everything referring to "SSO domain" to use "shared domain" (T379811)]], [[gerrit:1091841|Rename shared domain sso.wikimedia.org to auth.wikimedia.org (T379811)]], [[gerrit:1091842|Use DB name rather than server name in shared domain path prefix (T379811)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 21:29 |
<urbanecm> |
Add bvibber to wmf-deployment Gerrit group (existing deployer) |
[production] |
| 21:26 |
<urbanecm@deploy2002> |
Started scap sync-world: Backport for [[gerrit:1091839|Rename everything referring to "SSO domain" to use "shared domain" (T379811)]], [[gerrit:1091841|Rename shared domain sso.wikimedia.org to auth.wikimedia.org (T379811)]], [[gerrit:1091842|Use DB name rather than server name in shared domain path prefix (T379811)]] |
[production] |
| 21:21 |
<pt1979@cumin2002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on maps-test2001.codfw.wmnet with reason: host reimage |
[production] |
| 21:18 |
<pt1979@cumin2002> |
START - Cookbook sre.hosts.downtime for 2:00:00 on maps-test2001.codfw.wmnet with reason: host reimage |
[production] |