951-1000 of 10000 results (100ms)
2025-03-06 §
01:19 <jclark@cumin1002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1002" [production]
01:19 <jclark@cumin1002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1002" [production]
01:18 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host cloudelastic1008.eqiad.wmnet with OS bullseye [production]
00:58 <jclark@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1181.eqiad.wmnet with reason: host reimage [production]
00:55 <jclark@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1181.eqiad.wmnet with reason: host reimage [production]
00:40 <jclark@cumin1002> START - Cookbook sre.hosts.reimage for host an-worker1181.eqiad.wmnet with OS bullseye [production]
00:33 <zabe> zabe@mwmaint2002:~$ mwscript extensions/WikimediaMaintenance/migrateESRefToContentTableStage2.php commonswiki --delete /home/zabe/text_table_cleanup/commonswiki --sleep 0.5 # T183490 [production]
00:09 <tgr_> UTC late deploys done [production]
00:08 <tgr@deploy2002> Finished scap sync-world: Backport for [[gerrit:1124866|Roll out SUL3 signup to 1% of users on most group 1 wikis (T384007)]] (duration: 29m 13s) [production]
00:02 <tgr@deploy2002> tgr: Continuing with sync [production]
2025-03-05 §
23:42 <tgr@deploy2002> tgr: Backport for [[gerrit:1124866|Roll out SUL3 signup to 1% of users on most group 1 wikis (T384007)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
23:39 <tgr@deploy2002> Started scap sync-world: Backport for [[gerrit:1124866|Roll out SUL3 signup to 1% of users on most group 1 wikis (T384007)]] [production]
23:39 <jclark@cumin1002> START - Cookbook sre.hosts.reimage for host an-worker1181.eqiad.wmnet with OS bullseye [production]
23:36 <tgr@deploy2002> Finished scap sync-world: Backport for [[gerrit:1124860|Preserve usesul3 flag during autologin (T375788)]], [[gerrit:1124861|Preserve usesul3 flag during autologin (T375788)]], [[gerrit:1124865|Clean up SUL3 config (T384007)]] (duration: 18m 53s) [production]
23:29 <tgr@deploy2002> tgr: Continuing with sync [production]
23:20 <tgr@deploy2002> tgr: Backport for [[gerrit:1124860|Preserve usesul3 flag during autologin (T375788)]], [[gerrit:1124861|Preserve usesul3 flag during autologin (T375788)]], [[gerrit:1124865|Clean up SUL3 config (T384007)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
23:17 <bking@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudelastic1008.eqiad.wmnet with OS bullseye [production]
23:17 <tgr@deploy2002> Started scap sync-world: Backport for [[gerrit:1124860|Preserve usesul3 flag during autologin (T375788)]], [[gerrit:1124861|Preserve usesul3 flag during autologin (T375788)]], [[gerrit:1124865|Clean up SUL3 config (T384007)]] [production]
23:04 <tgr@deploy2002> Finished scap sync-world: Backport for [[gerrit:1124885|Revert^2 "Invert Parsoid read view wiktionary configs"]] (duration: 12m 13s) [production]
23:03 <ebernhardson@deploy2002> helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
23:02 <ebernhardson@deploy2002> helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
22:57 <tgr@deploy2002> tgr: Continuing with sync [production]
22:54 <tgr@deploy2002> tgr: Backport for [[gerrit:1124885|Revert^2 "Invert Parsoid read view wiktionary configs"]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
22:53 <ebernhardson@deploy2002> helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
22:53 <ebernhardson@deploy2002> helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
22:51 <tgr@deploy2002> Started scap sync-world: Backport for [[gerrit:1124885|Revert^2 "Invert Parsoid read view wiktionary configs"]] [production]
22:29 <tgr@deploy2002> Finished scap sync-world: Backport for [[gerrit:1124881|Revert^2 "Turn on Parsoid Read Views for 44 wiktionaries" (T387505)]] (duration: 12m 06s) [production]
22:24 <ebernhardson@deploy2002> helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
22:24 <ebernhardson@deploy2002> helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
22:23 <tgr@deploy2002> tgr: Continuing with sync [production]
22:20 <tgr@deploy2002> tgr: Backport for [[gerrit:1124881|Revert^2 "Turn on Parsoid Read Views for 44 wiktionaries" (T387505)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
22:17 <tgr@deploy2002> Started scap sync-world: Backport for [[gerrit:1124881|Revert^2 "Turn on Parsoid Read Views for 44 wiktionaries" (T387505)]] [production]
21:57 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host cloudelastic1008.eqiad.wmnet with OS bullseye [production]
21:56 <jclark@cumin1002> START - Cookbook sre.hosts.reimage for host an-worker1181.eqiad.wmnet with OS bullseye [production]
21:54 <bking@cumin2002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host cloudelastic1008.eqiad.wmnet with OS bullseye [production]
21:53 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host cloudelastic1008.eqiad.wmnet with OS bullseye [production]
21:50 <tgr@deploy2002> Finished scap sync-world: Backport for [[gerrit:1124874|Revert "Invert Parsoid read view wiktionary configs"]], [[gerrit:1124875|Revert "Turn on Parsoid Read Views for 44 wiktionaries"]] (duration: 09m 30s) [production]
21:44 <tgr@deploy2002> tgr: Continuing with sync [production]
21:44 <tgr@deploy2002> tgr: Backport for [[gerrit:1124874|Revert "Invert Parsoid read view wiktionary configs"]], [[gerrit:1124875|Revert "Turn on Parsoid Read Views for 44 wiktionaries"]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
21:41 <tgr@deploy2002> Started scap sync-world: Backport for [[gerrit:1124874|Revert "Invert Parsoid read view wiktionary configs"]], [[gerrit:1124875|Revert "Turn on Parsoid Read Views for 44 wiktionaries"]] [production]
21:29 <ebernhardson@deploy2002> helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
21:28 <ebernhardson@deploy2002> helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
21:26 <swfrench@deploy2002> helmfile [codfw] DONE helmfile.d/services/mw-web: apply [production]
21:26 <swfrench@deploy2002> helmfile [codfw] START helmfile.d/services/mw-web: apply [production]
21:25 <swfrench@deploy2002> helmfile [eqiad] DONE helmfile.d/services/mw-web: apply [production]
21:25 <swfrench@deploy2002> helmfile [eqiad] START helmfile.d/services/mw-web: apply [production]
21:24 <ebernhardson@deploy2002> helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
21:24 <ebernhardson@deploy2002> helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
21:23 <vriley@cumin1002> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host an-worker1181.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED [production]
21:22 <tgr@deploy2002> Finished scap sync-world: Backport for [[gerrit:1124551|Turn on Parsoid Read Views for 44 wiktionaries (T387505)]], [[gerrit:1124867|Invert Parsoid read view wiktionary configs]] (duration: 12m 23s) [production]