451-500 of 10000 results (110ms)
2024-11-25 ยง
21:42 <tgr@deploy2002> tgr, dani: Continuing with sync [production]
21:39 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db2131 (T380449)', diff saved to https://phabricator.wikimedia.org/P71153 and previous config saved to /var/cache/conftool/dbconfig/20241125-213904-ladsgroup.json [production]
21:38 <ladsgroup@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2131.codfw.wmnet with reason: Maintenance [production]
21:38 <ladsgroup@cumin1002> START - Cookbook sre.hosts.downtime for 6:00:00 on db2131.codfw.wmnet with reason: Maintenance [production]
21:38 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2115 (T380449)', diff saved to https://phabricator.wikimedia.org/P71152 and previous config saved to /var/cache/conftool/dbconfig/20241125-213841-ladsgroup.json [production]
21:37 <tgr@deploy2002> tgr, dani: Backport for [[gerrit:1094054|Reader Survey: Increase coverage on enwiki (T378660)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
21:33 <tgr@deploy2002> Started scap sync-world: Backport for [[gerrit:1094054|Reader Survey: Increase coverage on enwiki (T378660)]] [production]
21:31 <brett@cumin2002> START - Cookbook sre.hosts.reimage for host lvs7003.magru.wmnet with OS bullseye [production]
21:30 <brett@cumin2002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host lvs7003.magru.wmnet with OS bullseye [production]
21:29 <tgr@deploy2002> Finished scap sync-world: Backport for [[gerrit:1097417|Reader Survey: Fix yes/no messages (T378660)]] (duration: 16m 02s) [production]
21:23 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2115', diff saved to https://phabricator.wikimedia.org/P71151 and previous config saved to /var/cache/conftool/dbconfig/20241125-212334-ladsgroup.json [production]
21:22 <tgr@deploy2002> dani, tgr: Continuing with sync [production]
21:18 <sukhe@cumin1002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "testing - sukhe@cumin1002" [production]
21:18 <sukhe@cumin1002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "testing - sukhe@cumin1002" [production]
21:17 <tgr@deploy2002> dani, tgr: Backport for [[gerrit:1097417|Reader Survey: Fix yes/no messages (T378660)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
21:13 <tgr@deploy2002> Started scap sync-world: Backport for [[gerrit:1097417|Reader Survey: Fix yes/no messages (T378660)]] [production]
21:08 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2115', diff saved to https://phabricator.wikimedia.org/P71150 and previous config saved to /var/cache/conftool/dbconfig/20241125-210827-ladsgroup.json [production]
21:04 <brett@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Host reimage - brett@cumin2002 - brett@cumin2002" [production]
21:04 <brett@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Host reimage - brett@cumin2002 - brett@cumin2002" [production]
21:03 <eevans@cumin1002> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts restbase[2021-2023].codfw.wmnet [production]
21:03 <eevans@cumin1002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
21:03 <eevans@cumin1002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: restbase[2021-2023].codfw.wmnet decommissioned, removing all IPs except the asset tag one - eevans@cumin1002" [production]
21:03 <eevans@cumin1002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: restbase[2021-2023].codfw.wmnet decommissioned, removing all IPs except the asset tag one - eevans@cumin1002" [production]
20:59 <eevans@cumin1002> START - Cookbook sre.dns.netbox [production]
20:57 <brett@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cp7008.magru.wmnet with OS bullseye [production]
20:57 <brett@cumin2002> END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - brett@cumin2002" [production]
20:56 <robh@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - robh@cumin2002" [production]
20:53 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2115 (T380449)', diff saved to https://phabricator.wikimedia.org/P71149 and previous config saved to /var/cache/conftool/dbconfig/20241125-205320-ladsgroup.json [production]
20:51 <eevans@cumin1002> START - Cookbook sre.hosts.decommission for hosts restbase[2021-2023].codfw.wmnet [production]
20:51 <robh@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
20:51 <robh@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: magru reshuffle - robh@cumin2002" [production]
20:50 <robh@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: magru reshuffle - robh@cumin2002" [production]
20:45 <robh@cumin2002> START - Cookbook sre.dns.netbox [production]
20:45 <robh@cumin2002> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host dns7001 [production]
20:45 <robh@cumin2002> START - Cookbook sre.network.configure-switch-interfaces for host dns7001 [production]
20:44 <brett@cumin2002> START - Cookbook sre.hosts.reimage for host lvs7003.magru.wmnet with OS bullseye [production]
20:43 <brett@cumin2002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host lvs7003.magru.wmnet with OS bullseye [production]
20:40 <brett@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - brett@cumin2002" [production]
20:34 <robh@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ganeti7004.magru.wmnet with reason: host reimage [production]
20:31 <robh@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on ganeti7004.magru.wmnet with reason: host reimage [production]
20:26 <brett@cumin2002> START - Cookbook sre.hosts.reimage for host lvs7003.magru.wmnet with OS bullseye [production]
20:24 <brett@cumin2002> START - Cookbook sre.hosts.reimage for host cp7015.magru.wmnet with OS bullseye [production]
20:12 <brett@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cp7008.magru.wmnet with reason: host reimage [production]
20:09 <brett@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on cp7008.magru.wmnet with reason: host reimage [production]
20:00 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db2115 (T380449)', diff saved to https://phabricator.wikimedia.org/P71147 and previous config saved to /var/cache/conftool/dbconfig/20241125-200031-ladsgroup.json [production]
20:00 <ladsgroup@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2115.codfw.wmnet with reason: Maintenance [production]
20:00 <ladsgroup@cumin1002> START - Cookbook sre.hosts.downtime for 6:00:00 on db2115.codfw.wmnet with reason: Maintenance [production]
20:00 <robh@cumin2002> START - Cookbook sre.hosts.reimage for host ganeti7004.magru.wmnet with OS bookworm [production]
19:58 <robh@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ganeti7003.magru.wmnet with OS bookworm [production]
19:58 <robh@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - robh@cumin2002" [production]