2001-2050 of 10000 results (99ms)
2024-09-12 §
00:58 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2173 (T371742)', diff saved to https://phabricator.wikimedia.org/P68995 and previous config saved to /var/cache/conftool/dbconfig/20240912-005808-ladsgroup.json [production]
00:43 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2173', diff saved to https://phabricator.wikimedia.org/P68994 and previous config saved to /var/cache/conftool/dbconfig/20240912-004301-ladsgroup.json [production]
00:27 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2173', diff saved to https://phabricator.wikimedia.org/P68993 and previous config saved to /var/cache/conftool/dbconfig/20240912-002753-ladsgroup.json [production]
00:12 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2173 (T371742)', diff saved to https://phabricator.wikimedia.org/P68992 and previous config saved to /var/cache/conftool/dbconfig/20240912-001246-ladsgroup.json [production]
00:04 <eileen> civicrm upgraded from 929101dc to ac29ff45 [production]
2024-09-11 §
23:19 <jhancock@cumin2002> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host ganeti1041.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
23:18 <jhancock@cumin2002> START - Cookbook sre.hosts.provision for host ganeti1041.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
23:13 <jclark@cumin1002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host ganeti1052.eqiad.wmnet with OS bookworm [production]
23:13 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db2173 (T371742)', diff saved to https://phabricator.wikimedia.org/P68991 and previous config saved to /var/cache/conftool/dbconfig/20240911-231311-ladsgroup.json [production]
23:13 <ladsgroup@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2186.codfw.wmnet with reason: Maintenance [production]
23:12 <ladsgroup@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2186.codfw.wmnet with reason: Maintenance [production]
23:12 <ladsgroup@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2173.codfw.wmnet with reason: Maintenance [production]
23:12 <ladsgroup@cumin1002> START - Cookbook sre.hosts.downtime for 12:00:00 on db2173.codfw.wmnet with reason: Maintenance [production]
23:12 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2170 (T371742)', diff saved to https://phabricator.wikimedia.org/P68990 and previous config saved to /var/cache/conftool/dbconfig/20240911-231233-ladsgroup.json [production]
22:57 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2170', diff saved to https://phabricator.wikimedia.org/P68989 and previous config saved to /var/cache/conftool/dbconfig/20240911-225726-ladsgroup.json [production]
22:46 <jforrester@deploy1003> Finished scap sync-world: Backport for [[gerrit:1072305|SpecialExpandTemplates: Replace use of deprecated OutputPage::addCategoryLinks() (T373830)]], [[gerrit:1072306|SpecialExpandTemplates: Replace use of deprecated OutputPage::addCategoryLinks() (T373830)]] (duration: 07m 27s) [production]
22:42 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2170', diff saved to https://phabricator.wikimedia.org/P68988 and previous config saved to /var/cache/conftool/dbconfig/20240911-224218-ladsgroup.json [production]
22:41 <jforrester@deploy1003> jforrester: Continuing with sync [production]
22:41 <jforrester@deploy1003> jforrester: Backport for [[gerrit:1072305|SpecialExpandTemplates: Replace use of deprecated OutputPage::addCategoryLinks() (T373830)]], [[gerrit:1072306|SpecialExpandTemplates: Replace use of deprecated OutputPage::addCategoryLinks() (T373830)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
22:38 <jforrester@deploy1003> Started scap sync-world: Backport for [[gerrit:1072305|SpecialExpandTemplates: Replace use of deprecated OutputPage::addCategoryLinks() (T373830)]], [[gerrit:1072306|SpecialExpandTemplates: Replace use of deprecated OutputPage::addCategoryLinks() (T373830)]] [production]
22:28 <jclark@cumin1002> START - Cookbook sre.hosts.reimage for host ganeti1041.eqiad.wmnet with OS bookworm [production]
22:27 <jclark@cumin1002> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host ganeti1042.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
22:27 <jclark@cumin1002> START - Cookbook sre.hosts.provision for host ganeti1042.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
22:27 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2170 (T371742)', diff saved to https://phabricator.wikimedia.org/P68987 and previous config saved to /var/cache/conftool/dbconfig/20240911-222711-ladsgroup.json [production]
22:26 <jclark@cumin1002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti1041.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
22:26 <jclark@cumin1002> START - Cookbook sre.hosts.provision for host ganeti1041.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
22:21 <jclark@cumin1002> START - Cookbook sre.hosts.reimage for host ganeti1052.eqiad.wmnet with OS bookworm [production]
22:20 <jclark@cumin1002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti1052.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
22:19 <jclark@cumin1002> START - Cookbook sre.hosts.provision for host ganeti1052.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
22:19 <jclark@cumin1002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti1051.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
22:19 <jclark@cumin1002> START - Cookbook sre.hosts.provision for host ganeti1051.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
22:19 <jclark@cumin1002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti1050.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
22:18 <jclark@cumin1002> START - Cookbook sre.hosts.provision for host ganeti1050.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
22:18 <jclark@cumin1002> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host ganeti1044.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
22:18 <jclark@cumin1002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti1049.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
22:17 <jclark@cumin1002> START - Cookbook sre.hosts.provision for host ganeti1049.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
22:17 <jclark@cumin1002> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host ganeti1048.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
22:16 <jclark@cumin1002> START - Cookbook sre.hosts.provision for host ganeti1048.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
22:15 <jclark@cumin1002> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host ganeti1047.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
22:14 <jclark@cumin1002> START - Cookbook sre.hosts.provision for host ganeti1047.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
22:14 <jclark@cumin1002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti1046.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
22:13 <jclark@cumin1002> START - Cookbook sre.hosts.provision for host ganeti1046.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
22:13 <jclark@cumin1002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti1045.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
22:12 <jclark@cumin1002> START - Cookbook sre.hosts.provision for host ganeti1045.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
22:11 <jclark@cumin1002> START - Cookbook sre.hosts.provision for host ganeti1044.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
22:03 <jclark@cumin1002> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host ganeti1052 [production]
22:02 <jclark@cumin1002> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host ganeti1051 [production]
22:01 <jclark@cumin1002> START - Cookbook sre.network.configure-switch-interfaces for host ganeti1052 [production]
22:01 <jclark@cumin1002> START - Cookbook sre.network.configure-switch-interfaces for host ganeti1051 [production]
22:01 <jclark@cumin1002> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host ganeti1050 [production]