| 2024-09-12
      
      § | 
    
  | 04:21 | <ladsgroup@cumin1002> | dbctl commit (dc=all): 'Repooling after maintenance db2176', diff saved to https://phabricator.wikimedia.org/P69005 and previous config saved to /var/cache/conftool/dbconfig/20240912-042121-ladsgroup.json | [production] | 
            
  | 04:06 | <ladsgroup@cumin1002> | dbctl commit (dc=all): 'Repooling after maintenance db2176', diff saved to https://phabricator.wikimedia.org/P69004 and previous config saved to /var/cache/conftool/dbconfig/20240912-040613-ladsgroup.json | [production] | 
            
  | 03:51 | <ladsgroup@cumin1002> | dbctl commit (dc=all): 'Repooling after maintenance db2176 (T371742)', diff saved to https://phabricator.wikimedia.org/P69003 and previous config saved to /var/cache/conftool/dbconfig/20240912-035105-ladsgroup.json | [production] | 
            
  | 02:46 | <ladsgroup@cumin1002> | dbctl commit (dc=all): 'Depooling db2176 (T371742)', diff saved to https://phabricator.wikimedia.org/P69002 and previous config saved to /var/cache/conftool/dbconfig/20240912-024635-ladsgroup.json | [production] | 
            
  | 02:46 | <ladsgroup@cumin1002> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2176.codfw.wmnet with reason: Maintenance | [production] | 
            
  | 02:46 | <ladsgroup@cumin1002> | START - Cookbook sre.hosts.downtime for 12:00:00 on db2176.codfw.wmnet with reason: Maintenance | [production] | 
            
  | 02:46 | <ladsgroup@cumin1002> | dbctl commit (dc=all): 'Repooling after maintenance db2174 (T371742)', diff saved to https://phabricator.wikimedia.org/P69001 and previous config saved to /var/cache/conftool/dbconfig/20240912-024612-ladsgroup.json | [production] | 
            
  | 02:31 | <ladsgroup@cumin1002> | dbctl commit (dc=all): 'Repooling after maintenance db2174', diff saved to https://phabricator.wikimedia.org/P69000 and previous config saved to /var/cache/conftool/dbconfig/20240912-023105-ladsgroup.json | [production] | 
            
  | 02:15 | <ladsgroup@cumin1002> | dbctl commit (dc=all): 'Repooling after maintenance db2174', diff saved to https://phabricator.wikimedia.org/P68999 and previous config saved to /var/cache/conftool/dbconfig/20240912-021557-ladsgroup.json | [production] | 
            
  | 02:00 | <ladsgroup@cumin1002> | dbctl commit (dc=all): 'Repooling after maintenance db2174 (T371742)', diff saved to https://phabricator.wikimedia.org/P68998 and previous config saved to /var/cache/conftool/dbconfig/20240912-020050-ladsgroup.json | [production] | 
            
  | 00:58 | <ladsgroup@cumin1002> | dbctl commit (dc=all): 'Depooling db2174 (T371742)', diff saved to https://phabricator.wikimedia.org/P68996 and previous config saved to /var/cache/conftool/dbconfig/20240912-005830-ladsgroup.json | [production] | 
            
  | 00:58 | <ladsgroup@cumin1002> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2174.codfw.wmnet with reason: Maintenance | [production] | 
            
  | 00:58 | <ladsgroup@cumin1002> | START - Cookbook sre.hosts.downtime for 12:00:00 on db2174.codfw.wmnet with reason: Maintenance | [production] | 
            
  | 00:58 | <ladsgroup@cumin1002> | dbctl commit (dc=all): 'Repooling after maintenance db2173 (T371742)', diff saved to https://phabricator.wikimedia.org/P68995 and previous config saved to /var/cache/conftool/dbconfig/20240912-005808-ladsgroup.json | [production] | 
            
  | 00:43 | <ladsgroup@cumin1002> | dbctl commit (dc=all): 'Repooling after maintenance db2173', diff saved to https://phabricator.wikimedia.org/P68994 and previous config saved to /var/cache/conftool/dbconfig/20240912-004301-ladsgroup.json | [production] | 
            
  | 00:27 | <ladsgroup@cumin1002> | dbctl commit (dc=all): 'Repooling after maintenance db2173', diff saved to https://phabricator.wikimedia.org/P68993 and previous config saved to /var/cache/conftool/dbconfig/20240912-002753-ladsgroup.json | [production] | 
            
  | 00:12 | <ladsgroup@cumin1002> | dbctl commit (dc=all): 'Repooling after maintenance db2173 (T371742)', diff saved to https://phabricator.wikimedia.org/P68992 and previous config saved to /var/cache/conftool/dbconfig/20240912-001246-ladsgroup.json | [production] | 
            
  | 00:04 | <eileen> | civicrm upgraded from 929101dc to ac29ff45 | [production] | 
            
  
    | 2024-09-11
      
      § | 
    
  | 23:19 | <jhancock@cumin2002> | END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host ganeti1041.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART | [production] | 
            
  | 23:18 | <jhancock@cumin2002> | START - Cookbook sre.hosts.provision for host ganeti1041.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART | [production] | 
            
  | 23:13 | <jclark@cumin1002> | END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host ganeti1052.eqiad.wmnet with OS bookworm | [production] | 
            
  | 23:13 | <ladsgroup@cumin1002> | dbctl commit (dc=all): 'Depooling db2173 (T371742)', diff saved to https://phabricator.wikimedia.org/P68991 and previous config saved to /var/cache/conftool/dbconfig/20240911-231311-ladsgroup.json | [production] | 
            
  | 23:13 | <ladsgroup@cumin1002> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2186.codfw.wmnet with reason: Maintenance | [production] | 
            
  | 23:12 | <ladsgroup@cumin1002> | START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2186.codfw.wmnet with reason: Maintenance | [production] | 
            
  | 23:12 | <ladsgroup@cumin1002> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2173.codfw.wmnet with reason: Maintenance | [production] | 
            
  | 23:12 | <ladsgroup@cumin1002> | START - Cookbook sre.hosts.downtime for 12:00:00 on db2173.codfw.wmnet with reason: Maintenance | [production] | 
            
  | 23:12 | <ladsgroup@cumin1002> | dbctl commit (dc=all): 'Repooling after maintenance db2170 (T371742)', diff saved to https://phabricator.wikimedia.org/P68990 and previous config saved to /var/cache/conftool/dbconfig/20240911-231233-ladsgroup.json | [production] | 
            
  | 23:11 | <James_F> | Docker: [quibble-bullseye-php81] Re-build on Wikimedia-flavoured PHP, for T372507 | [releng] | 
            
  | 22:58 | <James_F> | Docker: [php81] Re-build based on Wikimedia-provided binaries, for T372507 | [releng] | 
            
  | 22:57 | <ladsgroup@cumin1002> | dbctl commit (dc=all): 'Repooling after maintenance db2170', diff saved to https://phabricator.wikimedia.org/P68989 and previous config saved to /var/cache/conftool/dbconfig/20240911-225726-ladsgroup.json | [production] | 
            
  | 22:46 | <jforrester@deploy1003> | Finished scap sync-world: Backport for [[gerrit:1072305|SpecialExpandTemplates: Replace use of deprecated OutputPage::addCategoryLinks() (T373830)]], [[gerrit:1072306|SpecialExpandTemplates: Replace use of deprecated OutputPage::addCategoryLinks() (T373830)]] (duration: 07m 27s) | [production] | 
            
  | 22:42 | <ladsgroup@cumin1002> | dbctl commit (dc=all): 'Repooling after maintenance db2170', diff saved to https://phabricator.wikimedia.org/P68988 and previous config saved to /var/cache/conftool/dbconfig/20240911-224218-ladsgroup.json | [production] | 
            
  | 22:41 | <jforrester@deploy1003> | jforrester: Continuing with sync | [production] | 
            
  | 22:41 | <jforrester@deploy1003> | jforrester: Backport for [[gerrit:1072305|SpecialExpandTemplates: Replace use of deprecated OutputPage::addCategoryLinks() (T373830)]], [[gerrit:1072306|SpecialExpandTemplates: Replace use of deprecated OutputPage::addCategoryLinks() (T373830)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) | [production] | 
            
  | 22:38 | <jforrester@deploy1003> | Started scap sync-world: Backport for [[gerrit:1072305|SpecialExpandTemplates: Replace use of deprecated OutputPage::addCategoryLinks() (T373830)]], [[gerrit:1072306|SpecialExpandTemplates: Replace use of deprecated OutputPage::addCategoryLinks() (T373830)]] | [production] | 
            
  | 22:28 | <jclark@cumin1002> | START - Cookbook sre.hosts.reimage for host ganeti1041.eqiad.wmnet with OS bookworm | [production] | 
            
  | 22:27 | <jclark@cumin1002> | END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host ganeti1042.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART | [production] | 
            
  | 22:27 | <jclark@cumin1002> | START - Cookbook sre.hosts.provision for host ganeti1042.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART | [production] | 
            
  | 22:27 | <ladsgroup@cumin1002> | dbctl commit (dc=all): 'Repooling after maintenance db2170 (T371742)', diff saved to https://phabricator.wikimedia.org/P68987 and previous config saved to /var/cache/conftool/dbconfig/20240911-222711-ladsgroup.json | [production] | 
            
  | 22:26 | <jclark@cumin1002> | END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti1041.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART | [production] | 
            
  | 22:26 | <jclark@cumin1002> | START - Cookbook sre.hosts.provision for host ganeti1041.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART | [production] | 
            
  | 22:21 | <jclark@cumin1002> | START - Cookbook sre.hosts.reimage for host ganeti1052.eqiad.wmnet with OS bookworm | [production] | 
            
  | 22:20 | <jclark@cumin1002> | END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti1052.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART | [production] | 
            
  | 22:19 | <jclark@cumin1002> | START - Cookbook sre.hosts.provision for host ganeti1052.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART | [production] | 
            
  | 22:19 | <jclark@cumin1002> | END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti1051.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART | [production] | 
            
  | 22:19 | <jclark@cumin1002> | START - Cookbook sre.hosts.provision for host ganeti1051.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART | [production] | 
            
  | 22:19 | <jclark@cumin1002> | END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti1050.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART | [production] | 
            
  | 22:18 | <jclark@cumin1002> | START - Cookbook sre.hosts.provision for host ganeti1050.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART | [production] | 
            
  | 22:18 | <jclark@cumin1002> | END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host ganeti1044.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART | [production] | 
            
  | 22:18 | <jclark@cumin1002> | END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti1049.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART | [production] |