| 
      
        2024-11-18
      
      ยง
     | 
  
    
  | 16:22 | 
  <cgoubert@cumin1002> | 
  END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker1312.eqiad.wmnet with OS bookworm | 
  [production] | 
            
  | 16:19 | 
  <cgoubert@cumin1002> | 
  END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker1306.eqiad.wmnet with OS bookworm | 
  [production] | 
            
  | 16:16 | 
  <cgoubert@cumin1002> | 
  END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker1308.eqiad.wmnet with OS bookworm | 
  [production] | 
            
  | 16:14 | 
  <cgoubert@cumin1002> | 
  END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker1309.eqiad.wmnet with OS bookworm | 
  [production] | 
            
  | 16:13 | 
  <jiji@cumin1002> | 
  END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host mc-gp1005.eqiad.wmnet | 
  [production] | 
            
  | 16:11 | 
  <cgoubert@cumin1002> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker1311.eqiad.wmnet with reason: host reimage | 
  [production] | 
            
  | 16:10 | 
  <cgoubert@cumin1002> | 
  END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker1307.eqiad.wmnet with OS bookworm | 
  [production] | 
            
  | 16:08 | 
  <cgoubert@cumin1002> | 
  END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker1305.eqiad.wmnet with OS bookworm | 
  [production] | 
            
  | 16:07 | 
  <cgoubert@cumin1002> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker1310.eqiad.wmnet with reason: host reimage | 
  [production] | 
            
  | 16:06 | 
  <jiji@cumin1002> | 
  START - Cookbook sre.hosts.reboot-single for host mc-gp1005.eqiad.wmnet | 
  [production] | 
            
  | 16:04 | 
  <cgoubert@cumin1002> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker1312.eqiad.wmnet with reason: host reimage | 
  [production] | 
            
  | 16:01 | 
  <cgoubert@cumin1002> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker1306.eqiad.wmnet with reason: host reimage | 
  [production] | 
            
  | 15:58 | 
  <Lucas_WMDE> | 
  UTC afternoon backport+config window done | 
  [production] | 
            
  | 15:58 | 
  <lucaswerkmeister-wmde@deploy2002> | 
  Finished scap sync-world: Backport for [[gerrit:1092259|Unified dashboard: Add UI for page collection recommendations (T368718)]] (duration: 27m 17s) | 
  [production] | 
            
  | 15:58 | 
  <cgoubert@cumin1002> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker1308.eqiad.wmnet with reason: host reimage | 
  [production] | 
            
  | 15:56 | 
  <cgoubert@cumin1002> | 
  START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker1312.eqiad.wmnet with reason: host reimage | 
  [production] | 
            
  | 15:55 | 
  <cgoubert@cumin1002> | 
  START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker1311.eqiad.wmnet with reason: host reimage | 
  [production] | 
            
  | 15:54 | 
  <cgoubert@cumin1002> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker1309.eqiad.wmnet with reason: host reimage | 
  [production] | 
            
  | 15:51 | 
  <cgoubert@cumin1002> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker1307.eqiad.wmnet with reason: host reimage | 
  [production] | 
            
  | 15:51 | 
  <cgoubert@cumin1002> | 
  START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker1310.eqiad.wmnet with reason: host reimage | 
  [production] | 
            
  | 15:50 | 
  <cgoubert@cumin1002> | 
  START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker1309.eqiad.wmnet with reason: host reimage | 
  [production] | 
            
  | 15:49 | 
  <cgoubert@cumin1002> | 
  START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker1308.eqiad.wmnet with reason: host reimage | 
  [production] | 
            
  | 15:49 | 
  <lucaswerkmeister-wmde@deploy2002> | 
  sbisson, lucaswerkmeister-wmde: Continuing with sync | 
  [production] | 
            
  | 15:48 | 
  <cgoubert@cumin1002> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker1305.eqiad.wmnet with reason: host reimage | 
  [production] | 
            
  | 15:48 | 
  <cgoubert@cumin1002> | 
  START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker1307.eqiad.wmnet with reason: host reimage | 
  [production] | 
            
  | 15:46 | 
  <cgoubert@cumin1002> | 
  START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker1306.eqiad.wmnet with reason: host reimage | 
  [production] | 
            
  | 15:45 | 
  <lucaswerkmeister-wmde@deploy2002> | 
  sbisson, lucaswerkmeister-wmde: Backport for [[gerrit:1092259|Unified dashboard: Add UI for page collection recommendations (T368718)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) | 
  [production] | 
            
  | 15:45 | 
  <cgoubert@cumin1002> | 
  START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker1305.eqiad.wmnet with reason: host reimage | 
  [production] | 
            
  | 15:36 | 
  <cgoubert@cumin1002> | 
  START - Cookbook sre.hosts.reimage for host wikikube-worker1312.eqiad.wmnet with OS bookworm | 
  [production] | 
            
  | 15:36 | 
  <cgoubert@cumin1002> | 
  START - Cookbook sre.hosts.reimage for host wikikube-worker1311.eqiad.wmnet with OS bookworm | 
  [production] | 
            
  | 15:31 | 
  <cgoubert@cumin1002> | 
  START - Cookbook sre.hosts.reimage for host wikikube-worker1310.eqiad.wmnet with OS bookworm | 
  [production] | 
            
  | 15:31 | 
  <cgoubert@cumin1002> | 
  START - Cookbook sre.hosts.reimage for host wikikube-worker1309.eqiad.wmnet with OS bookworm | 
  [production] | 
            
  | 15:30 | 
  <lucaswerkmeister-wmde@deploy2002> | 
  Started scap sync-world: Backport for [[gerrit:1092259|Unified dashboard: Add UI for page collection recommendations (T368718)]] | 
  [production] | 
            
  | 15:30 | 
  <cgoubert@cumin1002> | 
  START - Cookbook sre.hosts.reimage for host wikikube-worker1308.eqiad.wmnet with OS bookworm | 
  [production] | 
            
  | 15:29 | 
  <cgoubert@cumin1002> | 
  START - Cookbook sre.hosts.reimage for host wikikube-worker1307.eqiad.wmnet with OS bookworm | 
  [production] | 
            
  | 15:27 | 
  <cgoubert@cumin1002> | 
  START - Cookbook sre.hosts.reimage for host wikikube-worker1306.eqiad.wmnet with OS bookworm | 
  [production] | 
            
  | 15:26 | 
  <cgoubert@cumin1002> | 
  START - Cookbook sre.hosts.reimage for host wikikube-worker1305.eqiad.wmnet with OS bookworm | 
  [production] | 
            
  | 15:11 | 
  <lucaswerkmeister-wmde@deploy2002> | 
  Finished scap sync-world: Backport for [[gerrit:1091605|Revert "Allow other input and changes to trigger searchsuggestions to update" (T379983)]] (duration: 08m 14s) | 
  [production] | 
            
  | 15:07 | 
  <lucaswerkmeister-wmde@deploy2002> | 
  samtar, lucaswerkmeister-wmde: Continuing with sync | 
  [production] | 
            
  | 15:06 | 
  <lucaswerkmeister-wmde@deploy2002> | 
  samtar, lucaswerkmeister-wmde: Backport for [[gerrit:1091605|Revert "Allow other input and changes to trigger searchsuggestions to update" (T379983)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) | 
  [production] | 
            
  | 15:03 | 
  <lucaswerkmeister-wmde@deploy2002> | 
  Started scap sync-world: Backport for [[gerrit:1091605|Revert "Allow other input and changes to trigger searchsuggestions to update" (T379983)]] | 
  [production] | 
            
  | 15:00 | 
  <arnaudb@cumin1002> | 
  dbctl commit (dc=all): 'manual depool commit', diff saved to https://phabricator.wikimedia.org/P71077 and previous config saved to /var/cache/conftool/dbconfig/20241118-150020-arnaudb.json | 
  [production] | 
            
  | 14:59 | 
  <arnaudb@cumin1002> | 
  dbctl commit (dc=all): 'manual repool commit', diff saved to https://phabricator.wikimedia.org/P71076 and previous config saved to /var/cache/conftool/dbconfig/20241118-145946-arnaudb.json | 
  [production] | 
            
  | 14:56 | 
  <arnaudb@cumin1002> | 
  END (FAIL) - Cookbook sre.mysql.pool (exit_code=99) db2216 slowly with 10 steps - slow motion repool T380131 | 
  [production] | 
            
  | 14:56 | 
  <arnaudb@cumin1002> | 
  START - Cookbook sre.mysql.pool db2216 slowly with 10 steps - slow motion repool T380131 | 
  [production] | 
            
  | 14:52 | 
  <arnaudb@cumin1002> | 
  END (PASS) - Cookbook sre.mysql.pool (exit_code=0) db2150 slowly with 10 steps - slow repool db2150 T380117 | 
  [production] | 
            
  | 14:45 | 
  <dcaro@cloudcumin1001> | 
  END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component tools-webservice | 
  [tools] | 
            
  | 14:39 | 
  <dcaro@cloudcumin1001> | 
  START - Cookbook wmcs.toolforge.component.deploy for component tools-webservice | 
  [tools] | 
            
  | 14:35 | 
  <dcaro@cloudcumin1001> | 
  END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component tools-webservice | 
  [tools] | 
            
  | 14:33 | 
  <dcaro@cloudcumin1001> | 
  START - Cookbook wmcs.toolforge.component.deploy for component tools-webservice | 
  [tools] |