| 2024-09-20
      
      § | 
    
  | 10:08 | <ayounsi@cumin1002> | END (PASS) - Cookbook sre.network.tls (exit_code=0) for network device lsw1-e3-eqiad | [production] | 
            
  | 10:07 | <ayounsi@cumin1002> | START - Cookbook sre.network.tls for network device lsw1-e3-eqiad | [production] | 
            
  | 10:07 | <ayounsi@cumin1002> | END (PASS) - Cookbook sre.network.tls (exit_code=0) for network device lsw1-e2-eqiad | [production] | 
            
  | 10:07 | <ayounsi@cumin1002> | START - Cookbook sre.network.tls for network device lsw1-e2-eqiad | [production] | 
            
  | 10:03 | <ayounsi@cumin1002> | END (PASS) - Cookbook sre.network.tls (exit_code=0) for network device lsw1-e1-eqiad | [production] | 
            
  | 10:03 | <ayounsi@cumin1002> | START - Cookbook sre.network.tls for network device lsw1-e1-eqiad | [production] | 
            
  | 09:42 | <ayounsi@cumin1002> | END (PASS) - Cookbook sre.network.tls (exit_code=0) for network device cloudsw1-f4-eqiad | [production] | 
            
  | 09:40 | <ayounsi@cumin1002> | START - Cookbook sre.network.tls for network device cloudsw1-f4-eqiad | [production] | 
            
  | 09:38 | <ayounsi@cumin1002> | END (PASS) - Cookbook sre.network.tls (exit_code=0) for network device cloudsw1-e4-eqiad | [production] | 
            
  | 09:35 | <ayounsi@cumin1002> | START - Cookbook sre.network.tls for network device cloudsw1-e4-eqiad | [production] | 
            
  | 09:34 | <ayounsi@cumin1002> | END (PASS) - Cookbook sre.network.tls (exit_code=0) for network device cloudsw1-b1-codfw | [production] | 
            
  | 09:32 | <ayounsi@cumin1002> | START - Cookbook sre.network.tls for network device cloudsw1-b1-codfw | [production] | 
            
  | 08:54 | <btullis@cumin1002> | END (PASS) - Cookbook sre.hadoop.reboot-workers (exit_code=0) for Hadoop analytics cluster | [production] | 
            
  | 08:37 | <arnaudb@cumin1002> | dbctl commit (dc=all): 'depool es1022 - T375257', diff saved to https://phabricator.wikimedia.org/P69377 and previous config saved to /var/cache/conftool/dbconfig/20240920-083722-arnaudb.json | [production] | 
            
  | 08:32 | <jgleeson|away> | payments config changed from eee33e37 to e140fab5 | [production] | 
            
  | 07:50 | <tappof> | T375085 testing mtail 3.0.9 using debian testing package on centrallog2002 | [production] | 
            
  | 07:33 | <moritzm> | rebalance ganeti group D following the various switch maintenances T370630 | [production] | 
            
  | 07:28 | <jelto@cumin1002> | END (FAIL) - Cookbook sre.gitlab.upgrade (exit_code=99) on GitLab host gitlab1003.wikimedia.org with reason: test I946dd0b73b6be2d6b8093f03550f78d76188b92b with dummy upgrade | [production] | 
            
  | 07:27 | <jelto@cumin1002> | START - Cookbook sre.gitlab.upgrade on GitLab host gitlab1003.wikimedia.org with reason: test I946dd0b73b6be2d6b8093f03550f78d76188b92b with dummy upgrade | [production] | 
            
  | 07:18 | <vgutierrez> | rolling upgrade of purged on magru, drmrs, esams and eqiad - T334078 | [production] | 
            
  
    | 2024-09-19
      
      § | 
    
  | 21:15 | <Dreamy_Jazz> | Evening UTC backport window done | [production] | 
            
  | 21:15 | <dreamyjazz@deploy1003> | Finished scap sync-world: Backport for [[gerrit:1073871|Re-order arguments to DataAccess::addTrackingCategory]], [[gerrit:1074253|Add a "duplicate-ids" lint category (T200517)]] (duration: 29m 20s) | [production] | 
            
  | 21:02 | <dreamyjazz@deploy1003> | dreamyjazz, cscott: Continuing with sync | [production] | 
            
  | 21:00 | <dreamyjazz@deploy1003> | dreamyjazz, cscott: Backport for [[gerrit:1073871|Re-order arguments to DataAccess::addTrackingCategory]], [[gerrit:1074253|Add a "duplicate-ids" lint category (T200517)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) | [production] | 
            
  | 20:45 | <dreamyjazz@deploy1003> | Started scap sync-world: Backport for [[gerrit:1073871|Re-order arguments to DataAccess::addTrackingCategory]], [[gerrit:1074253|Add a "duplicate-ids" lint category (T200517)]] | [production] | 
            
  | 20:36 | <Dreamy_Jazz> | Running `foreachwikiindblist group1.dblist extensions/CheckUser/maintenance/populateCentralCheckUserIndexTables.php` on a tmux session for T375203 | [production] | 
            
  | 20:34 | <dreamyjazz@deploy1003> | Finished scap sync-world: Backport for [[gerrit:1074241|Add ::caller to queries in populateCentralCheckUserIndexTables.php (T375221)]] (duration: 06m 50s) | [production] | 
            
  | 20:29 | <dreamyjazz@deploy1003> | kharlan, dreamyjazz: Continuing with sync | [production] | 
            
  | 20:29 | <dreamyjazz@deploy1003> | kharlan, dreamyjazz: Backport for [[gerrit:1074241|Add ::caller to queries in populateCentralCheckUserIndexTables.php (T375221)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) | [production] | 
            
  | 20:27 | <dreamyjazz@deploy1003> | Started scap sync-world: Backport for [[gerrit:1074241|Add ::caller to queries in populateCentralCheckUserIndexTables.php (T375221)]] | [production] | 
            
  | 20:27 | <aqu@deploy1003> | Finished deploy [airflow-dags/analytics@e0d8d78]: Fix canary events generation schedule [airflow-dags/analytics@e0d8d78a] (duration: 00m 42s) | [production] | 
            
  | 20:26 | <aqu@deploy1003> | Started deploy [airflow-dags/analytics@e0d8d78]: Fix canary events generation schedule [airflow-dags/analytics@e0d8d78a] | [production] | 
            
  | 20:12 | <toyofuku@deploy1003> | Finished scap sync-world: Backport for [[gerrit:1073297|Deploy new donate link location to pilot wikis (take 2) (T373585)]] (duration: 08m 35s) | [production] | 
            
  | 20:08 | <toyofuku@deploy1003> | toyofuku: Continuing with sync | [production] | 
            
  | 20:06 | <toyofuku@deploy1003> | toyofuku: Backport for [[gerrit:1073297|Deploy new donate link location to pilot wikis (take 2) (T373585)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) | [production] | 
            
  | 20:04 | <toyofuku@deploy1003> | Started scap sync-world: Backport for [[gerrit:1073297|Deploy new donate link location to pilot wikis (take 2) (T373585)]] | [production] | 
            
  | 18:38 | <jnuche@deploy1003> | Finished scap sync-world: Backport for [[gerrit:1074209|DiscussionParser: Do not create User objects from subpages (T375212)]] (duration: 06m 38s) | [production] | 
            
  | 18:33 | <jnuche@deploy1003> | jnuche, kharlan: Continuing with sync | [production] | 
            
  | 18:33 | <jnuche@deploy1003> | jnuche, kharlan: Backport for [[gerrit:1074209|DiscussionParser: Do not create User objects from subpages (T375212)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) | [production] | 
            
  | 18:31 | <jnuche@deploy1003> | Started scap sync-world: Backport for [[gerrit:1074209|DiscussionParser: Do not create User objects from subpages (T375212)]] | [production] | 
            
  | 18:23 | <cmooney@cumin1002> | END (PASS) - Cookbook sre.dns.netbox (exit_code=0) | [production] | 
            
  | 18:23 | <cmooney@cumin1002> | END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: remove entries for sretest2002 - cmooney@cumin1002" | [production] | 
            
  | 18:23 | <cmooney@cumin1002> | START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: remove entries for sretest2002 - cmooney@cumin1002" | [production] | 
            
  | 18:20 | <cmooney@cumin1002> | START - Cookbook sre.dns.netbox | [production] | 
            
  | 18:02 | <swfrench-wmf> | scaling down mw-api-ext in eqiad after pre-switchover testing - T371273 | [production] | 
            
  | 18:02 | <swfrench@deploy1003> | helmfile [eqiad] DONE helmfile.d/services/mw-api-ext: apply | [production] | 
            
  | 18:02 | <swfrench@deploy1003> | helmfile [eqiad] START helmfile.d/services/mw-api-ext: apply | [production] | 
            
  | 18:02 | <swfrench-wmf> | scaling down mw-web in eqiad after pre-switchover testing - T371273 | [production] | 
            
  | 18:01 | <swfrench@deploy1003> | helmfile [eqiad] DONE helmfile.d/services/mw-web: apply | [production] | 
            
  | 18:01 | <swfrench@deploy1003> | helmfile [eqiad] START helmfile.d/services/mw-web: apply | [production] |