| 2024-10-23
      
      ยง | 
    
  | 14:40 | <brouberol@deploy2002> | helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-test-k8s: apply | [production] | 
            
  | 14:39 | <brouberol@deploy2002> | helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-test-k8s: apply | [production] | 
            
  | 14:39 | <lucaswerkmeister-wmde@deploy2002> | lucaswerkmeister-wmde, daimona: Continuing with sync | [production] | 
            
  | 14:29 | <brouberol@deploy2002> | helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-test-k8s: apply | [production] | 
            
  | 14:28 | <brouberol@deploy2002> | helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-test-k8s: apply | [production] | 
            
  | 14:28 | <lucaswerkmeister-wmde@deploy2002> | lucaswerkmeister-wmde, daimona: Backport for [[gerrit:1082082|Enable CampaignEvents collaboration list in testwiki and test2wiki (v2) (T376055)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) | [production] | 
            
  | 14:25 | <lucaswerkmeister-wmde@deploy2002> | Started scap sync-world: Backport for [[gerrit:1082082|Enable CampaignEvents collaboration list in testwiki and test2wiki (v2) (T376055)]] | [production] | 
            
  | 14:23 | <brouberol@deploy2002> | helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-test-k8s: apply | [production] | 
            
  | 14:22 | <brouberol@deploy2002> | helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-test-k8s: apply | [production] | 
            
  | 14:22 | <brouberol@deploy2002> | helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-test-k8s: apply | [production] | 
            
  | 14:21 | <brouberol@deploy2002> | helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-test-k8s: apply | [production] | 
            
  | 14:21 | <tgr@deploy2002> | Finished scap sync-world: Backport for [[gerrit:1082265|Auth: pass accountType to authevents log stream (T341650 T375510 T375505)]], [[gerrit:1082269|Auth: pass accountType to authevents log stream (T341650 T375510 T375505)]] (duration: 13m 23s) | [production] | 
            
  | 14:18 | <brouberol@deploy2002> | helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-test-k8s: apply | [production] | 
            
  | 14:18 | <sukhe> | sudo cumin 'O:alerting_host' 'run-puppet-agent' | [production] | 
            
  | 14:17 | <brouberol@deploy2002> | helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-test-k8s: apply | [production] | 
            
  | 14:16 | <tgr@deploy2002> | tgr: Continuing with sync | [production] | 
            
  | 14:14 | <sukhe> | sudo cumin 'A:dnsbox' 'run-puppet-agent' | [production] | 
            
  | 14:10 | <tgr@deploy2002> | tgr: Backport for [[gerrit:1082265|Auth: pass accountType to authevents log stream (T341650 T375510 T375505)]], [[gerrit:1082269|Auth: pass accountType to authevents log stream (T341650 T375510 T375505)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) | [production] | 
            
  | 14:07 | <tgr@deploy2002> | Started scap sync-world: Backport for [[gerrit:1082265|Auth: pass accountType to authevents log stream (T341650 T375510 T375505)]], [[gerrit:1082269|Auth: pass accountType to authevents log stream (T341650 T375510 T375505)]] | [production] | 
            
  | 13:58 | <jmm@cumin2002> | END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2040.codfw.wmnet | [production] | 
            
  | 13:54 | <sukhe@cumin1002> | END (PASS) - Cookbook sre.loadbalancer.restart-pybal (exit_code=0) rolling-restart of pybal on A:lvs and not A:ulsfo and A:lvs | [production] | 
            
  | 13:47 | <jmm@cumin2002> | START - Cookbook sre.hosts.reboot-single for host ganeti2040.codfw.wmnet | [production] | 
            
  | 13:39 | <brouberol@deploy2002> | helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-test-k8s: apply | [production] | 
            
  | 13:39 | <brouberol@deploy2002> | helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-test-k8s: apply | [production] | 
            
  | 13:37 | <moritzm> | instaling gdk-pixbuf security updates | [production] | 
            
  | 13:34 | <jgiannelos@deploy2002> | helmfile [eqiad] DONE helmfile.d/services/changeprop: apply | [production] | 
            
  | 13:34 | <jgiannelos@deploy2002> | helmfile [eqiad] START helmfile.d/services/changeprop: apply | [production] | 
            
  | 13:34 | <jgiannelos@deploy2002> | helmfile [codfw] DONE helmfile.d/services/changeprop: apply | [production] | 
            
  | 13:33 | <jgiannelos@deploy2002> | helmfile [codfw] START helmfile.d/services/changeprop: apply | [production] | 
            
  | 13:33 | <jgiannelos@deploy2002> | helmfile [staging] DONE helmfile.d/services/changeprop: apply | [production] | 
            
  | 13:33 | <jgiannelos@deploy2002> | helmfile [staging] START helmfile.d/services/changeprop: apply | [production] | 
            
  | 13:32 | <jgiannelos@deploy2002> | helmfile [staging] START helmfile.d/services/changeprop: apply | [production] | 
            
  | 13:31 | <lucaswerkmeister-wmde@deploy2002> | Finished scap sync-world: Backport for [[gerrit:1082277|WikiProjectIDLookup: use SparqlClient and make endpoint configurable (T377746)]] (duration: 07m 15s) | [production] | 
            
  | 13:27 | <lucaswerkmeister-wmde@deploy2002> | daimona, lucaswerkmeister-wmde: Continuing with sync | [production] | 
            
  | 13:27 | <lucaswerkmeister-wmde@deploy2002> | daimona, lucaswerkmeister-wmde: Backport for [[gerrit:1082277|WikiProjectIDLookup: use SparqlClient and make endpoint configurable (T377746)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) | [production] | 
            
  | 13:26 | <sukhe@cumin1002> | START - Cookbook sre.loadbalancer.restart-pybal rolling-restart of pybal on A:lvs and not A:ulsfo and A:lvs | [production] | 
            
  | 13:24 | <lucaswerkmeister-wmde@deploy2002> | Started scap sync-world: Backport for [[gerrit:1082277|WikiProjectIDLookup: use SparqlClient and make endpoint configurable (T377746)]] | [production] | 
            
  | 13:18 | <sukhe@cumin1002> | END (PASS) - Cookbook sre.loadbalancer.restart-pybal (exit_code=0) rolling-restart of pybal on A:lvs and A:ulsfo and A:lvs | [production] | 
            
  | 13:15 | <sukhe@cumin1002> | START - Cookbook sre.loadbalancer.restart-pybal rolling-restart of pybal on A:lvs and A:ulsfo and A:lvs | [production] | 
            
  | 13:13 | <brouberol@deploy2002> | helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-analytics-test: apply | [production] | 
            
  | 13:13 | <brouberol@deploy2002> | helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-analytics-test: apply | [production] | 
            
  | 13:12 | <moritzm> | installing qemu security updates | [production] | 
            
  | 13:12 | <brouberol@deploy2002> | helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-test-k8s: apply | [production] | 
            
  | 13:11 | <brouberol@deploy2002> | helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-test-k8s: apply | [production] | 
            
  | 13:09 | <sukhe> | running agent on A:lvs to roll out CR 1082238 | [production] | 
            
  | 13:02 | <jmm@cumin2002> | END (FAIL) - Cookbook sre.ganeti.addnode (exit_code=99) for new host ganeti2040.codfw.wmnet to cluster codfw and group C | [production] | 
            
  | 12:53 | <jmm@cumin2002> | START - Cookbook sre.ganeti.addnode for new host ganeti2040.codfw.wmnet to cluster codfw and group C | [production] | 
            
  | 12:26 | <jmm@cumin2002> | END (FAIL) - Cookbook sre.ganeti.addnode (exit_code=99) for new host ganeti2039.codfw.wmnet to cluster codfw and group C | [production] | 
            
  | 12:25 | <jmm@cumin2002> | START - Cookbook sre.ganeti.addnode for new host ganeti2039.codfw.wmnet to cluster codfw and group C | [production] | 
            
  | 12:16 | <jmm@cumin2002> | END (FAIL) - Cookbook sre.ganeti.addnode (exit_code=99) for new host ganeti2039.codfw.wmnet to cluster codfw and group C | [production] |