| 2024-11-20
      
      § | 
    
  | 15:23 | <apine@deploy2002> | helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 15:23 | <apine@deploy2002> | helmfile [codfw] START helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 15:22 | <apine@deploy2002> | helmfile [eqiad] DONE helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 15:22 | <apine@deploy2002> | helmfile [eqiad] START helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 15:19 | <apine@deploy2002> | helmfile [staging] DONE helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 15:19 | <apine@deploy2002> | helmfile [staging] START helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 15:15 | <apine@deploy2002> | helmfile [eqiad] DONE helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 15:15 | <dcaro@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway | [tools] | 
            
  | 15:14 | <apine@deploy2002> | helmfile [eqiad] START helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 15:13 | <apine@deploy2002> | helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 15:13 | <apine@deploy2002> | helmfile [codfw] START helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 15:10 | <apine@deploy2002> | helmfile [staging] DONE helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 15:09 | <apine@deploy2002> | helmfile [staging] START helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 15:09 | <urandom> | bootstrapping cassandra, restbase2037-{a,b,c} — T380236 | [production] | 
            
  | 15:09 | <dcaro@cloudcumin1001> | START - Cookbook wmcs.toolforge.component.deploy for component api-gateway | [tools] | 
            
  | 15:06 | <dcaro@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway | [toolsbeta] | 
            
  | 15:04 | <btullis@cumin1002> | END (PASS) - Cookbook sre.ceph.roll-restart-reboot-server (exit_code=0) rolling reboot on P{cephosd100[2-4].eqiad.wmnet} and (A:cephosd) | [production] | 
            
  | 15:01 | <dcaro@cloudcumin1001> | START - Cookbook wmcs.toolforge.component.deploy for component api-gateway | [toolsbeta] | 
            
  | 14:57 | <elukey@cumin1002> | END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host thanos-be1005.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART | [production] | 
            
  | 14:56 | <dcaro@cloudcumin1001> | END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component api-gateway | [toolsbeta] | 
            
  | 14:54 | <dcaro@cloudcumin1001> | START - Cookbook wmcs.toolforge.component.deploy for component api-gateway | [toolsbeta] | 
            
  | 14:53 | <JennH> | power cycling unresponsive mgmt switch in codfw: msw-c3-codfw | [production] | 
            
  | 14:53 | <dcaro@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api | [toolsbeta] | 
            
  | 14:50 | <btullis@cumin1002> | END (FAIL) - Cookbook sre.hadoop.roll-restart-workers (exit_code=99) restart workers for Hadoop analytics cluster: Roll restart of jvm daemons for openjdk upgrade. | [production] | 
            
  | 14:46 | <dcaro@cloudcumin1001> | START - Cookbook wmcs.toolforge.component.deploy for component components-api | [toolsbeta] | 
            
  | 14:43 | <elukey@cumin1002> | START - Cookbook sre.hosts.provision for host thanos-be1005.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART | [production] | 
            
  | 14:31 | <dcaro@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api | [toolsbeta] | 
            
  | 14:29 | <cdanis> | T380226 💙cdanis@mwmaint2002.codfw.wmnet ~ 🕤☕ mwscript sql.php --wiki=commonswiki  --cluster=extension1  /srv/mediawiki/php-1.44.0-wmf.4/extensions/JsonConfig/sql/mysql/tables-generated.sql | [production] | 
            
  | 14:25 | <sukhe@puppetserver1001> | conftool action : set/pooled=yes; selector: name=cp7007.magru.wmnet [reason: host reimaged] | [production] | 
            
  | 14:24 | <dcaro@cloudcumin1001> | START - Cookbook wmcs.toolforge.component.deploy for component components-api | [toolsbeta] | 
            
  | 14:24 | <btullis@cumin1002> | START - Cookbook sre.ceph.roll-restart-reboot-server rolling reboot on P{cephosd100[2-4].eqiad.wmnet} and (A:cephosd) | [production] | 
            
  | 14:23 | <jynus> | starting resharding of commons backup files into new host backup1010  T376892 | [production] | 
            
  | 14:23 | <sukhe> | running homer on asw*magru* | [production] | 
            
  | 14:21 | <dcaro@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component envvars-admission | [tools] | 
            
  | 14:15 | <dcaro@cloudcumin1001> | START - Cookbook wmcs.toolforge.component.deploy for component envvars-admission | [tools] | 
            
  | 14:14 | <dcaro@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component envvars-admission | [toolsbeta] | 
            
  | 14:08 | <dcaro@cloudcumin1001> | START - Cookbook wmcs.toolforge.component.deploy for component envvars-admission | [toolsbeta] | 
            
  | 14:08 | <dcaro@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component envvars-admission | [toolsbeta] | 
            
  | 14:06 | <jiji@deploy2002> | helmfile [aux-k8s-eqiad] DONE helmfile.d/admin 'apply'. | [production] | 
            
  | 14:05 | <jiji@deploy2002> | helmfile [aux-k8s-eqiad] START helmfile.d/admin 'apply'. | [production] | 
            
  | 14:05 | <jiji@deploy2002> | helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. | [production] | 
            
  | 14:05 | <jiji@deploy2002> | helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. | [production] | 
            
  | 14:05 | <jiji@deploy2002> | helmfile [ml-staging-codfw] DONE helmfile.d/admin 'apply'. | [production] | 
            
  | 14:04 | <jiji@deploy2002> | helmfile [ml-staging-codfw] START helmfile.d/admin 'apply'. | [production] | 
            
  | 14:04 | <jiji@deploy2002> | helmfile [ml-serve-codfw] DONE helmfile.d/admin 'apply'. | [production] | 
            
  | 14:04 | <jiji@deploy2002> | helmfile [ml-serve-codfw] START helmfile.d/admin 'apply'. | [production] | 
            
  | 14:04 | <jiji@deploy2002> | helmfile [ml-serve-eqiad] DONE helmfile.d/admin 'apply'. | [production] | 
            
  | 14:04 | <dcaro@cloudcumin1001> | START - Cookbook wmcs.toolforge.component.deploy for component envvars-admission | [toolsbeta] | 
            
  | 14:03 | <jiji@deploy2002> | helmfile [ml-serve-eqiad] START helmfile.d/admin 'apply'. | [production] | 
            
  | 14:03 | <jiji@deploy2002> | helmfile [staging-codfw] DONE helmfile.d/admin 'apply'. | [production] |