| 
      
        2025-09-16
      
      ยง
     | 
  
    
  | 21:28 | 
  <rzl@deploy1003> | 
  helmfile [eqiad] DONE helmfile.d/services/data-gateway: apply | 
  [production] | 
            
  | 21:27 | 
  <rzl@deploy1003> | 
  helmfile [eqiad] START helmfile.d/services/data-gateway: apply | 
  [production] | 
            
  | 21:26 | 
  <rzl@deploy1003> | 
  helmfile [codfw] DONE helmfile.d/services/cxserver: apply | 
  [production] | 
            
  | 21:26 | 
  <rzl@deploy1003> | 
  helmfile [codfw] START helmfile.d/services/cxserver: apply | 
  [production] | 
            
  | 21:25 | 
  <rzl@deploy1003> | 
  helmfile [eqiad] DONE helmfile.d/services/cxserver: apply | 
  [production] | 
            
  | 21:25 | 
  <rzl@deploy1003> | 
  helmfile [eqiad] START helmfile.d/services/cxserver: apply | 
  [production] | 
            
  | 21:25 | 
  <rzl@deploy1003> | 
  helmfile [codfw] DONE helmfile.d/services/citoid: apply | 
  [production] | 
            
  | 21:24 | 
  <rzl@deploy1003> | 
  helmfile [codfw] START helmfile.d/services/citoid: apply | 
  [production] | 
            
  | 21:24 | 
  <rzl@deploy1003> | 
  helmfile [eqiad] DONE helmfile.d/services/citoid: apply | 
  [production] | 
            
  | 21:24 | 
  <rzl@deploy1003> | 
  helmfile [eqiad] START helmfile.d/services/citoid: apply | 
  [production] | 
            
  | 21:23 | 
  <rzl@deploy1003> | 
  helmfile [codfw] DONE helmfile.d/services/cirrus-streaming-updater: apply | 
  [production] | 
            
  | 21:22 | 
  <rzl@deploy1003> | 
  helmfile [codfw] START helmfile.d/services/cirrus-streaming-updater: apply | 
  [production] | 
            
  | 21:21 | 
  <rzl@deploy1003> | 
  helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply | 
  [production] | 
            
  | 21:21 | 
  <rzl@deploy1003> | 
  helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply | 
  [production] | 
            
  | 21:21 | 
  <rzl@deploy1003> | 
  helmfile [codfw] DONE helmfile.d/services/chart-renderer: apply | 
  [production] | 
            
  | 21:21 | 
  <rzl@deploy1003> | 
  helmfile [codfw] START helmfile.d/services/chart-renderer: apply | 
  [production] | 
            
  | 21:20 | 
  <rzl@deploy1003> | 
  helmfile [eqiad] DONE helmfile.d/services/chart-renderer: apply | 
  [production] | 
            
  | 21:19 | 
  <rzl@deploy1003> | 
  helmfile [eqiad] START helmfile.d/services/chart-renderer: apply | 
  [production] | 
            
  | 21:19 | 
  <rzl@deploy1003> | 
  helmfile [eqiad] DONE helmfile.d/services/changeprop-jobqueue: apply | 
  [production] | 
            
  | 21:18 | 
  <rzl@deploy1003> | 
  helmfile [eqiad] START helmfile.d/services/changeprop-jobqueue: apply | 
  [production] | 
            
  | 21:17 | 
  <rzl@deploy1003> | 
  helmfile [eqiad] DONE helmfile.d/services/changeprop: apply | 
  [production] | 
            
  | 21:17 | 
  <rzl@deploy1003> | 
  helmfile [eqiad] START helmfile.d/services/changeprop: apply | 
  [production] | 
            
  | 21:16 | 
  <rzl@deploy1003> | 
  helmfile [codfw] DONE helmfile.d/services/api-gateway: apply | 
  [production] | 
            
  | 21:15 | 
  <rzl@deploy1003> | 
  helmfile [codfw] START helmfile.d/services/api-gateway: apply | 
  [production] | 
            
  | 21:15 | 
  <rzl@deploy1003> | 
  helmfile [eqiad] DONE helmfile.d/services/api-gateway: apply | 
  [production] | 
            
  | 21:14 | 
  <rzl@deploy1003> | 
  helmfile [eqiad] START helmfile.d/services/api-gateway: apply | 
  [production] | 
            
  | 21:06 | 
  <kharlan@deploy1003> | 
  Finished scap sync-world: Backport for [[gerrit:1187079|hCaptcha: Set wgHCaptchaApiUrlIntegrityHash and pin secure-api.js version (T404251)]] (duration: 13m 42s) | 
  [production] | 
            
  | 21:01 | 
  <kharlan@deploy1003> | 
  kharlan: Continuing with sync | 
  [production] | 
            
  | 20:58 | 
  <kharlan@deploy1003> | 
  kharlan: Backport for [[gerrit:1187079|hCaptcha: Set wgHCaptchaApiUrlIntegrityHash and pin secure-api.js version (T404251)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. | 
  [production] | 
            
  | 20:52 | 
  <kharlan@deploy1003> | 
  Started scap sync-world: Backport for [[gerrit:1187079|hCaptcha: Set wgHCaptchaApiUrlIntegrityHash and pin secure-api.js version (T404251)]] | 
  [production] | 
            
  | 20:49 | 
  <kharlan@deploy1003> | 
  Finished scap sync-world: Backport for [[gerrit:1188823|hCaptcha: Enable version pinning and subresource integrity (T404251)]], [[gerrit:1188824|hCaptcha: Enable version pinning and subresource integrity (T404251)]] (duration: 12m 31s) | 
  [production] | 
            
  | 20:43 | 
  <kharlan@deploy1003> | 
  kharlan: Continuing with sync | 
  [production] | 
            
  | 20:42 | 
  <kharlan@deploy1003> | 
  kharlan: Backport for [[gerrit:1188823|hCaptcha: Enable version pinning and subresource integrity (T404251)]], [[gerrit:1188824|hCaptcha: Enable version pinning and subresource integrity (T404251)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. | 
  [production] | 
            
  | 20:36 | 
  <kharlan@deploy1003> | 
  Started scap sync-world: Backport for [[gerrit:1188823|hCaptcha: Enable version pinning and subresource integrity (T404251)]], [[gerrit:1188824|hCaptcha: Enable version pinning and subresource integrity (T404251)]] | 
  [production] | 
            
  | 20:26 | 
  <jhancock@cumin1002> | 
  END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host dse-k8s-worker2003.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART | 
  [production] | 
            
  | 20:21 | 
  <jhuneidi@deploy1003> | 
  Finished scap sync-world: Backport for [[gerrit:1188493|Throttle exemption for Editathon by Wikimedistas en Cruce - 26 September 2025 (T404592)]] (duration: 12m 27s) | 
  [production] | 
            
  | 20:21 | 
  <jhancock@cumin1002> | 
  START - Cookbook sre.hosts.provision for host dse-k8s-worker2003.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART | 
  [production] | 
            
  | 20:16 | 
  <jhuneidi@deploy1003> | 
  jhuneidi, superpes: Continuing with sync | 
  [production] | 
            
  | 20:16 | 
  <jhuneidi@deploy1003> | 
  jhuneidi, superpes: Backport for [[gerrit:1188493|Throttle exemption for Editathon by Wikimedistas en Cruce - 26 September 2025 (T404592)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. | 
  [production] | 
            
  | 20:09 | 
  <jhuneidi@deploy1003> | 
  Started scap sync-world: Backport for [[gerrit:1188493|Throttle exemption for Editathon by Wikimedistas en Cruce - 26 September 2025 (T404592)]] | 
  [production] | 
            
  | 20:00 | 
  <swfrench-wmf> | 
  migrated shellbox (score) to PHP 8.3 - T403284 | 
  [production] | 
            
  | 19:59 | 
  <swfrench@deploy1003> | 
  helmfile [eqiad] DONE helmfile.d/services/shellbox: apply | 
  [production] | 
            
  | 19:58 | 
  <swfrench@deploy1003> | 
  helmfile [eqiad] START helmfile.d/services/shellbox: apply | 
  [production] | 
            
  | 19:52 | 
  <jhuneidi@deploy1003> | 
  rebuilt and synchronized wikiversions files: group0 to 1.45.0-wmf.19  refs T396380 | 
  [production] | 
            
  | 19:48 | 
  <jhancock@cumin1002> | 
  END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host dse-k8s-worker2003.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART | 
  [production] | 
            
  | 19:48 | 
  <jhancock@cumin1002> | 
  START - Cookbook sre.hosts.provision for host dse-k8s-worker2003.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART | 
  [production] | 
            
  | 19:47 | 
  <jhancock@cumin1002> | 
  END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host dse-k8s-worker2003.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART | 
  [production] | 
            
  | 19:43 | 
  <jhancock@cumin1002> | 
  START - Cookbook sre.hosts.provision for host dse-k8s-worker2003.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART | 
  [production] | 
            
  | 19:41 | 
  <jhancock@cumin1002> | 
  END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host dse-k8s-worker2003.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART | 
  [production] | 
            
  | 19:39 | 
  <eileen> | 
  civicrm upgraded from a847a79d to 636ba9d5 | 
  [production] |