| 2025-09-16
      
      ยง | 
    
  | 21:30 | <rzl@deploy1003> | helmfile [eqiad] DONE helmfile.d/services/developer-portal: apply | [production] | 
            
  | 21:29 | <rzl@deploy1003> | helmfile [eqiad] START helmfile.d/services/developer-portal: apply | [production] | 
            
  | 21:28 | <rzl@deploy1003> | helmfile [codfw] DONE helmfile.d/services/data-gateway: apply | [production] | 
            
  | 21:28 | <rzl@deploy1003> | helmfile [codfw] START helmfile.d/services/data-gateway: apply | [production] | 
            
  | 21:28 | <rzl@deploy1003> | helmfile [eqiad] DONE helmfile.d/services/data-gateway: apply | [production] | 
            
  | 21:27 | <rzl@deploy1003> | helmfile [eqiad] START helmfile.d/services/data-gateway: apply | [production] | 
            
  | 21:26 | <rzl@deploy1003> | helmfile [codfw] DONE helmfile.d/services/cxserver: apply | [production] | 
            
  | 21:26 | <rzl@deploy1003> | helmfile [codfw] START helmfile.d/services/cxserver: apply | [production] | 
            
  | 21:25 | <rzl@deploy1003> | helmfile [eqiad] DONE helmfile.d/services/cxserver: apply | [production] | 
            
  | 21:25 | <rzl@deploy1003> | helmfile [eqiad] START helmfile.d/services/cxserver: apply | [production] | 
            
  | 21:25 | <rzl@deploy1003> | helmfile [codfw] DONE helmfile.d/services/citoid: apply | [production] | 
            
  | 21:24 | <rzl@deploy1003> | helmfile [codfw] START helmfile.d/services/citoid: apply | [production] | 
            
  | 21:24 | <rzl@deploy1003> | helmfile [eqiad] DONE helmfile.d/services/citoid: apply | [production] | 
            
  | 21:24 | <rzl@deploy1003> | helmfile [eqiad] START helmfile.d/services/citoid: apply | [production] | 
            
  | 21:23 | <rzl@deploy1003> | helmfile [codfw] DONE helmfile.d/services/cirrus-streaming-updater: apply | [production] | 
            
  | 21:22 | <rzl@deploy1003> | helmfile [codfw] START helmfile.d/services/cirrus-streaming-updater: apply | [production] | 
            
  | 21:21 | <rzl@deploy1003> | helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply | [production] | 
            
  | 21:21 | <rzl@deploy1003> | helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply | [production] | 
            
  | 21:21 | <rzl@deploy1003> | helmfile [codfw] DONE helmfile.d/services/chart-renderer: apply | [production] | 
            
  | 21:21 | <rzl@deploy1003> | helmfile [codfw] START helmfile.d/services/chart-renderer: apply | [production] | 
            
  | 21:20 | <rzl@deploy1003> | helmfile [eqiad] DONE helmfile.d/services/chart-renderer: apply | [production] | 
            
  | 21:19 | <rzl@deploy1003> | helmfile [eqiad] START helmfile.d/services/chart-renderer: apply | [production] | 
            
  | 21:19 | <rzl@deploy1003> | helmfile [eqiad] DONE helmfile.d/services/changeprop-jobqueue: apply | [production] | 
            
  | 21:18 | <rzl@deploy1003> | helmfile [eqiad] START helmfile.d/services/changeprop-jobqueue: apply | [production] | 
            
  | 21:17 | <rzl@deploy1003> | helmfile [eqiad] DONE helmfile.d/services/changeprop: apply | [production] | 
            
  | 21:17 | <rzl@deploy1003> | helmfile [eqiad] START helmfile.d/services/changeprop: apply | [production] | 
            
  | 21:16 | <rzl@deploy1003> | helmfile [codfw] DONE helmfile.d/services/api-gateway: apply | [production] | 
            
  | 21:15 | <rzl@deploy1003> | helmfile [codfw] START helmfile.d/services/api-gateway: apply | [production] | 
            
  | 21:15 | <rzl@deploy1003> | helmfile [eqiad] DONE helmfile.d/services/api-gateway: apply | [production] | 
            
  | 21:14 | <rzl@deploy1003> | helmfile [eqiad] START helmfile.d/services/api-gateway: apply | [production] | 
            
  | 21:06 | <kharlan@deploy1003> | Finished scap sync-world: Backport for [[gerrit:1187079|hCaptcha: Set wgHCaptchaApiUrlIntegrityHash and pin secure-api.js version (T404251)]] (duration: 13m 42s) | [production] | 
            
  | 21:01 | <kharlan@deploy1003> | kharlan: Continuing with sync | [production] | 
            
  | 20:58 | <kharlan@deploy1003> | kharlan: Backport for [[gerrit:1187079|hCaptcha: Set wgHCaptchaApiUrlIntegrityHash and pin secure-api.js version (T404251)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. | [production] | 
            
  | 20:52 | <kharlan@deploy1003> | Started scap sync-world: Backport for [[gerrit:1187079|hCaptcha: Set wgHCaptchaApiUrlIntegrityHash and pin secure-api.js version (T404251)]] | [production] | 
            
  | 20:49 | <kharlan@deploy1003> | Finished scap sync-world: Backport for [[gerrit:1188823|hCaptcha: Enable version pinning and subresource integrity (T404251)]], [[gerrit:1188824|hCaptcha: Enable version pinning and subresource integrity (T404251)]] (duration: 12m 31s) | [production] | 
            
  | 20:43 | <kharlan@deploy1003> | kharlan: Continuing with sync | [production] | 
            
  | 20:42 | <kharlan@deploy1003> | kharlan: Backport for [[gerrit:1188823|hCaptcha: Enable version pinning and subresource integrity (T404251)]], [[gerrit:1188824|hCaptcha: Enable version pinning and subresource integrity (T404251)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. | [production] | 
            
  | 20:36 | <kharlan@deploy1003> | Started scap sync-world: Backport for [[gerrit:1188823|hCaptcha: Enable version pinning and subresource integrity (T404251)]], [[gerrit:1188824|hCaptcha: Enable version pinning and subresource integrity (T404251)]] | [production] | 
            
  | 20:26 | <jhancock@cumin1002> | END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host dse-k8s-worker2003.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART | [production] | 
            
  | 20:21 | <jhuneidi@deploy1003> | Finished scap sync-world: Backport for [[gerrit:1188493|Throttle exemption for Editathon by Wikimedistas en Cruce - 26 September 2025 (T404592)]] (duration: 12m 27s) | [production] | 
            
  | 20:21 | <jhancock@cumin1002> | START - Cookbook sre.hosts.provision for host dse-k8s-worker2003.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART | [production] | 
            
  | 20:16 | <jhuneidi@deploy1003> | jhuneidi, superpes: Continuing with sync | [production] | 
            
  | 20:16 | <jhuneidi@deploy1003> | jhuneidi, superpes: Backport for [[gerrit:1188493|Throttle exemption for Editathon by Wikimedistas en Cruce - 26 September 2025 (T404592)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. | [production] | 
            
  | 20:09 | <jhuneidi@deploy1003> | Started scap sync-world: Backport for [[gerrit:1188493|Throttle exemption for Editathon by Wikimedistas en Cruce - 26 September 2025 (T404592)]] | [production] | 
            
  | 20:00 | <swfrench-wmf> | migrated shellbox (score) to PHP 8.3 - T403284 | [production] | 
            
  | 19:59 | <swfrench@deploy1003> | helmfile [eqiad] DONE helmfile.d/services/shellbox: apply | [production] | 
            
  | 19:58 | <swfrench@deploy1003> | helmfile [eqiad] START helmfile.d/services/shellbox: apply | [production] | 
            
  | 19:52 | <jhuneidi@deploy1003> | rebuilt and synchronized wikiversions files: group0 to 1.45.0-wmf.19  refs T396380 | [production] | 
            
  | 19:48 | <jhancock@cumin1002> | END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host dse-k8s-worker2003.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART | [production] | 
            
  | 19:48 | <jhancock@cumin1002> | START - Cookbook sre.hosts.provision for host dse-k8s-worker2003.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART | [production] |