| 2024-09-09
      
      ยง | 
    
  | 21:37 | <bking@deploy1003> | helmfile [staging] START helmfile.d/services/rdf-streaming-updater: apply | [production] | 
            
  | 21:35 | <bking@deploy1003> | helmfile [staging] DONE helmfile.d/services/rdf-streaming-updater: apply | [production] | 
            
  | 21:35 | <bking@deploy1003> | helmfile [staging] START helmfile.d/services/rdf-streaming-updater: apply | [production] | 
            
  | 21:35 | <bking@deploy1003> | helmfile [staging] DONE helmfile.d/services/rdf-streaming-updater: apply | [production] | 
            
  | 21:35 | <bking@deploy1003> | helmfile [staging] START helmfile.d/services/rdf-streaming-updater: apply | [production] | 
            
  | 21:32 | <dcaro@cloudcumin1001> | END (FAIL) - Cookbook wmcs.ceph.osd.drain_node (exit_code=99) (T373986) | [admin] | 
            
  | 21:29 | <bking@deploy1003> | helmfile [staging] DONE helmfile.d/services/rdf-streaming-updater: apply | [production] | 
            
  | 21:29 | <bking@deploy1003> | helmfile [staging] START helmfile.d/services/rdf-streaming-updater: apply | [production] | 
            
  | 21:27 | <bking@deploy1003> | helmfile [staging] DONE helmfile.d/services/rdf-streaming-updater: apply | [production] | 
            
  | 21:26 | <bking@deploy1003> | helmfile [staging] START helmfile.d/services/rdf-streaming-updater: apply | [production] | 
            
  | 21:24 | <bking@deploy1003> | helmfile [staging] DONE helmfile.d/services/rdf-streaming-updater: apply | [production] | 
            
  | 21:24 | <bking@deploy1003> | helmfile [staging] START helmfile.d/services/rdf-streaming-updater: apply | [production] | 
            
  | 21:23 | <bking@deploy1003> | helmfile [staging] START helmfile.d/services/rdf-streaming-updater: apply | [production] | 
            
  | 21:19 | <bking@deploy1003> | helmfile [staging] DONE helmfile.d/services/rdf-streaming-updater: apply | [production] | 
            
  | 21:19 | <bking@deploy1003> | helmfile [staging] START helmfile.d/services/rdf-streaming-updater: apply | [production] | 
            
  | 21:19 | <bking@deploy1003> | helmfile [staging] DONE helmfile.d/services/rdf-streaming-updater: apply | [production] | 
            
  | 21:19 | <bking@deploy1003> | helmfile [staging] START helmfile.d/services/rdf-streaming-updater: apply | [production] | 
            
  | 21:18 | <bking@deploy1003> | helmfile [staging] DONE helmfile.d/services/rdf-streaming-updater: apply | [production] | 
            
  | 21:17 | <bking@deploy1003> | helmfile [staging] START helmfile.d/services/rdf-streaming-updater: apply | [production] | 
            
  | 21:15 | <bking@deploy1003> | helmfile [staging] DONE helmfile.d/services/rdf-streaming-updater: apply | [production] | 
            
  | 21:15 | <bking@deploy1003> | helmfile [staging] START helmfile.d/services/rdf-streaming-updater: apply | [production] | 
            
  | 21:02 | <kindrobot> | finished UTC late backport window. BACKPORTED: 1071281, 1071202, NOT backported (ran out of time): 1071037, 1070347, 1070354, 1070975 | [production] | 
            
  | 21:01 | <kindrobot@deploy1003> | Finished scap sync-world: Backport for [[gerrit:1071281|Release donate link to pilot wikis (French Wikipedia and Wikifunctions) (T373585)]], [[gerrit:1071202|Fix typo in browser vendor prefix (T374180)]] (duration: 12m 39s) | [production] | 
            
  | 20:57 | <kindrobot@deploy1003> | jforrester, toyofuku, kindrobot: Continuing with sync | [production] | 
            
  | 20:51 | <kindrobot@deploy1003> | jforrester, toyofuku, kindrobot: Backport for [[gerrit:1071281|Release donate link to pilot wikis (French Wikipedia and Wikifunctions) (T373585)]], [[gerrit:1071202|Fix typo in browser vendor prefix (T374180)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) | [production] | 
            
  | 20:49 | <kindrobot@deploy1003> | Started scap sync-world: Backport for [[gerrit:1071281|Release donate link to pilot wikis (French Wikipedia and Wikifunctions) (T373585)]], [[gerrit:1071202|Fix typo in browser vendor prefix (T374180)]] | [production] | 
            
  | 19:33 | <jforrester@deploy1003> | Finished scap sync-world: Backport for [[gerrit:1071690|ZObjectStructureValidator::validate: use set_time_limit() to limit in the case of run-away JsonSchema (T374241)]] (duration: 08m 19s) | [production] | 
            
  | 19:28 | <jforrester@deploy1003> | jforrester: Continuing with sync | [production] | 
            
  | 19:27 | <jforrester@deploy1003> | jforrester: Backport for [[gerrit:1071690|ZObjectStructureValidator::validate: use set_time_limit() to limit in the case of run-away JsonSchema (T374241)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) | [production] | 
            
  | 19:25 | <jforrester@deploy1003> | Started scap sync-world: Backport for [[gerrit:1071690|ZObjectStructureValidator::validate: use set_time_limit() to limit in the case of run-away JsonSchema (T374241)]] | [production] | 
            
  | 18:16 | <ejegg> | payments-wiki upgraded from 4f3500f7 to 672c9fb6 | [production] | 
            
  | 17:44 | <jforrester@deploy1003> | Finished scap sync-world: Backport for [[gerrit:1071266|ZObjectStore::findZTesterResult: Trim our own error so we don't break logstash (T374241)]] (duration: 12m 05s) | [production] | 
            
  | 17:39 | <jforrester@deploy1003> | jforrester: Continuing with sync | [production] | 
            
  | 17:33 | <jforrester@deploy1003> | jforrester: Backport for [[gerrit:1071266|ZObjectStore::findZTesterResult: Trim our own error so we don't break logstash (T374241)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) | [production] | 
            
  | 17:31 | <jforrester@deploy1003> | Started scap sync-world: Backport for [[gerrit:1071266|ZObjectStore::findZTesterResult: Trim our own error so we don't break logstash (T374241)]] | [production] | 
            
  | 17:15 | <ebernhardson@deploy1003> | helmfile [codfw] DONE helmfile.d/services/cirrus-streaming-updater: apply | [production] | 
            
  | 17:15 | <ebernhardson@deploy1003> | helmfile [codfw] START helmfile.d/services/cirrus-streaming-updater: apply | [production] | 
            
  | 17:08 | <ebernhardson@deploy1003> | helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply | [production] | 
            
  | 17:07 | <ebernhardson@deploy1003> | helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply | [production] | 
            
  | 17:05 | <pt1979@cumin2002> | END (PASS) - Cookbook sre.dns.netbox (exit_code=0) | [production] | 
            
  | 17:05 | <pt1979@cumin2002> | END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: add nes frack firewalls - pt1979@cumin2002" | [production] | 
            
  | 17:05 | <pt1979@cumin2002> | START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: add nes frack firewalls - pt1979@cumin2002" | [production] | 
            
  | 17:02 | <ebernhardson@deploy1003> | helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply | [production] | 
            
  | 17:02 | <ebernhardson@deploy1003> | helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply | [production] | 
            
  | 17:01 | <pt1979@cumin2002> | START - Cookbook sre.dns.netbox | [production] | 
            
  | 16:54 | <jmm@cumin2002> | END (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Manuel Merz (WMDE) out of all services on: 1552 hosts | [production] | 
            
  | 16:53 | <jmm@cumin2002> | START - Cookbook sre.idm.logout Logging Manuel Merz (WMDE) out of all services on: 1552 hosts | [production] | 
            
  | 16:52 | <jmm@cumin2002> | END (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Manuel Merz (WMDE) out of all services on: 677 hosts | [production] | 
            
  | 16:52 | <jmm@cumin2002> | START - Cookbook sre.idm.logout Logging Manuel Merz (WMDE) out of all services on: 677 hosts | [production] | 
            
  | 16:46 | <ejegg> | payments-wiki upgraded from e47e61cb to 4f3500f7 | [production] |