| 2024-09-13
      
      § | 
    
  | 07:34 | <jayme@deploy1003> | helmfile [ml-serve-eqiad] DONE helmfile.d/admin 'apply'. | [production] | 
            
  | 07:34 | <jayme@deploy1003> | helmfile [ml-serve-eqiad] START helmfile.d/admin 'apply'. | [production] | 
            
  | 07:34 | <jayme@deploy1003> | helmfile [staging-codfw] DONE helmfile.d/admin 'apply'. | [production] | 
            
  | 07:33 | <jayme@deploy1003> | helmfile [staging-codfw] START helmfile.d/admin 'apply'. | [production] | 
            
  | 07:33 | <jayme@deploy1003> | helmfile [staging-eqiad] DONE helmfile.d/admin 'apply'. | [production] | 
            
  | 07:33 | <jayme@deploy1003> | helmfile [staging-eqiad] START helmfile.d/admin 'apply'. | [production] | 
            
  | 07:33 | <jayme@deploy1003> | helmfile [codfw] DONE helmfile.d/admin 'apply'. | [production] | 
            
  | 07:32 | <jayme@deploy1003> | helmfile [codfw] START helmfile.d/admin 'apply'. | [production] | 
            
  | 07:32 | <jayme@deploy1003> | helmfile [eqiad] DONE helmfile.d/admin 'apply'. | [production] | 
            
  | 07:32 | <jayme@deploy1003> | helmfile [eqiad] START helmfile.d/admin 'apply'. | [production] | 
            
  | 07:27 | <jayme@cumin1002> | START - Cookbook sre.kafka.roll-restart-reboot-brokers rolling restart_daemons on A:kafka-main-codfw | [production] | 
            
  | 06:56 | <jayme@cumin1002> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on kafka-main[2005,2010].codfw.wmnet with reason: Hardware refresh | [production] | 
            
  | 06:56 | <jayme@cumin1002> | START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on kafka-main[2005,2010].codfw.wmnet with reason: Hardware refresh | [production] | 
            
  | 06:54 | <jayme> | evacuating leadership for all partitions assigned to broker id 2005 on kafka-main-codfw - T363210 | [production] | 
            
  
    | 2024-09-12
      
      § | 
    
  | 23:01 | <dduvall> | deployed https://gitlab.wikimedia.org/repos/releng/jenkins-deploy/-/merge_requests/79 and https://gitlab.wikimedia.org/repos/releng/jenkins-deploy/-/merge_requests/80 to releases-jenkins | [releng] | 
            
  | 23:00 | <dduvall@deploy1003> | Finished deploy [releng/jenkins-deploy@35befba] (releasing): (no justification provided) (duration: 00m 38s) | [production] | 
            
  | 22:59 | <dduvall@deploy1003> | Started deploy [releng/jenkins-deploy@35befba] (releasing): (no justification provided) | [production] | 
            
  | 22:34 | <dduvall@deploy1003> | Finished deploy [releng/jenkins-deploy@6e810dc] (releasing): (no justification provided) (duration: 00m 34s) | [production] | 
            
  | 22:33 | <dduvall@deploy1003> | Started deploy [releng/jenkins-deploy@6e810dc] (releasing): (no justification provided) | [production] | 
            
  | 22:30 | <dduvall@deploy1003> | deploy aborted: (no justification provided) (duration: 01m 43s) | [production] | 
            
  | 22:28 | <dduvall@deploy1003> | Started deploy [releng/jenkins-deploy@6e810dc] (releasing): (no justification provided) | [production] | 
            
  | 22:11 | <bd808@deploy1003> | helmfile [eqiad] DONE helmfile.d/services/toolhub: apply | [production] | 
            
  | 22:11 | <bd808@deploy1003> | helmfile [eqiad] START helmfile.d/services/toolhub: apply | [production] | 
            
  | 22:00 | <tzatziki> | removing 1 file for legal compliance | [production] | 
            
  | 21:56 | <tzatziki> | removing 6 files for legal compliance | [production] | 
            
  | 21:54 | <bd808@deploy1003> | helmfile [codfw] DONE helmfile.d/services/toolhub: apply | [production] | 
            
  | 21:52 | <ladsgroup@deploy1003> | Finished scap sync-world: Backport for [[gerrit:1072488|Fix night mode excepted Wikidata namespaces]] (duration: 07m 09s) | [production] | 
            
  | 21:52 | <bd808@deploy1003> | helmfile [codfw] START helmfile.d/services/toolhub: apply | [production] | 
            
  | 21:48 | <ladsgroup@deploy1003> | ladsgroup, ebrahim: Continuing with sync | [production] | 
            
  | 21:47 | <ladsgroup@deploy1003> | ladsgroup, ebrahim: Backport for [[gerrit:1072488|Fix night mode excepted Wikidata namespaces]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) | [production] | 
            
  | 21:45 | <ladsgroup@deploy1003> | Started scap sync-world: Backport for [[gerrit:1072488|Fix night mode excepted Wikidata namespaces]] | [production] | 
            
  | 21:43 | <bd808@deploy1003> | helmfile [staging] DONE helmfile.d/services/toolhub: apply | [production] | 
            
  | 21:40 | <cjming> | end of UTC late backport window | [production] | 
            
  | 21:37 | <bd808@deploy1003> | helmfile [staging] START helmfile.d/services/toolhub: apply | [production] | 
            
  | 21:37 | <cjming@deploy1003> | Finished scap sync-world: Backport for [[gerrit:1072614|Revert "Lift IP cap on this dates 10/09, 17/09, 24/09 for edit-a-thon for eswiki, commons and wikidata"]] (duration: 06m 37s) | [production] | 
            
  | 21:33 | <cjming@deploy1003> | cjming, gergesshamon: Continuing with sync | [production] | 
            
  | 21:32 | <cjming@deploy1003> | cjming, gergesshamon: Backport for [[gerrit:1072614|Revert "Lift IP cap on this dates 10/09, 17/09, 24/09 for edit-a-thon for eswiki, commons and wikidata"]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) | [production] | 
            
  | 21:30 | <cjming@deploy1003> | Started scap sync-world: Backport for [[gerrit:1072614|Revert "Lift IP cap on this dates 10/09, 17/09, 24/09 for edit-a-thon for eswiki, commons and wikidata"]] | [production] | 
            
  | 21:20 | <tzatziki> | removing 1 file for legal compliance | [production] | 
            
  | 21:16 | <cjming@deploy1003> | Finished scap sync-world: Backport for [[gerrit:1072606|Revert "Lift IP cap on this dates 10/09, 17/09, 24/09 for edit-a-thon for eswiki, commons and wikidata"]] (duration: 08m 57s) | [production] | 
            
  | 21:13 | <tzatziki> | removing 1 file for legal compliance | [production] | 
            
  | 21:12 | <cjming@deploy1003> | cjming, trainbranchbot: Continuing with sync | [production] | 
            
  | 21:09 | <cjming@deploy1003> | cjming, trainbranchbot: Backport for [[gerrit:1072606|Revert "Lift IP cap on this dates 10/09, 17/09, 24/09 for edit-a-thon for eswiki, commons and wikidata"]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) | [production] | 
            
  | 21:07 | <cjming@deploy1003> | Started scap sync-world: Backport for [[gerrit:1072606|Revert "Lift IP cap on this dates 10/09, 17/09, 24/09 for edit-a-thon for eswiki, commons and wikidata"]] | [production] | 
            
  | 21:05 | <cjming@deploy1003> | Sync cancelled. | [production] | 
            
  | 21:01 | <cjming@deploy1003> | cjming, gergesshamon: Backport for [[gerrit:1072604|Lift IP cap on this dates 10/09, 17/09, 24/09 for edit-a-thon for eswiki, commons and wikidata]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) | [production] | 
            
  | 20:59 | <cjming@deploy1003> | Started scap sync-world: Backport for [[gerrit:1072604|Lift IP cap on this dates 10/09, 17/09, 24/09 for edit-a-thon for eswiki, commons and wikidata]] | [production] | 
            
  | 20:53 | <cjming@deploy1003> | Finished scap sync-world: Backport for [[gerrit:1072227|eswiki, commonswiki, wikidata: lift IP cap for edit-a-thon (T374484)]] (duration: 09m 21s) | [production] | 
            
  | 20:49 | <cjming@deploy1003> | cjming, superzerocool: Continuing with sync | [production] | 
            
  | 20:46 | <cjming@deploy1003> | cjming, superzerocool: Backport for [[gerrit:1072227|eswiki, commonswiki, wikidata: lift IP cap for edit-a-thon (T374484)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) | [production] |