| 2024-09-17
      
      § | 
    
  | 01:00 | <raymond-ndibe@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-36 from 1.26.15 to 1.27.16 (T359641) | [tools] | 
            
  | 01:00 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-36 from 1.26.15 to 1.27.16 (T359641) | [tools] | 
            
  | 01:00 | <raymond-ndibe@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-32 from 1.26.15 to 1.27.16 (T359641) | [tools] | 
            
  | 00:59 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-31 (T359641) | [tools] | 
            
  | 00:59 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-32 from 1.26.15 to 1.27.16 (T359641) | [tools] | 
            
  | 00:59 | <raymond-ndibe@cloudcumin1001> | END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node tools-k8s-worker-nfs-31 from 1.26.15 to 1.27.16 (T359641) | [tools] | 
            
  | 00:58 | <raymond-ndibe@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-30 (T359641) | [tools] | 
            
  | 00:53 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-31 from 1.26.15 to 1.27.16 (T359641) | [tools] | 
            
  | 00:53 | <raymond-ndibe@cloudcumin1001> | END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node tools-k8s-worker-nfs-30 from 1.26.15 to 1.27.16 (T359641) | [tools] | 
            
  | 00:52 | <raymond-ndibe@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-29 (T359641) | [tools] | 
            
  | 00:47 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-29 (T359641) | [tools] | 
            
  | 00:47 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-30 from 1.26.15 to 1.27.16 (T359641) | [tools] | 
            
  | 00:47 | <raymond-ndibe@cloudcumin1001> | END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node tools-k8s-worker-nfs-29 from 1.26.15 to 1.27.16 (T359641) | [tools] | 
            
  | 00:47 | <raymond-ndibe@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-28 (T359641) | [tools] | 
            
  | 00:41 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-28 (T359641) | [tools] | 
            
  | 00:41 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-29 from 1.26.15 to 1.27.16 (T359641) | [tools] | 
            
  | 00:41 | <raymond-ndibe@cloudcumin1001> | END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node tools-k8s-worker-nfs-28 from 1.26.15 to 1.27.16 (T359641) | [tools] | 
            
  | 00:35 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-28 from 1.26.15 to 1.27.16 (T359641) | [tools] | 
            
  | 00:35 | <raymond-ndibe@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-27 from 1.26.15 to 1.27.16 (T359641) | [tools] | 
            
  | 00:34 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-27 from 1.26.15 to 1.27.16 (T359641) | [tools] | 
            
  | 00:34 | <raymond-ndibe@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-26 from 1.26.15 to 1.27.16 (T359641) | [tools] | 
            
  | 00:33 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-26 from 1.26.15 to 1.27.16 (T359641) | [tools] | 
            
  | 00:33 | <raymond-ndibe@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-22 from 1.26.15 to 1.27.16 (T359641) | [tools] | 
            
  | 00:32 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-22 from 1.26.15 to 1.27.16 (T359641) | [tools] | 
            
  | 00:32 | <raymond-ndibe@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-21 from 1.26.15 to 1.27.16 (T359641) | [tools] | 
            
  | 00:31 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-21 from 1.26.15 to 1.27.16 (T359641) | [tools] | 
            
  | 00:30 | <raymond-ndibe@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-60, tools-k8s-worker-nfs-61, tools-k8s-worker-nfs-62, tools-k8s-worker-nfs-63 (T359641) | [tools] | 
            
  | 00:26 | <raymond-ndibe@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-50, tools-k8s-worker-nfs-56, tools-k8s-worker-nfs-57, tools-k8s-worker-nfs-6 (T359641) | [tools] | 
            
  | 00:10 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-50, tools-k8s-worker-nfs-56, tools-k8s-worker-nfs-57, tools-k8s-worker-nfs-6 (T359641) | [tools] | 
            
  | 00:09 | <raymond-ndibe@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-38, tools-k8s-worker-nfs-46, tools-k8s-worker-nfs-49, tools-k8s-worker-nfs-50 (T359641) | [tools] | 
            
  | 00:09 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-60, tools-k8s-worker-nfs-61, tools-k8s-worker-nfs-62, tools-k8s-worker-nfs-63 (T359641) | [tools] | 
            
  | 00:04 | <raymond-ndibe@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-31, tools-k8s-worker-nfs-32, tools-k8s-worker-nfs-33, tools-k8s-worker-nfs-36 (T359641) | [tools] | 
            
  | 00:03 | <wmbot~bd808@tools-bastion-12> | Updated to 25edc86 (T374875) | [tools.stashbot] | 
            
  
    | 2024-09-16
      
      § | 
    
  | 22:49 | <Dreamy_Jazz> | Restarting MediaModeration scanning script - https://wikitech.wikimedia.org/wiki/MediaModeration | [production] | 
            
  | 20:53 | <jhathaway> | reloading puppetserver to enable strict mode | [production] | 
            
  | 20:47 | <pt1979@cumin2002> | END (PASS) - Cookbook sre.dns.netbox (exit_code=0) | [production] | 
            
  | 20:47 | <pt1979@cumin2002> | END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: add  frack new switches - pt1979@cumin2002" | [production] | 
            
  | 20:47 | <pt1979@cumin2002> | START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: add  frack new switches - pt1979@cumin2002" | [production] | 
            
  | 20:42 | <pt1979@cumin2002> | START - Cookbook sre.dns.netbox | [production] | 
            
  | 20:28 | <pt1979@cumin2002> | END (PASS) - Cookbook sre.dns.netbox (exit_code=0) | [production] | 
            
  | 20:28 | <pt1979@cumin2002> | END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: add  frack new switches - pt1979@cumin2002" | [production] | 
            
  | 20:28 | <pt1979@cumin2002> | START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: add  frack new switches - pt1979@cumin2002" | [production] | 
            
  | 20:25 | <pt1979@cumin2002> | START - Cookbook sre.dns.netbox | [production] | 
            
  | 20:25 | <toyofuku@deploy1003> | Finished scap sync-world: Backport for [[gerrit:1073277|Deploy Vector 2022 on small wikis (T374255)]], [[gerrit:1073279|Disable quick surveys (T374743)]] (duration: 08m 53s) | [production] | 
            
  | 20:24 | <pt1979@cumin2002> | END (FAIL) - Cookbook sre.dns.netbox (exit_code=99) | [production] | 
            
  | 20:21 | <pt1979@cumin2002> | START - Cookbook sre.dns.netbox | [production] | 
            
  | 20:20 | <toyofuku@deploy1003> | jdlrobson, toyofuku: Continuing with sync | [production] | 
            
  | 20:18 | <toyofuku@deploy1003> | jdlrobson, toyofuku: Backport for [[gerrit:1073277|Deploy Vector 2022 on small wikis (T374255)]], [[gerrit:1073279|Disable quick surveys (T374743)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) | [production] | 
            
  | 20:16 | <toyofuku@deploy1003> | Started scap sync-world: Backport for [[gerrit:1073277|Deploy Vector 2022 on small wikis (T374255)]], [[gerrit:1073279|Disable quick surveys (T374743)]] | [production] | 
            
  | 19:20 | <wmbot~lucaswerkmeister@tools-bastion-13> | deployed a82f584c44 (read toolsdb from envvars); commented out DATABASE section in config.yaml, should use envvars instead | [tools.quickcategories] |