| 2024-09-17
      
      ยง | 
    
  | 03:20 | <raymond-ndibe@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-ingress-8 from 1.26.15 to 1.27.16 (T359641) | [tools] | 
            
  | 03:19 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-ingress-8 from 1.26.15 to 1.27.16 (T359641) | [tools] | 
            
  | 03:18 | <raymond-ndibe@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-ingress-7 from 1.26.15 to 1.27.16 (T359641) | [tools] | 
            
  | 03:18 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-ingress-7 from 1.26.15 to 1.27.16 (T359641) | [tools] | 
            
  | 03:13 | <raymond-ndibe@cloudcumin1001> | END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=99) for host tools-k8s-worker-nfs-64 | [tools] | 
            
  | 03:09 | <raymond-ndibe@cloudcumin1001> | END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=99) for host tools-k8s-worker-nfs-63 | [tools] | 
            
  | 03:08 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-nfs-64 (T359641) | [tools] | 
            
  | 03:07 | <raymond-ndibe@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster | [tools] | 
            
  | 03:07 | <raymond-ndibe@cloudcumin1001> | Added a new k8s worker-nfs tools-k8s-worker-nfs-76.tools.eqiad1.wikimedia.cloud to the cluster | [tools] | 
            
  | 03:04 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-nfs-63 (T359641) | [tools] | 
            
  | 03:02 | <mwpresync@deploy1003> | Started scap sync-world: testwikis to 1.43.0-wmf.23  refs T373642 | [production] | 
            
  | 03:00 | <raymond-ndibe@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster | [tools] | 
            
  | 03:00 | <raymond-ndibe@cloudcumin1001> | Added a new k8s worker-nfs tools-k8s-worker-nfs-75.tools.eqiad1.wikimedia.cloud to the cluster | [tools] | 
            
  | 02:57 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster (T359641) | [tools] | 
            
  | 02:50 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster (T359641) | [tools] | 
            
  | 02:46 | <raymond-ndibe@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster | [tools] | 
            
  | 02:46 | <raymond-ndibe@cloudcumin1001> | Added a new k8s worker-nfs tools-k8s-worker-nfs-74.tools.eqiad1.wikimedia.cloud to the cluster | [tools] | 
            
  | 02:45 | <raymond-ndibe@cloudcumin1001> | END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=99) for host tools-k8s-worker-nfs-62 | [tools] | 
            
  | 02:45 | <raymond-ndibe@cloudcumin1001> | END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=99) for host tools-k8s-worker-nfs-60 | [tools] | 
            
  | 02:40 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-nfs-62 (T359641) | [tools] | 
            
  | 02:40 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-nfs-60 (T359641) | [tools] | 
            
  | 02:38 | <raymond-ndibe@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster | [tools] | 
            
  | 02:38 | <raymond-ndibe@cloudcumin1001> | Added a new k8s worker-nfs tools-k8s-worker-nfs-73.tools.eqiad1.wikimedia.cloud to the cluster | [tools] | 
            
  | 02:36 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster (T359641) | [tools] | 
            
  | 02:32 | <raymond-ndibe@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster | [tools] | 
            
  | 02:32 | <raymond-ndibe@cloudcumin1001> | Added a new k8s worker-nfs tools-k8s-worker-nfs-72.tools.eqiad1.wikimedia.cloud to the cluster | [tools] | 
            
  | 02:29 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster (T359641) | [tools] | 
            
  | 02:24 | <raymond-ndibe@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster | [tools] | 
            
  | 02:24 | <raymond-ndibe@cloudcumin1001> | Added a new k8s worker-nfs tools-k8s-worker-nfs-71.tools.eqiad1.wikimedia.cloud to the cluster | [tools] | 
            
  | 02:22 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster (T359641) | [tools] | 
            
  | 02:15 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster (T359641) | [tools] | 
            
  | 02:12 | <raymond-ndibe@cloudcumin1001> | END (ERROR) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=97) for a worker-nfs role in the tools cluster (T359641) | [tools] | 
            
  | 02:10 | <raymond-ndibe@cloudcumin1001> | END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=99) for host tools-k8s-worker-nfs-6 | [tools] | 
            
  | 02:10 | <raymond-ndibe@cloudcumin1001> | END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=99) for host tools-k8s-worker-nfs-56 | [tools] | 
            
  | 02:08 | <raymond-ndibe@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster | [tools] | 
            
  | 02:08 | <raymond-ndibe@cloudcumin1001> | Added a new k8s worker-nfs tools-k8s-worker-nfs-70.tools.eqiad1.wikimedia.cloud to the cluster | [tools] | 
            
  | 02:05 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-nfs-6 (T359641) | [tools] | 
            
  | 02:04 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-nfs-56 (T359641) | [tools] | 
            
  | 02:02 | <raymond-ndibe@cloudcumin1001> | END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=99) for host tools-k8s-worker-nfs-49 | [tools] | 
            
  | 02:02 | <raymond-ndibe@cloudcumin1001> | END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=99) for host tools-k8s-worker-nfs-31 | [tools] | 
            
  | 01:58 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster (T359641) | [tools] | 
            
  | 01:58 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster (T359641) | [tools] | 
            
  | 01:58 | <raymond-ndibe@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster | [tools] | 
            
  | 01:57 | <raymond-ndibe@cloudcumin1001> | Added a new k8s worker-nfs tools-k8s-worker-nfs-69.tools.eqiad1.wikimedia.cloud to the cluster | [tools] | 
            
  | 01:57 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-nfs-49 (T359641) | [tools] | 
            
  | 01:57 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-nfs-31 (T359641) | [tools] | 
            
  | 01:56 | <raymond-ndibe@cloudcumin1001> | END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=99) for host tools-k8s-worker-nfs-30 | [tools] | 
            
  | 01:54 | <raymond-ndibe@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-64 (T359641) | [tools] | 
            
  | 01:53 | <raymond-ndibe@cloudcumin1001> | END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=99) for host tools-k8s-worker-nfs-29 | [tools] | 
            
  | 01:50 | <raymond-ndibe@cloudcumin1001> | START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-nfs-30 (T359641) | [tools] |