| 2023-05-15
      
      § | 
    
  | 18:22 | <taavi> | clear mail queue | [tools] | 
            
  | 18:20 | <wm-bot2> | rebooted k8s node tools-k8s-worker-76 (T316544) - cookbook ran by dcaro@vulcanus | [tools] | 
            
  | 18:15 | <wm-bot2> | rebooted k8s node tools-k8s-worker-77 (T316544) - cookbook ran by dcaro@vulcanus | [tools] | 
            
  | 18:08 | <wm-bot2> | rebooted k8s node tools-k8s-worker-80 (T316544) - cookbook ran by dcaro@vulcanus | [tools] | 
            
  | 18:06 | <wm-bot2> | rebooted k8s node tools-k8s-worker-81 (T316544) - cookbook ran by dcaro@vulcanus | [tools] | 
            
  | 18:05 | <wm-bot2> | rebooted k8s node tools-k8s-worker-82 (T316544) - cookbook ran by dcaro@vulcanus | [tools] | 
            
  | 17:57 | <wm-bot2> | rebooted k8s node tools-k8s-worker-83 (T316544) - cookbook ran by dcaro@vulcanus | [tools] | 
            
  | 17:48 | <wm-bot2> | rebooted k8s node tools-k8s-worker-84 (T316544) - cookbook ran by dcaro@vulcanus | [tools] | 
            
  | 17:47 | <wm-bot2> | rebooted k8s node tools-k8s-worker-85 (T316544) - cookbook ran by dcaro@vulcanus | [tools] | 
            
  | 17:38 | <wm-bot2> | rebooted k8s node tools-k8s-worker-86 (T316544) - cookbook ran by dcaro@vulcanus | [tools] | 
            
  | 17:37 | <wm-bot2> | rebooted k8s node tools-k8s-worker-87 (T316544) - cookbook ran by dcaro@vulcanus | [tools] | 
            
  | 17:35 | <wm-bot2> | rebooted k8s node tools-k8s-worker-88 (T316544) - cookbook ran by dcaro@vulcanus | [tools] | 
            
  | 17:34 | <wm-bot2> | rebooting all the workers of tools k8s cluster (64 nodes) (T316544) - cookbook ran by dcaro@vulcanus | [tools] | 
            
  | 17:20 | <wm-bot2> | rebooted k8s node tools-k8s-worker-87 (T316544) - cookbook ran by dcaro@vulcanus | [tools] | 
            
  | 17:19 | <wm-bot2> | rebooted k8s node tools-k8s-worker-88 (T316544) - cookbook ran by dcaro@vulcanus | [tools] | 
            
  | 17:17 | <bd808> | Rebuilding bullseye and buster docker containers to pick up openssh-client package addition (T258841) | [tools] | 
            
  | 17:12 | <wm-bot2> | rebooting the whole tools k8s cluster (64 nodes) (T316544) - cookbook ran by dcaro@vulcanus | [tools] | 
            
  | 17:06 | <dcaro> | rebooting tools-sgegrid-shadow (T316544) | [tools] | 
            
  | 17:00 | <dcaro> | rebooting tools-sgegrid-master (T316544) | [tools] | 
            
  | 16:55 | <dcaro> | rebooting tools-sgeexec-10-20 (T316544) | [tools] | 
            
  | 16:53 | <dcaro> | rebooting tools-sgeweblight-10-18 (T316544) | [tools] | 
            
  | 16:53 | <dcaro> | rebooting tools-sgeweblight-10-25 (T316544) | [tools] | 
            
  | 16:53 | <dcaro> | rebooting tools-sgeweblight-10-20 (T316544) | [tools] | 
            
  | 16:52 | <dcaro> | rebooting tools-sgeweblight-10-21 (T316544) | [tools] | 
            
  | 16:52 | <dcaro> | rebooting tools-sgeexec-10-22 (T316544) | [tools] | 
            
  | 16:51 | <dcaro> | rebooting tools-sgeweblight-10-28 (T316544) | [tools] | 
            
  | 16:50 | <dcaro> | rebooting tools-sgeexec-10-17 (T316544) | [tools] | 
            
  | 16:48 | <dcaro> | rebooting tools-sgeexec-10-21 (T316544) | [tools] | 
            
  | 16:47 | <dcaro> | rebooting tools-sgeexec-10-19 (T316544) | [tools] | 
            
  | 16:45 | <dcaro> | rebooting tools-sgeexec-10-8 (T316544) | [tools] | 
            
  | 16:45 | <dcaro> | rebooting tools-sgeweblight-10-24 (T316544) | [tools] | 
            
  | 16:44 | <dcaro> | rebooting tools-sgewebgen-10-2 (T316544) | [tools] | 
            
  | 16:44 | <dcaro> | rebooting tools-sgeweblight-10-16 (T316544) | [tools] | 
            
  | 16:43 | <dcaro> | rebooting tools-sgeweblight-10-30 (T316544) | [tools] | 
            
  | 16:43 | <dcaro> | rebooting tools-sgeexec-10-18 (T316544) | [tools] | 
            
  | 16:42 | <dcaro> | rebooting tools-sgeexec-10-16 (T316544) | [tools] | 
            
  | 16:42 | <dcaro> | rebooting tools-sgeexec-10-14 (T316544) | [tools] | 
            
  | 16:41 | <dcaro> | rebooting tools-sgeweblight-10-32 (T316544) | [tools] | 
            
  | 16:40 | <dcaro> | rebooting tools-sgeweblight-10-22 (T316544) | [tools] | 
            
  | 16:39 | <dcaro> | rebooting tools-sgeweblight-10-17 (T316544) | [tools] | 
            
  | 16:32 | <dcaro> | rebooting tools-sgeexec-10-13.tools.eqiad1.wikimedia.cloud (T316544) | [tools] | 
            
  | 16:23 | <dcaro> | rebooting tools-sgeweblight-10-26 (T316544) | [tools] | 
            
  | 16:15 | <bd808> | Hard reboot of tools-sgebastion-11 via Horizon (done circa 16:11Z) | [tools] | 
            
  | 16:14 | <arturo> | rebooted a bunch of nodes to cleanup D procs and high load avg because NFS outage (result of T316544) | [tools] | 
            
  | 12:20 | <wm-bot2> | build & push docker image docker-registry.tools.wmflabs.org/builds-api:09f3b49-dev from https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-builds-api.git (32a8ae9) - cookbook ran by dcaro@vulcanus | [tools] | 
            
  | 09:11 | <wm-bot2> | build & push docker image docker-registry.tools.wmflabs.org/volume-admission:c64da5a from https://gerrit.wikimedia.org/r/cloud/toolforge/volume-admission-controller (c64da5a) - cookbook ran by dcaro@vulcanus | [tools] |