| 2021-05-05
      
      § | 
    
  | 10:39 | <wm-bot> | Safe reboot of 'cloudvirt1034.eqiad.wmnet' finished successfully. (T280641) - cookbook ran by dcaro@vulcanus | [admin] | 
            
  | 10:13 | <wm-bot> | Safe rebooting 'cloudvirt1034.eqiad.wmnet'. (T280641) - cookbook ran by dcaro@vulcanus | [admin] | 
            
  | 10:13 | <wm-bot> | Safe reboot of 'cloudvirt1033.eqiad.wmnet' finished successfully. (T280641) - cookbook ran by dcaro@vulcanus | [admin] | 
            
  | 09:47 | <wm-bot> | Safe rebooting 'cloudvirt1033.eqiad.wmnet'. (T280641) - cookbook ran by dcaro@vulcanus | [admin] | 
            
  | 09:47 | <wm-bot> | Safe reboot of 'cloudvirt1032.eqiad.wmnet' finished successfully. (T280641) - cookbook ran by dcaro@vulcanus | [admin] | 
            
  | 09:21 | <wm-bot> | Safe rebooting 'cloudvirt1032.eqiad.wmnet'. (T280641) - cookbook ran by dcaro@vulcanus | [admin] | 
            
  | 09:21 | <wm-bot> | Safe reboot of 'cloudvirt1031.eqiad.wmnet' finished successfully. (T280641) - cookbook ran by dcaro@vulcanus | [admin] | 
            
  | 08:45 | <wm-bot> | Safe rebooting 'cloudvirt1031.eqiad.wmnet'. (T280641) - cookbook ran by dcaro@vulcanus | [admin] | 
            
  | 08:45 | <wm-bot> | Safe reboot of 'cloudvirt1030.eqiad.wmnet' finished successfully. (T280641) - cookbook ran by dcaro@vulcanus | [admin] | 
            
  | 08:19 | <wm-bot> | Safe rebooting 'cloudvirt1030.eqiad.wmnet'. (T280641) - cookbook ran by dcaro@vulcanus | [admin] | 
            
  | 08:19 | <wm-bot> | Safe reboot of 'cloudvirt1029.eqiad.wmnet' finished successfully. (T280641) - cookbook ran by dcaro@vulcanus | [admin] | 
            
  | 08:02 | <wm-bot> | Safe rebooting 'cloudvirt1029.eqiad.wmnet'. (T280641) - cookbook ran by dcaro@vulcanus | [admin] | 
            
  
    | 2021-05-04
      
      § | 
    
  | 16:05 | <wm-bot> | Safe reboot of 'cloudvirt1028.eqiad.wmnet' finished successfully. (T280641) - cookbook ran by dcaro@vulcanus | [admin] | 
            
  | 15:45 | <wm-bot> | Safe rebooting 'cloudvirt1028.eqiad.wmnet'. (T280641) - cookbook ran by dcaro@vulcanus | [admin] | 
            
  | 15:44 | <wm-bot> | Safe reboot of 'cloudvirt1027.eqiad.wmnet' finished successfully. (T280641) - cookbook ran by dcaro@vulcanus | [admin] | 
            
  | 15:22 | <wm-bot> | Safe rebooting 'cloudvirt1027.eqiad.wmnet'. (T280641) - cookbook ran by dcaro@vulcanus | [admin] | 
            
  | 15:19 | <wm-bot> | Safe reboot of 'cloudvirt1026.eqiad.wmnet' finished successfully. (T280641) - cookbook ran by dcaro@vulcanus | [admin] | 
            
  | 15:15 | <wm-bot> | Safe rebooting 'cloudvirt1026.eqiad.wmnet'. (T280641) - cookbook ran by dcaro@vulcanus | [admin] | 
            
  | 13:19 | <dcaro> | rebooting cloudmetrics1002, got stuck again (T275605) | [admin] | 
            
  | 10:04 | <wm-bot> | Safe rebooting 'cloudvirt1026.eqiad.wmnet'. (T280641) - cookbook ran by dcaro@vulcanus | [admin] | 
            
  | 09:10 | <wm-bot> | Safe rebooting 'cloudvirt1026.eqiad.wmnet'. (T280641) - cookbook ran by dcaro@vulcanus | [admin] | 
            
  | 09:10 | <wm-bot> | Safe reboot of 'cloudvirt1025.eqiad.wmnet' finished successfully. (T280641) - cookbook ran by dcaro@vulcanus | [admin] | 
            
  | 08:34 | <wm-bot> | Safe rebooting 'cloudvirt1025.eqiad.wmnet'. (T280641) - cookbook ran by dcaro@vulcanus | [admin] | 
            
  | 08:20 | <wm-bot> | Safe reboot of 'cloudvirt1024.eqiad.wmnet' finished successfully. (T280641) - cookbook ran by dcaro@vulcanus | [admin] | 
            
  | 08:03 | <wm-bot> | Safe rebooting 'cloudvirt1024.eqiad.wmnet'. (T280641) - cookbook ran by dcaro@vulcanus | [admin] | 
            
  
    | 2021-05-03
      
      § | 
    
  | 23:53 | <bstorm> | running `maintain-dbusers harvest-replicas` on labstore1004 T281287 | [admin] | 
            
  | 23:51 | <bstorm> | running `maintain-dbusers harvest-replicas` on labstore1004 | [admin] | 
            
  | 16:34 | <wm-bot> | Safe reboot of 'cloudvirt1023.eqiad.wmnet' finished successfully. (T280641) - cookbook ran by dcaro@vulcanus | [admin] | 
            
  | 16:29 | <wm-bot> | Safe rebooting 'cloudvirt1023.eqiad.wmnet'. (T280641) - cookbook ran by dcaro@vulcanus | [admin] | 
            
  | 15:41 | <wm-bot> | Safe rebooting 'cloudvirt1023.eqiad.wmnet'. (T280641) - cookbook ran by dcaro@vulcanus | [admin] | 
            
  | 15:41 | <wm-bot> | Safe reboot of 'cloudvirt1022.eqiad.wmnet' finished successfully. (T280641) - cookbook ran by dcaro@vulcanus | [admin] | 
            
  | 15:13 | <wm-bot> | Safe rebooting 'cloudvirt1022.eqiad.wmnet'. (T280641) - cookbook ran by dcaro@vulcanus | [admin] | 
            
  | 10:31 | <wm-bot> | Safe rebooting 'cloudvirt1021.eqiad.wmnet'. (T280641 - cookbook ran by dcaro@vulcanus) | [admin] | 
            
  | 10:23 | <wm-bot> | (from a cookbook) | [admin] | 
            
  | 09:12 | <dcaro> | draining and rebooting coludvirt1021 (T280641) | [admin] | 
            
  | 08:26 | <dcaro> | draining and rebooting coludvirt1018 (T280641) | [admin] | 
            
  
    | 2021-04-28
      
      § | 
    
  | 21:11 | <andrewbogott> | cleaning up more references to deleted hypervisors with delete from services where topic='compute' and version != 53; | [admin] | 
            
  | 20:48 | <andrewbogott> | cleaning up references to deleted hypervisors with mysql:root@localhost [nova_eqiad1]> delete from compute_nodes where hypervisor_version != '5002000'; | [admin] | 
            
  | 19:40 | <andrewbogott> | putting cloudvirt1040 into the maintenance aggregate pending more info about T281399 | [admin] | 
            
  | 18:11 | <andrewbogott> | adding cloudvirt1040, 1041 and 1042 to the 'ceph' host aggregate -- T275081 | [admin] | 
            
  | 11:06 | <dcaro> | All ceph server side upgraded to Octopus! \o/ (T280641) | [admin] | 
            
  | 10:57 | <dcaro> | Got a PG getting stuck on 'remapping' after the OSD came up, had to unset the norebalance and then set it again to get it unstuck (T280641) | [admin] | 
            
  | 10:34 | <dcaro> | Slow/blocked opns from cloudcephmon03, "osd_failure(failed timeout osd.32..." (cloudcephosd1005), unset the cluster noout/norebalance and went away in a few secs, setting it again and continuing... (T280641) | [admin] |