| 2019-04-17
      
      § | 
    
  | 12:08 | <arturo> | T221225 rebooting bastions to clean sssd. We are back to nscd/nslcd until we figure out what's wrong here | [tools] | 
            
  | 11:58 | <arturo> | T221205 sssd was deployed successfully into all webgrid nodes | [tools] | 
            
  | 11:39 | <arturo> | deploy sssd to tools-sge-services-03/04 (includes reboot) | [tools] | 
            
  | 11:31 | <arturo> | reboot bastions for sssd deployment | [tools] | 
            
  | 11:30 | <arturo> | deploy sssd to bastions | [tools] | 
            
  | 11:24 | <arturo> | disable puppet in bastions to deploy sssd | [tools] | 
            
  | 09:52 | <arturo> | T221205 tools-sgewebgrid-lighttpd-0915 requires some manual intervention because issues in the dpkg database prevents deleting nscd/nslcd packages | [tools] | 
            
  | 09:45 | <arturo> | T221205 tools-sgewebgrid-lighttpd-0913 requires some manual intervention because unconfigured packages prevents a clean puppet agent run | [tools] | 
            
  | 09:12 | <arturo> | T221205 start deploying sssd to sgewebgrid nodes | [tools] | 
            
  | 09:00 | <arturo> | T221205 add `profile::ldap::client::labs::client_stack: sssd` in horizon for the puppet prefixes `tools-sgewebgrid-lighttpd` and `tools-sgewebgrid-generic` | [tools] | 
            
  | 08:56 | <arturo> | T221205 disable puppet in all tools-sgewebgrid-* nodes | [tools] | 
            
  
    | 2019-04-10
      
      § | 
    
  | 23:58 | <andrewbogott> | moving tools-clushmaster-02, tools-elastic-03 and tools-paws-worker-1001 to eqiad1-r | [tools] | 
            
  | 18:52 | <bstorm_> | depooled and rebooted tools-sgeexec-0929 because systemd was in a weird state | [tools] | 
            
  | 18:46 | <bstorm_> | depooled and rebooted tools-sgewebgrid-lighttpd-0913 because high load was caused by ancient lsof processes | [tools] | 
            
  | 14:49 | <bstorm_> | cleared E state from 5 queues | [tools] | 
            
  | 13:06 | <arturo> | T218126 hard reboot tools-sgeexec-0906 | [tools] | 
            
  | 12:31 | <arturo> | T218126 hard reboot tools-sgeexec-0926 | [tools] | 
            
  | 12:27 | <arturo> | T218126 hard reboot tools-sgeexec-0925 | [tools] | 
            
  | 12:06 | <arturo> | T218126 hard reboot tools-sgeexec-0901 | [tools] | 
            
  | 11:55 | <arturo> | T218126 hard reboot tools-sgeexec-0924 | [tools] | 
            
  | 11:47 | <arturo> | T218126 hard reboot tools-sgeexec-0921 | [tools] | 
            
  | 11:23 | <arturo> | T218126 hard reboot tools-sgeexec-0940 | [tools] | 
            
  | 11:03 | <arturo> | T218126 hard reboot tools-sgeexec-0928 | [tools] | 
            
  | 10:49 | <arturo> | T218126 hard reboot tools-sgeexec-0923 | [tools] | 
            
  | 10:43 | <arturo> | T218126 hard reboot tools-sgeexec-0915 | [tools] | 
            
  | 10:27 | <arturo> | T218126 hard reboot tools-sgeexec-0935 | [tools] | 
            
  | 10:19 | <arturo> | T218126 hard reboot tools-sgeexec-0914 | [tools] | 
            
  | 10:01 | <arturo> | T218126 hard reboot tools-sgeexec-0907 | [tools] | 
            
  | 09:40 | <arturo> | T218126 hard reboot tools-sgeexec-0918 | [tools] | 
            
  | 09:27 | <arturo> | T218126 hard reboot tools-sgeexec-0932 | [tools] |