| 
      
        2015-06-25
      
      §
     | 
  
    
  | 14:20 | 
  <jzerebecki> | 
  killing a fellows idle shell zuul@gallium:~$ kill 13602 | 
  [releng] | 
            
  | 11:03 | 
  <hashar> | 
  Rebooting  integration-raita and integration-vmbuilder-trusty | 
  [releng] | 
            
  | 11:01 | 
  <hashar> | 
  Unmounting /data/project and /home NFS mounts from integration-raita and integration-vmbuilder-trusty https://phabricator.wikimedia.org/T90610 | 
  [releng] | 
            
  | 10:45 | 
  <hashar> | 
  deployment-sca02 deleted /var/lib/puppet/state/agent_catalog_run.lock from June 5th | 
  [releng] | 
            
  | 08:57 | 
  <hashar> | 
  Fixed puppet "Can't dup Symbol" on deployment-pdf01  by deleting puppet, /var/lib/puppet and reinstalling it from scratch https://phabricator.wikimedia.org/T87197 | 
  [releng] | 
            
  | 08:39 | 
  <hashar> | 
  apt-get upgrade deployment-salt | 
  [releng] | 
            
  | 08:08 | 
  <hashar> | 
  deployment-pdf01 deleted /var/log/ocg/ content. Last entry is from July 25th 2014 and puppet complains with <tt>e[/var/log/ocg]: Not removing directory; use 'force' to override</tt> | 
  [releng] | 
            
  | 08:05 | 
  <hashar> | 
  apt-get upgrade deployment-pdf01 | 
  [releng] | 
            
  | 06:37 | 
  <Krinkle> | 
  Reloading Zuul to deploy https://gerrit.wikimedia.org/r/220712 | 
  [releng] | 
            
  | 06:33 | 
  <Krinkle> | 
  Reloading Zuul to deploy https://gerrit.wikimedia.org/r/220705 | 
  [releng] | 
            
  
    | 
      
        2015-06-24
      
      §
     | 
  
    
  | 19:31 | 
  <hashar> | 
  rebooting deployment-cache-upload02 | 
  [releng] | 
            
  | 19:28 | 
  <hashar> | 
  fixing DNS puppet etc on deployment-cache-upload02 | 
  [releng] | 
            
  | 19:24 | 
  <hashar> | 
  rebooting deployment-zookeeper to get rid of the /home NFS https://phabricator.wikimedia.org/T102169   | 
  [releng] | 
            
  | 19:06 | 
  <hashar> | 
  beta: salt 'i-00*' cmd.run "echo 'domain integration.eqiad.wmflabs\\nsearch integration.eqiad.wmflabs eqiad.wmflabs\\nnameserver 208.80.154.20\\noptions timeout:5' > /etc/resolv.conf" | 
  [releng] | 
            
  | 19:06 | 
  <hashar> | 
  fixing DNS / puppet and salt on i-000008d5.eqiad.wmflabs  i-000002de.eqiad.wmflabs i-00000958.eqiad.wmflabs | 
  [releng] | 
            
  | 15:35 | 
  <hashar> | 
  integration-dev recovered!  puppet hasn't run for ages but caught up with changes | 
  [releng] | 
            
  | 15:13 | 
  <hashar> | 
  removed /var/lib/puppet/state/agent_catalog_run.lock on integration-dev | 
  [releng] | 
            
  | 09:52 | 
  <hashar> | 
  Java 6 removed from gallium / lanthanum and CI labs slaves. | 
  [releng] | 
            
  | 09:18 | 
  <hashar> | 
  getting rid of java 6 on CI machines ( https://phabricator.wikimedia.org/T103491 ) | 
  [releng] | 
            
  | 07:58 | 
  <hashar> | 
  Bah puppet reenable NFS on deployment-parsoidcache02 for some reason | 
  [releng] | 
            
  | 07:57 | 
  <hashar> | 
  disabling NFS on deployment-parsoidcache02 | 
  [releng] | 
            
  | 00:38 | 
  <marxarelli> | 
  reloading zuul to deploy https://gerrit.wikimedia.org/r/#/c/219513/ | 
  [releng] | 
            
  | 00:32 | 
  <marxarelli> | 
  running `jenkins-jobs update` to create 'mwext-MobileFrontend-mw-selenium' with I7affe14e878d5c1fc4bcb4dfc7f2d1494cd795b7 | 
  [releng] | 
            
  | 00:20 | 
  <marxarelli> | 
  running `jenkins-jobs update` to create 'mediawiki-selenium-integration' with I7affe14e878d5c1fc4bcb4dfc7f2d1494cd795b7 | 
  [releng] | 
            
  
    | 
      
        2015-06-23
      
      §
     | 
  
    
  | 23:29 | 
  <Krinkle> | 
  Reloading Zuul to deploy https://gerrit.wikimedia.org/r/220350 | 
  [releng] | 
            
  | 21:34 | 
  <bd808> | 
  updated scap to 33f3002 (Ensure that the minimum batch size used by cluster_ssh is 1) | 
  [releng] | 
            
  | 19:53 | 
  <legoktm> | 
  deleted broken renames from centralauth.renameuser_status on beta cluster | 
  [releng] | 
            
  | 18:28 | 
  <jzerebecki> | 
  zuul reload for https://gerrit.wikimedia.org/r/#/c/219778/4 | 
  [releng] | 
            
  | 16:33 | 
  <bd808> | 
  updated scap to da64a65 (Cast pid read from file to an int) | 
  [releng] | 
            
  | 16:20 | 
  <bd808> | 
  updated scap to 947b93f (Fix reference to _get_apache_list) | 
  [releng] | 
            
  | 13:31 | 
  <cscott> | 
  fixed salt on deployment-pdf02, restarted OCG there. | 
  [releng] | 
            
  | 12:24 | 
  <hashar> | 
  rebooting integration-labvagrant (stuck) | 
  [releng] | 
            
  | 05:44 | 
  <cscott> | 
  stopped OCG service on deployment-pdf02, see https://phabricator.wikimedia.org/T103473 | 
  [releng] | 
            
  | 05:20 | 
  <cscott> | 
  updated OCG to version d7c698d5bf730d34057945e912ac75dc542dd788 ; restarted service. | 
  [releng] | 
            
  | 03:58 | 
  <cscott> | 
  stopped OCG on beta; redis 2.8.x is causing the service to crash on startup. | 
  [releng] | 
            
  | 00:07 | 
  <legoktm> | 
  deploying https://gerrit.wikimedia.org/r/220020 | 
  [releng] | 
            
  
    | 
      
        2015-06-22
      
      §
     | 
  
    
  | 22:23 | 
  <legoktm> | 
  deploying https://gerrit.wikimedia.org/r/219603 | 
  [releng] | 
            
  | 21:58 | 
  <andrewbogott> | 
  re-enabling puppet on deployment-videoscaler01 because no reason was given for disabling | 
  [releng] | 
            
  | 21:47 | 
  <bd808> | 
  scap emitting soft failures due to missing python-netifaces on deployment-videoscaler01; should be fixed by a current puppet run | 
  [releng] | 
            
  | 21:37 | 
  <bd808> | 
  Updated scap to 81b7c14 (Move dsh group file names to config) | 
  [releng] | 
            
  | 20:42 | 
  <cscott> | 
  updated OCG to version b482144f5bd8b427bcc64a3dd287247195aa1951 | 
  [releng] | 
            
  | 14:58 | 
  <hashar> | 
  disabled  sshd MAC/KEX hardening on beta (was https://gerrit.wikimedia.org/r/#/c/219828/ ) | 
  [releng] | 
            
  | 14:32 | 
  <hashar> | 
  restarting Jenkins | 
  [releng] | 
            
  | 14:30 | 
  <hashar> | 
  Reenable sshd MAC/KEX hardening on beta by cherry picking https://gerrit.wikimedia.org/r/#/c/219828/ | 
  [releng] | 
            
  | 13:17 | 
  <moritzm> | 
  activated firejail service containment for graphoid, citoid and mathoid in deployment-sca | 
  [releng] | 
            
  | 11:07 | 
  <hashar> | 
  fixing puppet on integration-zuul-server | 
  [releng] | 
            
  | 10:29 | 
  <hashar> | 
  rebooted deployment-kafka02 to get rid of /home NFS share | 
  [releng] | 
            
  | 10:25 | 
  <hashar> | 
  fixed puppet.conf on deployment-urldownloader | 
  [releng] | 
            
  | 10:20 | 
  <hashar> | 
  enabled puppet agent on deployment-urldownloader | 
  [releng] | 
            
  | 10:05 | 
  <hashar> | 
  removing puppet lock on deployment-elastic07 ( rm /var/lib/puppet/state/agent_catalog_run.lock ) | 
  [releng] |