| 
      
        2015-05-04
      
      §
     | 
  
    
  | 23:50 | 
  <hashar> | 
  restarted Jenkins (deadlock with deployment-bastion) | 
  [releng] | 
            
  | 23:49 | 
  <hashar> | 
  restarted Jenkins | 
  [releng] | 
            
  | 22:50 | 
  <hashar> | 
  Manually retriggering last change of operations/mediawiki-config.git with: <tt>zuul enqueue --trigger gerrit --pipeline postmerge --project operations/mediawiki-config --change 208822,1</tt> | 
  [releng] | 
            
  | 22:49 | 
  <hashar> | 
  restarted Zuul to clear out a bunch of operations/mediawiki-config.git jobs | 
  [releng] | 
            
  | 22:20 | 
  <hashar> | 
  restarting Jenkins from gallium :/ | 
  [releng] | 
            
  | 22:18 | 
  <thcipriani> | 
  jenkins restarted | 
  [releng] | 
            
  | 22:12 | 
  <thcipriani> | 
  preparing jenkins for shutdown | 
  [releng] | 
            
  | 21:59 | 
  <hashar> | 
  disconnected reconnected  Jenkins Gearman client | 
  [releng] | 
            
  | 21:41 | 
  <thcipriani> | 
  deployment-bastion still not accepting jobs from jenkins | 
  [releng] | 
            
  | 21:35 | 
  <thcipriani> | 
  disconnecting deployment-bastion and reconnecting, again | 
  [releng] | 
            
  | 20:54 | 
  <thcipriani> | 
  marking node deployment-bastion offline due to suck jenkins execution lock | 
  [releng] | 
            
  | 19:03 | 
  <legoktm> | 
  deploying https://gerrit.wikimedia.org/r/208339 | 
  [releng] | 
            
  | 17:46 | 
  <bd808> | 
  integration-slave-precise-1014 died trying to clone mediawiki/core.git with "fatal: destination path 'src' already exists and is not an empty directory." | 
  [releng] | 
            
  
    | 
      
        2015-04-29
      
      §
     | 
  
    
  | 21:03 | 
  <andrewbogott> | 
  suspending and shrinking disks of many instances | 
  [releng] | 
            
  | 17:15 | 
  <thcipriani> | 
  removed l10nupdate user from /etc/passwd on deployment-bastion | 
  [releng] | 
            
  | 15:00 | 
  <hashar> | 
  Instances are being moved out from labvirt1005 which has some faulty memory. List of instances at https://phabricator.wikimedia.org/T97521#1245217 | 
  [releng] | 
            
  | 14:25 | 
  <hashar> | 
  upgrading zuul on integration-slave-precise-1011 for  https://phabricator.wikimedia.org/T97106 | 
  [releng] | 
            
  | 14:11 | 
  <hashar> | 
  rebooting integration-saltmaster stalled. | 
  [releng] | 
            
  | 13:11 | 
  <hashar> | 
  Rebooting deployment-parsoid05 via wikitech interface. | 
  [releng] | 
            
  | 13:02 | 
  <hashar> | 
  labvirt1005 seems to have hardware issue. Impacts a bunch of beta cluster / integration instances as listed on https://phabricator.wikimedia.org/T97521#1245217 | 
  [releng] | 
            
  | 12:22 | 
  <hashar> | 
  deployment-parsoid05 slow down is https://phabricator.wikimedia.org/T97421  . Running apt-get upgrade and rebooting it but its slowness issue might be with the underlying hardware | 
  [releng] | 
            
  | 12:13 | 
  <hashar> | 
  killing puppet on deployment-parsoid05   eats all CPU for some reason | 
  [releng] | 
            
  | 02:40 | 
  <legoktm> | 
  deploying https://gerrit.wikimedia.org/r/207363 and https://gerrit.wikimedia.org/r/207368 | 
  [releng] |