| 2015-05-04
      
      § | 
    
  | 22:50 | <hashar> | Manually retriggering last change of operations/mediawiki-config.git with: <tt>zuul enqueue --trigger gerrit --pipeline postmerge --project operations/mediawiki-config --change 208822,1</tt> | [releng] | 
            
  | 22:49 | <hashar> | restarted Zuul to clear out a bunch of operations/mediawiki-config.git jobs | [releng] | 
            
  | 22:20 | <hashar> | restarting Jenkins from gallium :/ | [releng] | 
            
  | 22:18 | <thcipriani> | jenkins restarted | [releng] | 
            
  | 22:12 | <thcipriani> | preparing jenkins for shutdown | [releng] | 
            
  | 21:59 | <hashar> | disconnected reconnected  Jenkins Gearman client | [releng] | 
            
  | 21:41 | <thcipriani> | deployment-bastion still not accepting jobs from jenkins | [releng] | 
            
  | 21:35 | <thcipriani> | disconnecting deployment-bastion and reconnecting, again | [releng] | 
            
  | 20:54 | <thcipriani> | marking node deployment-bastion offline due to suck jenkins execution lock | [releng] | 
            
  | 19:03 | <legoktm> | deploying https://gerrit.wikimedia.org/r/208339 | [releng] | 
            
  | 17:46 | <bd808> | integration-slave-precise-1014 died trying to clone mediawiki/core.git with "fatal: destination path 'src' already exists and is not an empty directory." | [releng] | 
            
  
    | 2015-04-29
      
      § | 
    
  | 21:03 | <andrewbogott> | suspending and shrinking disks of many instances | [releng] | 
            
  | 17:15 | <thcipriani> | removed l10nupdate user from /etc/passwd on deployment-bastion | [releng] | 
            
  | 15:00 | <hashar> | Instances are being moved out from labvirt1005 which has some faulty memory. List of instances at https://phabricator.wikimedia.org/T97521#1245217 | [releng] | 
            
  | 14:25 | <hashar> | upgrading zuul on integration-slave-precise-1011 for  https://phabricator.wikimedia.org/T97106 | [releng] | 
            
  | 14:11 | <hashar> | rebooting integration-saltmaster stalled. | [releng] | 
            
  | 13:11 | <hashar> | Rebooting deployment-parsoid05 via wikitech interface. | [releng] | 
            
  | 13:02 | <hashar> | labvirt1005 seems to have hardware issue. Impacts a bunch of beta cluster / integration instances as listed on https://phabricator.wikimedia.org/T97521#1245217 | [releng] | 
            
  | 12:22 | <hashar> | deployment-parsoid05 slow down is https://phabricator.wikimedia.org/T97421  . Running apt-get upgrade and rebooting it but its slowness issue might be with the underlying hardware | [releng] | 
            
  | 12:13 | <hashar> | killing puppet on deployment-parsoid05   eats all CPU for some reason | [releng] | 
            
  | 02:40 | <legoktm> | deploying https://gerrit.wikimedia.org/r/207363 and https://gerrit.wikimedia.org/r/207368 | [releng] | 
            
  
    | 2015-04-28
      
      § | 
    
  | 23:33 | <hoo> | Ran foreachwiki extensions/Wikidata/extensions/Wikibase/lib/maintenance/populateSitesTable.php --load-from 'http://meta.wikimedia.beta.wmflabs.org/w/api.php' to fix all sites tables | [releng] | 
            
  | 23:18 | <hoo> | Ran mysql> INSERT INTO sites (SELECT * FROM wikidatawiki.sites); on enwikinews to populate the sites table | [releng] | 
            
  | 23:18 | <hoo> | Ran mysql> INSERT INTO sites (SELECT * FROM wikidatawiki.sites); on testwiki to populate the sites table | [releng] | 
            
  | 20:57 | <YuviPanda> | KILL KILL KILL DEPLOYMENT-LUCID-SALT WITH FIRE AND BRIMSTONE AND BAD THINGS | [releng] | 
            
  | 17:48 | <James_F> | Restarting grrrit-wm for config change. | [releng] | 
            
  | 16:24 | <bd808> | Updated scap to ef15380 (Make scap localization cache build $TMPDIR aware) | [releng] | 
            
  | 15:42 | <bd808> | Freed 5G on deployment-bastion by deleting abandoned /tmp/scap_l10n_* directories | [releng] | 
            
  | 14:01 | <marxarelli> | reloading zuul to deploy https://gerrit.wikimedia.org/r/#/c/206967/ | [releng] | 
            
  | 00:17 | <greg-g> | after the 3rd or so time doing it (while on the Golden Gate Bridge, btw) it worked | [releng] | 
            
  | 00:11 | <greg-g> | still nothing... | [releng] | 
            
  | 00:10 | <greg-g> | after disconnecting, marking temp offline, bringing back online, and launching slave agent: "Slave successfully connected and online" | [releng] | 
            
  | 00:07 | <greg-g> | deployment-bastion is idle, yet we have 3 pending jobs waiting for an executer on it - will disconnect/reconnect it in Jenkins | [releng] |