| 2015-07-29
      
      § | 
    
  | 23:55 | <marxarelli> | clearing disk space on integrations-slave-trusty-1012 with `find /mnt/jenkins-workspace/workspace -mindepth 1 -maxdepth 1 -type d -mtime +15 -exec rm -rf {} \;` | [releng] | 
            
  | 18:15 | <bd808> | upgraded nutcracker on deployment-jobrunner01 | [releng] | 
            
  | 18:14 | <bd808> | upgraded nutcracker on deployment-videoscaler01 | [releng] | 
            
  | 18:08 | <bd808> | rm deployment-fluorine:/a/mw-log/archive/*-201506* | [releng] | 
            
  | 18:08 | <bd808> | rm deployment-fluorine:/a/mw-log/archive/*-201505* | [releng] | 
            
  | 18:02 | <bd808> | rm deployment-videoscaler01:/var/log/atop.log.?* | [releng] | 
            
  | 16:49 | <thcipriani> | lots of "Error connecting to 10.68.16.193: Can't connect to MySQL server on '10.68.16.193'" deployment-db1 seems up and functional :( | [releng] | 
            
  | 16:27 | <thcipriani> | deployment-prep login timeouts, tried restarting apache, hhvm, and nutcracker on mediawiki{01..03} | [releng] | 
            
  | 14:38 | <bblack> | cherry-picked https://gerrit.wikimedia.org/r/#/c/215624 (updated to PS8) into deployment-puppetmaster ops/puppet | [releng] | 
            
  | 14:28 | <bblack> | cherry-picked https://gerrit.wikimedia.org/r/#/c/215624 into deployment-puppetmaster ops/puppet | [releng] | 
            
  | 12:38 | <hashar_> | salt minions are back somehow | [releng] | 
            
  | 12:36 | <hashar_> | salt on deployment-salt is missing most of the instances :-((( | [releng] | 
            
  | 03:00 | <ostriches> | deployment-bastion: please please someone rebuild me to not have a stupid 2G /var partition | [releng] | 
            
  | 02:59 | <ostriches> | deployment-bastion: purged a bunch of atop and pacct logs, and apt cache...clogging up /var again. | [releng] | 
            
  | 02:34 | <legoktm> | deploying https://gerrit.wikimedia.org/r/227640 | [releng] | 
            
  
    | 2015-07-28
      
      § | 
    
  | 23:43 | <marxarelli> | running `jenkins-jobs update config/ 'mwext-mw-selenium'` to deploy I7afa07e9f559bffeeebaf7454cc6b39a37e04063 | [releng] | 
            
  | 21:05 | <bd808> | upgraded nutcracker on mediawiki03 | [releng] | 
            
  | 21:04 | <bd808> | upgraded nutcracker on mediawiki02 | [releng] | 
            
  | 21:01 | <bd808> | upgraded nutcracker on mediawiki01 | [releng] | 
            
  | 19:49 | <jzerebecki> | reloading zuul b1b2cab..b02830e | [releng] | 
            
  | 11:18 | <hashar> | Assigning label "BetaClusterBastion" to https://integration.wikimedia.org/ci/computer/deployment-bastion.eqiad/ | [releng] | 
            
  | 11:12 | <hashar> | Jenkins jobs for the beta cluster ended up stuck again.  Found a workaround by removing the Jenkins label  on deployment-bastion node and reinstating it.  Seems to get rid of the deadlock ( ref: https://phabricator.wikimedia.org/T72597#1487801 ) | [releng] | 
            
  | 09:50 | <hashar> | deployment-apertium01 is back!  The ferm rules were outdated / not maintained by puppet, dropped ferm entirely. | [releng] | 
            
  | 09:40 | <hashar> | rebooting deployment-apertium01 to ensure its ferm rules are properly loaded on boot ( https://phabricator.wikimedia.org/T106658 ) | [releng] | 
            
  | 00:46 | <legoktm> | deploying https://gerrit.wikimedia.org/r/227383 | [releng] |