| 
      
        2015-04-27
      
      §
     | 
  
    
  | 21:45 | 
  <bd808> | 
  Manually triggered beta-mediawiki-config-update-eqiad for zuul build df1e789c726ad4aae60d7676e8a4fc8a2f6841fb | 
  [releng] | 
            
  | 21:20 | 
  <bd808> | 
  beta-scap-equad job green again after adding a /srv/ disk to deployment-jobrunner01 | 
  [releng] | 
            
  | 21:08 | 
  <bd808> | 
  Applied role::labs::lvm::srv on deployment-jobrunner01 and forced puppet run | 
  [releng] | 
            
  | 21:08 | 
  <bd808> | 
  Deleted deployment-jobrunner01:/srv/* in preparation for applying role::labs::lvm::srv | 
  [releng] | 
            
  | 21:06 | 
  <bd808> | 
  deployment-jobrunner01 missing role::labs::lvm::srv | 
  [releng] | 
            
  | 21:00 | 
  <bd808> | 
  Root partition full on deployment-jobrunner01 | 
  [releng] | 
            
  | 20:53 | 
  <bd808> | 
  removed mwdeploy user from deployment-bastion:/etc/passwd | 
  [releng] | 
            
  | 20:15 | 
  <Krinkle> | 
  Relaunched Gearman connection | 
  [releng] | 
            
  | 19:53 | 
  <Krinkle> | 
  Jenkins unable to re-create Gearman connection. (HTTP 503 error from /configure). Have to force restart Jenkins | 
  [releng] | 
            
  | 17:32 | 
  <Krinkle> | 
  Relauch slave agent on deployment-bastion | 
  [releng] | 
            
  | 17:31 | 
  <Krinkle> | 
  Jenkins slave deployment-bastion deadlock waiting for executors | 
  [releng] | 
            
  | 08:01 | 
  <_joe_> | 
  installed hhvm 3.6 on deployment-mediawiki02 | 
  [releng] | 
            
  
    | 
      
        2015-04-24
      
      §
     | 
  
    
  | 18:10 | 
  <Krinkle> | 
  cvn Promited Rxy from member to projectadmin | 
  [releng] | 
            
  | 18:01 | 
  <thcipriani> | 
  ran sudo chown -R mwdeploy:mwdeploy /srv/mediawiki on deployment-bastion to fix beta-scap-eqiad, hopefully | 
  [releng] | 
            
  | 17:26 | 
  <thcipriani> | 
  remove deployment-prep from domain in /etc/puppet/puppet.conf on deployment-stream, puppet now OK | 
  [releng] | 
            
  | 17:20 | 
  <thcipriani> | 
  rm stale lock on deployment-rsync01, puppet fine | 
  [releng] | 
            
  | 17:10 | 
  <thcipriani> | 
  gzip /var/log/account/pacct.0 on deployment-bastion: ought to revisit logrotate on that instance. | 
  [releng] | 
            
  | 17:00 | 
  <thcipriani> | 
  rm stale /var/lib/puppet/state/agent_catalog_run.lock on deployment-kafka02 | 
  [releng] | 
            
  | 14:25 | 
  <_joe_> | 
  installing hhvm 3.6.1 on mediawiki-deployment01 | 
  [releng] | 
            
  
    | 
      
        2015-04-22
      
      §
     | 
  
    
  | 23:57 | 
  <bd808> | 
  cherry-picked and applied https://gerrit.wikimedia.org/r/#/c/205968 (remove redis from logstash) | 
  [releng] | 
            
  | 23:33 | 
  <bd808> | 
  reset deployment-salt:/var/lib/git/operations/puppet HEAD to production; forced update with upstream; re-cherry-picked I46e422825af2cf6f972b64e6d50040220ab08995 | 
  [releng] | 
            
  | 23:29 | 
  <bd808> | 
  deployment-salt:/var/lib/git/operations/puppet in detached HEAD state; looks to be for cherry pick of I46e422825af2cf6f972b64e6d50040220ab08995 ? | 
  [releng] | 
            
  | 21:40 | 
  <thcipriani> | 
  restarted mariadb on deployment-db{1,2} | 
  [releng] | 
            
  | 20:20 | 
  <thcipriani> | 
  gzipped /var/log/pacct.0 on deployment-bastion | 
  [releng] | 
            
  | 19:50 | 
  <hashar> | 
  zuul/jenkins are back up (blame Jenkins) | 
  [releng] | 
            
  | 19:40 | 
  <hashar> | 
  reenabling Jenkins gearman client | 
  [releng] | 
            
  | 19:30 | 
  <hashar> | 
  Gearman went back. Reenabling Jenkins as a Gearman client | 
  [releng] | 
            
  | 19:27 | 
  <hashar> | 
  Zuul gearman is stalled.  Disabling Jenkins gearman client to free up connections | 
  [releng] | 
            
  | 17:58 | 
  <Krinkle> | 
  Creating integration-slave-trusty-1021 per T96629 (using ci1.medium type) | 
  [releng] | 
            
  | 14:34 | 
  <hashar> | 
  beta: failures on instances are due to them being moved on different openstack compute nodes (virt***) | 
  [releng] | 
            
  | 13:51 | 
  <jzerebecki> | 
  integration-slave-trusty-1015:~$ sudo -u jenkins-deploy rm -rf /mnt/jenkins-workspace/workspace/mwext-Wikibase-qunit/src/node_modules | 
  [releng] | 
            
  | 12:48 | 
  <andrewbogott> | 
  migrating to new labvirt nodes | 
  [releng] | 
            
  | 12:48 | 
  <hashar> | 
  beta: Andrew B. starting to migrate beta cluster instances on new virt servers | 
  [releng] | 
            
  | 11:34 | 
  <hashar> | 
  integration: apt-get upgrade on integration-slave-trusty* instances | 
  [releng] | 
            
  | 11:31 | 
  <hashar> | 
  integration: Zuul package has been uploaded for Trusty!   Deleting the .deb from /home/hashar/ | 
  [releng] |