| 
      
        2016-08-31
      
      §
     | 
  
    
  | 23:40 | 
  <bd808> | 
  forced puppet run on deployment-salt02. Had not run automatically for 8 hours | 
  [releng] | 
            
  | 23:36 | 
  <bd808> | 
  Deleted /data/scratch on integration-slave-trusty-1016 to fix puppet | 
  [releng] | 
            
  | 23:32 | 
  <bd808> | 
  Deleted /data/scratch on integration-slave-trusty-1013 to fix puppet | 
  [releng] | 
            
  | 23:22 | 
  <bd808> | 
  Deleted /data/scratch on integration-slave-trusty-1012 to fix puppet | 
  [releng] | 
            
  | 23:19 | 
  <bd808> | 
  Deleted /data/scratch on integration-slave-trusty-1011 to fix puppet | 
  [releng] | 
            
  | 23:15 | 
  <bd808> | 
  Deleted /data/scratch on integration-slave-precise-1012 to fix puppet | 
  [releng] | 
            
  | 23:11 | 
  <bd808> | 
  Deleted /data on integration-slave-precise-1011 to fix puppet | 
  [releng] | 
            
  | 23:08 | 
  <bd808> | 
  Deleted /data on integration-slave-jessie-1001 to fix puppet | 
  [releng] | 
            
  | 23:04 | 
  <bd808> | 
  Deleted empty /data, /data/project, and /data/scratch on integration-puppetmaster to fix puppet | 
  [releng] | 
            
  | 22:59 | 
  <bd808> | 
  Deleted empty /data, /data/project, and /data/scratch on integration-publisher to fix puppet | 
  [releng] | 
            
  | 01:44 | 
  <Krinkle> | 
  Reloading Zuul to deploy  https://gerrit.wikimedia.org/r/307670  | 
  [releng] | 
            
  
    | 
      
        2016-08-30
      
      §
     | 
  
    
  | 23:31 | 
  <yuvipanda> | 
  cherry-picking https://gerrit.wikimedia.org/r/#/c/307656/ fixed puppet on the elasticsearch machines! | 
  [releng] | 
            
  | 23:20 | 
  <Krenair> | 
  removed 'project_id' key from deployment-restbase02's metadata to fix compatibility with the new labsprojectfrommetadata code | 
  [releng] | 
            
  | 22:29 | 
  <yuvipanda> | 
  in lieu of blood sacrifice, restart puppetmaster on deployment-pupetmaster | 
  [releng] | 
            
  | 21:43 | 
  <yuvipanda> | 
  use clush to fix puppet.conf of all clients, realize also accidentally set a client's puppet.conf for the server, recover server's old conf file from a cat in shell history, restore, breathe sigh of relief | 
  [releng] | 
            
  | 21:37 | 
  <yuvipanda> | 
  sudo takes like 15s each time, is there no god? | 
  [releng] | 
            
  | 21:36 | 
  <yuvipanda> | 
  managed to get vim into a state where I can not quit it, probably recording a macro. I hate computers | 
  [releng] | 
            
  | 21:16 | 
  <yuvipanda> | 
  deployment-pdf01 fixed manually | 
  [releng] | 
            
  | 21:15 | 
  <yuvipanda> | 
  deployment-pdf02 has proper ssl certs mysteriously without me doing anything | 
  [releng] | 
            
  | 21:06 | 
  <yuvipanda> | 
  moved deployment-db[12], deployment-stream to not use role::puppet::self, attempting to semi-automate rest | 
  [releng] | 
            
  | 20:52 | 
  <yuvipanda> | 
  cherry-picked appropriate patch on deployment-puppetmaster for T120159, did https://wikitech.wikimedia.org/w/index.php?title=Hiera:Deployment-prep/host/deployment-puppetmaster&oldid=818847 to make sure the puppetmaster allows connections from elsewhere | 
  [releng] | 
            
  | 19:48 | 
  <legoktm> | 
  deploying https://gerrit.wikimedia.org/r/306710 | 
  [releng] | 
            
  | 19:13 | 
  <bd808> | 
  Fixed puppet runs on deployment-sca0[12] with cherry-pick of https://gerrit.wikimedia.org/r/#/c/307561 | 
  [releng] | 
            
  | 18:57 | 
  <bd808> | 
  Duplicate declaration: File[/srv/deployment] is already declared in file /etc/puppet/modules/contint/manifests/deployment_dir.pp:14; cannot redeclare at /etc/puppet/modules/service/manifests/deploy/common.pp:12 on node deployment-sca01.deployment-prep.eqiad.wmflabs | 
  [releng] |