| 
      
        2021-03-30
      
      §
     | 
  
    
  | 18:05 | 
  <Majavah> | 
  remove {trysty,precise}-deployment-prep repos from deployment-deploy01 aptly | 
  [releng] | 
            
  | 17:51 | 
  <Majavah> | 
  arm deployment-deploy01 keyholder with all the keys | 
  [releng] | 
            
  | 14:50 | 
  <Majavah> | 
  cherry pick 675807 675814 and 675815 to deployment-puppetmaster to unblock work on deployment-deploy03 until sre has merged those T278689 | 
  [releng] | 
            
  | 14:44 | 
  <Majavah> | 
  remove deployment-puppetmaster04 local patch adding releng/phatality to scap to see if it unbreaks deployment-deploy03 puppet runs | 
  [releng] | 
            
  | 13:55 | 
  <Majavah> | 
  cherry pick https://gerrit.wikimedia.org/r/c/operations/puppet/+/675802/ on beta to unblock my progress until merged | 
  [releng] | 
            
  | 13:35 | 
  <Majavah> | 
  create and install deployment-deploy03 T278689 | 
  [releng] | 
            
  | 13:17 | 
  <Majavah> | 
  armed deployment-cumin keyholder, found passphrase at deployment-puppetmaster04:/var/lib/git/labs/private/files/ssh/tin/cumin_rsa.passphrase | 
  [releng] | 
            
  | 07:26 | 
  <Majavah> | 
  shutoff deployment-mediawiki-09 T278664 | 
  [releng] | 
            
  | 06:25 | 
  <Majavah> | 
  switch w-beta.wmflabs.org web proxy to deployment-mediawiki11 | 
  [releng] | 
            
  | 06:18 | 
  <Majavah> | 
  restart restbase on deployment-restbase03 to pick up config changes to use deployment-mediawiki11 | 
  [releng] | 
            
  
    | 
      
        2021-03-29
      
      §
     | 
  
    
  | 15:37 | 
  <Majavah> | 
  hard reboot deployment-sessionstore03 T263617 | 
  [releng] | 
            
  | 15:16 | 
  <Majavah> | 
  manually run puppet on deployment-sessionstore03, starting Cassandra (which was stopped) T263617 | 
  [releng] | 
            
  | 13:04 | 
  <Majavah> | 
  cherry pick https://gerrit.wikimedia.org/r/c/operations/puppet/+/675503/ on deployment-puppetmaster04 (T278664), also apply same change on horizon. this will switch traffic from deployment-mediawiki-07 to deployment-mediawiki11 | 
  [releng] | 
            
  | 10:29 | 
  <Majavah> | 
  remove deployment-mediawiki10, too much live debugging, not in use | 
  [releng] | 
            
  | 09:56 | 
  <Majavah> | 
  taavi@deployment-mediawiki10:~$ sudo ln -s /usr/local/share/ca-certificates/Puppet_Internal_CA.crt /etc/ssl/certs/aeffde42.0 && sudo update-ca-certificates | 
  [releng] | 
            
  | 09:29 | 
  <Urbanecm> | 
  Manually run puppet on mediawiki10 | 
  [releng] | 
            
  | 09:28 | 
  <Urbanecm> | 
  Re-enable puppet on mediawiki10 | 
  [releng] | 
            
  | 08:49 | 
  <Urbanecm> | 
  DIsable puppet on mediawiki10 - investigating failing curl certificate check | 
  [releng] | 
            
  | 06:46 | 
  <Majavah> | 
  cherry-pick https://gerrit.wikimedia.org/r/c/operations/puppet/+/675357/ on deployment-puppetmaster04 - T278664 | 
  [releng] | 
            
  | 05:40 | 
  <Majavah> | 
  move role::labs::lvm::srv puppet classes from deployment-mediawiki- prefix to current individual appservers, T278664 | 
  [releng] |