| 
      
        2012-05-22
      
      §
     | 
  
    
  | 13:49 | 
  <hashar> | 
  Deleting jobrunner03 and 04, not going to need them afterall | 
  [releng] | 
            
  | 13:24 | 
  <hashar> | 
  deleting refreshLinks2 jobs from enwiki database | 
  [releng] | 
            
  | 12:52 | 
  <hashar> | 
  deleting deployment-jobrunner{3,4} installation failed I got permission denied. Will recreate them using same hostname | 
  [releng] | 
            
  | 11:55 | 
  <hashar> | 
  create two more job runner instances | 
  [releng] | 
            
  | 10:09 | 
  <hashar> | 
  Remove deployment-webs instance which was meant to emulate the HTTPS access. Hacky and low priority for now, we will need to setup a nginx proxy one day to properly replicate the production infrastructure. | 
  [releng] | 
            
  | 09:39 | 
  <labs-logs-bottie> | 
  hashar: rebooting jobrunner02 just to be sure it is properly loaded up | 
  [releng] | 
            
  | 09:30 | 
  <labs-logs-bottie> | 
  hashar: jobrunner logs are available in /home/wikipedia/logs/runJobs.log now | 
  [releng] | 
            
  | 09:25 | 
  <hashar> | 
  Fixed udp2log not able to add new log files in /home/wikipedia/log , that dir need to be writable by udp2log user!  See https://gerrit.wikimedia.org/r/8442 | https://bugzilla.wikimedia.org/37014 | 
  [releng] | 
            
  | 08:54 | 
  <hashar> | 
  purged all logs from /home/wikipedia/logs/archive/ just to be safe | 
  [releng] | 
            
  | 08:41 | 
  <hashar> | 
  restarted upd2log on -feed (again) | 
  [releng] | 
            
  | 08:23 | 
  <hashar> | 
  started job loop on deployment-job-runner02 | 
  [releng] | 
            
  | 07:49 | 
  <hashar> | 
  installing jobrunner2 | 
  [releng] | 
            
  | 05:11 | 
  <hashar> | 
  creating a second job runner instance deployment-jobrunner02 . Will apply puppet classes later on. | 
  [releng] | 
            
  | 03:47 | 
  <labs-logs-bottie> | 
  hashar: (Bug 36870) deleting deployment-web{,3,4,5} | 
  [releng] | 
            
  
    | 
      
        2012-05-21
      
      §
     | 
  
    
  | 21:11 | 
  <hashar> | 
  deployment-nfs-memc : fix user right for upload6 : chown apache /mnt/export/upload6 | 
  [releng] | 
            
  | 21:11 | 
  <hashar> | 
  On deployment-nfs-memc : added apache (uid 48) entry in /etc/passwd | 
  [releng] | 
            
  | 21:11 | 
  <hashar> | 
  Adding Faidon and Platonides to default CC list of "depoyment prep (beta)" component | 
  [releng] | 
            
  | 21:11 | 
  <hashar> | 
  In Bugzilla, I have removed Petr Bena as a default assignee of bugs opened for "deployment-prep (beta)" component. Default is now "Nobody", Petr is on CC. That will makes bug triage a bit easier. | 
  [releng] | 
            
  | 08:58 | 
  <hashar> | 
  rerebooting deployment-feed | 
  [releng] | 
            
  | 08:53 | 
  <hashar> | 
  Looks like -feed is dead : EXT3-fs: INFO: recovery required on readonly filesystem. | 
  [releng] | 
            
  | 08:49 | 
  <hashar> | 
  rebooting deployment-feed  | 
  [releng] | 
            
  | 08:22 | 
  <hashar> | 
  installing iotop on deployment-nfs-memc | 
  [releng] | 
            
  
    | 
      
        2012-05-18
      
      §
     | 
  
    
  | 17:39 | 
  <labs-logs-bottie> | 
  j: add /apache symlin on deployment-transcoding | 
  [releng] | 
            
  | 16:52 | 
  <hashar> | 
  added 'aawiki' to all.dblist and made a symbolic to it named wmflabs.dblist | 
  [releng] | 
            
  | 16:42 | 
  <hashar> | 
  added fake 'aawiki' entry to wikiversions.data | 
  [releng] | 
            
  | 16:37 | 
  <hashar> | 
  started mediawiki job runner on -jobrunner01 | 
  [releng] | 
            
  | 16:27 | 
  <hashar> | 
  Removed apache:: puppet class, uses the application::  ones instead | 
  [releng] | 
            
  | 15:24 | 
  <hashar> | 
  Well MaxSem fixed mobile Frontend :-D | 
  [releng] | 
            
  | 15:22 | 
  <hashar> | 
  rewinded MobileFrontend to before 9db8dc94b1b83999931fca3d0edf5e22ab1effb3 ( https://gerrit.wikimedia.org/r/#/c/7795/ ) | 
  [releng] | 
            
  | 15:05 | 
  <hashar> | 
  Running: foreachwiki update.php --quiet --quick | 
  [releng] | 
            
  | 15:00 | 
  <labs-logs-bottie> | 
  hashar: rebooting jobrunner01 | 
  [releng] | 
            
  | 14:55 | 
  <hashar> | 
  updating all extensions | 
  [releng] | 
            
  | 14:49 | 
  <hashar> | 
  /home/wikipedia/common/php-trunk now tracks mediawiki/core.git , branch master. So a simple 'git pull' will update it! | 
  [releng] | 
            
  | 14:47 | 
  <hashar> | 
  updating MediaWiki | 
  [releng] | 
            
  | 14:31 | 
  <hashar> | 
  adding puppet class applicationserver::jobrunner | 
  [releng] | 
            
  | 11:51 | 
  <hashar> | 
  puppet running again on -syslog \\o/ | 
  [releng] | 
            
  | 10:55 | 
  <hashar> | 
  ran apt-get clean on -syslog | 
  [releng] | 
            
  | 10:46 | 
  <hashar> | 
  On -feed, ran apt-get clean | 
  [releng] |