| 2012-05-22
      
      § | 
    
  | 13:49 | <hashar> | Deleting jobrunner03 and 04, not going to need them afterall | [releng] | 
            
  | 13:24 | <hashar> | deleting refreshLinks2 jobs from enwiki database | [releng] | 
            
  | 12:52 | <hashar> | deleting deployment-jobrunner{3,4} installation failed I got permission denied. Will recreate them using same hostname | [releng] | 
            
  | 11:55 | <hashar> | create two more job runner instances | [releng] | 
            
  | 10:09 | <hashar> | Remove deployment-webs instance which was meant to emulate the HTTPS access. Hacky and low priority for now, we will need to setup a nginx proxy one day to properly replicate the production infrastructure. | [releng] | 
            
  | 09:39 | <labs-logs-bottie> | hashar: rebooting jobrunner02 just to be sure it is properly loaded up | [releng] | 
            
  | 09:30 | <labs-logs-bottie> | hashar: jobrunner logs are available in /home/wikipedia/logs/runJobs.log now | [releng] | 
            
  | 09:25 | <hashar> | Fixed udp2log not able to add new log files in /home/wikipedia/log , that dir need to be writable by udp2log user!  See https://gerrit.wikimedia.org/r/8442 | https://bugzilla.wikimedia.org/37014 | [releng] | 
            
  | 08:54 | <hashar> | purged all logs from /home/wikipedia/logs/archive/ just to be safe | [releng] | 
            
  | 08:41 | <hashar> | restarted upd2log on -feed (again) | [releng] | 
            
  | 08:23 | <hashar> | started job loop on deployment-job-runner02 | [releng] | 
            
  | 07:49 | <hashar> | installing jobrunner2 | [releng] | 
            
  | 05:11 | <hashar> | creating a second job runner instance deployment-jobrunner02 . Will apply puppet classes later on. | [releng] | 
            
  | 03:47 | <labs-logs-bottie> | hashar: (Bug 36870) deleting deployment-web{,3,4,5} | [releng] | 
            
  
    | 2012-05-21
      
      § | 
    
  | 21:11 | <hashar> | deployment-nfs-memc : fix user right for upload6 : chown apache /mnt/export/upload6 | [releng] | 
            
  | 21:11 | <hashar> | On deployment-nfs-memc : added apache (uid 48) entry in /etc/passwd | [releng] | 
            
  | 21:11 | <hashar> | Adding Faidon and Platonides to default CC list of "depoyment prep (beta)" component | [releng] | 
            
  | 21:11 | <hashar> | In Bugzilla, I have removed Petr Bena as a default assignee of bugs opened for "deployment-prep (beta)" component. Default is now "Nobody", Petr is on CC. That will makes bug triage a bit easier. | [releng] | 
            
  | 08:58 | <hashar> | rerebooting deployment-feed | [releng] | 
            
  | 08:53 | <hashar> | Looks like -feed is dead : EXT3-fs: INFO: recovery required on readonly filesystem. | [releng] | 
            
  | 08:49 | <hashar> | rebooting deployment-feed | [releng] | 
            
  | 08:22 | <hashar> | installing iotop on deployment-nfs-memc | [releng] | 
            
  
    | 2012-05-18
      
      § | 
    
  | 17:39 | <labs-logs-bottie> | j: add /apache symlin on deployment-transcoding | [releng] | 
            
  | 16:52 | <hashar> | added 'aawiki' to all.dblist and made a symbolic to it named wmflabs.dblist | [releng] | 
            
  | 16:42 | <hashar> | added fake 'aawiki' entry to wikiversions.data | [releng] | 
            
  | 16:37 | <hashar> | started mediawiki job runner on -jobrunner01 | [releng] | 
            
  | 16:27 | <hashar> | Removed apache:: puppet class, uses the application::  ones instead | [releng] | 
            
  | 15:24 | <hashar> | Well MaxSem fixed mobile Frontend :-D | [releng] | 
            
  | 15:22 | <hashar> | rewinded MobileFrontend to before 9db8dc94b1b83999931fca3d0edf5e22ab1effb3 ( https://gerrit.wikimedia.org/r/#/c/7795/ ) | [releng] | 
            
  | 15:05 | <hashar> | Running: foreachwiki update.php --quiet --quick | [releng] | 
            
  | 15:00 | <labs-logs-bottie> | hashar: rebooting jobrunner01 | [releng] | 
            
  | 14:55 | <hashar> | updating all extensions | [releng] | 
            
  | 14:49 | <hashar> | /home/wikipedia/common/php-trunk now tracks mediawiki/core.git , branch master. So a simple 'git pull' will update it! | [releng] | 
            
  | 14:47 | <hashar> | updating MediaWiki | [releng] | 
            
  | 14:31 | <hashar> | adding puppet class applicationserver::jobrunner | [releng] | 
            
  | 11:51 | <hashar> | puppet running again on -syslog \\o/ | [releng] | 
            
  | 10:55 | <hashar> | ran apt-get clean on -syslog | [releng] | 
            
  | 10:46 | <hashar> | On -feed, ran apt-get clean | [releng] |