| 2013-04-30
      
      § | 
    
  | 21:35 | <hashar> | Fixed the git path in mediawiki/extensions.git local copy of -bastion | [releng] | 
            
  | 21:04 | <hashar> | both apaches are now serving content from the NFS cluster. | [releng] | 
            
  | 20:58 | <hashar> | Migrating apache-33 to use the new NFS server | [releng] | 
            
  | 20:58 | <hashar> | apache-32 running with NFS went from 560ms to 260ms when serving pages \\O/ | [releng] | 
            
  | 20:49 | <hashar> | Recreated wikiversions.cdb on bastion for the new NFS home dir | [releng] | 
            
  | 20:08 | <hashar> | migrating apache32 to new NFS server | [releng] | 
            
  | 20:06 | <hashar> | root@deployment-bastion:~# /etc/init.d/udp2log stop && /etc/init.d/udp2log-mw start | [releng] | 
            
  | 20:01 | <hashar> | applying role::labsnfs::client  on -bastion | [releng] | 
            
  | 19:45 | <hashar> | applying the very recent `role::labsnfs::client` class on deployment-integration | [releng] | 
            
  | 19:43 | <hashar> | Upgraded puppet manifests on deployment-integration and running puppet. | [releng] | 
            
  | 19:21 | <hashar> | Migrating homes to the new NFS server | [releng] | 
            
  | 18:27 | <hashar> | rsync to the NFS server are completed. There are most probably still some tiny files than need to be copied though | [releng] | 
            
  | 16:46 | <hashar> | Mounted new NFS server on /srv/project on instances: apache32, apache33, video05 and jobrunner08 | [releng] | 
            
  | 16:01 | <hashar> | Clearing out years old backup from /data/project such as copy of extensions, databases dumps and some old instances backups. | [releng] | 
            
  | 15:28 | <hashar> | Copying l10n cache to the new NFS server: rsync -av /home/wikipedia/common/php-master/cache /srv/project/apache/common/php-master | [releng] | 
            
  | 15:11 | <hashar> | syncing upload data from the Gluster share to labnfs server: rsync -avv /data/project/upload7 /srv/project | [releng] | 
            
  | 13:59 | <hashar> | bastion: created NFS mount point thanks to Coren. echo 1 >/sys/module/nfs/parameters/nfs4_disable_idmapping   ; mount -t nfs -o nfsvers=4,port=0,hard,rsize=65535,wsize=65536 labnfs.pmtpa.wmnet:/deployment-prep/project /srv/project | [releng] | 
            
  | 12:41 | <hashar> | Refreshed most extensions and running mw-update-l10n | [releng] | 
            
  
    | 2013-04-19
      
      § | 
    
  | 19:50 | <hashar> | The l10n cache was stalled since Mar 22 13:08 at least.   The files were owned by `mwdeploy` seems something changed and they are now owned by `l10nupdate` So I ran:  chown l10nupdate -R /home/wikipedia/common/php-master/cache/l10n/ | [releng] | 
            
  | 19:46 | <hashar> | Attempted to update the l10n cache (sudo -u mwdeploy mw-update-l10n )  got a permission deny on  /home/wikipedia/common/php-master/cache/l10n | [releng] | 
            
  | 19:43 | <hashar> | Gluster is broken on beta.  Extensions are no more updating nor the l10n update can run.   {{bug|47425}} | [releng] | 
            
  | 19:38 | <hashar> | root@deployment-bastion:~# /etc/init.d/udp2log stop && /etc/init.d/udp2log-mw start | [releng] | 
            
  | 19:37 | <hashar> | Rebooting bastion.  Seems GlusterFS can not allocate memory ( {{bug|47425}} ) | [releng] | 
            
  | 19:18 | <hashar> | manually updating mediawiki extensions | [releng] | 
            
  | 11:52 | <hashar> | Successfully added Mark Bergsma to deployment-prep. | [releng] | 
            
  | 09:00 | <hashar> | Updating puppet repositories on search01 and searchidx01. Running puppet on both of them. | [releng] |