3151-3200 of 4065 results (13ms)
2013-04-10 §
13:31 <hashar> rebooting deployment-bastion too : gluster issue [releng]
13:26 <hashar> Cluster is back up :-] [releng]
13:25 <hashar> rebooting both apaches. [releng]
13:24 <hashar> Gluster failure again /data/project/apache/conf/ has some files missing: www.wikipedia.conf en2.conf wikimedia.conf [releng]
13:23 <hashar> apache2: Syntax error on line 324 of /etc/apache2/apache2.conf: Syntax error on line 9 of /etc/apache2/wmf/all.conf: Could not open configuration file /etc/apache2/wmf/www.wikipedia.conf: No such file or directory [releng]
13:20 <hashar> apt-get upgraded apache32 and apache33 . Note that apache is down on them. [releng]
13:19 <hashar> no pages being served. Most probably a PHP Fatal error [releng]
13:13 <hashar> reran Jenkins job https://integration.wikimedia.org/ci/job/beta-mediawiki-config-update/ . Some git failures happened in /home/wikipedia/common . [releng]
06:45 <hashar> searchidx01 : restarted lucene-search-2 might have been killed by OOM killer (see {{bug|46459}} [releng]
06:39 <hashar> search01 : restarted lucene-search-2 , was not listening on port 8123. [releng]
2013-04-08 §
20:51 <hashar> deployment-search01 : /usr/bin/java -Xmx2000m :-] [releng]
20:50 <hashar> Changing lucene-search-2 memory usage from 20G to 2G by manually editing /etc/init.d/lucene-search-2 (see {{bug|46459}} ) [releng]
20:00 <hashar> deployment-search01 updating local puppet repo c345581..7d036cb [releng]
19:57 <hashar> deployment-searchidx01 updating local puppet repository 81f5a93..7d036cb [releng]
2013-03-29 §
12:00 <hashar> MobileFrontend should not let user login again ({{bug|46649}}, the issue was most probably caused by the lack of commonswiki on beta. [releng]
11:53 <hashar> restoring commonswiki on beta {{gerrit|56593}}. [releng]
10:40 <hashar> rebooting jobrunner08 and bastion. High network use too. [releng]
10:38 <hashar> rebooting both apaches instances. They consume ton of network, most probably related to Gluster [releng]
2013-03-25 §
19:46 <hashar> -bastion : restarting puppet. Restarting beta autoupdater. [releng]
15:01 <hashar> Updated database enwiki [releng]
14:26 <hashar> getting lazy, dropping Central Notice tables from enwiki and rerunning updater. [releng]
14:16 <hashar> Attempting to fix central notice database schema for enwiki [releng]
11:46 <hashar> removing local hack made to ArticleFeedback data/maintenance/DataModelPurgeCache.php [releng]
11:44 <hashar> /home/wikipedia/common/php-master/extensions : git remote update && git reset --hard origin/master && git submodule update --init [releng]
11:40 <hashar> Resetting the extensions checkout. Been broken for a few days because of extension renaming. [releng]
2013-03-22 §
21:11 <hashar> Search is back! Turns out that lucene-search2 service was not running on deployment-search01 despite puppet ensure => running on the service :( See also {{bug|46459}} [releng]
21:03 <hashar> Starting lucene-search-2 on deployment-search [releng]
15:00 <hashar> manually update puppet sources on -search01 and -searchidx01 [releng]
14:40 <hashar> manually refreshing extensions on -bastion [releng]
14:39 <hashar> Updated mediawiki/extensions.git which was lacking the Thanks extension {{gerrit|55263}} [releng]
14:39 <hashar> I have setup a Jenkins job to automatically update mediawiki-config. Dashboard is https://integration.wikimedia.org/ci/job/beta-mediawiki-config-update/ [releng]
2013-03-21 §
22:37 <hashar> -bastion : stopping udp2log that prevents udp2log-mw from running :/ See {{bug|38995}} [releng]
22:35 <hashar> both apaches gives out Error 500 so beta is now serving blank pages. [releng]
2013-03-19 §
09:23 <hashar> created sudo policy for jenkins-deploy user. That is the user for the Jenkins slave running deployment-bastion [releng]
2013-03-15 §
13:41 <hashar> -squid killed nrpe and restarted it (just to be sure) [releng]
13:39 <hashar> -squid started puppet service [releng]
13:38 <hashar> -squid ran puppet manually which deployed the new redirector from https://gerrit.wikimedia.org/r/53935 [releng]
13:34 <hashar> -squid shows a ton of stalled `redirector` processes. Killed them all. [releng]
2013-03-14 §
22:20 <hashar> deployment-bastion is now a jenkins slave of the production Jenkins machine [releng]
22:13 <hashar> manually installing openjdk-7-jre on -bastion [releng]
22:02 <hashar> Successfully added jenkins-deploy to deployment-prep. [releng]
21:59 <hashar> adding jenkins-deploy to the project [releng]
21:37 <hashar> removing restrictions from deployment-bastion . authorized_keys is not read when in labs :] (thx Ryan) [releng]
21:16 <hashar> -bastion changed restricted_to to (project-deployment-prep) (jenkins) [releng]
21:15 <hashar> on -bastion: Added group restrictions and set variable restricted_to = (project-jenkins) (jenkins) thanks ryan [releng]
21:02 <hashar> creating jenkins homedir manually on -bastion [releng]
20:38 <hashar> applying jenkins::user to deployment-bastion [releng]
19:03 <hashar> rebooting -bastion to find out whether the security rule is applied [releng]
19:01 <hashar> updated security rule to allow TCP port 22 connection from gallium.wikimedia.org [208.80.154.135/32] [releng]
04:22 <hashar> killed job runners on jobrunner08 and restarted service [releng]