2651-2700 of 4864 results (21ms)
2015-02-03 §
09:15 <hashar> Running puppet on deployment-eventlogging02 has been stalled for 3d15h. No log :-( [releng]
09:08 <hashar> cleaning /var/log on deployment-elastic06 and deployment-elastic07 [releng]
00:44 <Krinkle> Restarting Jenkins-Gearman connection [releng]
2015-02-02 §
21:39 <Krinkle> Deployed I94f65b56368 and reloading Zuul [releng]
13:37 <yuvipanda> added mx record to beta.wmflabs.org, for https://phabricator.wikimedia.org/T88215 via LDAP [releng]
2015-01-31 §
20:31 <hashar> canceling a bunch of browser tests jobs that are deadlocked waiting for SauceLabs. The http request has no timeout {{bug|T88221}} [releng]
2015-01-29 §
01:39 <James_F> Restarting Jenkins because deployment-bastion.eqiad isn't depooling even after restart. [releng]
00:47 <Krenair> running instructions at https://www.mediawiki.org/wiki/Continuous_integration/Jenkins#Hung_beta_code.2Fdb_update [releng]
00:26 <Krinkle> integration-slave1007 rm -rf /mnt/jenkins-workspace/workspace/oojs* [releng]
00:19 <Krinkle> Jenkins slave on deployment-bastion.eqiad has been stuck for the past 5 hours [releng]
2015-01-28 §
22:53 <Krinkle> rm -rf integration-slave1007 rm -rf /mnt/jenkins-workspace/workspace/mwext-DonationInterface-np* [releng]
22:43 <Krinkle> /srv/deployment/integration/slave-scripts got corrupted by puppet on labs slaves. No longer has the appropriate permission flags. [releng]
16:52 <marktraceur> restarting nginx on deployment-upload so beta images might work again [releng]
2015-01-27 §
18:54 <Krinkle> rm -rf integration-slave1007 mwext-VisualEditor-* [releng]
18:15 <andrewbogott> upgrading libc6 on all instances from deployment-salt [releng]
2015-01-26 §
23:22 <bd808> rm integration-slave1006:/mnt/jenkins-workspace/workspace/mediawiki-phpunit-hhvm/src/.git/HEAD.lock (file was timestamped Jan 22 23:55) [releng]
21:06 <bd808> I just merged a scap change that probably will break the beta-recomile-math-textvc-eqiad job -- https://gerrit.wikimedia.org/r/#/c/186808/ [releng]
2015-01-24 §
01:05 <hashar> restarting Jenkins (deadlock on deployment-bastion slave) [releng]
2015-01-20 §
18:50 <Krinkle> Reconfigure Jenkins default language back to 'en' as it was set to Turkish [releng]
02:30 <YuviPanda> created deployment-mediawiki04 to test roles [releng]
2015-01-17 §
20:20 <James_F> Brought deployment-bastion.eqiad back online, but without effect AFAICS. [releng]
20:19 <James_F> Marking deployment-bastion.eqiad as temporarily offline to try to fix the backlog. [releng]
2015-01-16 §
23:26 <bd808> cherry-picked https://gerrit.wikimedia.org/r/#/c/185570/ to fix puppet errors on deployment-prep [releng]
12:43 <_joe_> added hhvm.pcre_cache_type = "lru" to beta hhvm config [releng]
12:32 <_joe_> installing the new HHVM package on mediawiki hosts [releng]
11:59 <akosiaris> removed ferm from all beta hosts via salt [releng]
2015-01-15 §
17:06 <greg-g> turned off the beta-scap-eqiad jenkins job due to the persistent failing (https://phabricator.wikimedia.org/T86901) and the impending labs outage [releng]
14:50 <hashar> beta-scap-eqiad broken since ~ 7:52am UTC. Depends on mwdeploy user homedir to be fixed in LDAP https://phabricator.wikimedia.org/T86903 [releng]
10:55 <hashar> https://integration.wikimedia.org/ci/job/beta-scap-eqiad/ is broken since roughly 7:52am UTC. [releng]
2015-01-14 §
23:22 <mutante> cherry-picked I1e5f9f7bcbbe6c4 on deployment-bastion [releng]
20:37 <hashar> Restarting Zuul [releng]
20:36 <hashar> Zuul applied Ori patch to fix a git lock contention in Zuul-cloner {{bug|T86730}} . Tagged wmf-deploy-20150114-1 [releng]
16:58 <greg-g> rm -rf'd the Wikigrok checkout in integration-slave1006:/mnt/jenkins-workspace/workspace/mediawiki-extensions-hhvm/src/extensions to (hopefully) fix https://phabricator.wikimedia.org/T86730 [releng]
14:56 <anomie> Cherry-pick https://gerrit.wikimedia.org/r/#/c/173336/11/ to Beta Labs [releng]
02:05 <bd808> There is some kind of race / conflict with the mediawiki-extensions-hhvm; I cleaned up the same error for a different extension yesterday [releng]
2015-01-13 §
22:37 <hashar> Restarted Zuul, deadlocked waiting for Gerrit [releng]
21:38 <ori> deployment-prep upgraded nutcracker on mw1/mw2 to 0.4.0+dfsg-1+wm1 [releng]
17:43 <hashar> Restarted deadlocked Zuul , which drops ALL events. Reason is Gerrit lost connection with its database which is not handled by Zuul . See https://wikitech.wikimedia.org/wiki/Incident_documentation/20150106-Zuul [releng]
17:32 <James_F> No effect from restarting Gearman. Getting Timo to restart Zuul. [releng]
17:30 <James_F> No effect. Restarting Gearman. [releng]
17:27 <James_F> Trying a shutdown/re-enable of Jenkins. [releng]
13:59 <YuviPanda> running scap via jenkins, hitting buttons on https://integration.wikimedia.org/ci/job/beta-scap-eqiad/ [releng]
13:58 <YuviPanda> scap failed [releng]
13:58 <YuviPanda> running scap, because why not [releng]
13:58 <YuviPanda> modified PrivateSettings.php to make it use wikiadmin user rather than mw user [releng]
13:51 <YuviPanda> created user wikiadmin on deployment-db1 [releng]
04:31 <James_F> Zuul now appears fixed. [releng]
04:29 <marktraceur> FORCE RESTART ZUUL (James_F told me to) [releng]
04:28 <marktraceur> Attempting graceful zuul restart [releng]
04:26 <marktraceur> Reloaded zuul to see if it will help [releng]