|
2016-04-28
§
|
| 20:33 |
<urandom> |
Cassandra on deployment-restbase01.deployment-prep started : T126629 |
[releng] |
| 20:25 |
<urandom> |
Restarting Cassandra on deployment-restbase01.deployment-prep : T126629 |
[releng] |
| 20:14 |
<urandom> |
Re-enable puppet on deployment-restbase01.deployment-prep, and force a run : T126629 |
[releng] |
| 20:12 |
<urandom> |
cherry-picking https://gerrit.wikimedia.org/r/#/c/284078/ to deployment-puppetmaster : T126629 |
[releng] |
| 20:06 |
<urandom> |
Disabling puppet on deployment-restbase0[1-2].deployment-prep : T126629 |
[releng] |
| 14:43 |
<hashar> |
Rebuild Nodepool Jessie image. Comes with hhvm |
[releng] |
| 12:52 |
<hashar> |
Puppet is happy on deployment-changeprop |
[releng] |
| 12:47 |
<hashar> |
apt-get upgrade deployment-changeprop (outdated exim package) |
[releng] |
| 12:42 |
<hashar> |
Rebuild Nodepool Trusty instance to include the PHP wrapper script T126211 |
[releng] |
|
2016-04-27
§
|
| 23:57 |
<thcipriani> |
nodepool instances running again after an openstack rabbitmq restart by andrewbogott |
[releng] |
| 22:51 |
<duploktm> |
also ran openstack server delete ci-jessie-wikimedia-85342 |
[releng] |
| 22:42 |
<legoktm> |
nodepool delete 85342 |
[releng] |
| 22:41 |
<matt_flaschen> |
Deployed https://gerrit.wikimedia.org/r/#/c/285765/ to enable External Store everywhere on Beta Cluster |
[releng] |
| 22:38 |
<legoktm> |
stop/started nodepool |
[releng] |
| 22:36 |
<thcipriani> |
I don't have permission to restart nodepool |
[releng] |
| 22:35 |
<thcipriani> |
restarting nodepool |
[releng] |
| 22:18 |
<matt_flaschen> |
Deployed https://gerrit.wikimedia.org/r/#/c/282440/ to switch Beta Cluster to use External Store for new testwiki writes |
[releng] |
| 20:59 |
<hashar> |
thcipriani downgraded git plugins successfully (we wanted to rule out their upgrade for some weird issue) |
[releng] |
| 20:13 |
<cscott> |
updated OCG to version e39e06570083877d5498da577758cf8d162c1af4 |
[releng] |
| 14:10 |
<hashar> |
restarting Jenkins |
[releng] |
| 14:09 |
<hashar> |
Jenkins upgrading credential plugin 1.24 > 1.27 And Credentials binding plugin 1.6 > 1.7 |
[releng] |
| 14:07 |
<hashar> |
Jenkins upgrading git plugin 2.4.1 > 2.4.4 |
[releng] |
| 14:01 |
<hashar> |
Jenkins upgrading git client plugin 1.19.1. > 1.19.6 |
[releng] |
| 13:13 |
<jzerebecki> |
reloading zuul for 81a1f1a..0993349 |
[releng] |
| 11:43 |
<hashar> |
fixed puppet on deployment-cache-text04 T132689 |
[releng] |
| 10:38 |
<hashar> |
Rebuild Image ci-trusty-wikimedia-1461753210 in wmflabs-eqiad is ready |
[releng] |
| 09:43 |
<hashar> |
tmh01.deployment-prep.eqiad.wmflabs denies mwdeploy user breaking https://integration.wikimedia.org/ci/job/beta-scap-eqiad/ |
[releng] |
|
2016-04-26
§
|
| 20:45 |
<hashar> |
Regenerating Nodepool Jessie snapshot to include composer and HHVM | T128092 |
[releng] |
| 20:23 |
<jzerebecki> |
reloading zuul for eb480d8..81a1f1a |
[releng] |
| 19:25 |
<jzerebecki> |
reload zuul for 4675213..eb480d8 |
[releng] |
| 19:25 |
<jzerebecki> |
4675213..eb480d8 |
[releng] |
| 14:18 |
<hashar> |
Applied security patches to 1.27.0-wmf.22 | T131556 |
[releng] |
| 12:39 |
<hashar> |
starting cut of 1.27.0-wmf.22 branch ( poke ostriches ) |
[releng] |
| 10:29 |
<hashar> |
restored integration/phpunit on CI slaves due to https://integration.wikimedia.org/ci/job/operations-mw-config-phpunit/ failling |
[releng] |
| 09:11 |
<hashar> |
CI is back up! |
[releng] |
| 08:20 |
<hashar> |
shutoff instance castor, does not seem to be able to start again :( | T133652 |
[releng] |
| 08:12 |
<hashar> |
hard rebooting castor instance | T133652 |
[releng] |
| 08:10 |
<hashar> |
soft rebooting castor instance | T133652 |
[releng] |
| 08:06 |
<hashar> |
CI jobs deadlocked due to castor being unavailable | https://phabricator.wikimedia.org/T133652 |
[releng] |
| 00:46 |
<thcipriani> |
temporary keyholder fix in place in beta |
[releng] |
| 00:18 |
<thcipriani> |
beta-scap-eqiad failure due to bad keyholder-auth.d fingerprints |
[releng] |
|
2016-04-25
§
|
| 20:58 |
<cscott> |
updated OCG to version 58a720508deb368abfb7652e6a8c7225f95402d2 |
[releng] |
| 19:46 |
<hashar> |
Nodepool now has a couple trusty instances intended to experiment with Zend 5.5 / HHVM migration . https://phabricator.wikimedia.org/T133203#2236625 |
[releng] |
| 13:34 |
<hashar> |
Nodepool is attempting to create a Trusty snapshot with name ci-trusty-wikimedia-1461591203 | T133203 |
[releng] |
| 13:15 |
<hashar> |
openstack image create --file /home/hashar/image-trusty-20160425T124552Z.qcow2 ci-trusty-wikimedia --disk-format qcow2 --property show=true # T133203 |
[releng] |
| 10:38 |
<hashar> |
Refreshing Nodepool Jessie snapshot based on new image |
[releng] |
| 10:35 |
<hashar> |
Refreshed Nodepool Jessie image ( image-jessie-20160425T100035Z ) |
[releng] |
| 09:24 |
<hashar> |
beta / scap failure filled as T133521 |
[releng] |
| 09:20 |
<hashar> |
Keyholder / mwdeploy ssh keys have been messed up on beta cluster somehow :-( |
[releng] |
| 08:47 |
<hashar> |
mwdeploy@deployment-tin has lost ssh host keys file :( |
[releng] |