|
2015-03-05
§
|
| 20:48 |
<Krinkle> |
Re-establishing Gearman connection from Jenkins |
[releng] |
| 20:44 |
<Krinkle> |
Deleting integration-slave1201-integration-slave1404, and integration-slave1401-integration-slave1404. |
[releng] |
| 20:18 |
<Krinkle> |
Finished creation and provisioning of integration-slave1405 |
[releng] |
| 19:34 |
<legoktm> |
deploying https://gerrit.wikimedia.org/r/194461, lots of new jobs |
[releng] |
| 18:50 |
<Krinkle> |
Re-creating integration-slave1405 |
[releng] |
| 17:52 |
<twentyafterfour> |
pushed wmf/1.25wmf20 branch to submodule repos |
[releng] |
| 16:18 |
<greg-g> |
now there are jobs running on the zuul status page |
[releng] |
| 16:16 |
<greg-g> |
getting "/zuul/status.json: Service Temporarily Unavailable" after the zuul restart |
[releng] |
| 16:12 |
<^d> |
restarted zuul |
[releng] |
| 16:06 |
<greg-g> |
jenkins doesn't have anything queued and is processing jobs apparently, not sure why zuul is showing two jobs queued for almost 2 hours (one with all tests passing, the other with nothing tested yet) |
[releng] |
| 16:04 |
<greg-g> |
not sure it helped |
[releng] |
| 16:02 |
<greg-g> |
about to disconnect/reconnect gearman per https://www.mediawiki.org/wiki/Continuous_integration/Zuul#Known_issues |
[releng] |
| 00:34 |
<legoktm> |
deployed https://gerrit.wikimedia.org/r/194421 |
[releng] |
|
2015-03-04
§
|
| 17:34 |
<Krinkle> |
Depooling all new integation-slave12xx and integration-slave14xx instances again |
[releng] |
| 17:34 |
<Krinkle> |
"ImportError: Entry point ('console_scripts', 'tox') not found" on integration-slave12xx instances for operations-puppet-tox-data_admin_lint |
[releng] |
| 17:33 |
<Krinkle> |
"/usr/local/bin/tox: Permission denied" on integration-slave14xx instances |
[releng] |
| 17:11 |
<Krinkle> |
Pooled integration-slave1201, integration-slave1202, integration-slave1203, integration-slave1204 |
[releng] |
| 17:06 |
<Krinkle> |
Pooled integration-slave1402, integration-slave1403, integration-slave1404, integration-slave1405 |
[releng] |
| 16:56 |
<Krinkle> |
Pooled integration-slave1401 |
[releng] |
| 16:26 |
<Krinkle> |
integration-slave14xx are now provisioned and being added to the pool. Old trusty slaves will be depooled later and eventually deleted. |
[releng] |
|
2015-03-03
§
|
| 22:00 |
<hashar> |
reboot integration-puppetmaster in case it solves a NFS mount issue |
[releng] |
| 20:33 |
<legoktm> |
manually created centralauth.users_to_rename table |
[releng] |
| 18:48 |
<Krinkle> |
New builds run fine, but there's 30 stuck builds occupying executors. Their output is finished and Zuul/Gerrit got the event already, but they won't die. |
[releng] |
| 18:30 |
<Krinkle> |
The stuck builds have a line of "Finished: .." in them, but are still showing a loader spinner on the bottom of their build log 10 minutes later. |
[releng] |
| 18:28 |
<Krinkle> |
Lots of Jenkins builds are stuck half-way executing. No clear cause. Everything seems up. |
[releng] |
| 17:18 |
<Krinkle> |
Reloading Zuul to deploy Icad0a26dc8 and Icac172b16 |
[releng] |
| 15:39 |
<hashar> |
cancelled logrotate update of all jobs since that seems to kill the Jenkins/Zuul gearman connection. Probably because all jobs are registered on each config change. |
[releng] |
| 15:31 |
<hashar> |
updating all jobs in Jenkins based on PS2 of https://gerrit.wikimedia.org/r/194109 |
[releng] |
| 10:56 |
<hashar> |
Created instance i-000008fb with image "ubuntu-14.04-trusty" and hostname i-000008fb.eqiad.wmflabs. |
[releng] |
| 10:52 |
<hashar> |
deleting integration-puppetmaster to recreate it with a new image {bug|T87484} . Will have to reapply I5335ea7cbfba33e84b3ddc6e3dd83a7232b8acfd and I30e5bfeac398e0f88e538c75554439fe82fcc1cf |
[releng] |
| 03:47 |
<Krinkle> |
git-deploy: Deploying integration/slave-scripts 05a5593..1e64ed9 |
[releng] |
| 01:11 |
<marxarelli> |
gzip'd /var/log/account/pacct.0 on deployment-bastion to free space |
[releng] |
|
2015-02-26
§
|
| 23:24 |
<Krinkle> |
integration-puppetmaster /var disk is full (1.8 of 1.9GB) - /var/log/puppet/reports is 1.1GB - purging |
[releng] |
| 23:23 |
<Krinkle> |
Puppet failing on new instances due to "Error 400 on SERVER: cannot generate tempfile `/var/lib/puppet/yaml/" |
[releng] |
| 13:27 |
<Krinkle> |
Provisioning the new integration-slave12xx and integration-slave14xx instances |
[releng] |
| 11:53 |
<YuviPanda> |
created deployment-parsoid01-test to test patch to use role::parsoid on labs |
[releng] |
| 05:05 |
<legoktm> |
deployed https://gerrit.wikimedia.org/r/192980 |
[releng] |
| 03:48 |
<Krinkle> |
Creating integration-slave1201,02,03,04 and integration-slave1401,02,03,04,05 per T74011 (not yet setup/provisioned, keep depooled) |
[releng] |
| 03:39 |
<Krinkle> |
Cleaned up and re-pooled integration-slave1006 (was depooled since yesterday) |
[releng] |