901-950 of 1282 results (16ms)
2016-07-22 §
17:30 <YuviPanda> repool tools-worker-1018 [tools]
14:04 <chasemp> reboot tools-worker-1015 as stuck w/ high iowait warning seconds ago. I cannot ssh in as root. [tools]
2016-07-21 §
22:42 <chasemp> reboot tools-worker-1018 as stuck T141017 [tools]
2016-07-20 §
21:26 <andrewbogott> rebooting tools-k8s-etcd-01 [tools]
11:14 <Guest9334> rebooted tools-worker-1004 [tools]
2016-07-19 §
01:06 <bd808> Upgraded Elasticsearch on tools-elastic-* to 2.3.4 [tools]
2016-07-18 §
21:50 <YuviPanda> force downgrade hhvm on tools-webgrid-lighttpd-1408 to fix puppet issues [tools]
21:40 <YuviPanda> bind mount and kill files in /var/lib/docker that were monuted over by proper mount on lvm on tools-worker-1004 [tools]
21:40 <YuviPanda> bind mount and kill files in /var/lib/docker that were monuted over by proper mount on lvm [tools]
21:37 <YuviPanda> killed tools-pastion-01, no longer in use [tools]
20:59 <bd808> Disabled puppet on tools-elastic-0[123]. Elasticsearch needs to be upgraded. [tools]
15:15 <YuviPanda> kill 8807036 for Luke081515 [tools]
12:48 <YuviPanda> reboot tools-flannel-etcd-03 for T140256 [tools]
12:41 <YuviPanda> reboot tools-k8s-etcd-02 for T140256 [tools]
2016-07-15 §
10:24 <yuvipanda> depool tools-exec-1402 for T138447 [tools]
10:24 <yuvipanda> reboot tools-exec-1402 for T138447 [tools]
10:16 <yuvipanda> depooling tools-webgrid-lighttpd-1402 and -1412 since they seem to be suffering from T138447 [tools]
10:08 <yuvipanda> reboot tools-webgrid-lighttpd-1402 and 1412 [tools]
2016-07-14 §
23:12 <bd808> Added Madhuvishy to project "roots" sudoer list [tools]
22:58 <bd808> Added Madhuvishy as projectadmin [tools]
21:25 <chasemp> change perms for tools.readmore to correct bot [tools]
2016-07-13 §
11:40 <yuvipanda> cold-migrate tools-worker-1014 off labvirt1010 to see if that improves the ksoftirqd situation [tools]
11:19 <yuvipanda> drained tools-worker-1004 - high ksoftirqd usage even with no load [tools]
11:13 <yuvipanda> depool tools-worker-1014 - unusable, totally in iowait [tools]
11:13 <yuvipanda> reboot tools-worker-1004, was unresponsive [tools]
2016-07-12 §
18:07 <yuvipanda> reboot tools-worker-1012, it seems to have failed LDAP connectivity :| [tools]
2016-07-08 §
12:38 <yuvipanda> starting up tools-web-static-02 again [tools]
2016-07-07 §
12:45 <yuvipanda> start deployment of k8s 1.3.0wmf4 for T139259 [tools]
2016-07-06 §
13:09 <yuvipanda> associated a floating IP with tools-k8s-master-01 for T139461 [tools]
11:47 <yuvipanda> moved tools-checker-0[12] to use tools-puppetmaster-01 as puppetmaster so they get appropriate CA for use when talking to kubernetes API [tools]
2016-07-04 §
11:13 <yuvipanda> delete tools-prometheus-01 to free up resources on labvirt1010 [tools]
11:11 <yuvipanda> actually deleted instance tools-cron-02 to free up resources on labvirt1010 - was large and not currently used, and failover process takes a while anyway, so we can recreate if needed [tools]
11:10 <yuvipanda> stopped instance tools-cron-02 to free up some resources on labvirt1010 [tools]
2016-07-03 §
17:09 <yuvipanda> run qstat -u '*' | grep 'dr ' | awk '{ print $1;}' | xargs -L1 qdel -f to clean out jobs stuck in dr state [tools]
16:58 <yuvipanda> migrate tools-web-static-02 to labvirt1011 to provide more breathing room [tools]
16:56 <yuvipanda> delete temp-test-trusty-package to provide more breathing room on labvirt1010 [tools]
13:49 <yuvipanda> reboot tools-exec-1219 [tools]
13:37 <yuvipanda> migrating tools-exec-1216 to labvirt1011 [tools]
13:07 <yuvipanda> delete tools-bastion-01 which was shut down anyway [tools]
13:04 <yuvipanda> attempt to reboot tools-exec-1212 [tools]
2016-06-28 §
15:25 <bd808> Signed client cert for tools-worker-1019.tools.eqiad.wmflabs on tools-puppetmaster-01.tools.eqiad.wmflabs [tools]
2016-06-21 §
16:49 <bd808> Updated jobutils to v1.14 for T138178 [tools]
2016-06-17 §
06:17 <yuvipanda> forced deletion of 7033590 for dykbot for shubinator [tools]
2016-06-09 §
00:19 <mutante> is a valid project [tools]
2016-06-08 §
20:31 <yuvipanda> start tools-bastion-03 was stuck in 'stopped' state [tools]
20:31 <yuvipanda> reboot tools-bastion-03 [tools]
2016-05-31 §
17:35 <valhallasw`cloud> re-enabled queues on tools-exec-1407, tools-exec-1216, tools-exec-1219 [tools]
13:13 <chasemp> reboot of tools-exec-1203 see T136495 all jobs seem gone now [tools]
2016-05-30 §
13:06 <valhallasw`cloud> rebooting tools-exec-1221 [tools]
11:53 <godog> cherry-pick https://gerrit.wikimedia.org/r/#/c/280652 https://gerrit.wikimedia.org/r/#/c/290479 https://gerrit.wikimedia.org/r/#/c/291710/ on tools-puppetmaster-01 [tools]