8751-8800 of 8958 results (41ms)
2016-01-08 §
17:23 <andrewbogott> killing tools.icelab as per https://wikitech.wikimedia.org/wiki/User_talk:Torin#Running_queries_on_tools-dev_.28tools-bastion-02.29 [tools]
2015-12-30 §
04:06 <YuviPanda> delete all webgrid jobs to start with a clean slate [tools]
03:54 <YuviPanda> qmod -rj all tools in the continuous queue, they are all orphaned [tools]
03:22 <YuviPanda> stop cron on tools-submit, wait for webservices to come back up [tools]
02:39 <YuviPanda> remove lbenedix and ebekebe from tools.hcclab [tools]
00:40 <YuviPanda> restarted master on grid-master [tools]
00:40 <YuviPanda> copied and cleaned out spooldb [tools]
00:10 <YuviPanda> reboot tools-grid-shadow [tools]
00:08 <YuviPanda> attempt to stop shadowd [tools]
00:03 <YuviPanda> attempting to start gridengine-master on tools-grid-shadow [tools]
00:00 <YuviPanda> kill -9'd gridengine master [tools]
2015-12-29 §
23:31 <YuviPanda> rebooting tools-grid-master [tools]
23:22 <YuviPanda> restart gridengine-master on tools-grid-master [tools]
00:18 <YuviPanda> shut down redis on tools-redis-01 [tools]
2015-12-28 §
22:34 <chasemp> attempt to unmount nfs volumes on tools-redis-01 to debug but it hands (I am on console and see root at console hang on login) [tools]
22:31 <YuviPanda> disable NFS on tools-redis-1001 and 1002 [tools]
21:32 <YuviPanda> disable puppet on tools-redis-01 and -02 [tools]
21:27 <YuviPanda> created tools-redis-1001 [tools]
2015-12-23 §
21:21 <YuviPanda> deleted tools-worker-01 to -05, creating tools-worker-1001 to 1005 [tools]
21:19 <valhallasw`cloud> tools-proxy-01: umount /home /data/project /data/scratch /public/dumps [tools]
19:01 <valhallasw`cloud> ah, connections that are kept open. A new incognito window is routed correctly. [tools]
18:59 <valhallasw`cloud> switched to -02, worked correctly, switched back. Switching back does not seem to fully work?! [tools]
18:40 <valhallasw`cloud> scratch that, first going to eat dinner [tools]
18:38 <valhallasw`cloud> dynamicproxy ban system deployed on tools-proxy-02 working correctly for localhost; switching over users there by moving the external IP. [tools]
14:42 <valhallasw`cloud> toollabs homepage is unhappy because tools.xtools-articleinfo is using a lot of cpu on tools-webgrid-lighttpd-1409. Checking to see what's happening there. [tools]
10:46 <YuviPanda> migrate tools-worker-01 to 3.19 kernel [tools]
2015-12-22 §
18:30 <YuviPanda> rescheduling all webservices [tools]
18:17 <YuviPanda> failed over active proxy to proxy-01 [tools]
18:12 <YuviPanda> upgraded kernel and rebooted tools-proxy-01 [tools]
01:42 <YuviPanda> rebooting tools-worker-08 [tools]
2015-12-21 §
18:44 <YuviPanda> reboot tools-proxy-01 [tools]
18:31 <YuviPanda> failover proxy to tools-proxy-02 [tools]
2015-12-20 §
00:00 <YuviPanda> tools-worker-08 stuck again :| [tools]
2015-12-18 §
15:16 <andrewbogott> rebooting locked up host tools-exec-1409 [tools]
2015-12-16 §
23:14 <andrewbogott> rebooting tools-exec-1407, unresponsive [tools]
22:48 <YuviPanda> run qmod -c '*' to clear error state on gridengine [tools]
21:28 <andrewbogott> deleted tools-docker-registry-01 [tools]
16:24 <andrewbogott> rebooting tools-exec-1221 as it was in kernel lockup [tools]
08:38 <YuviPanda> reboot tools-bastion-01 [tools]
2015-12-12 §
10:08 <YuviPanda> restarted cron on tools-submit [tools]
2015-12-10 §
17:06 <valhallasw`cloud> revoked mjmbr membership of 'weather' tool; replaced with tools.admin [tools]
12:47 <valhallasw`cloud> broke tools-proxy-02 login (for valhallasw, root still works) by restarting nslcd. Restarting; current proxy is -01. [tools]
2015-12-07 §
13:46 <Coren> The new grid masters are happy, killing the old ones (-shadow, -master) [tools]
10:46 <YuviPanda> restarted nscd on tools-proxy-01 [tools]
2015-12-06 §
10:29 <YuviPanda> did webservice start on tool 'derivative', was missing service.manifest [tools]
2015-12-04 §
19:33 <Coren> switching master role to tools-grid-master [tools]
04:42 <YuviPanda> disabled puppet on tools-puppetmaster-01 because everything sucks [tools]
04:09 <bd808> Cherry-picked https://gerrit.wikimedia.org/r/#/c/256618 to tools-puppetmaster-01 [tools]
2015-12-02 §
18:29 <Coren> switching gridmaster activity to tools-grid-shadow [tools]
05:13 <YuviPanda> increased security groups quota to 50 because why not [tools]