6601-6650 of 7543 results (31ms)
2017-10-31 §
16:50 <bd808> tools-bastion-03 (tools-login, login.tools) is overloaded [tools]
2017-10-30 §
17:35 <madhuvishy> Clear dns caches across tools hosts `sudo nscd -i hosts` [tools]
16:08 <arturo> depool tools-exec-1401.tools.eqiad.wmflabs [tools]
2017-10-24 §
18:09 <madhuvishy> Disable puppet on tools-package-builder-01 temporarily (T178920) [tools]
13:22 <chasemp> start admin webservice [tools]
13:22 <chasemp> stop admin webservice [tools]
2017-10-23 §
14:49 <chasemp> wall message and scheduled reboot in 5m for bastion-03 [tools]
2017-10-18 §
21:36 <chasemp> stop basebot -- it is going crazy and spamming email w/ failing to log to error.log. Need to figure out how to notify but it's clearly in a failure loop. [tools]
14:04 <chasemp> add strephit creds to elasticsearch per T178310 [tools]
2017-10-12 §
16:57 <bd808> Rebuilding all Kubernetes Docker images to include toollabs-webservice 0.38 [tools]
16:53 <bd808> Upgraded toollabs-webservice to 0.38 [tools]
2017-10-06 §
15:33 <bd808> Upgrade jobutils to 1.25 (T177614) [tools]
00:27 <bd808> Updated misctools to 1.24 [tools]
2017-10-05 §
22:47 <bd808> Updated misctools to 1.23 [tools]
22:42 <bd808> Updated jobutils to 1.23 [tools]
15:46 <chasemp> tools-bastion-03 has tons of local tools running long lived NFS intensive processes. I'm rebooting rather than playing whackamole. [tools]
2017-10-03 §
19:30 <bd808> `kubectl --namespace=prod delete pod --all` on tools-paws-master-01 [tools]
2017-10-01 §
21:46 <madhuvishy> Cold migrating tools-clushmaster-01 from labvirt1015 to labvirt1017 [tools]
2017-09-29 §
19:49 <andrewbogott> migration tools-clushmaster-01 to labvirt1015 [tools]
2017-09-25 §
15:14 <andrewbogott> rebooting tools-paws-worker-1006 since I can't access it [tools]
14:57 <chasemp> OS_TENANT_NAME=tools openstack server reboot 2c0cf363-c7c3-42ad-94bd-e586f2492321 (unresponsive) [tools]
2017-09-20 §
16:52 <madhuvishy> apt-get install --only-upgrade apache2; service apache2 restart on tools-puppetmaster-01 [tools]
2017-09-19 §
15:22 <chasemp> tools-clushmaster-01:~$ clush -f 5 -g all 'sudo puppet agent --test' [tools]
13:39 <chasemp> bastion-03 someone dropped 8.6G in /tmp which is /not/ seemingly on a temp file system [tools]
13:25 <chasemp> wall Bastion disk is full and needs attention and reboot in 60 [tools]
2017-09-18 §
18:02 <bd808> Updated PHP5.6 images for Kubernetes (T172358) [tools]
2017-09-13 §
15:34 <bd808> Running inbound message purge via clush to @tools-exec [tools]
15:15 <bd808> Running outbound message purge via clush to @tools-exec [tools]
13:57 <bd808> apt-get install nginx-common on tools-static-1[01] [tools]
13:31 <bd808> static down due to apparent nginx package upgrade/config change [tools]
02:10 <bd808> Really disabled puppet on tools-mail [tools]
01:51 <bd808> Nuked all messages in the exim spool on tools-mail [tools]
01:09 <bd808> Removed user WiktCAPT from project [tools]
00:55 <bd808> Archived and then purged /var/spool/exim4/input on tools-mail [tools]
00:47 <bd808> Archived and then purged /var/spool/exim4/msglog on tools-mail [tools]
00:43 <bd808> Stopped exim on tools-mail [tools]
00:43 <bd808> Disabled puppet on tools-mail [tools]
00:15 <chasemp> forced to clean out exim queue as the filesystem used up all inodes [tools]
2017-08-31 §
20:33 <madhuvishy> Updated certs and ran puppet, restarted nginx on tools-proxy-* and tools-static-* (T174611) [tools]
20:25 <madhuvishy> Merging new cert https://gerrit.wikimedia.org/r/#/c/374873/ (T174611) [tools]
20:24 <madhuvishy> Disabling puppet on tools-proxy-* and tools-static-* for star.wmflabs.org SSL cert update (T174611) [tools]
20:23 <madhuvishy> Disabling puppet on tools-proxy-* and tools-static-* for star.wmflabs.org SSL cert update [tools]
2017-08-24 §
19:59 <bd808> restarted nslcd and nscd on tools-bastion-03 [tools]
19:59 <bd808> restarted nslcd and nscd on tools-bastion-02 [tools]
2017-08-22 §
19:20 <andrewbogott> deleted tools-puppetmaster-02, it was replaced a month ago by -01 [tools]
2017-08-12 §
18:38 <chasemp> retart admin webservice [tools]
2017-08-11 §
16:09 <chasemp> qdel -f -j 7441503 [tools]
2017-08-10 §
14:59 <chasemp> 'become stimmberechtigung && restart' && 'become intersect-contribs && restart' [tools]
2017-08-09 §
17:28 <chasemp> webservices restart tools.orphantalk [tools]
2017-08-03 §
00:47 <bd808> tools-bastion-03 not usably responsive to interactive commands; will reboot [tools]