5651-5700 of 6235 results (39ms)
2017-02-28 §
13:30 <elukey> stopping camus as prep step for the CDH upgrade [analytics]
2017-02-23 §
12:18 <joal> Restart cassandra-coord-pageview-per-project-hourly 2017-02-23T07, 08, 09 to recover from cassandra issue - Worked ! [analytics]
11:19 <joal> Restart cassandra-coord-pageview-per-project-hourly 2017-02-23T07 and 08 to recover from cassandra issue [analytics]
2017-02-22 §
08:06 <elukey> restart Hue on an1027 for openssl upgrades [analytics]
2017-02-16 §
13:22 <elukey> updated firewall rules for Analytics VLAN [analytics]
2017-02-15 §
13:55 <elukey> disabled apache mod_deflate on bohrium (piwik test) [analytics]
09:01 <elukey> restarted Piwik with bulk_requests_use_transaction=0 to try to fix the SQL deadlock issue (https://github.com/piwik/piwik/issues/6398#issuecomment-91093146) [analytics]
2017-02-13 §
21:38 <elukey> Restarted webrequest-load-coord-upload 19:00 - failed and Hue returning 500s [analytics]
2017-02-11 §
00:13 <joal> Restartedwebrequest-load-wf-text-2017-2-10-20 [analytics]
2017-02-10 §
09:53 <elukey> re-enabled oozie bundles after maintenance [analytics]
09:51 <elukey> restarted Hive-* and oozie on analytics1003 [analytics]
09:40 <elukey> suspending oozie bundles to allow oozie/hive maintenance [analytics]
2017-02-09 §
13:02 <mforns> Restarted webrequest-load-bundle and pageview-hourly-coord [analytics]
12:46 <mforns> Deployed refinery using scap, then deployed onto hdfs [analytics]
12:00 <elukey> added Marcel as superuser in Hue [analytics]
11:56 <elukey> stopped webrequest-load-bundle from hue [analytics]
11:06 <mforns> Deployed refinery-source using jenkins [analytics]
10:48 <elukey> restarting druid daemons for Java upgrades [analytics]
10:05 <elukey> re-enabled oozie bundles after maintenance [analytics]
10:04 <elukey> performed master failover from an1001 to an1002 (and vice-versa) for java upgrades [analytics]
10:04 <elukey> restarted oozie, hive-server and metastore for java upgrades [analytics]
09:49 <elukey> suspended oozie bundles temporarily to allow graceful restarts [analytics]
2017-02-08 §
18:05 <ottomata> restarting pivot [analytics]
17:52 <ottomata> restarting pivot [analytics]
15:35 <elukey> restarted all the failed oozie cassandra load jobs [analytics]
2017-02-07 §
20:24 <joal> Resubmit cassandra-coord-pageview-per-project-hourly for 2017-02-07T18:00 [analytics]
14:36 <elukey> restarted webrequest-load-wf-text-2017-2-7-13 [analytics]
2017-02-04 §
13:18 <joal> Restarted mediacounts-archive job for day 2017-02-03 (had failed) [analytics]
2017-02-02 §
12:07 <joal> Restarted daily and monthly pageview druid loading jobs [analytics]
12:03 <joal> Deployed refinery to correct bug introduced in https://gerrit.wikimedia.org/r/#/c/335067/ [analytics]
10:13 <joal> Killed-Restarted last access uniques monthly jobs to pick up new config -0097552-161121120201437-oozie-oozi-C [analytics]
2017-02-01 §
19:01 <joal> Killed-Restarted Mobile apps Uniques monthly jobs to pick up new config - 0096638-161121120201437-oozie-oozi-C [analytics]
18:47 <joal> Deploy refinery for uniques monthly patches [analytics]
17:27 <joal> Restarting 2 webrequest-load text jobs that failed during NM restart (2016-02-01T11:00 and T13:00) [analytics]
13:12 <elukey> restarted pageview-druid-monthly-coord and pageview-druid-daily-coord oozie coordinators after deployment [analytics]
12:17 <elukey> deployed Refinery via scap and then executed the hdfs copies on stat1002 [analytics]
2017-01-31 §
16:11 <elukey> started Cassandra nodetool cleanup for aqs1007-a [analytics]
16:04 <elukey> started Cassandra nodetool cleanup for aqs1004-b [analytics]
08:31 <elukey> started Cassandra nodetool cleanup for aqs1004-a [analytics]
2017-01-26 §
19:20 <joal> Restart webrequest-lood-coord-text 2017-01-26T15:00 after cluster shake [analytics]
19:18 <elukey> restored an1001 as RM and HDFS master [analytics]
2017-01-24 §
21:30 <ottomata> restarted hadoop-mapreduce-historyserver on analytics1001. it died to do OOM [analytics]
2017-01-22 §
13:27 <joal> Rerun pageview-druid-daily-wf-2017-1-20 trying to see if it fixes automagically [analytics]
2017-01-19 §
15:51 <joal> Launched 0080172-161121120201437-oozie-oozi-B to recover from missing webrequest-load 2017-01-18 19:00 with a correct setup this time [analytics]
15:39 <joal> Launched 0080149-161121120201437-oozie-oozi-B to recover from missing webrequest-load 2017-01-18 19:00 [analytics]
2017-01-17 §
11:16 <joal> Remove mediawiki-history-beta datasource from druid [analytics]
09:51 <elukey> restarted mediacounts-archive-wf-2017-01-16 [analytics]
2017-01-11 §
19:23 <joal> Start mediawiki history reconstruction job on newly sqooped data [analytics]
18:25 <joal> Replace /wmf/data/raw/mediawiki/tables/ with newly sqooped data [analytics]
2017-01-10 §
15:30 <joal> Restart 0024519-160420145651441-oozie-oozi-C for day 2017-01-09 to see if it fails again [analytics]