51-100 of 10000 results (14ms)
2025-01-17 ยง
14:34 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ganeti2027.codfw.wmnet with OS bookworm [production]
14:33 <marostegui@cumin1002> dbctl commit (dc=all): 'es1046 (re)pooling @ 75%: Repooling for the first time', diff saved to https://phabricator.wikimedia.org/P72146 and previous config saved to /var/cache/conftool/dbconfig/20250117-143318-root.json [production]
14:18 <marostegui@cumin1002> dbctl commit (dc=all): 'es1046 (re)pooling @ 50%: Repooling for the first time', diff saved to https://phabricator.wikimedia.org/P72145 and previous config saved to /var/cache/conftool/dbconfig/20250117-141813-root.json [production]
14:15 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ganeti2027.codfw.wmnet with reason: host reimage [production]
14:12 <jmm@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on ganeti2027.codfw.wmnet with reason: host reimage [production]
14:03 <marostegui@cumin1002> dbctl commit (dc=all): 'es1046 (re)pooling @ 25%: Repooling for the first time', diff saved to https://phabricator.wikimedia.org/P72144 and previous config saved to /var/cache/conftool/dbconfig/20250117-140308-root.json [production]
14:01 <brouberol@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-test-k8s: apply [production]
14:01 <brouberol@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-test-k8s: apply [production]
14:00 <brouberol@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-analytics-test: apply [production]
14:00 <brouberol@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-analytics-test: apply [production]
13:51 <brouberol@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-analytics-test: apply [production]
13:50 <brouberol@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-analytics-test: apply [production]
13:48 <marostegui@cumin1002> dbctl commit (dc=all): 'es1046 (re)pooling @ 10%: Repooling for the first time', diff saved to https://phabricator.wikimedia.org/P72143 and previous config saved to /var/cache/conftool/dbconfig/20250117-134801-root.json [production]
13:36 <brouberol@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-analytics-test: apply [production]
13:35 <brouberol@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-analytics-test: apply [production]
13:35 <brouberol@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-test-k8s: apply [production]
13:35 <brouberol@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-test-k8s: apply [production]
13:32 <marostegui@cumin1002> dbctl commit (dc=all): 'es1046 (re)pooling @ 5%: Repooling for the first time', diff saved to https://phabricator.wikimedia.org/P72142 and previous config saved to /var/cache/conftool/dbconfig/20250117-133256-root.json [production]
13:17 <marostegui@cumin1002> dbctl commit (dc=all): 'es1046 (re)pooling @ 4%: Repooling for the first time', diff saved to https://phabricator.wikimedia.org/P72140 and previous config saved to /var/cache/conftool/dbconfig/20250117-131751-root.json [production]
13:02 <marostegui@cumin1002> dbctl commit (dc=all): 'es1046 (re)pooling @ 3%: Repooling for the first time', diff saved to https://phabricator.wikimedia.org/P72139 and previous config saved to /var/cache/conftool/dbconfig/20250117-130245-root.json [production]
12:56 <hashar> Tag Quibble 1.12.0 @ 633099ead3ec72180e7890e1980074b4fde56c26 # T365978, T383243 [releng]
12:47 <marostegui@cumin1002> dbctl commit (dc=all): 'es1046 (re)pooling @ 2%: Repooling for the first time', diff saved to https://phabricator.wikimedia.org/P72138 and previous config saved to /var/cache/conftool/dbconfig/20250117-124740-root.json [production]
12:32 <marostegui@cumin1002> dbctl commit (dc=all): 'es1046 (re)pooling @ 1%: Repooling for the first time', diff saved to https://phabricator.wikimedia.org/P72137 and previous config saved to /var/cache/conftool/dbconfig/20250117-123235-root.json [production]
12:31 <marostegui@cumin1002> dbctl commit (dc=all): 'Add es1046 to dbctl depooled T382569', diff saved to https://phabricator.wikimedia.org/P72136 and previous config saved to /var/cache/conftool/dbconfig/20250117-123153-marostegui.json [production]
12:25 <jynus@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2141.codfw.wmnet with OS bookworm [production]
12:07 <kamila@cumin1002> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[1111-1116].eqiad.wmnet [production]
12:07 <kamila@cumin1002> START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[1111-1116].eqiad.wmnet [production]
12:02 <hnowlan@deploy1003> helmfile [codfw] DONE helmfile.d/services/eventstreams: apply [production]
12:02 <hnowlan@deploy1003> helmfile [codfw] START helmfile.d/services/eventstreams: apply [production]
12:01 <jynus@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2141.codfw.wmnet with reason: host reimage [production]
12:00 <hnowlan@deploy1003> helmfile [eqiad] DONE helmfile.d/services/eventstreams: apply [production]
12:00 <hnowlan@deploy1003> helmfile [eqiad] START helmfile.d/services/eventstreams: apply [production]
11:58 <moritzm> installing Linux 6.1.124 on Bookworm hosts [production]
11:57 <jynus@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on db2141.codfw.wmnet with reason: host reimage [production]
11:42 <jmm@cumin2002> START - Cookbook sre.hosts.reimage for host ganeti2027.codfw.wmnet with OS bookworm [production]
11:41 <jynus@cumin2002> START - Cookbook sre.hosts.reimage for host db2141.codfw.wmnet with OS bookworm [production]
11:37 <jynus@cumin2002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2141.codfw.wmnet with reason: reimage [production]
11:31 <kamila@cumin1002> END (FAIL) - Cookbook sre.k8s.pool-depool-node (exit_code=99) pool for host wikikube-worker[1111-1116].eqiad.wmnet [production]
11:31 <kamila@cumin1002> START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[1111-1116].eqiad.wmnet [production]
10:58 <jelto@cumin1002> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[2240-2241].codfw.wmnet [production]
10:58 <jelto@cumin1002> START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[2240-2241].codfw.wmnet [production]
10:55 <jelto> homer 'cr*codfw*' commit 'T377877' [production]
10:54 <jelto> homer 'lsw1-b3-codfw*' commit 'T377877' [production]
10:51 <jelto@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2241.codfw.wmnet with OS bookworm [production]
10:31 <jelto@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2241.codfw.wmnet with reason: host reimage [production]
10:26 <jelto@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2241.codfw.wmnet with reason: host reimage [production]
10:19 <jelto@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2240.codfw.wmnet with OS bookworm [production]
10:08 <jelto@cumin1002> END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host wikikube-worker2241 [production]
10:08 <jelto@cumin1002> START - Cookbook sre.hosts.move-vlan for host wikikube-worker2241 [production]
10:08 <jelto@cumin1002> START - Cookbook sre.hosts.reimage for host wikikube-worker2241.codfw.wmnet with OS bookworm [production]