9901-9950 of 10000 results (118ms)
2024-09-09 §
07:33 <jayme@cumin1002> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host kubestage2001.codfw.wmnet [production]
07:33 <jayme@cumin1002> START - Cookbook sre.k8s.pool-depool-node pool for host kubestage2001.codfw.wmnet [production]
07:17 <moritzm> installing Linux 5.10.223 on bullseye hosts [production]
07:06 <moritzm> roll out debmonitor-client 0.4.0-2+deb11u1 on bullseye hosts [production]
06:56 <moritzm> installing aom security updates [production]
2024-09-07 §
14:44 <ebernhardson@deploy1003> helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
14:44 <ebernhardson@deploy1003> helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
14:38 <dani@deploy1003> helmfile [codfw] DONE helmfile.d/services/miscweb: apply [production]
14:38 <dani@deploy1003> helmfile [codfw] START helmfile.d/services/miscweb: apply [production]
14:38 <dani@deploy1003> helmfile [eqiad] DONE helmfile.d/services/miscweb: apply [production]
14:38 <dani@deploy1003> helmfile [eqiad] START helmfile.d/services/miscweb: apply [production]
14:38 <dani@deploy1003> helmfile [staging] DONE helmfile.d/services/miscweb: apply [production]
14:38 <dani@deploy1003> helmfile [staging] START helmfile.d/services/miscweb: apply [production]
14:34 <dani@deploy1003> helmfile [codfw] DONE helmfile.d/services/miscweb: apply [production]
14:34 <dani@deploy1003> helmfile [codfw] START helmfile.d/services/miscweb: apply [production]
14:34 <dani@deploy1003> helmfile [eqiad] DONE helmfile.d/services/miscweb: apply [production]
14:34 <dani@deploy1003> helmfile [eqiad] START helmfile.d/services/miscweb: apply [production]
14:34 <dani@deploy1003> helmfile [staging] DONE helmfile.d/services/miscweb: apply [production]
14:34 <dani@deploy1003> helmfile [staging] START helmfile.d/services/miscweb: apply [production]
10:28 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4 days, 0:00:00 on db1246.eqiad.wmnet with reason: https://phabricator.wikimedia.org/T374215 → server depooled has hardware issues [production]
10:28 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 4 days, 0:00:00 on db1246.eqiad.wmnet with reason: https://phabricator.wikimedia.org/T374215 → server depooled has hardware issues [production]
2024-09-06 §
22:35 <ebernhardson@deploy1003> helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
22:35 <ebernhardson@deploy1003> helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
19:07 <dduvall@deploy1003> Finished deploy [releng/jenkins-deploy@6ca00a7] (releasing): (no justification provided) (duration: 00m 43s) [production]
19:06 <dduvall@deploy1003> Started deploy [releng/jenkins-deploy@6ca00a7] (releasing): (no justification provided) [production]
19:01 <ebernhardson@deploy1003> helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
19:00 <ebernhardson@deploy1003> helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
18:57 <ebernhardson@deploy1003> helmfile [codfw] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
18:57 <ebernhardson@deploy1003> helmfile [codfw] START helmfile.d/services/cirrus-streaming-updater: apply [production]
18:32 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 7:00:00 on db2200.codfw.wmnet with reason: Maintenance [production]
18:32 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 2 days, 7:00:00 on db2200.codfw.wmnet with reason: Maintenance [production]
18:12 <brett> Import ncmonitor 1.2.1-1 into bookworm-wikimedia apt archive [production]
17:55 <brett> Import corto 0.3.1-1 into bookworm-wikimedia apt archive [production]
16:46 <kamila_> ran homer on cr*codfw* for T372878 [production]
16:30 <kamila@cumin1002> END (PASS) - Cookbook sre.k8s.renumber-node (exit_code=0) Renumbering for host wikikube-worker2103.codfw.wmnet [production]
16:30 <kamila@cumin1002> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2103.codfw.wmnet [production]
16:30 <kamila@cumin1002> START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2103.codfw.wmnet [production]
16:24 <kamila@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2103.codfw.wmnet with OS bullseye [production]
16:06 <dzahn@cumin2002> END (PASS) - Cookbook sre.dns.roll-restart-reboot-durum (exit_code=0) rolling reboot on A:durum and A:durum [production]
16:04 <hnowlan@cumin1002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker2095.codfw.wmnet with OS bullseye [production]
16:02 <kamila@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2103.codfw.wmnet with reason: host reimage [production]
15:59 <kamila@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2103.codfw.wmnet with reason: host reimage [production]
15:52 <elukey@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host sretest2001.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART [production]
15:49 <elukey@cumin2002> START - Cookbook sre.hosts.provision for host sretest2001.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART [production]
15:44 <dduvall@deploy1003> Finished deploy [releng/jenkins-deploy@ad2c434] (releasing): (no justification provided) (duration: 00m 41s) [production]
15:43 <dduvall@deploy1003> Started deploy [releng/jenkins-deploy@ad2c434] (releasing): (no justification provided) [production]
15:43 <cmooney@cumin1002> END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for lvs1017.eqiad.wmnet [production]
15:43 <cmooney@cumin1002> START - Cookbook sre.hosts.remove-downtime for lvs1017.eqiad.wmnet [production]
15:42 <topranks> enabling PyBal on lvs1017 to make primary again after repairing faulty fiber link T374247 [production]
15:42 <elukey> install spicerack 8.13.0 on cumin1002 [production]