1-50 of 8498 results (35ms)
2025-05-12 §
19:48 <raymond-ndibe@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api [tools]
19:35 <raymond-ndibe@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component jobs-api [tools]
16:18 <dcaro@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-cli [tools]
16:06 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component builds-cli [tools]
13:46 <dcaro@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-api [tools]
13:34 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component builds-api [tools]
13:23 <dcaro@cloudcumin1001> END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component builds-api [tools]
13:22 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component builds-api [tools]
13:04 <arturo> add container image to docker registry docker-registry.tools.wmflabs.org/tofu-provisioning:20250512 (T393686) [tools]
11:51 <dcaro@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component registry-admission [tools]
11:39 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component registry-admission [tools]
11:26 <dcaro@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component ingress-admission [tools]
11:14 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component ingress-admission [tools]
10:44 <dcaro@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-api [tools]
10:32 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component builds-api [tools]
10:00 <dcaro@cloudcumin1001> END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component builds-api [tools]
09:57 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component builds-api [tools]
09:29 <dcaro@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component envvars-api [tools]
09:17 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component envvars-api [tools]
08:59 <dcaro@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component volume-admission [tools]
08:47 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component volume-admission [tools]
02:36 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-19 [tools]
02:32 <andrew@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-19 [tools]
2025-05-10 §
17:35 <lucaswerkmeister> root@tools-bastion-13:~# systemctl restart sssd-sudo{,.socket} # looks like the reset-failed didn’t work properly, systemd didn’t even try to start the service again afaict (T393732) [tools]
17:33 <lucaswerkmeister> root@tools-bastion-13:~# systemctl reset-failed sssd-{pam,sudo}.service && systemctl restart sssd-pam{,-priv}.socket # try to reset the rate limits this way (T393732) [tools]
16:22 <lucaswerkmeister> systemctl restart sssd-{pam{,-priv},sudo}.socket # service-start-limit-hit, T393732? [tools]
14:10 <lucaswerkmeister> root@tools-bastion-13:~# systemctl restart sssd-sudo.socket # service-start-limit-hit, T393732? [tools]
11:53 <lucaswerkmeister> T393732 note: restart of sssd-pam.service actually failed, “may be requested by dependency only”; overall it still seems to have worked though (so next time restarting the sockets is probably sufficient) [tools]
11:52 <lucaswerkmeister> root@tools-bastion-13:~# systemctl restart sssd-pam{,{,-priv}.socket} # all three failed with start-limit-hit / Start request repeated too quickly; T393732? [tools]
2025-05-09 §
12:31 <arturo> hard-reboot tools-bastion-13 (login.toolforge.org) because unresponsive (out of memory) -- previous reboot was for tools-bastion-12 (dev.t.o) by mistake [tools]
12:29 <arturo> hard-reboot tools-bastion-12 (login.toolforge.org) because unresponsive (out of memory) [tools]
07:10 <taavi> kill bunch of unwanted processes off of tools-bastion-13 T393732, please run your things as jobs [tools]
2025-05-08 §
17:41 <dcaro@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-emailer [tools]
17:40 <raymond-ndibe@cloudcumin1001> END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component envvars-admission [tools]
17:40 <raymond-ndibe@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component envvars-admission [tools]
17:39 <raymond-ndibe@cloudcumin1001> END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component envvars-admission [tools]
17:39 <raymond-ndibe@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component envvars-admission [tools]
17:29 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component jobs-emailer [tools]
17:05 <dcaro@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component envvars-admission [tools]
16:54 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component envvars-admission [tools]
16:48 <raymond-ndibe@cloudcumin1001> END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component envvars-admission [tools]
16:47 <raymond-ndibe@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component envvars-admission [tools]
16:47 <raymond-ndibe@cloudcumin1001> END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component envvars-admission [tools]
16:46 <raymond-ndibe@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component envvars-admission [tools]
16:46 <raymond-ndibe@cloudcumin1001> END (ERROR) - Cookbook wmcs.toolforge.component.deploy (exit_code=97) for component envvars-admission [tools]
16:38 <raymond-ndibe@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component envvars-admission [tools]
13:26 <dcaro@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway [tools]
13:13 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component api-gateway [tools]
12:18 <dcaro@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api [tools]
12:06 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component jobs-api [tools]