101-150 of 5617 results (32ms)
2023-12-08 §
11:57 <wm-bot2> dcaro@urcuchillay END (ERROR) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=97) on etcd-discovery-1.cloudinfra-codfw1dev.codfw1dev.wikimedia.cloud [admin]
11:57 <wm-bot2> dcaro@urcuchillay START - Cookbook wmcs.vps.refresh_puppet_certs on etcd-discovery-1.cloudinfra-codfw1dev.codfw1dev.wikimedia.cloud [admin]
09:38 <wm-bot2> dcaro@urcuchillay END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) (T345084) [admin]
09:32 <dcaro> restarting nova and keystone as they are getting too slow (T345084) [admin]
09:32 <wm-bot2> dcaro@urcuchillay START - Cookbook wmcs.openstack.restart_openstack (T345084) [admin]
09:32 <wm-bot2> dcaro@urcuchillay END (FAIL) - Cookbook wmcs.openstack.restart_openstack (exit_code=99) (T345084) [admin]
09:31 <wm-bot2> dcaro@urcuchillay START - Cookbook wmcs.openstack.restart_openstack (T345084) [admin]
2023-12-07 §
13:12 <dcaro> rebooting cloudcephosd1001 to make sure puppet7 migration went ok [admin]
2023-12-04 §
00:07 <andrewbogott> rebooting cloudcontrol1006 to recover from full disk error [admin]
2023-12-03 §
09:05 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) [admin]
09:05 <taavi@cloudcumin1001> START - Cookbook wmcs.openstack.restart_openstack [admin]
2023-12-02 §
12:27 <taavi> powercycle cloudvirt1063 T352595 [admin]
11:28 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker role in the tools cluster [admin]
11:28 <taavi@cloudcumin1001> Added a new k8s worker tools-k8s-worker-97.tools.eqiad1.wikimedia.cloud to the cluster [admin]
11:02 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker role in the tools cluster [admin]
11:02 <taavi@cloudcumin1001> Added a new k8s worker tools-k8s-worker-96.tools.eqiad1.wikimedia.cloud to the cluster [admin]
10:50 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker role in the tools cluster [admin]
10:50 <taavi@cloudcumin1001> Added a new k8s worker tools-k8s-worker-95.tools.eqiad1.wikimedia.cloud to the cluster [admin]
00:21 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker role in the tools cluster [admin]
00:21 <taavi@cloudcumin1001> Added a new k8s worker tools-k8s-worker-94.tools.eqiad1.wikimedia.cloud to the cluster [admin]
00:18 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker role in the tools cluster [admin]
00:18 <taavi@cloudcumin1001> Added a new k8s worker tools-k8s-worker-93.tools.eqiad1.wikimedia.cloud to the cluster [admin]
00:15 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker role in the tools cluster [admin]
00:15 <taavi@cloudcumin1001> Added a new k8s worker tools-k8s-worker-92.tools.eqiad1.wikimedia.cloud to the cluster [admin]
00:06 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker role in the tools cluster [admin]
00:06 <taavi@cloudcumin1001> Added a new k8s worker tools-k8s-worker-91.tools.eqiad1.wikimedia.cloud to the cluster [admin]
00:05 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker role in the tools cluster [admin]
00:05 <taavi@cloudcumin1001> Added a new k8s worker tools-k8s-worker-90.tools.eqiad1.wikimedia.cloud to the cluster [admin]
00:01 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker role in the tools cluster [admin]
00:01 <taavi@cloudcumin1001> Added a new k8s worker tools-k8s-worker-89.tools.eqiad1.wikimedia.cloud to the cluster [admin]
2023-12-01 §
17:43 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) [admin]
17:37 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.restart_openstack [admin]
17:01 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) [admin]
16:57 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.restart_openstack [admin]
16:48 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) [admin]
16:46 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.restart_openstack [admin]
16:19 <wm-bot2> fran@wmf3169 END (PASS) - Cookbook wmcs.openstack.cloudvirt.unset_maintenance (exit_code=0) (T351171) [admin]
16:19 <wm-bot2> fran@wmf3169 START - Cookbook wmcs.openstack.cloudvirt.unset_maintenance (T351171) [admin]
15:48 <andrewbogott> reimaging cloudcontrol1005 due to widespread misbehavior [admin]
14:24 <wm-bot2> fran@wmf3169 END (PASS) - Cookbook wmcs.openstack.cloudvirt.unset_maintenance (exit_code=0) (T351171) [admin]
14:20 <wm-bot2> fran@wmf3169 START - Cookbook wmcs.openstack.cloudvirt.unset_maintenance (T351171) [admin]
14:19 <fnegri@cloudcumin1001> END (ERROR) - Cookbook wmcs.openstack.cloudvirt.unset_maintenance (exit_code=97) (T351171) [admin]
14:18 <fnegri@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.unset_maintenance (T351171) [admin]
13:55 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) [admin]
13:54 <taavi@cloudcumin1001> START - Cookbook wmcs.openstack.restart_openstack [admin]
11:57 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) [admin]
11:56 <taavi@cloudcumin1001> START - Cookbook wmcs.openstack.restart_openstack [admin]
11:55 <taavi> restart neutron-rpc-server.service on eqiad1 cloudcontrols [admin]
2023-11-30 §
20:44 <andrewbogott> generating to application credentials for the tests that run on tf-infra-test [admin]
19:54 <andrewbogott> reimaged cloudrabbit100[23] after https://gerrit.wikimedia.org/r/c/operations/puppet/+/979127. I didn't reimage 1001 because that will require rebuilding the whole cluster but I did remove the related packages. [admin]