7351-7400 of 10000 results (57ms)
2022-10-20 §
19:25 <wm-bot2> Safe reboot of 'cloudvirt1052.eqiad.wmnet' finished successfully. - cookbook ran by andrew@bullseye [admin]
19:25 <wm-bot2> Unset cloudvirt 'cloudvirt1052.eqiad.wmnet' maintenance. - cookbook ran by andrew@bullseye [admin]
19:21 <wm-bot2> Drained 'cloudvirt1052.eqiad.wmnet'. - cookbook ran by andrew@bullseye [admin]
19:04 <wm-bot2> Set cloudvirt 'cloudvirt1052.eqiad.wmnet' maintenance (downtime id: 18fae8d8-7353-4f67-90d7-8df9b3fb1ccb, use this to unset). - cookbook ran by andrew@bullseye [admin]
19:03 <wm-bot2> Draining 'cloudvirt1052.eqiad.wmnet'. - cookbook ran by andrew@bullseye [admin]
19:03 <wm-bot2> Safe rebooting 'cloudvirt1052.eqiad.wmnet'. - cookbook ran by andrew@bullseye [admin]
19:01 <wm-bot2> Set cloudvirt 'cloudvirt1053.eqiad.wmnet' maintenance (downtime id: 3d3cffa3-abc6-4901-83d9-3bcc4b02fd3c, use this to unset). - cookbook ran by andrew@bullseye [admin]
19:00 <wm-bot2> Draining 'cloudvirt1053.eqiad.wmnet'. - cookbook ran by andrew@bullseye [admin]
19:00 <wm-bot2> Safe rebooting 'cloudvirt1053.eqiad.wmnet'. - cookbook ran by andrew@bullseye [admin]
18:54 <wm-bot2> Draining 'cloudvirt1053.eqiad.wmnet'. - cookbook ran by andrew@bullseye [admin]
18:54 <wm-bot2> Safe rebooting 'cloudvirt1053.eqiad.wmnet'. - cookbook ran by andrew@bullseye [admin]
18:52 <wm-bot2> Draining 'cloudvirt1053.eqiad.wmnet'. - cookbook ran by andrew@bullseye [admin]
18:52 <wm-bot2> Safe rebooting 'cloudvirt1053.eqiad.wmnet'. - cookbook ran by andrew@bullseye [admin]
18:51 <wm-bot2> Draining 'cloudvirt1053.eqiad.wmnet'. - cookbook ran by andrew@bullseye [admin]
18:51 <wm-bot2> Safe rebooting 'cloudvirt1053.eqiad.wmnet'. - cookbook ran by andrew@bullseye [admin]
18:51 <wm-bot2> Draining 'cloudvirt1053.eqiad.wmnet'. - cookbook ran by andrew@bullseye [admin]
18:51 <wm-bot2> Safe rebooting 'cloudvirt1053.eqiad.wmnet'. - cookbook ran by andrew@bullseye [admin]
18:51 <wm-bot2> Draining 'cloudvirt1053.eqiad.wmnet'. - cookbook ran by andrew@bullseye [admin]
18:50 <wm-bot2> Safe rebooting 'cloudvirt1053.eqiad.wmnet'. - cookbook ran by andrew@bullseye [admin]
18:50 <wm-bot2> Draining 'cloudvirt1053.eqiad.wmnet'. - cookbook ran by andrew@bullseye [admin]
18:50 <wm-bot2> Safe rebooting 'cloudvirt1053.eqiad.wmnet'. - cookbook ran by andrew@bullseye [admin]
18:46 <wm-bot2> Draining 'cloudvirt1053.eqiad.wmnet'. - cookbook ran by andrew@bullseye [admin]
18:46 <wm-bot2> Safe rebooting 'cloudvirt1053.eqiad.wmnet'. - cookbook ran by andrew@bullseye [admin]
2022-10-15 §
17:38 <taavi> taavi@cloudweb1003 ~ $ mwscript extensions/OATHAuth/maintenance/disableOATHAuthForUser.php --wiki=labswiki Slevinski # T320867 [admin]
2022-10-13 §
12:19 <wm-bot2> OSDs (['cloudcephosd1027', 'cloudcephosd1028', 'cloudcephosd1029', 'cloudcephosd1030', 'cloudcephosd1031', 'cloudcephosd1032', 'cloudcephosd1033', 'cloudcephosd1034']) upgraded successfully B-) (T309786) - cookbook ran by dcaro@vulcanus [admin]
11:31 <wm-bot2> Upgrading OSDs and rebooting the nodes ['cloudcephosd1027', 'cloudcephosd1028', 'cloudcephosd1029', 'cloudcephosd1030', 'cloudcephosd1031', 'cloudcephosd1032', 'cloudcephosd1033', 'cloudcephosd1034'] (T309786) - cookbook ran by dcaro@vulcanus [admin]
11:30 <wm-bot2> OSDs (['cloudcephosd1025', 'cloudcephosd1026']) upgraded successfully B-) (T309786) - cookbook ran by dcaro@vulcanus [admin]
2022-10-10 §
14:01 <dcaro> test2 [admin]
01:22 <andrewbogott> restarting designate-sink on cloudservices100[45], possible example of T316614 [admin]
2022-10-09 §
12:04 <taavi> taavi@cloudweb1003 ~ $ mwscript extensions/OATHAuth/maintenance/disableOATHAuthForUser.php --wiki=labswiki DatGuy # T320301 [admin]
2022-10-07 §
13:40 <andrewbogott> dhinus is resetting rabbitmq cluster in an attempt to resolve a suspected (by Andrew) split-brain [admin]
11:33 <arturo> rabbitmq-server.service @ cloudrabbit1002 is again up and running (T320232) [admin]
10:24 <arturo> stopping rabbitmq-server.service @ cloudrabbit1002 (T320232) [admin]
10:19 <arturo> restarting nova-conductor in all 3 cloudcontrols (T320232) [admin]
09:45 <arturo> restarting rabbitmq-server.service @ cloudrabbit1002 (T320232) [admin]
2022-10-06 §
15:55 <arturo> cloudnet1005 & cloudnet1006 now in service. Secom cloudnet1003 & cloudnet1004. Drop neutron agents, etc. (T316284) [admin]
11:54 <arturo> rebooting cloudnet1005/1006 to see if they have the right network config (T316284) [admin]
11:50 <arturo> set neutron l3 agents on cloudnet1005/1006 as down `root@cloudcontrol1005:~# neutron agent-update --admin-state-down <uuid>` (T316284) [admin]
11:40 <arturo> [codfw1dev] rebooting both network nodes to test https://gerrit.wikimedia.org/r/c/operations/puppet/+/839492 [admin]
10:14 <arturo> [codfw1dev] restart neutron-l3-agent on cloudnet2006-dev, it was dead [admin]
2022-10-05 §
14:40 <wm-bot2> Adding OSD cloudcephosd1021.eqiad.wmnet... (1/1) (T319418) - cookbook ran by fran@wmf3169 [admin]
14:40 <wm-bot2> Adding new OSDs ['cloudcephosd1021.eqiad.wmnet'] to the cluster (T319418) - cookbook ran by fran@wmf3169 [admin]
14:28 <arturo> adding cloudinstances2b-gw router to l3 agents on cloudnet1005/1006 (T316284) [admin]
13:11 <wm-bot2> Added 1 new OSDs ['cloudcephosd1034.eqiad.wmnet'] (T314870) - cookbook ran by fran@wmf3169 [admin]
13:11 <wm-bot2> Added OSD cloudcephosd1034.eqiad.wmnet... (1/1) (T314870) - cookbook ran by fran@wmf3169 [admin]
13:02 <wm-bot2> Finished rebooting node cloudcephosd1034.eqiad.wmnet (T314870) - cookbook ran by fran@wmf3169 [admin]
12:58 <wm-bot2> Rebooting node cloudcephosd1034.eqiad.wmnet (T314870) - cookbook ran by fran@wmf3169 [admin]
12:58 <wm-bot2> Adding OSD cloudcephosd1034.eqiad.wmnet... (1/1) (T314870) - cookbook ran by fran@wmf3169 [admin]
12:58 <wm-bot2> Adding new OSDs ['cloudcephosd1034.eqiad.wmnet'] to the cluster (T314870) - cookbook ran by fran@wmf3169 [admin]
2022-10-04 §
16:40 <wm-bot2> Added 1 new OSDs ['cloudcephosd1033.eqiad.wmnet'] (T314870) - cookbook ran by fran@wmf3169 [admin]