51-100 of 160 results (31ms)
2024-04-03 §
11:27 <wmbot~taavi@runko> END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance mx-out04 [cloudinfra]
11:27 <wmbot~taavi@runko> START - Cookbook wmcs.vps.remove_instance for instance mx-out04 [cloudinfra]
2024-03-27 §
12:23 <taavi> hard reboot cloudinfra-cloudvps-puppetserver-1 [cloudinfra]
2024-03-25 §
18:34 <andrewbogott> moved mx records to point to service names (mx-out-a.wmcloud.org and mx-out-b.wmcloud.org) and assoviated those service names with new mx servers mx-out05 and mx-out06 [cloudinfra]
2024-02-21 §
14:08 <taavi> update cloudinfra-idp-1 to java 17 to fix CAS class loading errors on idp.wmcloud.org T357749 [cloudinfra]
2023-12-01 §
09:52 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.vps.remove_user_from_project (exit_code=0) for user 'jbond' (T352508) [cloudinfra]
09:52 <taavi@cloudcumin1001> START - Cookbook wmcs.vps.remove_user_from_project for user 'jbond' (T352508) [cloudinfra]
2023-09-29 §
09:26 <wm-bot2> dcaro@urcuchillay START - Cookbook wmcs.openstack.cloudvirt.vm_console [cloudinfra]
08:34 <wm-bot2> dcaro@urcuchillay END (PASS) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=0) [cloudinfra]
08:32 <wm-bot2> dcaro@urcuchillay START - Cookbook wmcs.openstack.cloudvirt.vm_console [cloudinfra]
08:31 <wm-bot2> dcaro@urcuchillay END (PASS) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=0) [cloudinfra]
08:30 <wm-bot2> dcaro@urcuchillay START - Cookbook wmcs.openstack.cloudvirt.vm_console [cloudinfra]
08:30 <wm-bot2> dcaro@urcuchillay END (PASS) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=0) [cloudinfra]
08:28 <wm-bot2> dcaro@urcuchillay START - Cookbook wmcs.openstack.cloudvirt.vm_console [cloudinfra]
08:28 <wm-bot2> dcaro@urcuchillay END (ERROR) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=255) [cloudinfra]
08:25 <wm-bot2> dcaro@urcuchillay START - Cookbook wmcs.openstack.cloudvirt.vm_console [cloudinfra]
08:25 <wm-bot2> dcaro@urcuchillay END (ERROR) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=97) [cloudinfra]
08:25 <wm-bot2> dcaro@urcuchillay START - Cookbook wmcs.openstack.cloudvirt.vm_console [cloudinfra]
08:13 <wm-bot2> dcaro@urcuchillay END (PASS) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=0) [cloudinfra]
08:08 <wm-bot2> dcaro@urcuchillay START - Cookbook wmcs.openstack.cloudvirt.vm_console [cloudinfra]
07:41 <wm-bot2> dcaro@urcuchillay END (PASS) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=0) [cloudinfra]
07:37 <wm-bot2> dcaro@urcuchillay START - Cookbook wmcs.openstack.cloudvirt.vm_console [cloudinfra]
2023-07-04 §
12:32 <taavi> configure DKIM signing for wmflabs.org and wmcloud.org on mx-out servers T208281 [cloudinfra]
2023-06-19 §
08:39 <arturo> update delegation for codfw1dev.wmcloud.org. to point to ns0/ns1 T307357 [cloudinfra]
2023-06-12 §
15:56 <arturo> live-hacking cloudinfra-internal-puppetmaster-02 with https://gerrit.wikimedia.org/r/c/operations/puppet/+/929357 [cloudinfra]
10:40 <arturo> update codfw1dev.wmcloud.org NS records to point to ns.openstack.wikimediacloud.org (T307357) [cloudinfra]
2023-06-08 §
08:05 <taavi> [codfw1dev] start mariadb.service on cloudinfra-db-01 [cloudinfra]
2023-06-01 §
13:27 <dhinus> cloud-cumin-04 successfully restored from backup (T332734) [cloudinfra]
13:09 <dhinus> shutting down cloud-cumin-04 and restoring from backup, to test backups are working (T332734) [cloudinfra]
2023-05-10 §
17:16 <wm-bot2> Increased quotas by 8 cores, 16384 ram (T336423) - cookbook ran by taavi@runko [cloudinfra]
2023-04-17 §
15:05 <taavi> add firewall rules allowing public (authenticated) access to the enc api - T317478 [cloudinfra]
2023-02-16 §
15:49 <arturo> aborrero@cloud-cumin-03:~$ sudo keyholder arm (password in pw) [cloudinfra]
2023-02-14 §
08:09 <taavi> arm keyholder on enc-2 T329589 [cloudinfra]
00:18 <bd808> enc-2.cloudinfra.eqiad1.wikimedia.cloud: `shutdown -r now` (T329589) [cloudinfra]
00:13 <bd808> enc-2.cloudinfra.eqiad1.wikimedia.cloud: `service puppet-enc-git-worker restart` (T329589) [cloudinfra]
2023-02-13 §
23:59 <bd808> enc-1.cloudinfra.eqiad1.wikimedia.cloud: `service uwsgi-puppet-enc restart` (T329589) [cloudinfra]
2023-01-03 §
15:17 <taavi> enc: create git update sql tables for https://gerrit.wikimedia.org/r/c/operations/puppet/+/871298 [cloudinfra]
2022-12-21 §
13:58 <taavi> re-enable puppet on cloud-cumin-03 [cloudinfra]
2022-10-14 §
14:07 <wm-bot2> Increased quotas by 1 instances - cookbook ran by dcaro@vulcanus [cloudinfra]
2022-07-20 §
07:03 <dcaro> manually running fsck through costole on puppetmaster-03 do to disk errors (T313380) [cloudinfra]
06:54 <dcaro> rebooting puppetmaster-03 do to disk errors [cloudinfra]
2022-02-16 §
14:44 <taavi> delete old cloudinfra-db instances (cloudinfra-db01 cloudinfra-db02) [cloudinfra]
12:10 <taavi> enable gtid (master_use_gtid=current_pos) on cloudinfra-db04 [cloudinfra]
2022-02-13 §
17:14 <taavi> encapi db switchover: cloudinfra-db01 -> cloudinfra-db03 [cloudinfra]
11:38 <taavi> generate new profile::mariadb::grants::cloudinfra::repl_pass, previous one seems to have been lost in the 2020-06-04 labs/private incident [cloudinfra]
10:48 <taavi> shutdown and delete ntp-01 and ntp-02 running stretch, replaced by -03 and -04 [cloudinfra]
2022-02-11 §
15:41 <taavi> switch floating ip 185.15.56.3 from ntp-01 to ntp-03 [cloudinfra]
15:18 <taavi> switch floating ip 185.15.56.27 from ntp-02 to ntp-04 [cloudinfra]
2022-02-09 §
18:33 <taavi> deleted cloud-cumin-01 T255980 [cloudinfra]
18:31 <taavi> deleted cloud-cumin-02 T255980 [cloudinfra]