1501-1550 of 10000 results (17ms)
2025-03-10 §
07:38 <jmm@cumin2002> DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Jdcc-berkman out of all services on: 1284 hosts [production]
07:37 <jmm@cumin2002> DONE (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Jdcc-berkman out of all services on: 961 hosts [production]
02:30 <wmbot~anticomposite@tools-bastion-13> stewardbots/StewardBot/manage.sh restart # replaying old EventStreams [tools.stewardbots]
2025-03-09 §
20:01 <Daniuu> Restarted stewardbot to recapture IRC event stream [tools.stewardbots]
10:13 <elukey@puppetserver1001> conftool action : set/weight=5; selector: name=wikikube-worker2.*,dc=codfw,cluster=maps,service=kartotherian-k8s-ssl [production]
10:12 <elukey@puppetserver1001> conftool action : set/weight=5; selector: name=wikikube-worker1.*,dc=eqiad,cluster=maps,service=kartotherian-k8s-ssl [production]
10:12 <elukey@puppetserver1001> conftool action : set/pooled=yes; selector: name=maps2005.codfw.wmnet,dc=codfw,cluster=maps,service=kartotherian-k8s-ssl [production]
10:12 <elukey@puppetserver1001> conftool action : set/pooled=yes; selector: name=maps1005.eqiad.wmnet,dc=eqiad,cluster=maps,service=kartotherian-k8s-ssl [production]
04:52 <wmbot~anticomposite@tools-bastion-13> stewardbots/StewardBot/manage.sh restart # Not reading RC after restart from IRC [tools.stewardbots]
2025-03-08 §
11:33 <moritzm> truncated /var/log/syslog on seaborgium and bounced slapd [production]
01:40 <wmbot~anticomposite@tools-bastion-13> Attempt to mitigate EventStreams replay issues, leaving uncommitted changes to test for now [tools.stewardbots]
01:22 <James_F> Zuul: [php-session-serializer] Enable PHP 8.4 as voting, for T368270 [releng]
00:34 <tzatziki> removing 3 files for legal compliance [production]
2025-03-07 §
22:42 <inflatador> bking@cloudelastic1009 exclude `cloudelastic1010` from master voting T387904 [production]
22:17 <ryankemper> [Cloudelastic] Doing a `/_cluster/reroute?retry_failed=true` of all 3 elastic/opensearch clusters [production]
22:02 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudelastic1009.eqiad.wmnet with OS bullseye [production]
22:02 <bking@cumin2002> END (PASS) - Cookbook sre.elasticsearch.ban (exit_code=0) Unbanning all hosts in cloudelastic [production]
22:01 <bking@cumin2002> START - Cookbook sre.elasticsearch.ban Unbanning all hosts in cloudelastic [production]
21:36 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudelastic1009.eqiad.wmnet with reason: host reimage [production]
21:32 <bking@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on cloudelastic1009.eqiad.wmnet with reason: host reimage [production]
21:13 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host cloudelastic1009.eqiad.wmnet with OS bullseye [production]
21:12 <bking@cumin2002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host cloudelastic1009.eqiad.wmnet with OS bullseye [production]
21:00 <James_F> Zuul: [mediawiki/libs/Shellbox] Enable PHP 8.4 as voting, for T386570 [releng]
20:58 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host cloudelastic1009.eqiad.wmnet with OS bullseye [production]
20:53 <James_F> Zuul: [wikipeg] Enable PHP 8.4 as voting, for T386570 [releng]
20:49 <bking@cumin2002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=1) upgrade firmware for hosts ['cloudelastic1009.eqiad.wmnet'] [production]
20:44 <bking@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['cloudelastic1009.eqiad.wmnet'] [production]
20:42 <bking@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['cloudelastic1009.eqiad.wmnet'] [production]
20:32 <bking@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['cloudelastic1009.eqiad.wmnet'] [production]
20:32 <bking@cumin2002> END (ERROR) - Cookbook sre.hardware.upgrade-firmware (exit_code=97) upgrade firmware for hosts ['cloudelastic1009.eqiad.wmnet'] [production]
20:32 <bking@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['cloudelastic1009.eqiad.wmnet'] [production]
20:17 <bking@cumin2002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts ['cloudelastic1009.eqiad.wmnet'] [production]
20:17 <bking@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['cloudelastic1009.eqiad.wmnet'] [production]
20:17 <bking@cumin2002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=1) upgrade firmware for hosts ['cloudelastic1009.eqiad.wmnet'] [production]
20:16 <bking@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['cloudelastic1009.eqiad.wmnet'] [production]
20:16 <bking@cumin2002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts ['cloudelastic1009.eqiad.wmnet'] [production]
20:08 <bking@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['cloudelastic1009.eqiad.wmnet'] [production]
20:07 <James_F> Zuul: [mediawiki/libs/Equivset] Enable PHP 8.4 as voting, for T387806 [releng]
20:06 <bking@cumin2002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host cloudelastic1009.eqiad.wmnet with OS bullseye [production]
19:53 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host cloudelastic1009.eqiad.wmnet with OS bullseye [production]
19:53 <bking@cumin2002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host cloudelastic1009.eqiad.wmnet with OS bullseye [production]
19:28 <wmbot~lucaswerkmeister@tools-bastion-13> deployed 5ced6327b8 (l10n updates: nl) [tools.ranker]
19:27 <wmbot~lucaswerkmeister@tools-bastion-13> deployed b8022cddca (l10n updates: tr) [tools.wd-image-positions]
19:20 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host cloudelastic1009.eqiad.wmnet with OS bullseye [production]
17:38 <cmooney@cumin1002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
17:38 <cmooney@cumin1002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: dns additions for eqiad E8/F8 links to new switches - cmooney@cumin1002" [production]
17:38 <cmooney@cumin1002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: dns additions for eqiad E8/F8 links to new switches - cmooney@cumin1002" [production]
17:32 <cmooney@cumin1002> START - Cookbook sre.dns.netbox [production]
17:26 <jclark@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1202.eqiad.wmnet with OS bullseye [production]
17:26 <jclark@cumin1002> END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1002" [production]