1-50 of 10000 results (14ms)
2026-11-04 ยง
09:31 <marostegui@cumin1004> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host dbprov2006.codfw.wmnet with OS trixie [production]
09:21 <brouberol@deploy2003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-growthbook: apply [production]
09:21 <brouberol@deploy2003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-growthbook: apply [production]
09:14 <brouberol@deploy2003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-growthbook: apply [production]
09:14 <brouberol@deploy2003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-growthbook: apply [production]
09:13 <jnuche@deploy2003> rebuilt and synchronized wikiversions files: group1 to 1.48.0-wmf.3 refs T442221 [production]
09:12 <marostegui@cumin1004> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on dbprov2006.codfw.wmnet with reason: host reimage [production]
09:08 <cwilliams@cumin1004> START - Cookbook sre.mysql.pool pool es2036: Maintenance [production]
09:07 <marostegui@cumin1004> START - Cookbook sre.hosts.downtime for 2:00:00 on dbprov2006.codfw.wmnet with reason: host reimage [production]
09:07 <arnaudb@cumin1004> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3:00:00 on gerrit2002.wikimedia.org with reason: T440177 offline reindex test [production]
09:05 <marostegui@cumin1004> END (PASS) - Cookbook sre.hosts.dhcp (exit_code=0) for host dbprov2006.codfw.wmnet (remove config only) [production]
09:05 <marostegui@cumin1004> START - Cookbook sre.hosts.dhcp for host dbprov2006.codfw.wmnet (remove config only) [production]
08:51 <marostegui@cumin1004> END (PASS) - Cookbook sre.hosts.dhcp (exit_code=0) for host dbprov2006.codfw.wmnet (push config only) [production]
08:51 <marostegui@cumin1004> START - Cookbook sre.hosts.dhcp for host dbprov2006.codfw.wmnet (push config only) [production]
08:50 <marostegui@cumin1004> START - Cookbook sre.hosts.reimage for host dbprov2006.codfw.wmnet with OS trixie [production]
08:15 <Tran> Deployed patch for T441775 [production]
08:11 <marostegui@cumin1004> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host dbprov2005.codfw.wmnet with OS trixie [production]
08:10 <jmm@cumin2003> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast3007.wikimedia.org [production]
08:06 <jmm@cumin2003> START - Cookbook sre.hosts.reboot-single for host bast3007.wikimedia.org [production]
08:00 <marostegui@cumin1004> END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1274: after reboot [production]
07:53 <marostegui@cumin1004> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on dbprov2005.codfw.wmnet with reason: host reimage [production]
07:47 <marostegui@cumin1004> START - Cookbook sre.hosts.downtime for 2:00:00 on dbprov2005.codfw.wmnet with reason: host reimage [production]
07:45 <marostegui@cumin1004> END (PASS) - Cookbook sre.hosts.dhcp (exit_code=0) for host dbprov2005.codfw.wmnet (remove config only) [production]
07:45 <marostegui@cumin1004> START - Cookbook sre.hosts.dhcp for host dbprov2005.codfw.wmnet (remove config only) [production]
07:31 <marostegui@cumin1004> END (PASS) - Cookbook sre.hosts.dhcp (exit_code=0) for host dbprov2005.codfw.wmnet (push config only) [production]
07:31 <marostegui@cumin1004> START - Cookbook sre.hosts.dhcp for host dbprov2005.codfw.wmnet (push config only) [production]
07:30 <marostegui@cumin1004> START - Cookbook sre.hosts.reimage for host dbprov2005.codfw.wmnet with OS trixie [production]
07:15 <marostegui@cumin1004> START - Cookbook sre.mysql.pool pool db1274: after reboot [production]
07:06 <ladsgroup@cumin1004> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on dbstore1008.eqiad.wmnet with reason: Maintenance [production]
07:06 <ladsgroup@cumin1004> dbctl commit (dc=all): 'Repooling after maintenance db1288 (T441633)', diff saved to https://phabricator.wikimedia.org/P97242 and previous config saved to /var/cache/conftool/dbconfig/20261104-070600-ladsgroup.json [production]
06:55 <ladsgroup@cumin1004> dbctl commit (dc=all): 'Repooling after maintenance db1288', diff saved to https://phabricator.wikimedia.org/P97241 and previous config saved to /var/cache/conftool/dbconfig/20261104-065558-ladsgroup.json [production]
06:55 <marostegui@cumin1004> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1274.eqiad.wmnet with reason: Server didn't come back [production]
06:45 <ladsgroup@cumin1004> dbctl commit (dc=all): 'Repooling after maintenance db1288', diff saved to https://phabricator.wikimedia.org/P97240 and previous config saved to /var/cache/conftool/dbconfig/20261104-064556-ladsgroup.json [production]
06:35 <ladsgroup@cumin1004> dbctl commit (dc=all): 'Repooling after maintenance db1288 (T441633)', diff saved to https://phabricator.wikimedia.org/P97239 and previous config saved to /var/cache/conftool/dbconfig/20261104-063554-ladsgroup.json [production]
06:30 <ladsgroup@cumin1004> dbctl commit (dc=all): 'Depooling db1288 (T441633)', diff saved to https://phabricator.wikimedia.org/P97238 and previous config saved to /var/cache/conftool/dbconfig/20261104-063055-ladsgroup.json [production]
06:30 <ladsgroup@cumin1004> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1288.eqiad.wmnet with reason: Maintenance [production]
06:27 <ladsgroup@cumin1004> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1285.eqiad.wmnet with reason: Maintenance [production]
06:27 <ladsgroup@cumin1004> dbctl commit (dc=all): 'Repooling after maintenance db1284 (T441633)', diff saved to https://phabricator.wikimedia.org/P97237 and previous config saved to /var/cache/conftool/dbconfig/20261104-062712-ladsgroup.json [production]
06:17 <ladsgroup@cumin1004> dbctl commit (dc=all): 'Repooling after maintenance db1284', diff saved to https://phabricator.wikimedia.org/P97236 and previous config saved to /var/cache/conftool/dbconfig/20261104-061710-ladsgroup.json [production]
06:07 <ladsgroup@cumin1004> dbctl commit (dc=all): 'Repooling after maintenance db1284', diff saved to https://phabricator.wikimedia.org/P97235 and previous config saved to /var/cache/conftool/dbconfig/20261104-060708-ladsgroup.json [production]
05:57 <ladsgroup@cumin1004> dbctl commit (dc=all): 'Repooling after maintenance db1284 (T441633)', diff saved to https://phabricator.wikimedia.org/P97234 and previous config saved to /var/cache/conftool/dbconfig/20261104-055706-ladsgroup.json [production]
05:52 <ladsgroup@cumin1004> dbctl commit (dc=all): 'Depooling db1284 (T441633)', diff saved to https://phabricator.wikimedia.org/P97233 and previous config saved to /var/cache/conftool/dbconfig/20261104-055201-ladsgroup.json [production]
05:52 <ladsgroup@cumin1004> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1284.eqiad.wmnet with reason: Maintenance [production]
05:51 <ladsgroup@cumin1004> dbctl commit (dc=all): 'Repooling after maintenance db1273 (T441633)', diff saved to https://phabricator.wikimedia.org/P97232 and previous config saved to /var/cache/conftool/dbconfig/20261104-055141-ladsgroup.json [production]
05:41 <ladsgroup@cumin1004> dbctl commit (dc=all): 'Repooling after maintenance db1273', diff saved to https://phabricator.wikimedia.org/P97231 and previous config saved to /var/cache/conftool/dbconfig/20261104-054139-ladsgroup.json [production]
05:31 <ladsgroup@cumin1004> dbctl commit (dc=all): 'Repooling after maintenance db1273', diff saved to https://phabricator.wikimedia.org/P97230 and previous config saved to /var/cache/conftool/dbconfig/20261104-053137-ladsgroup.json [production]
05:21 <ladsgroup@cumin1004> dbctl commit (dc=all): 'Repooling after maintenance db1273 (T441633)', diff saved to https://phabricator.wikimedia.org/P97229 and previous config saved to /var/cache/conftool/dbconfig/20261104-052135-ladsgroup.json [production]
05:14 <ladsgroup@cumin1004> dbctl commit (dc=all): 'Depooling db1273 (T441633)', diff saved to https://phabricator.wikimedia.org/P97228 and previous config saved to /var/cache/conftool/dbconfig/20261104-051423-ladsgroup.json [production]
05:14 <ladsgroup@cumin1004> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on 7 hosts with reason: Maintenance [production]
05:14 <ladsgroup@cumin1004> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1273.eqiad.wmnet with reason: Maintenance [production]