|
2025-05-04
§
|
| 23:27 |
<andrew@cumin1002> |
END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts cloudcephosd1003.eqiad.wmnet |
[production] |
| 23:27 |
<andrew@cumin1002> |
END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: cloudcephosd1003.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - andrew@cumin1002" |
[production] |
| 23:27 |
<andrew@cumin1002> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 23:27 |
<andrew@cumin1002> |
START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: cloudcephosd1003.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - andrew@cumin1002" |
[production] |
| 23:22 |
<andrew@cumin1002> |
START - Cookbook sre.dns.netbox |
[production] |
| 23:16 |
<andrew@cumin1002> |
START - Cookbook sre.hosts.decommission for hosts cloudcephosd1003.eqiad.wmnet |
[production] |
| 23:15 |
<andrew@cumin1002> |
END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts cloudcephosd1002.eqiad.wmnet |
[production] |
| 23:15 |
<andrew@cumin1002> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 23:15 |
<andrew@cumin1002> |
END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: cloudcephosd1002.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - andrew@cumin1002" |
[production] |
| 23:14 |
<andrew@cumin1002> |
START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: cloudcephosd1002.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - andrew@cumin1002" |
[production] |
| 23:08 |
<andrew@cumin1002> |
START - Cookbook sre.dns.netbox |
[production] |
| 23:02 |
<andrew@cumin1002> |
START - Cookbook sre.hosts.decommission for hosts cloudcephosd1002.eqiad.wmnet |
[production] |
| 23:02 |
<andrew@cumin1002> |
END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts cloudcephosd1001.eqiad.wmnet |
[production] |
| 23:02 |
<andrew@cumin1002> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 23:02 |
<andrew@cumin1002> |
END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: cloudcephosd1001.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - andrew@cumin1002" |
[production] |
| 23:01 |
<andrew@cumin1002> |
START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: cloudcephosd1001.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - andrew@cumin1002" |
[production] |
| 22:57 |
<andrew@cumin1002> |
START - Cookbook sre.dns.netbox |
[production] |
| 22:52 |
<andrew@cumin1002> |
START - Cookbook sre.hosts.decommission for hosts cloudcephosd1001.eqiad.wmnet |
[production] |
| 20:29 |
<bking@cumin2002> |
END (PASS) - Cookbook sre.elasticsearch.ban (exit_code=0) Unbanning all hosts in search_eqiad |
[production] |
| 20:29 |
<bking@cumin2002> |
START - Cookbook sre.elasticsearch.ban Unbanning all hosts in search_eqiad |
[production] |
| 20:07 |
<bking@cumin2002> |
END (PASS) - Cookbook sre.elasticsearch.ban (exit_code=0) Banning hosts: elastic1056*,elastic1063* for host appears to have hot shards - bking@cumin2002 |
[production] |
| 20:06 |
<bking@cumin2002> |
START - Cookbook sre.elasticsearch.ban Banning hosts: elastic1056*,elastic1063* for host appears to have hot shards - bking@cumin2002 |
[production] |
| 19:43 |
<bking@cumin2002> |
END (PASS) - Cookbook sre.elasticsearch.ban (exit_code=0) Banning hosts: elastic1063* for host appears to have hot shards - bking@cumin2002 |
[production] |
| 19:43 |
<bking@cumin2002> |
START - Cookbook sre.elasticsearch.ban Banning hosts: elastic1063* for host appears to have hot shards - bking@cumin2002 |
[production] |
| 19:35 |
<bking@cumin2002> |
END (PASS) - Cookbook sre.elasticsearch.ban (exit_code=0) Banning hosts: elastic1062* for hosts appear to have hot shards - bking@cumin2002 |
[production] |
| 19:35 |
<bking@cumin2002> |
START - Cookbook sre.elasticsearch.ban Banning hosts: elastic1062* for hosts appear to have hot shards - bking@cumin2002 |
[production] |
| 19:10 |
<bking@cumin2002> |
END (PASS) - Cookbook sre.elasticsearch.ban (exit_code=0) Banning hosts: elastic1057*,elastic1058* for hosts appear to have hot shards - bking@cumin2002 |
[production] |
| 19:10 |
<bking@cumin2002> |
START - Cookbook sre.elasticsearch.ban Banning hosts: elastic1057*,elastic1058* for hosts appear to have hot shards - bking@cumin2002 |
[production] |
| 19:04 |
<bking@cumin2002> |
END (PASS) - Cookbook sre.elasticsearch.ban (exit_code=0) Banning hosts: elastic1057* for host appears to have hot shards - bking@cumin2002 |
[production] |
| 19:04 |
<bking@cumin2002> |
START - Cookbook sre.elasticsearch.ban Banning hosts: elastic1057* for host appears to have hot shards - bking@cumin2002 |
[production] |
| 19:04 |
<bking@cumin2002> |
END (PASS) - Cookbook sre.elasticsearch.ban (exit_code=0) Banning hosts: elastic1064* for host appears to have hot shards - bking@cumin2002 |
[production] |
| 19:03 |
<bking@cumin2002> |
START - Cookbook sre.elasticsearch.ban Banning hosts: elastic1064* for host appears to have hot shards - bking@cumin2002 |
[production] |
| 10:36 |
<krinkle@deploy1003> |
Finished scap sync-world: Backport for [[gerrit:1141460|actions: Fix handling of redirects to known (non-existing) pages]] (duration: 30m 22s) |
[production] |
| 10:26 |
<krinkle@deploy1003> |
krinkle: Continuing with sync |
[production] |
| 10:22 |
<krinkle@deploy1003> |
krinkle: Backport for [[gerrit:1141460|actions: Fix handling of redirects to known (non-existing) pages]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 10:06 |
<krinkle@deploy1003> |
Started scap sync-world: Backport for [[gerrit:1141460|actions: Fix handling of redirects to known (non-existing) pages]] |
[production] |
|
2025-05-03
§
|
| 20:09 |
<taavi@cumin1002> |
dbctl commit (dc=all): 'depool db1246', diff saved to https://phabricator.wikimedia.org/P75739 and previous config saved to /var/cache/conftool/dbconfig/20250503-200910-taavi.json |
[production] |
| 18:35 |
<hnowlan> |
delete a stuck thumbor pod in codfw |
[production] |
| 13:53 |
<krinkle@deploy1003> |
Finished scap sync-world: Backport for [[gerrit:1140653|multiversion: Remove getMWConfigForCacheing() as identical to getConfigGlobals() (T169821)]], [[gerrit:1140658|tests: Move buildLogoHTML.php to tests/ alongside buildConfigCache.php]], [[gerrit:1140707|multiversion: Separate wmf-config reading from actual Multiversion (T169821)]] (duration: 16m 22s) |
[production] |
| 13:46 |
<krinkle@deploy1003> |
krinkle: Continuing with sync |
[production] |
| 13:41 |
<krinkle@deploy1003> |
krinkle: Backport for [[gerrit:1140653|multiversion: Remove getMWConfigForCacheing() as identical to getConfigGlobals() (T169821)]], [[gerrit:1140658|tests: Move buildLogoHTML.php to tests/ alongside buildConfigCache.php]], [[gerrit:1140707|multiversion: Separate wmf-config reading from actual Multiversion (T169821)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 13:36 |
<krinkle@deploy1003> |
Started scap sync-world: Backport for [[gerrit:1140653|multiversion: Remove getMWConfigForCacheing() as identical to getConfigGlobals() (T169821)]], [[gerrit:1140658|tests: Move buildLogoHTML.php to tests/ alongside buildConfigCache.php]], [[gerrit:1140707|multiversion: Separate wmf-config reading from actual Multiversion (T169821)]] |
[production] |
| 12:19 |
<reedy@deploy1003> |
Synchronized wmf-config/InitialiseSettings-labs.php: Allow all users to use 2FA on beta (duration: 11m 14s) |
[production] |