|
2026-06-09
§
|
| 05:37 |
<marostegui@dns1004> |
START - running authdns-update |
[production] |
| 05:27 |
<marostegui@cumin1003> |
END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1237: Upgrading db1237.eqiad.wmnet |
[production] |
| 05:27 |
<marostegui@cumin1003> |
START - Cookbook sre.mysql.depool depool db1237: Upgrading db1237.eqiad.wmnet |
[production] |
| 05:27 |
<marostegui@cumin1003> |
START - Cookbook sre.mysql.major-upgrade |
[production] |
| 05:24 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Depool db1237 T428158', diff saved to https://phabricator.wikimedia.org/P93935 and previous config saved to /var/cache/conftool/dbconfig/20260609-052420-marostegui.json |
[production] |
| 05:23 |
<marostegui@dns1004> |
START - running authdns-update |
[production] |
| 05:23 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Promote db1220 to x1 primary and set section read-write T428158', diff saved to https://phabricator.wikimedia.org/P93934 and previous config saved to /var/cache/conftool/dbconfig/20260609-052311-marostegui.json |
[production] |
| 05:22 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Set x1 eqiad as read-only for maintenance - T428158', diff saved to https://phabricator.wikimedia.org/P93933 and previous config saved to /var/cache/conftool/dbconfig/20260609-052253-marostegui.json |
[production] |
| 05:22 |
<marostegui> |
Starting x1 eqiad failover from db1237 to db1220 - T428158 |
[production] |
| 05:19 |
<marostegui@cumin1003> |
dbctl commit (dc=all): 'Set db1220 with weight 0 T428158', diff saved to https://phabricator.wikimedia.org/P93932 and previous config saved to /var/cache/conftool/dbconfig/20260609-051859-marostegui.json |
[production] |
| 05:18 |
<marostegui@cumin1003> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 16 hosts with reason: Primary switchover x1 T428158 |
[production] |
| 04:02 |
<mwpresync@deploy1003> |
Pruned MediaWiki: 1.47.0-wmf.3 (duration: 02m 43s) |
[production] |
| 03:40 |
<mwpresync@deploy1003> |
Finished scap sync-world: testwikis to 1.47.0-wmf.6 refs T423915 (duration: 37m 16s) |
[production] |
| 03:03 |
<mwpresync@deploy1003> |
Started scap sync-world: testwikis to 1.47.0-wmf.6 refs T423915 |
[production] |
| 02:08 |
<mwpresync@deploy1003> |
Finished scap build-images: Publishing wmf/next image (duration: 06m 38s) |
[production] |
| 02:01 |
<mwpresync@deploy1003> |
Started scap build-images: Publishing wmf/next image |
[production] |
|
2026-06-08
§
|
| 22:00 |
<reedy@deploy1003> |
Finished scap sync-world: Backport for [[gerrit:1298915|CommonSettings: Set $wgScoreSafeMode = false (T428484)]] (duration: 07m 42s) |
[production] |
| 21:56 |
<reedy@deploy1003> |
reedy: Continuing with deployment |
[production] |
| 21:54 |
<reedy@deploy1003> |
reedy: Backport for [[gerrit:1298915|CommonSettings: Set $wgScoreSafeMode = false (T428484)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 21:52 |
<reedy@deploy1003> |
Started scap sync-world: Backport for [[gerrit:1298915|CommonSettings: Set $wgScoreSafeMode = false (T428484)]] |
[production] |
| 21:12 |
<mlitn@deploy1003> |
Finished scap sync-world: Backport for [[gerrit:1298891|OOUIHTMLForm: Avoid treating form header as a clickable label (T428359)]] (duration: 08m 10s) |
[production] |
| 21:07 |
<mlitn@deploy1003> |
mlitn, neriah: Continuing with deployment |
[production] |
| 21:05 |
<mlitn@deploy1003> |
mlitn, neriah: Backport for [[gerrit:1298891|OOUIHTMLForm: Avoid treating form header as a clickable label (T428359)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 21:03 |
<mlitn@deploy1003> |
Started scap sync-world: Backport for [[gerrit:1298891|OOUIHTMLForm: Avoid treating form header as a clickable label (T428359)]] |
[production] |
| 20:43 |
<mlitn@deploy1003> |
Finished scap sync-world: Backport for [[gerrit:1297162|MultimediaViewer: enable image carousel as a beta feature on Wikipedias]], [[gerrit:1298841|Squashed diff to master]] (duration: 07m 05s) |
[production] |
| 20:39 |
<mlitn@deploy1003> |
mlitn: Continuing with deployment |
[production] |
| 20:38 |
<mlitn@deploy1003> |
mlitn: Backport for [[gerrit:1297162|MultimediaViewer: enable image carousel as a beta feature on Wikipedias]], [[gerrit:1298841|Squashed diff to master]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 20:36 |
<mlitn@deploy1003> |
Started scap sync-world: Backport for [[gerrit:1297162|MultimediaViewer: enable image carousel as a beta feature on Wikipedias]], [[gerrit:1298841|Squashed diff to master]] |
[production] |
| 20:29 |
<mlitn@deploy1003> |
Finished scap sync-world: Backport for [[gerrit:1298390|English Wikibooks: update FlaggedRevs configuration (T428329)]], [[gerrit:1298328|English Wikiversity: Add new user group "autopatrolled" (T428269)]] (duration: 08m 58s) |
[production] |
| 20:25 |
<mlitn@deploy1003> |
mlitn, vadymts1: Continuing with deployment |
[production] |
| 20:22 |
<mlitn@deploy1003> |
mlitn, vadymts1: Backport for [[gerrit:1298390|English Wikibooks: update FlaggedRevs configuration (T428329)]], [[gerrit:1298328|English Wikiversity: Add new user group "autopatrolled" (T428269)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 20:20 |
<mlitn@deploy1003> |
Started scap sync-world: Backport for [[gerrit:1298390|English Wikibooks: update FlaggedRevs configuration (T428329)]], [[gerrit:1298328|English Wikiversity: Add new user group "autopatrolled" (T428269)]] |
[production] |
| 20:03 |
<kharlan@deploy1003> |
Finished scap sync-world: Backport for [[gerrit:1298879|SimpleCaptcha: Re-render captcha when edit form is redisplayed (T428437)]] (duration: 37m 43s) |
[production] |
| 19:43 |
<jhancock@cumin2002> |
END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host dse-k8s-wdqs2004.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART |
[production] |
| 19:43 |
<jhancock@cumin2002> |
START - Cookbook sre.hosts.provision for host dse-k8s-wdqs2004.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART |
[production] |
| 19:31 |
<kharlan@deploy1003> |
kharlan: Continuing with deployment |
[production] |
| 19:30 |
<jhancock@cumin2002> |
END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host dse-k8s-wdqs2002.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART |
[production] |
| 19:30 |
<jhancock@cumin2002> |
START - Cookbook sre.hosts.provision for host dse-k8s-wdqs2002.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART |
[production] |
| 19:29 |
<kharlan@deploy1003> |
kharlan: Backport for [[gerrit:1298879|SimpleCaptcha: Re-render captcha when edit form is redisplayed (T428437)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 19:28 |
<jhancock@cumin2002> |
END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host dse-k8s-wdqs2003.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART |
[production] |
| 19:27 |
<jhancock@cumin2002> |
START - Cookbook sre.hosts.provision for host dse-k8s-wdqs2003.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART |
[production] |
| 19:25 |
<kharlan@deploy1003> |
Started scap sync-world: Backport for [[gerrit:1298879|SimpleCaptcha: Re-render captcha when edit form is redisplayed (T428437)]] |
[production] |
| 19:24 |
<aokoth@deploy1003> |
Finished deploy [phabricator/deployment@939557b]: deploy phab (duration: 01m 32s) |
[production] |
| 19:23 |
<jhancock@cumin2002> |
END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host dse-k8s-wdqs2002.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART |
[production] |
| 19:22 |
<aokoth@deploy1003> |
Started deploy [phabricator/deployment@939557b]: deploy phab |
[production] |
| 19:20 |
<aokoth@deploy1003> |
Finished deploy [phabricator/deployment@939557b]: deploy phab (duration: 01m 40s) |
[production] |
| 19:19 |
<aokoth@deploy1003> |
Started deploy [phabricator/deployment@939557b]: deploy phab |
[production] |
| 19:16 |
<jhancock@cumin2002> |
START - Cookbook sre.hosts.provision for host dse-k8s-wdqs2002.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART |
[production] |
| 19:14 |
<jhancock@cumin2002> |
END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host dse-k8s-wdqs2002.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART |
[production] |
| 19:07 |
<jhancock@cumin2002> |
START - Cookbook sre.hosts.provision for host dse-k8s-wdqs2002.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART |
[production] |