351-400 of 10000 results (116ms)
2026-06-23 §
04:02 <mwpresync@deploy1003> Pruned MediaWiki: 1.47.0-wmf.5 (duration: 02m 37s) [production]
03:44 <mwpresync@deploy1003> Finished scap sync-world: testwikis to 1.47.0-wmf.8 refs T423917 (duration: 39m 23s) [production]
03:08 <mwpresync@deploy1003> Started scap sync-world: testwikis to 1.47.0-wmf.8 refs T423917 [production]
02:08 <mwpresync@deploy1003> Finished scap build-images: Publishing wmf/next image (duration: 07m 25s) [production]
02:00 <mwpresync@deploy1003> Started scap build-images: Publishing wmf/next image [production]
01:41 <egardner@deploy1003> Finished scap sync-world: Backport for [[gerrit:1304859|Remove multimediaviewer-beta]] (duration: 11m 14s) [production]
01:34 <egardner@deploy1003> egardner, ksarabia: Continuing with deployment [production]
01:34 <egardner@deploy1003> egardner, ksarabia: Backport for [[gerrit:1304859|Remove multimediaviewer-beta]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
01:30 <egardner@deploy1003> Started scap sync-world: Backport for [[gerrit:1304859|Remove multimediaviewer-beta]] [production]
01:24 <egardner@deploy1003> Finished scap sync-world: Backport for [[gerrit:1304914|Inject service RepoGroup into Hooks]], [[gerrit:1304909|MMV Beta Viewer: Improve loading/navigation UX (T429193)]], [[gerrit:1304910|Take the feature out of beta (T429509)]] (duration: 33m 22s) [production]
01:12 <egardner@deploy1003> egardner: Continuing with deployment [production]
01:10 <egardner@deploy1003> egardner: Backport for [[gerrit:1304914|Inject service RepoGroup into Hooks]], [[gerrit:1304909|MMV Beta Viewer: Improve loading/navigation UX (T429193)]], [[gerrit:1304910|Take the feature out of beta (T429509)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
01:10 <brett@dns7002> END - running authdns-update [production]
01:08 <brett@dns7002> START - running authdns-update [production]
00:57 <brett@dns7002> END - running authdns-update [production]
00:56 <brett@dns7002> START - running authdns-update [production]
00:51 <egardner@deploy1003> Started scap sync-world: Backport for [[gerrit:1304914|Inject service RepoGroup into Hooks]], [[gerrit:1304909|MMV Beta Viewer: Improve loading/navigation UX (T429193)]], [[gerrit:1304910|Take the feature out of beta (T429509)]] [production]
00:46 <swfrench-wmf> manually started scap-clean-images.service on deply1003 to reclaim /srv space (93% -> 66% utilization) [production]
00:39 <mutante> codesearch10: systemctl start codesearch-write-config; systemctl restart hound-operations (gerrit:1304848) (T429819) [production]
00:33 <jclark@cumin1003> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host clouddb1027.eqiad.wmnet with OS trixie [production]
00:33 <jclark@cumin1003> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1003" [production]
00:33 <jclark@cumin1003> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1003" [production]
00:30 <jclark@cumin1003> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host clouddb1033.eqiad.wmnet with OS trixie [production]
00:30 <jclark@cumin1003> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1003" [production]
00:29 <jclark@cumin1003> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1003" [production]
00:24 <jclark@cumin1003> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host clouddb1026.eqiad.wmnet with OS trixie [production]
00:24 <jclark@cumin1003> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1003" [production]
00:24 <jclark@cumin1003> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1003" [production]
00:20 <jclark@cumin1003> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host clouddb1028.eqiad.wmnet with OS trixie [production]
00:20 <jclark@cumin1003> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1003" [production]
00:20 <jclark@cumin1003> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1003" [production]
00:17 <jclark@cumin1003> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on clouddb1027.eqiad.wmnet with reason: host reimage [production]
00:15 <egardner@deploy1003> sync-world failed: <CalledProcessError> Command 'sudo -u mwbuilder /srv/mwbuilder/release/make-container-image/build-images.py --http-proxy http://webproxy:8080 --https-proxy http://webproxy:8080 /srv/mediawiki-staging/scap/image-build --staging-dir /srv/mediawiki-staging --mediawiki-versions 1.47.0-wmf.6,1.47.0-wmf.7,next --multiversion-image-basename docker-registry.discovery.wmnet/restricted/me [production]
00:14 <jclark@cumin1003> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on clouddb1033.eqiad.wmnet with reason: host reimage [production]
00:11 <egardner@deploy1003> Started scap sync-world: Backport for [[gerrit:1304914|Inject service RepoGroup into Hooks]], [[gerrit:1304909|MMV Beta Viewer: Improve loading/navigation UX (T429193)]], [[gerrit:1304910|Take the feature out of beta (T429509)]] [production]
00:10 <jclark@cumin1003> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host clouddb1029.eqiad.wmnet with OS trixie [production]
00:10 <jclark@cumin1003> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1003" [production]
00:10 <jclark@cumin1003> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1003" [production]
00:09 <jclark@cumin1003> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on clouddb1026.eqiad.wmnet with reason: host reimage [production]
00:08 <egardner@deploy1003> sync-world failed: <CalledProcessError> Command 'sudo -u mwbuilder /srv/mwbuilder/release/make-container-image/build-images.py --http-proxy http://webproxy:8080 --https-proxy http://webproxy:8080 /srv/mediawiki-staging/scap/image-build --staging-dir /srv/mediawiki-staging --mediawiki-versions 1.47.0-wmf.6,1.47.0-wmf.7,next --multiversion-image-basename docker-registry.discovery.wmnet/restricted/me [production]
00:07 <jclark@cumin1003> START - Cookbook sre.hosts.downtime for 2:00:00 on clouddb1027.eqiad.wmnet with reason: host reimage [production]
00:05 <jclark@cumin1003> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on clouddb1028.eqiad.wmnet with reason: host reimage [production]
00:03 <egardner@deploy1003> Started scap sync-world: Backport for [[gerrit:1304914|Inject service RepoGroup into Hooks]], [[gerrit:1304909|MMV Beta Viewer: Improve loading/navigation UX (T429193)]], [[gerrit:1304910|Take the feature out of beta (T429509)]] [production]
00:03 <jclark@cumin1003> START - Cookbook sre.hosts.downtime for 2:00:00 on clouddb1033.eqiad.wmnet with reason: host reimage [production]
2026-06-22 §
23:57 <jclark@cumin1003> START - Cookbook sre.hosts.downtime for 2:00:00 on clouddb1026.eqiad.wmnet with reason: host reimage [production]
23:57 <jclark@cumin1003> START - Cookbook sre.hosts.reimage for host clouddb1027.eqiad.wmnet with OS trixie [production]
23:57 <jclark@cumin1003> START - Cookbook sre.hosts.downtime for 2:00:00 on clouddb1028.eqiad.wmnet with reason: host reimage [production]
23:55 <jclark@cumin1003> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on clouddb1029.eqiad.wmnet with reason: host reimage [production]
23:53 <jclark@cumin1003> START - Cookbook sre.hosts.reimage for host clouddb1033.eqiad.wmnet with OS trixie [production]
23:52 <jclark@cumin1003> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host clouddb1033.eqiad.wmnet with OS trixie [production]