|
2025-05-06
§
|
| 03:45 |
<ryankemper@cumin2002> |
START - Cookbook sre.wdqs.data-transfer (T388134, bring new main graph hosts into service) xfer wikidata_main from wdqs2021.codfw.wmnet -> wdqs2015.codfw.wmnet, repooling source-only afterwards |
[production] |
| 03:45 |
<ryankemper@cumin2002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 15:00:00 on wdqs[2008,2014-2015].codfw.wmnet,wdqs[1011,1016].eqiad.wmnet with reason: T388134 |
[production] |
| 03:44 |
<ryankemper@cumin2002> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) (T388134, bring new main graph hosts into service) xfer wikidata_main from wdqs2021.codfw.wmnet -> wdqs2014.codfw.wmnet, repooling source-only afterwards |
[production] |
| 03:36 |
<ryankemper@cumin2002> |
START - Cookbook sre.wdqs.data-transfer (T388134, bring new main graph hosts into service) xfer wikidata_main from wdqs2021.codfw.wmnet -> wdqs2014.codfw.wmnet, repooling source-only afterwards |
[production] |
| 03:35 |
<ryankemper@cumin2002> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) (T388134, bring new main graph hosts into service) xfer wikidata_main from wdqs2021.codfw.wmnet -> wdqs2008.codfw.wmnet, repooling source-only afterwards |
[production] |
| 03:27 |
<ryankemper@cumin2002> |
START - Cookbook sre.wdqs.data-transfer (T388134, bring new main graph hosts into service) xfer wikidata_main from wdqs2021.codfw.wmnet -> wdqs2008.codfw.wmnet, repooling source-only afterwards |
[production] |
| 03:25 |
<ryankemper@cumin2002> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) (T388134, bring new main graph hosts into service) xfer wikidata_main from wdqs1021.eqiad.wmnet -> wdqs1011.eqiad.wmnet, repooling source-only afterwards |
[production] |
| 03:18 |
<ryankemper@cumin2002> |
START - Cookbook sre.wdqs.data-transfer (T388134, bring new main graph hosts into service) xfer wikidata_main from wdqs1021.eqiad.wmnet -> wdqs1011.eqiad.wmnet, repooling source-only afterwards |
[production] |
| 03:18 |
<ryankemper@deploy1003> |
Finished deploy [wdqs/wdqs@fe88851]: deploy to freshly reimaged host (duration: 00m 12s) |
[production] |
| 03:17 |
<ryankemper@deploy1003> |
Started deploy [wdqs/wdqs@fe88851]: deploy to freshly reimaged host |
[production] |
| 03:17 |
<ryankemper@deploy1003> |
Finished deploy [wdqs/wdqs@fe88851]: deploy to freshly reimaged host (duration: 00m 13s) |
[production] |
| 03:17 |
<ryankemper@deploy1003> |
Started deploy [wdqs/wdqs@fe88851]: deploy to freshly reimaged host |
[production] |
| 03:17 |
<ryankemper@deploy1003> |
Finished deploy [wdqs/wdqs@fe88851]: deploy to freshly reimaged host (duration: 00m 13s) |
[production] |
| 03:16 |
<ryankemper@deploy1003> |
Started deploy [wdqs/wdqs@fe88851]: deploy to freshly reimaged host |
[production] |
| 03:16 |
<ryankemper@deploy1003> |
Finished deploy [wdqs/wdqs@fe88851]: deploy to freshly reimaged host (duration: 00m 14s) |
[production] |
| 03:16 |
<ryankemper@deploy1003> |
Started deploy [wdqs/wdqs@fe88851]: deploy to freshly reimaged host |
[production] |
| 03:05 |
<ryankemper@cumin2002> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) (T388134, bring new main graph hosts into service) xfer wikidata_main from wdqs1021.eqiad.wmnet -> wdqs1016.eqiad.wmnet, repooling source-only afterwards |
[production] |
| 03:03 |
<mwpresync@deploy1003> |
Started scap sync-world: testwikis to 1.44.0-wmf.28 refs T386223 |
[production] |
| 03:00 |
<ryankemper@cumin2002> |
START - Cookbook sre.wdqs.data-transfer (T388134, bring new main graph hosts into service) xfer wikidata_main from wdqs1021.eqiad.wmnet -> wdqs1016.eqiad.wmnet, repooling source-only afterwards |
[production] |
| 02:52 |
<ryankemper@cumin2002> |
END (FAIL) - Cookbook sre.wdqs.data-transfer (exit_code=99) (T388134, bring new main graph hosts into service) xfer wikidata from wdqs1021.eqiad.wmnet -> wdqs1016.eqiad.wmnet, repooling source-only afterwards |
[production] |
| 02:49 |
<ryankemper@cumin2002> |
START - Cookbook sre.wdqs.data-transfer (T388134, bring new main graph hosts into service) xfer wikidata from wdqs1021.eqiad.wmnet -> wdqs1016.eqiad.wmnet, repooling source-only afterwards |
[production] |
| 02:43 |
<ryankemper@cumin2002> |
END (FAIL) - Cookbook sre.wdqs.data-transfer (exit_code=99) (T388134, bring new main graph hosts into service) xfer wikidata from wdqs1021.eqiad.wmnet -> wdqs1016.eqiad.wmnet, repooling source-only afterwards |
[production] |
| 02:41 |
<ryankemper@cumin2002> |
START - Cookbook sre.wdqs.data-transfer (T388134, bring new main graph hosts into service) xfer wikidata from wdqs1021.eqiad.wmnet -> wdqs1016.eqiad.wmnet, repooling source-only afterwards |
[production] |
| 02:32 |
<ryankemper@cumin2002> |
END (FAIL) - Cookbook sre.wdqs.data-transfer (exit_code=99) (T388134, bring new main graph hosts into service) xfer wdqs-all from wdqs1021.eqiad.wmnet -> wdqs1016.eqiad.wmnet, repooling source-only afterwards |
[production] |
| 02:27 |
<ryankemper@cumin2002> |
START - Cookbook sre.wdqs.data-transfer (T388134, bring new main graph hosts into service) xfer wdqs-all from wdqs1021.eqiad.wmnet -> wdqs1016.eqiad.wmnet, repooling source-only afterwards |
[production] |
| 02:24 |
<ryankemper@cumin2002> |
END (FAIL) - Cookbook sre.wdqs.data-transfer (exit_code=99) (T388134, bring new main graph hosts into service) xfer wdqs-all from wdqs1021.eqiad.wmnet -> wdqs1016.eqiad.wmnet, repooling source-only afterwards |
[production] |
| 02:24 |
<ryankemper@cumin2002> |
START - Cookbook sre.wdqs.data-transfer (T388134, bring new main graph hosts into service) xfer wdqs-all from wdqs1021.eqiad.wmnet -> wdqs1016.eqiad.wmnet, repooling source-only afterwards |
[production] |
| 02:24 |
<ryankemper@deploy1003> |
Finished deploy [wdqs/wdqs@fe88851]: deploy to freshly reimaged host (duration: 00m 12s) |
[production] |
| 02:24 |
<ryankemper@deploy1003> |
Started deploy [wdqs/wdqs@fe88851]: deploy to freshly reimaged host |
[production] |
| 02:22 |
<ryankemper@cumin2002> |
END (FAIL) - Cookbook sre.wdqs.data-transfer (exit_code=99) (T388134, bring new main graph hosts into service) xfer wdqs-all from wdqs1021.eqiad.wmnet -> wdqs1016.eqiad.wmnet, repooling source-only afterwards |
[production] |
| 02:22 |
<ryankemper@cumin2002> |
START - Cookbook sre.wdqs.data-transfer (T388134, bring new main graph hosts into service) xfer wdqs-all from wdqs1021.eqiad.wmnet -> wdqs1016.eqiad.wmnet, repooling source-only afterwards |
[production] |
| 00:24 |
<ryankemper@cumin2002> |
END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wdqs1017.eqiad.wmnet with OS bullseye |
[production] |
|
2025-05-05
§
|
| 23:32 |
<ryankemper@cumin2002> |
START - Cookbook sre.hosts.reimage for host wdqs1017.eqiad.wmnet with OS bullseye |
[production] |
| 23:29 |
<eileen> |
civicrm upgraded from 5a1f3e8e to 6ffbde61 |
[production] |
| 23:14 |
<zabe> |
zabe@mwmaint1002:~$ mwscript extensions/WikimediaMaintenance/migrateESRefToContentTableStage2.php enwiki --delete /home/zabe/afl_text_table_deletedump/enwiki --sleep 0.3 # T381599 |
[production] |
| 23:04 |
<zabe@deploy1003> |
Finished scap sync-world: Backport for [[gerrit:1140661|core-Permissions: refactor enwiki wgRemoveGroups]] (duration: 11m 13s) |
[production] |
| 23:01 |
<ryankemper@cumin2002> |
END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=1) upgrade firmware for hosts ['wdqs1017.eqiad.wmnet'] |
[production] |
| 23:01 |
<ryankemper@cumin2002> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['wdqs1017.eqiad.wmnet'] |
[production] |
| 23:00 |
<ryankemper@cumin2002> |
END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts ['wdqs1017.eqiad.wmnet'] |
[production] |
| 22:59 |
<ryankemper@cumin2002> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['wdqs1017.eqiad.wmnet'] |
[production] |
| 22:59 |
<ryankemper@cumin2002> |
END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=1) upgrade firmware for hosts ['wdqs1017.eqiad.wmnet'] |
[production] |
| 22:57 |
<zabe@deploy1003> |
zabe, novemlinguae: Continuing with sync |
[production] |
| 22:57 |
<zabe@deploy1003> |
zabe, novemlinguae: Backport for [[gerrit:1140661|core-Permissions: refactor enwiki wgRemoveGroups]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 22:52 |
<zabe@deploy1003> |
Started scap sync-world: Backport for [[gerrit:1140661|core-Permissions: refactor enwiki wgRemoveGroups]] |
[production] |
| 22:47 |
<ryankemper@cumin2002> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['wdqs1017.eqiad.wmnet'] |
[production] |
| 22:46 |
<ryankemper@cumin2002> |
END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts ['wdqs1017.eqiad.wmnet'] |
[production] |
| 22:46 |
<ryankemper@cumin2002> |
START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['wdqs1017.eqiad.wmnet'] |
[production] |
| 22:46 |
<ryankemper@cumin2002> |
END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wdqs1017.eqiad.wmnet with OS bullseye |
[production] |
| 22:35 |
<vriley@cumin1002> |
END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host apus-fe1003.wikimedia.org with OS bookworm |
[production] |
| 22:12 |
<sbassett> |
Deployed security fix (2) for T392341 |
[production] |