801-850 of 10000 results (120ms)
2025-05-06 ยง
19:23 <ryankemper@cumin2002> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) (T388134, bring new main graph hosts into service) xfer wikidata_main from wdqs1021.eqiad.wmnet -> wdqs1011.eqiad.wmnet w/ force delete existing files, repooling source-only afterwards [production]
19:23 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db1251 (T382778)', diff saved to https://phabricator.wikimedia.org/P75816 and previous config saved to /var/cache/conftool/dbconfig/20250506-192333-ladsgroup.json [production]
19:23 <ladsgroup@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on db1251.eqiad.wmnet with reason: Maintenance [production]
19:22 <ladsgroup@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on db1240.eqiad.wmnet with reason: Maintenance [production]
19:21 <ladsgroup@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on db1239.eqiad.wmnet with reason: Maintenance [production]
19:21 <ryankemper@cumin2002> START - Cookbook sre.wdqs.data-transfer (T388134, bring new main graph hosts into service) xfer wikidata_main from wdqs2014.codfw.wmnet -> wdqs2015.codfw.wmnet w/ force delete existing files, repooling neither afterwards [production]
19:20 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1235 (T382778)', diff saved to https://phabricator.wikimedia.org/P75815 and previous config saved to /var/cache/conftool/dbconfig/20250506-192054-ladsgroup.json [production]
19:20 <ryankemper@cumin2002> START - Cookbook sre.wdqs.data-transfer (T388134, bring new main graph hosts into service) xfer wikidata_main from wdqs2021.codfw.wmnet -> wdqs2008.codfw.wmnet w/ force delete existing files, repooling source-only afterwards [production]
19:18 <ryankemper@cumin2002> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) (T388134, bring new main graph hosts into service) xfer wikidata_main from wdqs2021.codfw.wmnet -> wdqs2014.codfw.wmnet w/ force delete existing files, repooling source-only afterwards [production]
19:05 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1235', diff saved to https://phabricator.wikimedia.org/P75814 and previous config saved to /var/cache/conftool/dbconfig/20250506-190547-ladsgroup.json [production]
18:50 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1235', diff saved to https://phabricator.wikimedia.org/P75813 and previous config saved to /var/cache/conftool/dbconfig/20250506-185040-ladsgroup.json [production]
18:35 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1235 (T382778)', diff saved to https://phabricator.wikimedia.org/P75812 and previous config saved to /var/cache/conftool/dbconfig/20250506-183533-ladsgroup.json [production]
18:32 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db1235 (T382778)', diff saved to https://phabricator.wikimedia.org/P75811 and previous config saved to /var/cache/conftool/dbconfig/20250506-183222-ladsgroup.json [production]
18:32 <ladsgroup@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on db1235.eqiad.wmnet with reason: Maintenance [production]
18:31 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1234 (T382778)', diff saved to https://phabricator.wikimedia.org/P75810 and previous config saved to /var/cache/conftool/dbconfig/20250506-183159-ladsgroup.json [production]
18:25 <ryankemper@cumin2002> START - Cookbook sre.wdqs.data-transfer (T388134, bring new main graph hosts into service) xfer wikidata_main from wdqs2021.codfw.wmnet -> wdqs2014.codfw.wmnet w/ force delete existing files, repooling source-only afterwards [production]
18:25 <ryankemper@cumin2002> START - Cookbook sre.wdqs.data-transfer (T388134, bring new main graph hosts into service) xfer wikidata_main from wdqs1021.eqiad.wmnet -> wdqs1011.eqiad.wmnet w/ force delete existing files, repooling source-only afterwards [production]
18:24 <ryankemper@cumin2002> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) (T388134, bring new main graph hosts into service) xfer categories from wdqs2021.codfw.wmnet -> wdqs2008.codfw.wmnet w/ force delete existing files, repooling source-only afterwards [production]
18:23 <ryankemper@cumin2002> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) (T388134, bring new main graph hosts into service) xfer categories from wdqs1021.eqiad.wmnet -> wdqs1011.eqiad.wmnet w/ force delete existing files, repooling source-only afterwards [production]
18:17 <jhuneidi@deploy1003> rebuilt and synchronized wikiversions files: group0 to 1.44.0-wmf.28 refs T386223 [production]
18:16 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1234', diff saved to https://phabricator.wikimedia.org/P75808 and previous config saved to /var/cache/conftool/dbconfig/20250506-181652-ladsgroup.json [production]
18:13 <ryankemper@cumin2002> START - Cookbook sre.wdqs.data-transfer (T388134, bring new main graph hosts into service) xfer categories from wdqs2021.codfw.wmnet -> wdqs2008.codfw.wmnet w/ force delete existing files, repooling source-only afterwards [production]
18:12 <ryankemper@cumin2002> START - Cookbook sre.wdqs.data-transfer (T388134, bring new main graph hosts into service) xfer categories from wdqs1021.eqiad.wmnet -> wdqs1011.eqiad.wmnet w/ force delete existing files, repooling source-only afterwards [production]
18:01 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1234', diff saved to https://phabricator.wikimedia.org/P75807 and previous config saved to /var/cache/conftool/dbconfig/20250506-180146-ladsgroup.json [production]
17:54 <ryankemper@cumin2002> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) (T388134, bring new main graph hosts into service) xfer categories from wdqs2021.codfw.wmnet -> wdqs2015.codfw.wmnet, repooling source-only afterwards [production]
17:54 <ryankemper@cumin2002> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) (T388134, bring new main graph hosts into service) xfer categories from wdqs1021.eqiad.wmnet -> wdqs1017.eqiad.wmnet, repooling source-only afterwards [production]
17:49 <ryankemper@cumin2002> START - Cookbook sre.wdqs.data-transfer (T388134, bring new main graph hosts into service) xfer categories from wdqs1021.eqiad.wmnet -> wdqs1017.eqiad.wmnet, repooling source-only afterwards [production]
17:49 <ryankemper@cumin2002> START - Cookbook sre.wdqs.data-transfer (T388134, bring new main graph hosts into service) xfer categories from wdqs2021.codfw.wmnet -> wdqs2015.codfw.wmnet, repooling source-only afterwards [production]
17:49 <ryankemper@cumin2002> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) (T388134, bring new main graph hosts into service) xfer categories from wdqs1021.eqiad.wmnet -> wdqs1016.eqiad.wmnet, repooling source-only afterwards [production]
17:49 <ryankemper@cumin2002> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) (T388134, bring new main graph hosts into service) xfer categories from wdqs2021.codfw.wmnet -> wdqs2014.codfw.wmnet, repooling source-only afterwards [production]
17:46 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1234 (T382778)', diff saved to https://phabricator.wikimedia.org/P75806 and previous config saved to /var/cache/conftool/dbconfig/20250506-174639-ladsgroup.json [production]
17:44 <ryankemper@cumin2002> START - Cookbook sre.wdqs.data-transfer (T388134, bring new main graph hosts into service) xfer categories from wdqs1021.eqiad.wmnet -> wdqs1016.eqiad.wmnet, repooling source-only afterwards [production]
17:44 <ryankemper@cumin2002> START - Cookbook sre.wdqs.data-transfer (T388134, bring new main graph hosts into service) xfer categories from wdqs2021.codfw.wmnet -> wdqs2014.codfw.wmnet, repooling source-only afterwards [production]
17:44 <ryankemper@cumin2002> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) (T388134, bring new main graph hosts into service) xfer categories from wdqs1021.eqiad.wmnet -> wdqs1011.eqiad.wmnet, repooling source-only afterwards [production]
17:43 <ryankemper@cumin2002> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) (T388134, bring new main graph hosts into service) xfer categories from wdqs2021.codfw.wmnet -> wdqs2008.codfw.wmnet, repooling source-only afterwards [production]
17:43 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db1234 (T382778)', diff saved to https://phabricator.wikimedia.org/P75805 and previous config saved to /var/cache/conftool/dbconfig/20250506-174325-ladsgroup.json [production]
17:43 <ladsgroup@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on db1234.eqiad.wmnet with reason: Maintenance [production]
17:43 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1232 (T382778)', diff saved to https://phabricator.wikimedia.org/P75804 and previous config saved to /var/cache/conftool/dbconfig/20250506-174313-ladsgroup.json [production]
17:41 <ryankemper@cumin2002> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) (T388134, bring new main graph hosts into service) xfer wikidata_main from wdqs1021.eqiad.wmnet -> wdqs1017.eqiad.wmnet, repooling source-only afterwards [production]
17:40 <ryankemper@cumin2002> START - Cookbook sre.wdqs.data-transfer (T388134, bring new main graph hosts into service) xfer categories from wdqs1021.eqiad.wmnet -> wdqs1011.eqiad.wmnet, repooling source-only afterwards [production]
17:39 <ryankemper@cumin2002> START - Cookbook sre.wdqs.data-transfer (T388134, bring new main graph hosts into service) xfer categories from wdqs2021.codfw.wmnet -> wdqs2008.codfw.wmnet, repooling source-only afterwards [production]
17:31 <ryankemper@cumin2002> START - Cookbook sre.wdqs.data-transfer (T388134, bring new main graph hosts into service) xfer wikidata_main from wdqs1021.eqiad.wmnet -> wdqs1017.eqiad.wmnet, repooling source-only afterwards [production]
17:30 <hnowlan@deploy1003> helmfile [eqiad] DONE helmfile.d/services/mw-cron: apply [production]
17:30 <hnowlan@deploy1003> helmfile [eqiad] START helmfile.d/services/mw-cron: apply [production]
17:29 <ryankemper@deploy1003> Finished deploy [wdqs/wdqs@fe88851]: deploy to freshly reimaged host (duration: 00m 11s) [production]
17:29 <ryankemper@deploy1003> Started deploy [wdqs/wdqs@fe88851]: deploy to freshly reimaged host [production]
17:28 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1232', diff saved to https://phabricator.wikimedia.org/P75803 and previous config saved to /var/cache/conftool/dbconfig/20250506-172807-ladsgroup.json [production]
17:13 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1232', diff saved to https://phabricator.wikimedia.org/P75802 and previous config saved to /var/cache/conftool/dbconfig/20250506-171259-ladsgroup.json [production]
17:12 <hnowlan@deploy1003> helmfile [eqiad] DONE helmfile.d/services/mw-cron: apply [production]
17:11 <hnowlan@deploy1003> helmfile [eqiad] START helmfile.d/services/mw-cron: apply [production]