|
2026-10-28
ยง
|
| 19:54 |
<cmooney@cumin1004> |
START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update subint entries for row B interfaces on eqiad CRs - cmooney@cumin1004" |
[production] |
| 19:52 |
<akhatun@deploy1003> |
Finished scap sync-world: Backport for [[gerrit:1359949|Prefetch the LAC revision_feature_counts artifact on group1 wikipedias ]] (duration: 11m 48s) |
[production] |
| 19:52 |
<pt1979@cumin1004> |
START - Cookbook sre.hosts.dhcp for host ml-serve1016.eqiad.wmnet (live debug) |
[production] |
| 19:49 |
<cmooney@cumin1004> |
START - Cookbook sre.dns.netbox |
[production] |
| 19:46 |
<akhatun@deploy1003> |
akhatun, otto: Continuing with deployment |
[production] |
| 19:45 |
<jasmine@cumin2003> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[1284-1289,1291-1297,1299].eqiad.wmnet |
[production] |
| 19:45 |
<akhatun@deploy1003> |
akhatun, otto: Backport for [[gerrit:1359949|Prefetch the LAC revision_feature_counts artifact on group1 wikipedias ]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. |
[production] |
| 19:40 |
<akhatun@deploy1003> |
Started scap sync-world: Backport for [[gerrit:1359949|Prefetch the LAC revision_feature_counts artifact on group1 wikipedias ]] |
[production] |
| 19:37 |
<jasmine@cumin2003> |
START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[1284-1289,1291-1297,1299].eqiad.wmnet |
[production] |
| 19:37 |
<jasmine@cumin2003> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[1269-1283].eqiad.wmnet |
[production] |
| 19:37 |
<jasmine@cumin2003> |
START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[1269-1283].eqiad.wmnet |
[production] |
| 19:30 |
<akhatun@deploy1003> |
helmfile [codfw] DONE helmfile.d/services/lac-lambdas: apply |
[production] |
| 19:29 |
<akhatun@deploy1003> |
helmfile [codfw] START helmfile.d/services/lac-lambdas: apply |
[production] |
| 19:28 |
<jasmine@cumin2003> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[1269-1283].eqiad.wmnet |
[production] |
| 19:26 |
<akhatun@deploy1003> |
helmfile [eqiad] DONE helmfile.d/services/lac-lambdas: apply |
[production] |
| 19:26 |
<akhatun@deploy1003> |
helmfile [eqiad] START helmfile.d/services/lac-lambdas: apply |
[production] |
| 19:23 |
<akhatun@deploy1003> |
helmfile [staging] DONE helmfile.d/services/lac-lambdas: apply |
[production] |
| 19:22 |
<akhatun@deploy1003> |
helmfile [staging] START helmfile.d/services/lac-lambdas: apply |
[production] |
| 19:21 |
<inflatador> |
[bking@deploy1003] `curl -XPOST -u $U -s https://opensearch-ttmserver.svc.eqiad.wmnet:30443/_cluster/reroute` Fix yellow status |
[production] |
| 19:18 |
<jasmine@cumin2003> |
START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[1269-1283].eqiad.wmnet |
[production] |
| 19:18 |
<jasmine@cumin2003> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[1254-1268].eqiad.wmnet |
[production] |
| 19:18 |
<jasmine@cumin2003> |
START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[1254-1268].eqiad.wmnet |
[production] |
| 19:09 |
<jasmine@cumin2003> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[1254-1268].eqiad.wmnet |
[production] |
| 18:59 |
<jasmine@cumin2003> |
START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[1254-1268].eqiad.wmnet |
[production] |
| 18:59 |
<jasmine@cumin2003> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[1165,1240-1253].eqiad.wmnet |
[production] |
| 18:59 |
<dancy@deploy1003> |
Installation of scap version "4.307.0" completed for 2 hosts |
[production] |
| 18:59 |
<jasmine@cumin2003> |
START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[1165,1240-1253].eqiad.wmnet |
[production] |
| 18:58 |
<dancy@deploy1003> |
Installing scap version "4.307.0" for 2 host(s) |
[production] |
| 18:57 |
<Dreamy_Jazz> |
Running `foreachwikiindblist group1.dblist cleanupWatchlistTemporaryAccountRows.php` for T439059 |
[production] |
| 18:50 |
<jasmine@cumin2003> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[1165,1240-1253].eqiad.wmnet |
[production] |
| 18:45 |
<dancy@deploy1003> |
Installing scap version "4.307.0" for 155 host(s) |
[production] |
| 18:41 |
<jasmine@cumin2003> |
START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[1165,1240-1253].eqiad.wmnet |
[production] |
| 18:41 |
<jasmine@cumin2003> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[1149-1151,1153-1164].eqiad.wmnet |
[production] |
| 18:41 |
<jasmine@cumin2003> |
START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[1149-1151,1153-1164].eqiad.wmnet |
[production] |
| 18:32 |
<jasmine@cumin2003> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host wikikube-worker[1149-1151,1153-1164].eqiad.wmnet |
[production] |
| 18:31 |
<rzl> |
rzl@apt1002:~$ sudo -i reprepro copy bullseye-wikimedia bookworm-wikimedia envoyproxy # backwards, intended |
[production] |
| 18:31 |
<rzl> |
rzl@apt1002:~$ sudo -i reprepro copy trixie-wikimedia bookworm-wikimedia envoyproxy |
[production] |
| 18:30 |
<rzl> |
rzl@apt1002:~$ sudo -i reprepro -C main includedeb bookworm-wikimedia /srv/wikimedia/pool/component/envoy-future/e/envoyproxy/envoyproxy_1.40.1-1_amd64.deb |
[production] |
| 18:26 |
<tchin@deploy1003> |
helmfile [codfw] DONE helmfile.d/services/eventstreams: apply |
[production] |
| 18:25 |
<dduvall@deploy1003> |
rebuilt and synchronized wikiversions files: group1 to 1.48.0-wmf.1 refs T438223 |
[production] |
| 18:25 |
<tchin@deploy1003> |
helmfile [codfw] START helmfile.d/services/eventstreams: apply |
[production] |
| 18:25 |
<tchin@deploy1003> |
helmfile [codfw] DONE helmfile.d/services/eventstreams: apply |
[production] |
| 18:19 |
<tchin@deploy1003> |
helmfile [codfw] START helmfile.d/services/eventstreams: apply |
[production] |
| 18:19 |
<rzl@deploy1003> |
helmfile [staging] DONE helmfile.d/services/rest-gateway: apply |
[production] |
| 18:19 |
<rzl@deploy1003> |
helmfile [staging] START helmfile.d/services/rest-gateway: apply |
[production] |
| 18:19 |
<jasmine@cumin2003> |
START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker[1149-1151,1153-1164].eqiad.wmnet |
[production] |
| 18:18 |
<jasmine@cumin2003> |
END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[1134-1148].eqiad.wmnet |
[production] |
| 18:18 |
<jasmine@cumin2003> |
START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[1134-1148].eqiad.wmnet |
[production] |
| 18:17 |
<tchin@deploy1003> |
helmfile [codfw] DONE helmfile.d/services/eventstreams: apply |
[production] |
| 18:10 |
<tchin@deploy1003> |
helmfile [codfw] START helmfile.d/services/eventstreams: apply |
[production] |