| … | … | @@ -40,26 +40,83 @@ At prep time this was **124 commits**. Open a PR from `feat/service-http-wiring` |
| 40 | 40 | rather than fast-forwarding — `rc.3` was previously cut from `main` by mistake |
| 41 | 41 | without the feature tree, and that is exactly the failure this gap invites. |
| 42 | 42 | |
| 43 | | ### 1b. No signing key is configured |
| 43 | ### 1b. The release signing key is not on this machine |
| 44 | 44 | |
| 45 | | The updater refuses any tag that is not an annotated tag signed by a key in |
| 46 | | `/etc/hiverelay/allowed-signers`. It fails closed: a box that cannot verify the |
| 47 | | tag stays on its current version. This repo currently has no signing config: |
| 45 | The fleet trusts exactly one signer, in both `fleet/allowed-signers` and the copy |
| 46 | installed on the provisioned boxes (verified identical on miami): |
| 47 | |
| 48 | ``` |
| 49 | bigdestiny2@users.noreply.github.com ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGxvfmV3/yKO0C5HMCGRiOMiqfEwkzr6/aMlz4TUfojO hiverelay-release |
| 50 | ``` |
| 51 | |
| 52 | None of the local keys match it (`cloudzy_hiverelay`, `id_ed25519`, |
| 53 | `peerit_vps_deploy`), the ssh-agent is empty, and git has no signing config |
| 54 | (`user.signingkey`, `gpg.format`, `tag.gpgsign` all unset). **Sign on whichever |
| 55 | machine or token holds `hiverelay-release`**, or rotate the release key and |
| 56 | redistribute `/etc/hiverelay/allowed-signers` fleet-wide. |
| 57 | |
| 58 | Prior tags *are* signed — `git tag -v v0.24.3` errors only because local |
| 59 | verification is unconfigured, not because the signature is missing. To verify |
| 60 | locally: |
| 48 | 61 | |
| 49 | 62 | ```bash |
| 50 | | git config --get user.signingkey # empty |
| 51 | | git config --get gpg.format # empty |
| 52 | | git config --get tag.gpgsign # empty |
| 63 | git config gpg.ssh.allowedSignersFile "$PWD/fleet/allowed-signers" |
| 64 | git config gpg.format ssh |
| 65 | git tag -v v0.24.3 |
| 53 | 66 | ``` |
| 54 | 67 | |
| 55 | | Configure it, then tag. **An unsigned tag is refused by every box.** |
| 68 | ### 1c. `main`'s canary channel is parked at v0.24.3 |
| 69 | |
| 70 | `origin/main:fleet/channels.json` currently reads `canary: v0.24.3` — PR #203 |
| 71 | (`ops/canary-hold-after-rc4`) deliberately held it after rc.4. utah is on rc.4 |
| 72 | only because it was placed there by hand. Promoting means moving canary forward |
| 73 | from `v0.24.3`, not from `v0.25.0-rc.4`. |
| 74 | |
| 75 | ### 1d. The pull-based automation is not actually provisioned (read this first) |
| 76 | |
| 77 | Surveyed 2026-07-28. The fleet is in three states, and the "boxes self-update |
| 78 | within ~15 min" model in `fleet/README.md` does not currently hold: |
| 79 | |
| 80 | | State | Boxes | Agent | Signed-tag gate | Timer | |
| 81 | |-------|-------|-------|-----------------|-------| |
| 82 | | **Provisioned** | miami, sydney, amsterdam | 13796 B (current) | ✅ `verify_tag`, signers installed | active | |
| 83 | | **Stale agent** | utah-us, utah-2gb-a, utah-0.5gb, sing-1, sing-2, dubai | 6799 B (2026-06-22) | ❌ no `verify_tag`, no signers | active | |
| 84 | | **Automation off** | utah, utah-8gb, dallas | stale / none | ❌ | inactive | |
| 85 | |
| 86 | Three consequences: |
| 87 | |
| 88 | 1. **The supply-chain gate is not deployed on 9 of 12 boxes.** The six |
| 89 | stale-agent boxes would check out *any* tag the channel names, signed or not. |
| 90 | `fleet/README.md` describes this as fail-closed; that is currently true only |
| 91 | for miami, sydney, and amsterdam. |
| 92 | 2. **utah — the designated canary — has its updater timer off**, so a canary |
| 93 | promotion would not reach it at all. The only canary box that would move is |
| 94 | **utah-0.5gb**, the 0.5 GB box that was ABRT-looping under the full plugin |
| 95 | suite. |
| 96 | 3. **sydney is one of the three boxes that WILL enforce the gate.** Pinning it to |
| 97 | `hold` (§3) is still the right move, but note it is already the strictest box |
| 98 | in the fleet, not the loosest. |
| 99 | |
| 100 | **Recommended order: provision before promoting.** Reinstall the current agent and |
| 101 | allowed-signers on the nine boxes that lack them, and re-enable the timers you |
| 102 | intend to be automatic, *then* cut the tag and bump the channel. Otherwise the |
| 103 | promotion is a no-op on the intended canary and an unverified checkout on six |
| 104 | others. |
| 105 | |
| 106 | ```bash |
| 107 | # per box, from a workstation with SSH: |
| 108 | ssh root@<box> 'bash -s' < fleet/install-updater.sh <canary|stable> |
| 109 | scp fleet/allowed-signers root@<box>:/etc/hiverelay/allowed-signers |
| 110 | ``` |
| 56 | 111 | |
| 57 | 112 | --- |
| 58 | 113 | |
| 59 | 114 | ## 2. Cut the release |
| 60 | 115 | |
| 116 | Run from the repo, not `~`: |
| 117 | |
| 61 | 118 | ```bash |
| 62 | | git tag -s v0.25.0-rc.5 -m "v0.25.0-rc.5 — restore orphaned lifecycle/gateway features" |
| 119 | cd /Users/localllm/Projects/pear-ecosystem/00-core/hiverelay && git tag -s v0.25.0-rc.5 -m "v0.25.0-rc.5 — restore orphaned lifecycle/gateway features" |
| 63 | 120 | ``` |
| 64 | 121 | |
| 65 | 122 | Verify before pushing — this is the exact check each box performs: |