| … | … | @@ -6,6 +6,56 @@ documented here. Dates in YYYY-MM-DD. |
| 6 | 6 | |
| 7 | 7 | The packages are versioned in lockstep. |
| 8 | 8 | |
| 9 | ## [Unreleased] — 2026-07-29 | |
| 10 | ||
| 11 | **Post-RC7 release repair. Requires a new prerelease identity before any fleet | |
| 12 | deployment.** | |
| 13 | ||
| 14 | ### Fixed — runtime and persistence | |
| 15 | ||
| 16 | - Relay app lifecycle now serializes recovery, seed, update, and retirement on | |
| 17 | one canonical mutation authority; preserves exact finite storage commitments; | |
| 18 | retains failed teardown owners for retry; and no longer acknowledges work | |
| 19 | before durable state is complete. The test suites no longer use a finalizer | |
| 20 | that could force a passing exit while asynchronous work remained. | |
| 21 | - OutboxLog partitioned-Hypercore journals bind storage admission before append, | |
| 22 | settle journal operations before close, and keep the configured aggregate | |
| 23 | byte commitment authoritative across restart. | |
| 24 | - Service management now persists to `<storage>/services.json`, rolls back API | |
| 25 | state on write failure, accepts the shipped `storage-proof` builtin, and | |
| 26 | honors an explicit persisted `enabled: false` over package defaults. | |
| 27 | - The Node plugin lifecycle now applies `config.vrfBeacon`, making the existing | |
| 28 | beacon implementation and HTTP routes reachable on configured Node relays. | |
| 29 | ||
| 30 | ### Fixed — wake, mailbox, and privacy | |
| 31 | ||
| 32 | - Notify watches prefer the exact signed `notify-outbox-lane`; the global | |
| 33 | `notify-feed-head` compatibility source is marked deprecated and omitted from | |
| 34 | preferred signed watch sources when exact-lane support is present. | |
| 35 | - Tor restricted-discovery readiness now performs an unauthenticated negative | |
| 36 | probe against an exposed onion vport. A restricted endpoint that accepts the | |
| 37 | probe, or cannot prove rejection, is degraded and cannot be signed-advertised. | |
| 38 | - Client connection error containment is installed before replication and peer | |
| 39 | identity parsing, closing an early Noise/session crash window. | |
| 40 | - Operator and fleet dashboards render untrusted relay/app data through text | |
| 41 | nodes and validate links instead of constructing executable HTML. | |
| 42 | ||
| 43 | ### Fixed — packaging and fleet evidence | |
| 44 | ||
| 45 | - Docker, systemd, and Umbrel service/Tor environment values are now real Node | |
| 46 | first-boot defaults. Explicit CLI values, persisted `config.json`, and the | |
| 47 | storage-local services authority retain precedence. | |
| 48 | - Packaged OutboxLog uses `hypercore-outboxes` with a finite 512 MB local | |
| 49 | journal commitment. Shard custody is opt-in instead of silently consuming | |
| 50 | disk on every Docker/Umbrel node. Real wake credentials can be mounted in an | |
| 51 | owner-readable JSON descriptor selected by `HIVERELAY_NOTIFY_PUSH_CONFIG`. | |
| 52 | - Fleet health and service probes support exact include/exclude scopes before | |
| 53 | SSH, report exact prerelease versions and updater/channel drift, inspect | |
| 54 | signed capability evidence, and avoid exposing private inventory/key paths. | |
| 55 | - StartOS and Umbrel metadata, release-evidence tests, and operator docs are | |
| 56 | synchronized to the current prerelease baseline and distinguish configured | |
| 57 | defaults from signed runtime proof. | |
| 58 | ||
| 9 | 59 | ## [0.25.0-rc.7] — 2026-07-28 |
| 10 | 60 | |
| 11 | 61 | **Security release. Supersedes rc.6 — deploy this.** |
| … | … | @@ -194,7 +194,8 @@ replayed). It backs `client.proveSeeded(driveKey, { relay, samples })`. |
| 194 | 194 | It is **OFF by default.** Enable it per node: |
| 195 | 195 | |
| 196 | 196 | - **Node runtime** — add `storage-proof` to `config.plugins` (or the |
| 197 | **Services** tab / `services.json`): | |
| 197 | **Services** tab / `services.json`), or set the first-boot environment | |
| 198 | default `HIVERELAY_STORAGE_PROOF=1`: | |
| 198 | 199 | |
| 199 | 200 | ```json |
| 200 | 201 | { |
| … | … | @@ -202,8 +203,8 @@ It is **OFF by default.** Enable it per node: |
| 202 | 203 | } |
| 203 | 204 | ``` |
| 204 | 205 | |
| 205 | - **Bare / appliance runtime** — add `storage-proof` to `config.services`, | |
| 206 | or set the env var: | |
| 206 | - **Bare runtime** — add `storage-proof` to `config.services`, or set the same | |
| 207 | env var: | |
| 207 | 208 | |
| 208 | 209 | ```bash |
| 209 | 210 | HIVERELAY_STORAGE_PROOF=1 |
| … | … | @@ -6,7 +6,6 @@ |
| 6 | 6 | */ |
| 7 | 7 | |
| 8 | 8 | const fs = require('fs') |
| 9 | const path = require('path') | |
| 10 | 9 | |
| 11 | 10 | function defaultPolicy () { |
| 12 | 11 | return { |
| … | … | @@ -200,6 +199,8 @@ const REMOTE = { |
| 200 | 199 | 'set +e', |
| 201 | 200 | 'echo "=== doctor ==="', |
| 202 | 201 | 'SW=$(free -m | awk "/Swap/{print \\$2}")', |
| 202 | // Shell parameter expansion, not a JavaScript template placeholder. | |
| 203 | // eslint-disable-next-line no-template-curly-in-string | |
| 203 | 204 | 'if [ "${SW:-0}" = "0" ]; then', |
| 204 | 205 | ' fallocate -l 2G /swapfile 2>/dev/null || dd if=/dev/zero of=/swapfile bs=1M count=2048', |
| 205 | 206 | ' chmod 600 /swapfile; mkswap /swapfile; swapon /swapfile', |
| … | … | @@ -232,8 +233,12 @@ const REMOTE = { |
| 232 | 233 | ' cat > /usr/local/bin/hiverelay-health-watchdog.sh << "WATCH"', |
| 233 | 234 | '#!/usr/bin/env bash', |
| 234 | 235 | 'set -uo pipefail', |
| 236 | // Shell parameter expansions, not JavaScript template placeholders. | |
| 237 | // eslint-disable-next-line no-template-curly-in-string | |
| 235 | 238 | 'URL="${HIVERELAY_HEALTH_URL:-http://127.0.0.1:9100/health}"', |
| 239 | // eslint-disable-next-line no-template-curly-in-string | |
| 236 | 240 | 'TIMEOUT="${HIVERELAY_HEALTH_TIMEOUT:-5}"', |
| 241 | // eslint-disable-next-line no-template-curly-in-string | |
| 237 | 242 | 'NEED_FAILS="${HIVERELAY_HEALTH_FAILS:-2}"', |
| 238 | 243 | 'STATE_FILE=/run/hiverelay-health-watchdog.failcount', |
| 239 | 244 | 'UNIT=hiverelay', |
| … | … | @@ -1,3 +1,4 @@ |
| 1 | /* global EventSource, alert, confirm */ | |
| 1 | 2 | /* HiveRelay · Dashboard Admin Operator — live fleet control plane */ |
| 2 | 3 | |
| 3 | 4 | let state = null |
| … | … | @@ -6,18 +7,28 @@ let expandedNode = null |
| 6 | 7 | |
| 7 | 8 | const REGION_NAMES = { NA: 'North America', EU: 'Europe', APAC: 'Asia-Pacific', ME: 'Middle East' } |
| 8 | 9 | const FEATURE_LABELS = { |
| 9 | coreRelay: 'Core Relay', seeding: 'Seeding', services: 'Service Layer', | |
| 10 | dhtRelayWs: 'DHT Relay (WS)', tor: 'Tor Transport', holesail: 'Holesail', | |
| 11 | outboxlog: 'OutboxLog', forwardRelay: 'Forward Relay', signedDirectory: 'Signed Directory', | |
| 12 | vrf: 'VRF (RFC 9381)', poker: 'Poker Substrate', zk: 'ZK Proofs', | |
| 13 | arbitration: 'Arbitration', payment: 'Payments', eviction: 'Auto-Eviction', | |
| 14 | autoHeal: 'Auto-Heal', distributedDrive: 'Distributed Drive' | |
| 10 | coreRelay: 'Core Relay', | |
| 11 | seeding: 'Seeding', | |
| 12 | services: 'Service Layer', | |
| 13 | dhtRelayWs: 'DHT Relay (WS)', | |
| 14 | tor: 'Tor Transport', | |
| 15 | holesail: 'Holesail', | |
| 16 | outboxlog: 'OutboxLog', | |
| 17 | forwardRelay: 'Forward Relay', | |
| 18 | signedDirectory: 'Signed Directory', | |
| 19 | vrf: 'VRF (RFC 9381)', | |
| 20 | poker: 'Poker Substrate', | |
| 21 | zk: 'ZK Proofs', | |
| 22 | arbitration: 'Arbitration', | |
| 23 | payment: 'Payments', | |
| 24 | eviction: 'Auto-Eviction', | |
| 25 | autoHeal: 'Auto-Heal', | |
| 26 | distributedDrive: 'Distributed Drive' | |
| 15 | 27 | } |
| 16 | 28 | const TOGGLEABLE = ['dhtRelayWs', 'outboxlog', 'forwardRelay', 'signedDirectory', 'services', 'eviction'] |
| 17 | 29 | |
| 18 | 30 | async function fetchState () { |
| 19 | try { state = await (await fetch('/api/state')).json(); render() } | |
| 20 | catch { document.getElementById('app').innerHTML = '<div class="loading">Connection lost...</div>' } | |
| 31 | try { state = await (await fetch('/api/state')).json(); render() } catch { document.getElementById('app').innerHTML = '<div class="loading">Connection lost...</div>' } | |
| 21 | 32 | } |
| 22 | 33 | |
| 23 | 34 | function connectSSE () { |
| … | … | @@ -63,7 +74,8 @@ function computeSummary (nodes) { |
| 63 | 74 | else regions[r].degraded++ |
| 64 | 75 | } |
| 65 | 76 | return { |
| 66 | total: nodes.length, healthy: nodes.filter(n => n.status === 'healthy').length, | |
| 77 | total: nodes.length, | |
| 78 | healthy: nodes.filter(n => n.status === 'healthy').length, | |
| 67 | 79 | degraded: nodes.filter(n => n.status === 'degraded').length, |
| 68 | 80 | offline: nodes.filter(n => n.status === 'offline').length, |
| 69 | 81 | running: nodes.filter(n => n.running === true).length, |
| … | … | @@ -78,7 +90,7 @@ function computeSummary (nodes) { |
| 78 | 90 | |
| 79 | 91 | function fmtBytes (b) { |
| 80 | 92 | if (!b || b === 0) return '0 B' |
| 81 | const u = ['B','KB','MB','GB','TB','PB'] | |
| 93 | const u = ['B', 'KB', 'MB', 'GB', 'TB', 'PB'] | |
| 82 | 94 | const i = Math.floor(Math.log(b) / Math.log(1024)) |
| 83 | 95 | return (b / Math.pow(1024, i)).toFixed(1) + ' ' + u[i] |
| 84 | 96 | } |
| … | … | @@ -112,13 +124,13 @@ function render () { |
| 112 | 124 | ${renderActionLog(state.actions)} |
| 113 | 125 | </div> |
| 114 | 126 | <div id="toast-container"></div> |
| 115 | <div class="footer"><div class="footer-meta">Admin Operator · full fleet control · probe ${state.interval||60}s · auto-control ${state.control?.autoControl ? 'ON' : 'OFF'} · ${new Date().toISOString()}</div></div> | |
| 127 | <div class="footer"><div class="footer-meta">Admin Operator · full fleet control · probe ${state.interval || 60}s · auto-control ${state.control?.autoControl ? 'ON' : 'OFF'} · ${new Date().toISOString()}</div></div> | |
| 116 | 128 | ` |
| 117 | 129 | } |
| 118 | 130 | |
| 119 | 131 | function renderControlPanel (ctrl) { |
| 120 | 132 | if (!ctrl) { |
| 121 | return `<div class="control-panel"><div class="control-panel-head"><strong>Fleet control</strong><span class="dim">waiting for first probe…</span></div></div>` | |
| 133 | return '<div class="control-panel"><div class="control-panel-head"><strong>Fleet control</strong><span class="dim">waiting for first probe…</span></div></div>' | |
| 122 | 134 | } |
| 123 | 135 | const auto = !!ctrl.autoControl |
| 124 | 136 | const cycle = ctrl.lastCycle |
| … | … | @@ -153,21 +165,25 @@ function renderControlPanel (ctrl) { |
| 153 | 165 | <span class="range-chip">large ≥4GB: mem≤4.8GB</span> |
| 154 | 166 | <span class="range-chip">swap ≥256MB · watchdog required · disk warn 75%</span> |
| 155 | 167 | </div> |
| 156 | ${violators.length ? ` | |
| 168 | ${violators.length | |
| 169 | ? ` | |
| 157 | 170 | <div class="violations"> |
| 158 | 171 | <div class="section-title" style="margin:8px 0 6px">Out of range <span class="count">${violators.length}</span></div> |
| 159 | 172 | ${violators.map(v => ` |
| 160 | 173 | <div class="violation-row"> |
| 161 | 174 | <strong>${esc(v.name)}</strong> |
| 162 | <span class="badge ${esc(v.tier||'')}">${esc(v.tier||'?')}</span> | |
| 163 | ${(v.violations||[]).map(x => `<span class="vchip ${x.severity}">${esc(x.code)}: ${esc(x.msg)}</span>`).join('')} | |
| 164 | ${(v.actions||[]).map(a => `<span class="auto-act">→ ${esc(a.type)}</span>`).join('')} | |
| 175 | <span class="badge ${esc(v.tier || '')}">${esc(v.tier || '?')}</span> | |
| 176 | ${(v.violations || []).map(x => `<span class="vchip ${x.severity}">${esc(x.code)}: ${esc(x.msg)}</span>`).join('')} | |
| 177 | ${(v.actions || []).map(a => `<span class="auto-act">→ ${esc(a.type)}</span>`).join('')} | |
| 165 | 178 | </div>`).join('')} |
| 166 | </div>` : `<div class="in-range-ok">All reachable nodes within healthy ranges (or offline only).</div>`} | |
| 167 | ${remediations.length ? ` | |
| 179 | </div>` | |
| 180 | : '<div class="in-range-ok">All reachable nodes within healthy ranges (or offline only).</div>'} | |
| 181 | ${remediations.length | |
| 182 | ? ` | |
| 168 | 183 | <div class="remed-log"> |
| 169 | ${remediations.map(r => `<div class="remed-item ${r.ok?'ok':'fail'}">${esc(r.node)} · auto:${esc(r.type)} · ${esc(r.reason)} · ${r.ok?'ok':'fail'} ${esc(r.detail||'')}</div>`).join('')} | |
| 170 | </div>` : ''} | |
| 184 | ${remediations.map(r => `<div class="remed-item ${r.ok ? 'ok' : 'fail'}">${esc(r.node)} · auto:${esc(r.type)} · ${esc(r.reason)} · ${r.ok ? 'ok' : 'fail'} ${esc(r.detail || '')}</div>`).join('')} | |
| 185 | </div>` | |
| 186 | : ''} | |
| 171 | 187 | </div>` |
| 172 | 188 | } |
| 173 | 189 | |
| … | … | @@ -190,7 +206,7 @@ async function runControlNow () { |
| 190 | 206 | const res = await fetch('/api/control/run', { method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify({ force: true }) }) |
| 191 | 207 | const data = await res.json() |
| 192 | 208 | if (data.error) alert(data.error) |
| 193 | else showToast({ node: 'fleet', action: 'control cycle', detail: `done · ${data.cycle?.remediations?.length||0} remediations` }) | |
| 209 | else showToast({ node: 'fleet', action: 'control cycle', detail: `done · ${data.cycle?.remediations?.length || 0} remediations` }) | |
| 194 | 210 | await fetchState() |
| 195 | 211 | } catch (e) { alert(e.message) } |
| 196 | 212 | } |
| … | … | @@ -225,14 +241,14 @@ function renderKPIs (s) { |
| 225 | 241 | function renderRegions (regions) { |
| 226 | 242 | if (!regions) return '' |
| 227 | 243 | return `<div class="regions">${Object.entries(regions).map(([code, r]) => ` |
| 228 | <div class="region-card"><div class="region-name">${REGION_NAMES[code]||code}</div> | |
| 229 | <div class="region-stats"><span class="up">${r.healthy} up</span>${r.degraded?`<span class="deg">${r.degraded} deg</span>`:''}${r.offline?`<span class="down">${r.offline} down</span>`:''}<span class="total">${r.total}</span></div></div>`).join('')}</div>` | |
| 244 | <div class="region-card"><div class="region-name">${REGION_NAMES[code] || code}</div> | |
| 245 | <div class="region-stats"><span class="up">${r.healthy} up</span>${r.degraded ? `<span class="deg">${r.degraded} deg</span>` : ''}${r.offline ? `<span class="down">${r.offline} down</span>` : ''}<span class="total">${r.total}</span></div></div>`).join('')}</div>` | |
| 230 | 246 | } |
| 231 | 247 | |
| 232 | 248 | function renderNodeTable (nodes) { |
| 233 | 249 | const sorted = [...nodes].sort((a, b) => { |
| 234 | 250 | const order = { offline: 3, degraded: 2, healthy: 1 } |
| 235 | return (order[a.status]||3) - (order[b.status]||3) || a.name.localeCompare(b.name) | |
| 251 | return (order[a.status] || 3) - (order[b.status] || 3) || a.name.localeCompare(b.name) | |
| 236 | 252 | }) |
| 237 | 253 | return ` |
| 238 | 254 | <div class="section"> |
| … | … | @@ -257,19 +273,19 @@ function renderRow (n) { |
| 257 | 273 | <td><span class="chevron ${isExpanded ? 'open' : ''}">▶</span></td> |
| 258 | 274 | <td><span class="indicator"><span class="dot ${n.status}"></span></span> <strong>${esc(n.name)}</strong></td> |
| 259 | 275 | <td class="mono">${esc(n.publicIp)}</td> |
| 260 | <td class="${verOk?'version-ok':'version-bad'} mono-val">${esc(n.version||'—')}</td> | |
| 261 | <td>${n.running&&n.healthOk!==false?'<span class="ok-text">running</span>':(n.running?'<span class="err-text">hung?</span>':'<span class="err-text">stopped</span>')}</td> | |
| 276 | <td class="${verOk ? 'version-ok' : 'version-bad'} mono-val">${esc(n.version || '—')}</td> | |
| 277 | <td>${n.running && n.healthOk !== false ? '<span class="ok-text">running</span>' : (n.running ? '<span class="err-text">hung?</span>' : '<span class="err-text">stopped</span>')}</td> | |
| 262 | 278 | <td>${renderDiskBar(n)}</td> |
| 263 | <td class="mono-val">${mem!=null?mem+'M':'—'}</td> | |
| 279 | <td class="mono-val">${mem != null ? mem + 'M' : '—'}</td> | |
| 264 | 280 | <td>${renderRamBar(n)}</td> |
| 265 | <td class="mono">${n.loadAvg!=null?Number(n.loadAvg).toFixed(2):'—'}</td> | |
| 266 | <td class="mono-val">${n.seededApps||'—'}</td> | |
| 267 | <td class="mono-val">${n.connections||'—'}</td> | |
| 268 | <td>${wdOk?'<span class="ok-text">on</span>':'<span class="err-text">off</span>'}</td> | |
| 269 | <td>${svcCount?`<span class="svc-count">${svcCount}</span>`:'—'}</td> | |
| 270 | <td><span class="badge ${esc(n.channel||'stable')}">${esc(n.channel||'stable')}</span></td> | |
| 271 | <td>${esc(n.region||'—')}</td> | |
| 272 | <td class="mono">${n.vcpu||'?'}c/${n.ramGB||'?'}G/${n.diskGB||'?'}G</td> | |
| 281 | <td class="mono">${n.loadAvg != null ? Number(n.loadAvg).toFixed(2) : '—'}</td> | |
| 282 | <td class="mono-val">${n.seededApps || '—'}</td> | |
| 283 | <td class="mono-val">${n.connections || '—'}</td> | |
| 284 | <td>${wdOk ? '<span class="ok-text">on</span>' : '<span class="err-text">off</span>'}</td> | |
| 285 | <td>${svcCount ? `<span class="svc-count">${svcCount}</span>` : '—'}</td> | |
| 286 | <td><span class="badge ${esc(n.channel || 'stable')}">${esc(n.channel || 'stable')}</span></td> | |
| 287 | <td>${esc(n.region || '—')}</td> | |
| 288 | <td class="mono">${n.vcpu || '?'}c/${n.ramGB || '?'}G/${n.diskGB || '?'}G</td> | |
| 273 | 289 | </tr>` |
| 274 | 290 | } |
| 275 | 291 | |
| … | … | @@ -287,13 +303,13 @@ function renderDetailRow (n) { |
| 287 | 303 | const feat = f[key] || {} |
| 288 | 304 | const isOn = feat.enabled === true |
| 289 | 305 | const canToggle = TOGGLEABLE.includes(key) && !offline |
| 290 | return `<div class="feature-item ${isOn?'on':'off'}"> | |
| 291 | <span class="feat-status ${isOn?'on':'off'}">${isOn?'ON':'OFF'}</span> | |
| 306 | return `<div class="feature-item ${isOn ? 'on' : 'off'}"> | |
| 307 | <span class="feat-status ${isOn ? 'on' : 'off'}">${isOn ? 'ON' : 'OFF'}</span> | |
| 292 | 308 | <span class="feat-label">${label}</span> |
| 293 | ${feat.running != null ? `<span class="feat-extra">${feat.running?'running':'stopped'}</span>` : ''} | |
| 309 | ${feat.running != null ? `<span class="feat-extra">${feat.running ? 'running' : 'stopped'}</span>` : ''} | |
| 294 | 310 | ${feat.count != null ? `<span class="feat-extra">${feat.count} svc</span>` : ''} |
| 295 | 311 | ${feat.coresSeeded != null ? `<span class="feat-extra">${feat.coresSeeded} cores</span>` : ''} |
| 296 | ${canToggle ? `<button class="feat-toggle ${isOn?'off':'on'}" onclick="toggleFeature('${esc(n.name)}','${key}',${!isOn})">${isOn?'Disable':'Enable'}</button>` : ''} | |
| 312 | ${canToggle ? `<button class="feat-toggle ${isOn ? 'off' : 'on'}" onclick="toggleFeature('${esc(n.name)}','${key}',${!isOn})">${isOn ? 'Disable' : 'Enable'}</button>` : ''} | |
| 297 | 313 | </div>` |
| 298 | 314 | }).join('')} |
| 299 | 315 | </div> |
| … | … | @@ -315,31 +331,35 @@ function renderDetailRow (n) { |
| 315 | 331 | ${renderStat('Blocks served', n.blocksServed)} |
| 316 | 332 | </div> |
| 317 | 333 | |
| 318 | ${n.services && n.services.length > 0 ? ` | |
| 334 | ${n.services && n.services.length > 0 | |
| 335 | ? ` | |
| 319 | 336 | <div class="detail-section-title" style="margin-top:16px">Service Layer (${n.services.length})</div> |
| 320 | 337 | <div class="svc-list"> |
| 321 | ${n.services.map(s => `<div class="svc-item"><span class="svc-name">${esc(s.name)}</span><span class="svc-ver">v${esc(s.version)}</span><span class="svc-caps">${(s.capabilities||[]).length} caps</span></div>`).join('')} | |
| 322 | </div>` : ''} | |
| 338 | ${n.services.map(s => `<div class="svc-item"><span class="svc-name">${esc(s.name)}</span><span class="svc-ver">v${esc(s.version)}</span><span class="svc-caps">${(s.capabilities || []).length} caps</span></div>`).join('')} | |
| 339 | </div>` | |
| 340 | : ''} | |
| 323 | 341 | |
| 324 | ${n.transports ? ` | |
| 342 | ${n.transports | |
| 343 | ? ` | |
| 325 | 344 | <div class="detail-section-title" style="margin-top:16px">Transports</div> |
| 326 | 345 | <div class="stats-grid"> |
| 327 | 346 | ${renderTransport('DHT WS', n.transports.dhtRelayWs)} |
| 328 | 347 | ${renderTransport('Tor', n.transports.tor)} |
| 329 | 348 | ${renderTransport('Holesail', n.transports.holesail)} |
| 330 | </div>` : ''} | |
| 349 | </div>` | |
| 350 | : ''} | |
| 331 | 351 | </div> |
| 332 | 352 | <div class="detail-col"> |
| 333 | 353 | <div class="detail-section-title">Controls</div> |
| 334 | 354 | <div class="control-list"> |
| 335 | <button class="ctrl-btn" onclick="nodeAction('${esc(n.name)}','restart')" ${offline?'disabled':''}>Restart Relay</button> | |
| 336 | <button class="ctrl-btn" onclick="nodeAction('${esc(n.name)}','update')" ${offline?'disabled':''}>Update (git pull + restart)</button> | |
| 337 | <button class="ctrl-btn" onclick="nodeAction('${esc(n.name)}','doctor')" ${offline?'disabled':''}>Doctor (swap + locks + cleanup)</button> | |
| 355 | <button class="ctrl-btn" onclick="nodeAction('${esc(n.name)}','restart')" ${offline ? 'disabled' : ''}>Restart Relay</button> | |
| 356 | <button class="ctrl-btn" onclick="nodeAction('${esc(n.name)}','update')" ${offline ? 'disabled' : ''}>Update (git pull + restart)</button> | |
| 357 | <button class="ctrl-btn" onclick="nodeAction('${esc(n.name)}','doctor')" ${offline ? 'disabled' : ''}>Doctor (swap + locks + cleanup)</button> | |
| 338 | 358 | </div> |
| 339 | 359 | |
| 340 | 360 | <div class="detail-section-title" style="margin-top:16px">Environment</div> |
| 341 | 361 | <div class="env-list"> |
| 342 | ${Object.entries(n.env||{}).map(([k,v]) => `<div class="env-item"><span class="env-key">${esc(k)}</span><span class="env-val">${esc(k.includes('KEY')?'[redacted]':v)}</span></div>`).join('') || '<div class="env-empty">No env vars</div>'} | |
| 362 | ${Object.entries(n.env || {}).map(([k, v]) => `<div class="env-item"><span class="env-key">${esc(k)}</span><span class="env-val">${esc(k.includes('KEY') ? '[redacted]' : v)}</span></div>`).join('') || '<div class="env-empty">No env vars</div>'} | |
| 343 | 363 | </div> |
| 344 | 364 | </div> |
| 345 | 365 | </div> |
| … | … | @@ -352,13 +372,13 @@ function renderStat (label, value) { |
| 352 | 372 | |
| 353 | 373 | function renderTransport (name, t) { |
| 354 | 374 | if (!t) return `<div class="stat-item"><span class="stat-label">${name}</span><span class="stat-value">disabled</span></div>` |
| 355 | return `<div class="stat-item"><span class="stat-label">${name}</span><span class="stat-value">${t.running?'running':'stopped'} ${t.activeConnections!=null?`(${t.activeConnections} active)`:''}</span></div>` | |
| 375 | return `<div class="stat-item"><span class="stat-label">${name}</span><span class="stat-value">${t.running ? 'running' : 'stopped'} ${t.activeConnections != null ? `(${t.activeConnections} active)` : ''}</span></div>` | |
| 356 | 376 | } |
| 357 | 377 | |
| 358 | 378 | function renderActionLog (actions) { |
| 359 | 379 | if (!actions || actions.length === 0) return '' |
| 360 | return `<div class="section"><div class="section-title">Action Log <span class="count">last ${Math.min(actions.length,10)}</span></div> | |
| 361 | <div class="action-log">${actions.slice(0,10).map(a => `<div class="action-item"><span class="action-time">${new Date(a.timestamp).toLocaleTimeString()}</span><span class="action-node">${esc(a.node)}</span><span class="action-type">${esc(a.action)}</span><span class="action-detail">${esc(a.detail||'')}</span></div>`).join('')}</div></div>` | |
| 380 | return `<div class="section"><div class="section-title">Action Log <span class="count">last ${Math.min(actions.length, 10)}</span></div> | |
| 381 | <div class="action-log">${actions.slice(0, 10).map(a => `<div class="action-item"><span class="action-time">${new Date(a.timestamp).toLocaleTimeString()}</span><span class="action-node">${esc(a.node)}</span><span class="action-type">${esc(a.action)}</span><span class="action-detail">${esc(a.detail || '')}</span></div>`).join('')}</div></div>` | |
| 362 | 382 | } |
| 363 | 383 | |
| 364 | 384 | function renderDiskBar (n) { |
| … | … | @@ -397,10 +417,10 @@ async function nodeAction (name, action) { |
| 397 | 417 | } |
| 398 | 418 | |
| 399 | 419 | async function toggleFeature (name, feature, enabled) { |
| 400 | if (!confirm(`${enabled?'Enable':'Disable'} ${feature} on ${name}?`)) return | |
| 420 | if (!confirm(`${enabled ? 'Enable' : 'Disable'} ${feature} on ${name}?`)) return | |
| 401 | 421 | try { |
| 402 | showToast({ node: name, action: `${enabled?'enable':'disable'} ${feature}`, detail: 'applying...' }) | |
| 403 | const res = await fetch(`/api/node/${name}/toggle`, { method: 'POST', headers: {'content-type':'application/json'}, body: JSON.stringify({ feature, enabled }) }) | |
| 422 | showToast({ node: name, action: `${enabled ? 'enable' : 'disable'} ${feature}`, detail: 'applying...' }) | |
| 423 | const res = await fetch(`/api/node/${name}/toggle`, { method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify({ feature, enabled }) }) | |
| 404 | 424 | const data = await res.json() |
| 405 | 425 | if (data.error) alert(data.error) |
| 406 | 426 | await fetchState() |
| … | … | @@ -487,9 +507,21 @@ async function submitAddRelay () { |
| 487 | 507 | function showToast (data) { |
| 488 | 508 | const el = document.createElement('div') |
| 489 | 509 | el.className = 'toast' |
| 490 | el.innerHTML = `<strong>${esc(data.node)}</strong> · ${esc(data.action)} · ${esc(data.detail||'')}` | |
| 510 | el.innerHTML = `<strong>${esc(data.node)}</strong> · ${esc(data.action)} · ${esc(data.detail || '')}` | |
| 491 | 511 | document.getElementById('toast-container')?.appendChild(el) |
| 492 | 512 | setTimeout(() => el.remove(), 5000) |
| 493 | 513 | } |
| 494 | 514 | |
| 515 | Object.assign(globalThis, { | |
| 516 | setAutoControl, | |
| 517 | runControlNow, | |
| 518 | stabilizeAll, | |
| 519 | toggleDetail, | |
| 520 | nodeAction, | |
| 521 | toggleFeature, | |
| 522 | reprobe, | |
| 523 | showAddRelay, | |
| 524 | submitAddRelay | |
| 525 | }) | |
| 526 | ||
| 495 | 527 | fetchState().then(() => connectSSE()) |
| … | … | @@ -1,3 +1,4 @@ |
| 1 | /* global EventSource, alert, confirm */ | |
| 1 | 2 | /* HiveRelay · Dashboard Admin Operator — live fleet control plane */ |
| 2 | 3 | |
| 3 | 4 | let state = null |
| … | … | @@ -6,18 +7,28 @@ let expandedNode = null |
| 6 | 7 | |
| 7 | 8 | const REGION_NAMES = { NA: 'North America', EU: 'Europe', APAC: 'Asia-Pacific', ME: 'Middle East' } |
| 8 | 9 | const FEATURE_LABELS = { |
| 9 | coreRelay: 'Core Relay', seeding: 'Seeding', services: 'Service Layer', | |
| 10 | dhtRelayWs: 'DHT Relay (WS)', tor: 'Tor Transport', holesail: 'Holesail', | |
| 11 | outboxlog: 'OutboxLog', forwardRelay: 'Forward Relay', signedDirectory: 'Signed Directory', | |
| 12 | vrf: 'VRF (RFC 9381)', poker: 'Poker Substrate', zk: 'ZK Proofs', | |
| 13 | arbitration: 'Arbitration', payment: 'Payments', eviction: 'Auto-Eviction', | |
| 14 | autoHeal: 'Auto-Heal', distributedDrive: 'Distributed Drive' | |
| 10 | coreRelay: 'Core Relay', | |
| 11 | seeding: 'Seeding', | |
| 12 | services: 'Service Layer', | |
| 13 | dhtRelayWs: 'DHT Relay (WS)', | |
| 14 | tor: 'Tor Transport', | |
| 15 | holesail: 'Holesail', | |
| 16 | outboxlog: 'OutboxLog', | |
| 17 | forwardRelay: 'Forward Relay', | |
| 18 | signedDirectory: 'Signed Directory', | |
| 19 | vrf: 'VRF (RFC 9381)', | |
| 20 | poker: 'Poker Substrate', | |
| 21 | zk: 'ZK Proofs', | |
| 22 | arbitration: 'Arbitration', | |
| 23 | payment: 'Payments', | |
| 24 | eviction: 'Auto-Eviction', | |
| 25 | autoHeal: 'Auto-Heal', | |
| 26 | distributedDrive: 'Distributed Drive' | |
| 15 | 27 | } |
| 16 | 28 | const TOGGLEABLE = ['dhtRelayWs', 'outboxlog', 'forwardRelay', 'signedDirectory', 'services', 'eviction'] |
| 17 | 29 | |
| 18 | 30 | async function fetchState () { |
| 19 | try { state = await (await fetch('/api/state')).json(); render() } | |
| 20 | catch { document.getElementById('app').innerHTML = '<div class="loading">Connection lost...</div>' } | |
| 31 | try { state = await (await fetch('/api/state')).json(); render() } catch { document.getElementById('app').innerHTML = '<div class="loading">Connection lost...</div>' } | |
| 21 | 32 | } |
| 22 | 33 | |
| 23 | 34 | function connectSSE () { |
| … | … | @@ -63,7 +74,8 @@ function computeSummary (nodes) { |
| 63 | 74 | else regions[r].degraded++ |
| 64 | 75 | } |
| 65 | 76 | return { |
| 66 | total: nodes.length, healthy: nodes.filter(n => n.status === 'healthy').length, | |
| 77 | total: nodes.length, | |
| 78 | healthy: nodes.filter(n => n.status === 'healthy').length, | |
| 67 | 79 | degraded: nodes.filter(n => n.status === 'degraded').length, |
| 68 | 80 | offline: nodes.filter(n => n.status === 'offline').length, |
| 69 | 81 | running: nodes.filter(n => n.running === true).length, |
| … | … | @@ -78,7 +90,7 @@ function computeSummary (nodes) { |
| 78 | 90 | |
| 79 | 91 | function fmtBytes (b) { |
| 80 | 92 | if (!b || b === 0) return '0 B' |
| 81 | const u = ['B','KB','MB','GB','TB','PB'] | |
| 93 | const u = ['B', 'KB', 'MB', 'GB', 'TB', 'PB'] | |
| 82 | 94 | const i = Math.floor(Math.log(b) / Math.log(1024)) |
| 83 | 95 | return (b / Math.pow(1024, i)).toFixed(1) + ' ' + u[i] |
| 84 | 96 | } |
| … | … | @@ -112,13 +124,13 @@ function render () { |
| 112 | 124 | ${renderActionLog(state.actions)} |
| 113 | 125 | </div> |
| 114 | 126 | <div id="toast-container"></div> |
| 115 | <div class="footer"><div class="footer-meta">Admin Operator · full fleet control · probe ${state.interval||60}s · auto-control ${state.control?.autoControl ? 'ON' : 'OFF'} · ${new Date().toISOString()}</div></div> | |
| 127 | <div class="footer"><div class="footer-meta">Admin Operator · full fleet control · probe ${state.interval || 60}s · auto-control ${state.control?.autoControl ? 'ON' : 'OFF'} · ${new Date().toISOString()}</div></div> | |
| 116 | 128 | ` |
| 117 | 129 | } |
| 118 | 130 | |
| 119 | 131 | function renderControlPanel (ctrl) { |
| 120 | 132 | if (!ctrl) { |
| 121 | return `<div class="control-panel"><div class="control-panel-head"><strong>Fleet control</strong><span class="dim">waiting for first probe…</span></div></div>` | |
| 133 | return '<div class="control-panel"><div class="control-panel-head"><strong>Fleet control</strong><span class="dim">waiting for first probe…</span></div></div>' | |
| 122 | 134 | } |
| 123 | 135 | const auto = !!ctrl.autoControl |
| 124 | 136 | const cycle = ctrl.lastCycle |
| … | … | @@ -153,21 +165,25 @@ function renderControlPanel (ctrl) { |
| 153 | 165 | <span class="range-chip">large ≥4GB: mem≤4.8GB</span> |
| 154 | 166 | <span class="range-chip">swap ≥256MB · watchdog required · disk warn 75%</span> |
| 155 | 167 | </div> |
| 156 | ${violators.length ? ` | |
| 168 | ${violators.length | |
| 169 | ? ` | |
| 157 | 170 | <div class="violations"> |
| 158 | 171 | <div class="section-title" style="margin:8px 0 6px">Out of range <span class="count">${violators.length}</span></div> |
| 159 | 172 | ${violators.map(v => ` |
| 160 | 173 | <div class="violation-row"> |
| 161 | 174 | <strong>${esc(v.name)}</strong> |
| 162 | <span class="badge ${esc(v.tier||'')}">${esc(v.tier||'?')}</span> | |
| 163 | ${(v.violations||[]).map(x => `<span class="vchip ${x.severity}">${esc(x.code)}: ${esc(x.msg)}</span>`).join('')} | |
| 164 | ${(v.actions||[]).map(a => `<span class="auto-act">→ ${esc(a.type)}</span>`).join('')} | |
| 175 | <span class="badge ${esc(v.tier || '')}">${esc(v.tier || '?')}</span> | |
| 176 | ${(v.violations || []).map(x => `<span class="vchip ${x.severity}">${esc(x.code)}: ${esc(x.msg)}</span>`).join('')} | |
| 177 | ${(v.actions || []).map(a => `<span class="auto-act">→ ${esc(a.type)}</span>`).join('')} | |
| 165 | 178 | </div>`).join('')} |
| 166 | </div>` : `<div class="in-range-ok">All reachable nodes within healthy ranges (or offline only).</div>`} | |
| 167 | ${remediations.length ? ` | |
| 179 | </div>` | |
| 180 | : '<div class="in-range-ok">All reachable nodes within healthy ranges (or offline only).</div>'} | |
| 181 | ${remediations.length | |
| 182 | ? ` | |
| 168 | 183 | <div class="remed-log"> |
| 169 | ${remediations.map(r => `<div class="remed-item ${r.ok?'ok':'fail'}">${esc(r.node)} · auto:${esc(r.type)} · ${esc(r.reason)} · ${r.ok?'ok':'fail'} ${esc(r.detail||'')}</div>`).join('')} | |
| 170 | </div>` : ''} | |
| 184 | ${remediations.map(r => `<div class="remed-item ${r.ok ? 'ok' : 'fail'}">${esc(r.node)} · auto:${esc(r.type)} · ${esc(r.reason)} · ${r.ok ? 'ok' : 'fail'} ${esc(r.detail || '')}</div>`).join('')} | |
| 185 | </div>` | |
| 186 | : ''} | |
| 171 | 187 | </div>` |
| 172 | 188 | } |
| 173 | 189 | |
| … | … | @@ -190,7 +206,7 @@ async function runControlNow () { |
| 190 | 206 | const res = await fetch('/api/control/run', { method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify({ force: true }) }) |
| 191 | 207 | const data = await res.json() |
| 192 | 208 | if (data.error) alert(data.error) |
| 193 | else showToast({ node: 'fleet', action: 'control cycle', detail: `done · ${data.cycle?.remediations?.length||0} remediations` }) | |
| 209 | else showToast({ node: 'fleet', action: 'control cycle', detail: `done · ${data.cycle?.remediations?.length || 0} remediations` }) | |
| 194 | 210 | await fetchState() |
| 195 | 211 | } catch (e) { alert(e.message) } |
| 196 | 212 | } |
| … | … | @@ -225,14 +241,14 @@ function renderKPIs (s) { |
| 225 | 241 | function renderRegions (regions) { |
| 226 | 242 | if (!regions) return '' |
| 227 | 243 | return `<div class="regions">${Object.entries(regions).map(([code, r]) => ` |
| 228 | <div class="region-card"><div class="region-name">${REGION_NAMES[code]||code}</div> | |
| 229 | <div class="region-stats"><span class="up">${r.healthy} up</span>${r.degraded?`<span class="deg">${r.degraded} deg</span>`:''}${r.offline?`<span class="down">${r.offline} down</span>`:''}<span class="total">${r.total}</span></div></div>`).join('')}</div>` | |
| 244 | <div class="region-card"><div class="region-name">${REGION_NAMES[code] || code}</div> | |
| 245 | <div class="region-stats"><span class="up">${r.healthy} up</span>${r.degraded ? `<span class="deg">${r.degraded} deg</span>` : ''}${r.offline ? `<span class="down">${r.offline} down</span>` : ''}<span class="total">${r.total}</span></div></div>`).join('')}</div>` | |
| 230 | 246 | } |
| 231 | 247 | |
| 232 | 248 | function renderNodeTable (nodes) { |
| 233 | 249 | const sorted = [...nodes].sort((a, b) => { |
| 234 | 250 | const order = { offline: 3, degraded: 2, healthy: 1 } |
| 235 | return (order[a.status]||3) - (order[b.status]||3) || a.name.localeCompare(b.name) | |
| 251 | return (order[a.status] || 3) - (order[b.status] || 3) || a.name.localeCompare(b.name) | |
| 236 | 252 | }) |
| 237 | 253 | return ` |
| 238 | 254 | <div class="section"> |
| … | … | @@ -257,19 +273,19 @@ function renderRow (n) { |
| 257 | 273 | <td><span class="chevron ${isExpanded ? 'open' : ''}">▶</span></td> |
| 258 | 274 | <td><span class="indicator"><span class="dot ${n.status}"></span></span> <strong>${esc(n.name)}</strong></td> |
| 259 | 275 | <td class="mono">${esc(n.publicIp)}</td> |
| 260 | <td class="${verOk?'version-ok':'version-bad'} mono-val">${esc(n.version||'—')}</td> | |
| 261 | <td>${n.running&&n.healthOk!==false?'<span class="ok-text">running</span>':(n.running?'<span class="err-text">hung?</span>':'<span class="err-text">stopped</span>')}</td> | |
| 276 | <td class="${verOk ? 'version-ok' : 'version-bad'} mono-val">${esc(n.version || '—')}</td> | |
| 277 | <td>${n.running && n.healthOk !== false ? '<span class="ok-text">running</span>' : (n.running ? '<span class="err-text">hung?</span>' : '<span class="err-text">stopped</span>')}</td> | |
| 262 | 278 | <td>${renderDiskBar(n)}</td> |
| 263 | <td class="mono-val">${mem!=null?mem+'M':'—'}</td> | |
| 279 | <td class="mono-val">${mem != null ? mem + 'M' : '—'}</td> | |
| 264 | 280 | <td>${renderRamBar(n)}</td> |
| 265 | <td class="mono">${n.loadAvg!=null?Number(n.loadAvg).toFixed(2):'—'}</td> | |
| 266 | <td class="mono-val">${n.seededApps||'—'}</td> | |
| 267 | <td class="mono-val">${n.connections||'—'}</td> | |
| 268 | <td>${wdOk?'<span class="ok-text">on</span>':'<span class="err-text">off</span>'}</td> | |
| 269 | <td>${svcCount?`<span class="svc-count">${svcCount}</span>`:'—'}</td> | |
| 270 | <td><span class="badge ${esc(n.channel||'stable')}">${esc(n.channel||'stable')}</span></td> | |
| 271 | <td>${esc(n.region||'—')}</td> | |
| 272 | <td class="mono">${n.vcpu||'?'}c/${n.ramGB||'?'}G/${n.diskGB||'?'}G</td> | |
| 281 | <td class="mono">${n.loadAvg != null ? Number(n.loadAvg).toFixed(2) : '—'}</td> | |
| 282 | <td class="mono-val">${n.seededApps || '—'}</td> | |
| 283 | <td class="mono-val">${n.connections || '—'}</td> | |
| 284 | <td>${wdOk ? '<span class="ok-text">on</span>' : '<span class="err-text">off</span>'}</td> | |
| 285 | <td>${svcCount ? `<span class="svc-count">${svcCount}</span>` : '—'}</td> | |
| 286 | <td><span class="badge ${esc(n.channel || 'stable')}">${esc(n.channel || 'stable')}</span></td> | |
| 287 | <td>${esc(n.region || '—')}</td> | |
| 288 | <td class="mono">${n.vcpu || '?'}c/${n.ramGB || '?'}G/${n.diskGB || '?'}G</td> | |
| 273 | 289 | </tr>` |
| 274 | 290 | } |
| 275 | 291 | |
| … | … | @@ -287,13 +303,13 @@ function renderDetailRow (n) { |
| 287 | 303 | const feat = f[key] || {} |
| 288 | 304 | const isOn = feat.enabled === true |
| 289 | 305 | const canToggle = TOGGLEABLE.includes(key) && !offline |
| 290 | return `<div class="feature-item ${isOn?'on':'off'}"> | |
| 291 | <span class="feat-status ${isOn?'on':'off'}">${isOn?'ON':'OFF'}</span> | |
| 306 | return `<div class="feature-item ${isOn ? 'on' : 'off'}"> | |
| 307 | <span class="feat-status ${isOn ? 'on' : 'off'}">${isOn ? 'ON' : 'OFF'}</span> | |
| 292 | 308 | <span class="feat-label">${label}</span> |
| 293 | ${feat.running != null ? `<span class="feat-extra">${feat.running?'running':'stopped'}</span>` : ''} | |
| 309 | ${feat.running != null ? `<span class="feat-extra">${feat.running ? 'running' : 'stopped'}</span>` : ''} | |
| 294 | 310 | ${feat.count != null ? `<span class="feat-extra">${feat.count} svc</span>` : ''} |
| 295 | 311 | ${feat.coresSeeded != null ? `<span class="feat-extra">${feat.coresSeeded} cores</span>` : ''} |
| 296 | ${canToggle ? `<button class="feat-toggle ${isOn?'off':'on'}" onclick="toggleFeature('${esc(n.name)}','${key}',${!isOn})">${isOn?'Disable':'Enable'}</button>` : ''} | |
| 312 | ${canToggle ? `<button class="feat-toggle ${isOn ? 'off' : 'on'}" onclick="toggleFeature('${esc(n.name)}','${key}',${!isOn})">${isOn ? 'Disable' : 'Enable'}</button>` : ''} | |
| 297 | 313 | </div>` |
| 298 | 314 | }).join('')} |
| 299 | 315 | </div> |
| … | … | @@ -315,31 +331,35 @@ function renderDetailRow (n) { |
| 315 | 331 | ${renderStat('Blocks served', n.blocksServed)} |
| 316 | 332 | </div> |
| 317 | 333 | |
| 318 | ${n.services && n.services.length > 0 ? ` | |
| 334 | ${n.services && n.services.length > 0 | |
| 335 | ? ` | |
| 319 | 336 | <div class="detail-section-title" style="margin-top:16px">Service Layer (${n.services.length})</div> |
| 320 | 337 | <div class="svc-list"> |
| 321 | ${n.services.map(s => `<div class="svc-item"><span class="svc-name">${esc(s.name)}</span><span class="svc-ver">v${esc(s.version)}</span><span class="svc-caps">${(s.capabilities||[]).length} caps</span></div>`).join('')} | |
| 322 | </div>` : ''} | |
| 338 | ${n.services.map(s => `<div class="svc-item"><span class="svc-name">${esc(s.name)}</span><span class="svc-ver">v${esc(s.version)}</span><span class="svc-caps">${(s.capabilities || []).length} caps</span></div>`).join('')} | |
| 339 | </div>` | |
| 340 | : ''} | |
| 323 | 341 | |
| 324 | ${n.transports ? ` | |
| 342 | ${n.transports | |
| 343 | ? ` | |
| 325 | 344 | <div class="detail-section-title" style="margin-top:16px">Transports</div> |
| 326 | 345 | <div class="stats-grid"> |
| 327 | 346 | ${renderTransport('DHT WS', n.transports.dhtRelayWs)} |
| 328 | 347 | ${renderTransport('Tor', n.transports.tor)} |
| 329 | 348 | ${renderTransport('Holesail', n.transports.holesail)} |
| 330 | </div>` : ''} | |
| 349 | </div>` | |
| 350 | : ''} | |
| 331 | 351 | </div> |
| 332 | 352 | <div class="detail-col"> |
| 333 | 353 | <div class="detail-section-title">Controls</div> |
| 334 | 354 | <div class="control-list"> |
| 335 | <button class="ctrl-btn" onclick="nodeAction('${esc(n.name)}','restart')" ${offline?'disabled':''}>Restart Relay</button> | |
| 336 | <button class="ctrl-btn" onclick="nodeAction('${esc(n.name)}','update')" ${offline?'disabled':''}>Update (git pull + restart)</button> | |
| 337 | <button class="ctrl-btn" onclick="nodeAction('${esc(n.name)}','doctor')" ${offline?'disabled':''}>Doctor (swap + locks + cleanup)</button> | |
| 355 | <button class="ctrl-btn" onclick="nodeAction('${esc(n.name)}','restart')" ${offline ? 'disabled' : ''}>Restart Relay</button> | |
| 356 | <button class="ctrl-btn" onclick="nodeAction('${esc(n.name)}','update')" ${offline ? 'disabled' : ''}>Update (git pull + restart)</button> | |
| 357 | <button class="ctrl-btn" onclick="nodeAction('${esc(n.name)}','doctor')" ${offline ? 'disabled' : ''}>Doctor (swap + locks + cleanup)</button> | |
| 338 | 358 | </div> |
| 339 | 359 | |
| 340 | 360 | <div class="detail-section-title" style="margin-top:16px">Environment</div> |
| 341 | 361 | <div class="env-list"> |
| 342 | ${Object.entries(n.env||{}).map(([k,v]) => `<div class="env-item"><span class="env-key">${esc(k)}</span><span class="env-val">${esc(k.includes('KEY')?'[redacted]':v)}</span></div>`).join('') || '<div class="env-empty">No env vars</div>'} | |
| 362 | ${Object.entries(n.env || {}).map(([k, v]) => `<div class="env-item"><span class="env-key">${esc(k)}</span><span class="env-val">${esc(k.includes('KEY') ? '[redacted]' : v)}</span></div>`).join('') || '<div class="env-empty">No env vars</div>'} | |
| 343 | 363 | </div> |
| 344 | 364 | </div> |
| 345 | 365 | </div> |
| … | … | @@ -352,13 +372,13 @@ function renderStat (label, value) { |
| 352 | 372 | |
| 353 | 373 | function renderTransport (name, t) { |
| 354 | 374 | if (!t) return `<div class="stat-item"><span class="stat-label">${name}</span><span class="stat-value">disabled</span></div>` |
| 355 | return `<div class="stat-item"><span class="stat-label">${name}</span><span class="stat-value">${t.running?'running':'stopped'} ${t.activeConnections!=null?`(${t.activeConnections} active)`:''}</span></div>` | |
| 375 | return `<div class="stat-item"><span class="stat-label">${name}</span><span class="stat-value">${t.running ? 'running' : 'stopped'} ${t.activeConnections != null ? `(${t.activeConnections} active)` : ''}</span></div>` | |
| 356 | 376 | } |
| 357 | 377 | |
| 358 | 378 | function renderActionLog (actions) { |
| 359 | 379 | if (!actions || actions.length === 0) return '' |
| 360 | return `<div class="section"><div class="section-title">Action Log <span class="count">last ${Math.min(actions.length,10)}</span></div> | |
| 361 | <div class="action-log">${actions.slice(0,10).map(a => `<div class="action-item"><span class="action-time">${new Date(a.timestamp).toLocaleTimeString()}</span><span class="action-node">${esc(a.node)}</span><span class="action-type">${esc(a.action)}</span><span class="action-detail">${esc(a.detail||'')}</span></div>`).join('')}</div></div>` | |
| 380 | return `<div class="section"><div class="section-title">Action Log <span class="count">last ${Math.min(actions.length, 10)}</span></div> | |
| 381 | <div class="action-log">${actions.slice(0, 10).map(a => `<div class="action-item"><span class="action-time">${new Date(a.timestamp).toLocaleTimeString()}</span><span class="action-node">${esc(a.node)}</span><span class="action-type">${esc(a.action)}</span><span class="action-detail">${esc(a.detail || '')}</span></div>`).join('')}</div></div>` | |
| 362 | 382 | } |
| 363 | 383 | |
| 364 | 384 | function renderDiskBar (n) { |
| … | … | @@ -397,10 +417,10 @@ async function nodeAction (name, action) { |
| 397 | 417 | } |
| 398 | 418 | |
| 399 | 419 | async function toggleFeature (name, feature, enabled) { |
| 400 | if (!confirm(`${enabled?'Enable':'Disable'} ${feature} on ${name}?`)) return | |
| 420 | if (!confirm(`${enabled ? 'Enable' : 'Disable'} ${feature} on ${name}?`)) return | |
| 401 | 421 | try { |
| 402 | showToast({ node: name, action: `${enabled?'enable':'disable'} ${feature}`, detail: 'applying...' }) | |
| 403 | const res = await fetch(`/api/node/${name}/toggle`, { method: 'POST', headers: {'content-type':'application/json'}, body: JSON.stringify({ feature, enabled }) }) | |
| 422 | showToast({ node: name, action: `${enabled ? 'enable' : 'disable'} ${feature}`, detail: 'applying...' }) | |
| 423 | const res = await fetch(`/api/node/${name}/toggle`, { method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify({ feature, enabled }) }) | |
| 404 | 424 | const data = await res.json() |
| 405 | 425 | if (data.error) alert(data.error) |
| 406 | 426 | await fetchState() |
| … | … | @@ -487,9 +507,21 @@ async function submitAddRelay () { |
| 487 | 507 | function showToast (data) { |
| 488 | 508 | const el = document.createElement('div') |
| 489 | 509 | el.className = 'toast' |
| 490 | el.innerHTML = `<strong>${esc(data.node)}</strong> · ${esc(data.action)} · ${esc(data.detail||'')}` | |
| 510 | el.innerHTML = `<strong>${esc(data.node)}</strong> · ${esc(data.action)} · ${esc(data.detail || '')}` | |
| 491 | 511 | document.getElementById('toast-container')?.appendChild(el) |
| 492 | 512 | setTimeout(() => el.remove(), 5000) |
| 493 | 513 | } |
| 494 | 514 | |
| 515 | Object.assign(globalThis, { | |
| 516 | setAutoControl, | |
| 517 | runControlNow, | |
| 518 | stabilizeAll, | |
| 519 | toggleDetail, | |
| 520 | nodeAction, | |
| 521 | toggleFeature, | |
| 522 | reprobe, | |
| 523 | showAddRelay, | |
| 524 | submitAddRelay | |
| 525 | }) | |
| 526 | ||
| 495 | 527 | fetchState().then(() => connectSSE()) |
| … | … | @@ -56,8 +56,8 @@ function presentedToken (req) { |
| 56 | 56 | return '' |
| 57 | 57 | } |
| 58 | 58 | } |
| 59 | const POLICY_PATH = process.env.HIVERELAY_CONTROL_POLICY | |
| 60 | || path.join(__dirname, 'control-policy.json') | |
| 59 | const POLICY_PATH = process.env.HIVERELAY_CONTROL_POLICY || | |
| 60 | path.join(__dirname, 'control-policy.json') | |
| 61 | 61 | |
| 62 | 62 | function expandHome (p) { |
| 63 | 63 | if (!p) return p |
| … | … | @@ -120,7 +120,7 @@ function probeNode (relay) { |
| 120 | 120 | : [] |
| 121 | 121 | |
| 122 | 122 | const remote = [ |
| 123 | 'V=$(grep -m1 \'"version"\' ~/hiverelay/package.json 2>/dev/null | tr -dc \'0-9.\' || echo \'none\')', | |
| 123 | 'V=$(node -p \'require(process.env.HOME + "/hiverelay/package.json").version\' 2>/dev/null || echo \'none\')', | |
| 124 | 124 | 'D=$(df -h / | awk \'NR==2{print $5}\' | tr -dc \'0-9\')', |
| 125 | 125 | 'DG=$(df -h / | awk \'NR==2{print $2}\' | tr -dc \'0-9.\')', |
| 126 | 126 | 'SW=$(free -m 2>/dev/null | awk \'/Swap/{print $2}\' | head -1)', |
| … | … | @@ -142,6 +142,8 @@ function probeNode (relay) { |
| 142 | 142 | 'HEALTH=$(curl -fsS --max-time 4 http://127.0.0.1:9100/health 2>/dev/null || echo \'\')', |
| 143 | 143 | 'WD=$(systemctl is-enabled hiverelay-health-watchdog.timer 2>/dev/null || echo missing)', |
| 144 | 144 | 'HO=0; echo "$HEALTH" | grep -q \'"ok"[[:space:]]*:[[:space:]]*true\' && HO=1', |
| 145 | // Shell parameter expansions, not JavaScript template placeholders. | |
| 146 | // eslint-disable-next-line no-template-curly-in-string | |
| 145 | 147 | 'printf \'{"version":"%s","diskPct":%s,"diskGB":%s,"swapMB":%s,"ramUsedPct":%s,"loadAvg":%s,"uptime":"%s","restarts":%s,"memCurrentMB":%s,"tasks":%s,"env":"%s","watchdog":"%s","healthOk":%s,"config":%s,"status":%s,"cap":%s}\\n\' "$V" "${D:-0}" "${DG:-0}" "${SW:-0}" "${RM:-0}" "${LA:-0}" "$UP" "${RT:-0}" "${MC:-0}" "${TS:-0}" "$ENV" "$WD" "$HO" "$CFG" "$STATUS" "$CAP"' |
| 146 | 148 | ].join('\n') |
| 147 | 149 | |
| … | … | @@ -379,7 +381,7 @@ const server = http.createServer(async (req, res) => { |
| 379 | 381 | |
| 380 | 382 | // SSE |
| 381 | 383 | if (req.url === '/api/events') { |
| 382 | res.writeHead(200, { 'content-type': 'text/event-stream', 'cache-control': 'no-cache', 'connection': 'keep-alive' }) | |
| 384 | res.writeHead(200, { 'content-type': 'text/event-stream', 'cache-control': 'no-cache', connection: 'keep-alive' }) | |
| 383 | 385 | res.write(`data: ${JSON.stringify({ type: 'fleet', data: fleetState, timestamp: lastProbe })}\n\n`) |
| 384 | 386 | res.write(`data: ${JSON.stringify({ type: 'actions', data: actionLog })}\n\n`) |
| 385 | 387 | sseClients.push(res) |
| … | … | @@ -554,7 +556,6 @@ const server = http.createServer(async (req, res) => { |
| 554 | 556 | if (idx >= 0) fleetState[idx] = updated |
| 555 | 557 | broadcast({ type: 'fleet', data: fleetState, timestamp: new Date().toISOString() }) |
| 556 | 558 | return sendJson(res, 200, { ok: true, state: updated }) |
| 557 | ||
| 558 | 559 | } else if (action === 'update') { |
| 559 | 560 | const output = await sshAction(relay, 'cd ~/hiverelay && git pull --ff-only 2>&1 && npm install --production 2>&1 | tail -5 && systemctl restart hiverelay && sleep 5 && systemctl is-active hiverelay', 'update relay') |
| 560 | 561 | const updated = await probeNode(relay) |
| … | … | @@ -562,7 +563,6 @@ const server = http.createServer(async (req, res) => { |
| 562 | 563 | if (idx >= 0) fleetState[idx] = updated |
| 563 | 564 | broadcast({ type: 'fleet', data: fleetState, timestamp: new Date().toISOString() }) |
| 564 | 565 | return sendJson(res, 200, { ok: true, output: output.slice(0, 500), state: updated }) |
| 565 | ||
| 566 | 566 | } else if (action === 'doctor') { |
| 567 | 567 | const cmds = [ |
| 568 | 568 | 'echo "=== Swap ===" && free -m | grep Swap', |
| … | … | @@ -574,7 +574,6 @@ const server = http.createServer(async (req, res) => { |
| 574 | 574 | ].join('; ') |
| 575 | 575 | const output = await sshAction(relay, cmds, 'doctor relay') |
| 576 | 576 | return sendJson(res, 200, { ok: true, output }) |
| 577 | ||
| 578 | 577 | } else if (action === 'toggle') { |
| 579 | 578 | const body = await readBody(req) |
| 580 | 579 | const { feature, enabled } = body |
| … | … | @@ -728,12 +727,17 @@ function computeSummary () { |
| 728 | 727 | else regions[r].degraded++ |
| 729 | 728 | } |
| 730 | 729 | return { |
| 731 | total: nodes.length, healthy: nodes.filter(n => n.status === 'healthy').length, | |
| 730 | total: nodes.length, | |
| 731 | healthy: nodes.filter(n => n.status === 'healthy').length, | |
| 732 | 732 | degraded: nodes.filter(n => n.status === 'degraded').length, |
| 733 | 733 | offline: nodes.filter(n => n.status === 'offline').length, |
| 734 | 734 | running: nodes.filter(n => n.running === true).length, |
| 735 | 735 | diskWarnings: nodes.filter(n => n.diskPct && n.diskPct > 75).length, |
| 736 | totalRamGB: Math.round(totalRamGB), totalDiskGB, totalSeeded, totalConns, regions | |
| 736 | totalRamGB: Math.round(totalRamGB), | |
| 737 | totalDiskGB, | |
| 738 | totalSeeded, | |
| 739 | totalConns, | |
| 740 | regions | |
| 737 | 741 | } |
| 738 | 742 | } |
| 739 | 743 | |
| … | … | @@ -760,7 +764,7 @@ async function main () { |
| 760 | 764 | }) |
| 761 | 765 | server.on('error', reject) |
| 762 | 766 | }) |
| 763 | console.log(`[admin-operator] Running initial probe...`) | |
| 767 | console.log('[admin-operator] Running initial probe...') | |
| 764 | 768 | await probeAll() |
| 765 | 769 | const s = computeSummary() |
| 766 | 770 | console.log(`[admin-operator] Done: ${s.healthy} healthy, ${s.degraded} degraded, ${s.offline} offline`) |
| … | … | @@ -660,12 +660,6 @@ function fetchCatalogNames() { |
| 660 | 660 | }).catch(function() {}); |
| 661 | 661 | } |
| 662 | 662 | |
| 663 | function escapeHtml(s) { | |
| 664 | if (s === null || s === undefined) return ''; | |
| 665 | return String(s).replace(/[&<>"']/g, function(ch) { | |
| 666 | return { '&': '&', '<': '<', '>': '>', '"': '"', "'": ''' }[ch]; | |
| 667 | }); | |
| 668 | } | |
| 669 | 663 | function clearNode(node) { |
| 670 | 664 | while (node.firstChild) node.removeChild(node.firstChild); |
| 671 | 665 | node.textContent = ''; |
| … | … | @@ -677,6 +671,44 @@ function appendEl(parent, tag, className, text) { |
| 677 | 671 | parent.appendChild(el); |
| 678 | 672 | return el; |
| 679 | 673 | } |
| 674 | function renderNamespaceRows(parent, list, attention) { | |
| 675 | clearNode(parent); | |
| 676 | if (!list.length) { | |
| 677 | var emptyRow = appendEl(parent, 'div', 'r'); | |
| 678 | var emptyLabel = appendEl(emptyRow, 'span', '', 'No namespaces configured'); | |
| 679 | emptyLabel.style.color = 'var(--text-faint)'; | |
| 680 | appendEl(emptyRow, 'span'); | |
| 681 | appendEl(emptyRow, 'span'); | |
| 682 | return; | |
| 683 | } | |
| 684 | list.forEach(function(ns) { | |
| 685 | var cap = ns.capBytes24h != null ? formatBytes(ns.capBytes24h) : '∞'; | |
| 686 | var pct = (ns.capBytes24h && ns.capBytes24h > 0) | |
| 687 | ? Math.min(100, Math.round((ns.bytes24h || 0) / ns.capBytes24h * 100)) + '%' | |
| 688 | : ''; | |
| 689 | if (ns.capBytes24h && ns.bytes24h != null && ns.bytes24h / ns.capBytes24h >= 0.8) { | |
| 690 | attention.push({ text: 'Namespace ' + String(ns.name || '') + ' near daily byte cap', sev: 'warn' }); | |
| 691 | } | |
| 692 | var row = appendEl(parent, 'div', 'r'); | |
| 693 | var nameCell = appendEl(row, 'span'); | |
| 694 | appendEl(nameCell, 'span', '', ns.name || 'unnamed'); | |
| 695 | if (ns.blind) { | |
| 696 | nameCell.appendChild(document.createTextNode(' ')); | |
| 697 | var blind = appendEl(nameCell, 'span', '', 'blind'); | |
| 698 | blind.style.color = 'var(--accent)'; | |
| 699 | } | |
| 700 | appendEl(row, 'span', '', (ns.writers || 0) + ' writers'); | |
| 701 | appendEl(row, 'span', '', formatBytes(ns.bytes24h || 0) + ' / ' + cap + (pct ? ' (' + pct + ')' : '')); | |
| 702 | }); | |
| 703 | } | |
| 704 | function renderAttentionList(parent, attention) { | |
| 705 | clearNode(parent); | |
| 706 | attention.forEach(function(a) { | |
| 707 | var row = appendEl(parent, 'li'); | |
| 708 | appendEl(row, 'span', '', a.text); | |
| 709 | appendEl(row, 'span', 'sev ' + (a.sev === 'ok' ? 'ok' : a.sev === 'bad' ? 'bad' : ''), a.sev === 'ok' ? 'ok' : a.sev === 'bad' ? 'crit' : 'warn'); | |
| 710 | }); | |
| 711 | } | |
| 680 | 712 | function safeHttpUrl(value) { |
| 681 | 713 | var raw = String(value || '').trim(); |
| 682 | 714 | if (!/^https?:\/\//i.test(raw)) return ''; |
| … | … | @@ -1132,24 +1164,7 @@ function updateOperatorPulse(overview) { |
| 1132 | 1164 | var nsBody = document.getElementById('namespacesBody'); |
| 1133 | 1165 | if (nsPanel && nsBody) { |
| 1134 | 1166 | nsPanel.style.display = ''; |
| 1135 | if (!list.length) { | |
| 1136 | nsBody.innerHTML = '<div class="r"><span style="color:var(--text-faint)">No namespaces configured</span><span></span><span></span></div>'; | |
| 1137 | } else { | |
| 1138 | nsBody.innerHTML = list.map(function (ns) { | |
| 1139 | var cap = ns.capBytes24h != null ? formatBytes(ns.capBytes24h) : '∞'; | |
| 1140 | var pct = (ns.capBytes24h && ns.capBytes24h > 0) | |
| 1141 | ? Math.min(100, Math.round((ns.bytes24h || 0) / ns.capBytes24h * 100)) + '%' | |
| 1142 | : ''; | |
| 1143 | if (ns.capBytes24h && ns.bytes24h != null && ns.bytes24h / ns.capBytes24h >= 0.8) { | |
| 1144 | attention.push({ text: 'Namespace ' + ns.name + ' near daily byte cap', sev: 'warn' }); | |
| 1145 | } | |
| 1146 | return '<div class="r"><span>' + escapeHtml(ns.name) + | |
| 1147 | (ns.blind ? ' <span style="color:var(--accent)">blind</span>' : '') + | |
| 1148 | '</span><span>' + (ns.writers || 0) + ' writers</span><span>' + | |
| 1149 | formatBytes(ns.bytes24h || 0) + ' / ' + cap + (pct ? ' (' + pct + ')' : '') + | |
| 1150 | '</span></div>'; | |
| 1151 | }).join(''); | |
| 1152 | } | |
| 1167 | renderNamespaceRows(nsBody, list, attention); | |
| 1153 | 1168 | } |
| 1154 | 1169 | } |
| 1155 | 1170 | if (shard) { |
| … | … | @@ -1313,12 +1328,7 @@ function updateOperatorPulse(overview) { |
| 1313 | 1328 | |
| 1314 | 1329 | var att = document.getElementById('attentionList'); |
| 1315 | 1330 | if (att) { |
| 1316 | att.innerHTML = attention.map(function (a) { | |
| 1317 | return '<li><span>' + escapeHtml(a.text) + '</span><span class="sev ' + | |
| 1318 | (a.sev === 'ok' ? 'ok' : a.sev === 'bad' ? 'bad' : '') + '">' + | |
| 1319 | (a.sev === 'ok' ? 'ok' : a.sev === 'bad' ? 'crit' : 'warn') + | |
| 1320 | '</span></li>'; | |
| 1321 | }).join(''); | |
| 1331 | renderAttentionList(att, attention); | |
| 1322 | 1332 | } |
| 1323 | 1333 | } |
| 1324 | 1334 |
| … | … | @@ -51,7 +51,9 @@ services: |
| 51 | 51 | - HIVERELAY_TOR_CONTROL_HOST=hiverelay-tor |
| 52 | 52 | - HIVERELAY_TOR_CONTROL_PORT=9051 |
| 53 | 53 | - HIVERELAY_TOR_KEY_FILE=/config/tor-onion-key.blob |
| 54 | - HIVERELAY_TOR_CLIENT_AUTH_DIR=/config/tor-client-auth | |
| 54 | - HIVERELAY_TOR_MIN_DAEMON_VERSION=0.4.9.5 | |
| 55 | # Optional restricted-discovery roster (one base32 x25519 pubkey/line): | |
| 56 | # - HIVERELAY_TOR_ROSTER_FILE=/config/tor-client-auth-roster.txt | |
| 55 | 57 | # |
| 56 | 58 | # ─── Utility services (enabled by default) ─────────────────── |
| 57 | 59 | # These make the relay useful to all apps on the substrate: |
| … | … | @@ -64,8 +66,13 @@ services: |
| 64 | 66 | # repairticket = self-healing work orders |
| 65 | 67 | - HIVERELAY_ENABLE_SERVICES=1 |
| 66 | 68 | - HIVERELAY_OUTBOXLOG=1 |
| 69 | - HIVERELAY_OUTBOXLOG_JOURNAL=hypercore-outboxes | |
| 70 | - HIVERELAY_OUTBOXLOG_MAX_JOURNAL_STORAGE=512MB | |
| 67 | 71 | - HIVERELAY_NOTIFY=1 |
| 68 | - HIVERELAY_SHARD_STORE=1 | |
| 72 | # Real APNs/FCM/WebPush egress requires a private JSON descriptor: | |
| 73 | # - HIVERELAY_NOTIFY_PUSH_CONFIG=/config/notify-push.json | |
| 74 | # Shard custody is deliberately opt-in because it consumes operator disk: | |
| 75 | # - HIVERELAY_SHARD_STORE=1 | |
| 69 | 76 | - HIVERELAY_STORAGE_PROOF=1 |
| 70 | 77 | - HIVERELAY_VRF=1 |
| 71 | 78 | - HIVERELAY_WITNESSLOG=1 |
| … | … | @@ -123,10 +130,12 @@ services: |
| 123 | 130 | - HIVERELAY_TOR=1 |
| 124 | 131 | - HIVERELAY_TOR_SOCKS_HOST=hiverelay-na-tor |
| 125 | 132 | - HIVERELAY_TOR_CONTROL_HOST=hiverelay-na-tor |
| 133 | - HIVERELAY_TOR_MIN_DAEMON_VERSION=0.4.9.5 | |
| 126 | 134 | - HIVERELAY_ENABLE_SERVICES=1 |
| 127 | 135 | - HIVERELAY_OUTBOXLOG=1 |
| 136 | - HIVERELAY_OUTBOXLOG_JOURNAL=hypercore-outboxes | |
| 137 | - HIVERELAY_OUTBOXLOG_MAX_JOURNAL_STORAGE=512MB | |
| 128 | 138 | - HIVERELAY_NOTIFY=1 |
| 129 | - HIVERELAY_SHARD_STORE=1 | |
| 130 | 139 | - HIVERELAY_STORAGE_PROOF=1 |
| 131 | 140 | - HIVERELAY_VRF=1 |
| 132 | 141 | - HIVERELAY_WITNESSLOG=1 |
| … | … | @@ -169,10 +178,12 @@ services: |
| 169 | 178 | - HIVERELAY_TOR=1 |
| 170 | 179 | - HIVERELAY_TOR_SOCKS_HOST=hiverelay-eu-tor |
| 171 | 180 | - HIVERELAY_TOR_CONTROL_HOST=hiverelay-eu-tor |
| 181 | - HIVERELAY_TOR_MIN_DAEMON_VERSION=0.4.9.5 | |
| 172 | 182 | - HIVERELAY_ENABLE_SERVICES=1 |
| 173 | 183 | - HIVERELAY_OUTBOXLOG=1 |
| 184 | - HIVERELAY_OUTBOXLOG_JOURNAL=hypercore-outboxes | |
| 185 | - HIVERELAY_OUTBOXLOG_MAX_JOURNAL_STORAGE=512MB | |
| 174 | 186 | - HIVERELAY_NOTIFY=1 |
| 175 | - HIVERELAY_SHARD_STORE=1 | |
| 176 | 187 | - HIVERELAY_STORAGE_PROOF=1 |
| 177 | 188 | - HIVERELAY_VRF=1 |
| 178 | 189 | - HIVERELAY_WITNESSLOG=1 |
| … | … | @@ -215,10 +226,12 @@ services: |
| 215 | 226 | - HIVERELAY_TOR=1 |
| 216 | 227 | - HIVERELAY_TOR_SOCKS_HOST=hiverelay-as-tor |
| 217 | 228 | - HIVERELAY_TOR_CONTROL_HOST=hiverelay-as-tor |
| 229 | - HIVERELAY_TOR_MIN_DAEMON_VERSION=0.4.9.5 | |
| 218 | 230 | - HIVERELAY_ENABLE_SERVICES=1 |
| 219 | 231 | - HIVERELAY_OUTBOXLOG=1 |
| 232 | - HIVERELAY_OUTBOXLOG_JOURNAL=hypercore-outboxes | |
| 233 | - HIVERELAY_OUTBOXLOG_MAX_JOURNAL_STORAGE=512MB | |
| 220 | 234 | - HIVERELAY_NOTIFY=1 |
| 221 | - HIVERELAY_SHARD_STORE=1 | |
| 222 | 235 | - HIVERELAY_STORAGE_PROOF=1 |
| 223 | 236 | - HIVERELAY_VRF=1 |
| 224 | 237 | - HIVERELAY_WITNESSLOG=1 |
| … | … | @@ -11,6 +11,18 @@ what is built but undeployed — and a concrete fleet enablement plan. |
| 11 | 11 | | Fleet snapshot | `docs/html/fleet-live.json`, `generatedAt 2026-07-27 10:17 UTC`, 12 relays (`summary.total: 12`) | |
| 12 | 12 | | Method | Read-only. No SSH, no fleet mutation. Every claim carries `file:line` or a quoted command. | |
| 13 | 13 | |
| 14 | > Repair status (2026-07-29, post-RC7 worktree): this document preserves the | |
| 15 | > audited RC6 baseline below. The release repair now translates the shipped | |
| 16 | > service/Tor environment into Node first-boot defaults, keeps persisted config | |
| 17 | > and `services.json` authoritative, persists management API edits to | |
| 18 | > `<storage>/services.json`, accepts `storage-proof`, configures packaged | |
| 19 | > OutboxLog with a finite partitioned-Hypercore journal, and leaves shard-store | |
| 20 | > opt-in. Focused regression tests cover precedence, explicit disable, Tor | |
| 21 | > fields, bounded journal bytes, and private notify descriptor loading. These | |
| 22 | > fixes require a new prerelease and are not yet deployed. The Node plugin | |
| 23 | > lifecycle now also consumes `config.vrfBeacon`, closing the audited | |
| 24 | > beacon-config dead end. | |
| 25 | ||
| 14 | 26 | ### Evidence-quality caveat, stated once and load-bearing throughout |
| 15 | 27 | |
| 16 | 28 | `docs/html/fleet-live.json` records `services_source: "config"` for **10 of 12** relays. |
| … | … | @@ -57,7 +69,7 @@ It exists only in the gitignored `fleet/relays.local.json:77-81`. The real fleet |
| 57 | 69 | | App-origin HTTPS gateway runtime | `packages/core/gateway/{hive-host,exact-app-context,public-app-admission}.js`, wired `gateway-server.js:23,75,115-150,238-246`; `git ls-tree v0.25.0-rc.5 packages/core/gateway/` lists all three; 34/34 unit tests pass | Merged but opt-in-off (no `hiveAppHostSuffix` default) | |
| 58 | 70 | | Fleet update path fix | `CHANGELOG.md:9-42` — `patch-package` moved to prod deps; `rollback_to_previous()` gains `--force` | **rc.6 only, and rc.6 is not in any channel** | |
| 59 | 71 | |
| 60 | ### 1.3 The two headline claims are both inert | |
| 72 | ### 1.3 Audited RC6 baseline: the two headline claims were inert | |
| 61 | 73 | |
| 62 | 74 | Every `HIVERELAY_*` service/Tor variable in the three shipped packagings is read by **zero** lines |
| 63 | 75 | on the runtime those packagings launch. Measured directly: |
| … | … | @@ -167,6 +179,8 @@ appears to work and is **silently discarded at the next restart** by `_loadServi |
| 167 | 179 | `docs/STORAGE-CAP-SAFETY.md` self-flags its own gap accurately. The spec-drift problem is concentrated |
| 168 | 180 | in `PROTOCOL-SPEC.md` and a handful of Tor/services sentences. |
| 169 | 181 | |
| 182 | > Resolution note (2026-07-29): the Tor negative-probe finding above records the audited baseline. The release-repair branch now produces `negativeProbe`, automatically probes an exposed vport, degrades restricted endpoints that accept anonymous access, and refuses to sign-advertise a restricted endpoint without a successful negative proof. Live/fleet evidence is still pending. | |
| 183 | ||
| 170 | 184 | --- |
| 171 | 185 | |
| 172 | 186 | ## 4. Built but not deployed — per-box gap analysis |
| … | … | @@ -210,8 +210,9 @@ cryptographically precluded. Sealed PoRep is out of scope. (v1 proves the |
| 210 | 210 | drive's **metadata** core; blobs-core proofs are a follow-up.) |
| 211 | 211 | |
| 212 | 212 | **Privacy gate (blind drives).** The relay-side service is opt-in and **off |
| 213 | by default** (Node: `config.plugins` / Services tab; Bare/appliance: | |
| 214 | `config.services` or `HIVERELAY_STORAGE_PROOF=1`). Critically, a blind or | |
| 213 | by default** (Node: `config.plugins` / Services tab or first-boot | |
| 214 | `HIVERELAY_STORAGE_PROOF=1`; Bare: `config.services` or the same env flag). | |
| 215 | Critically, a blind or | |
| 215 | 216 | privacy-redacted drive returns `NOT_SEEDED` — **indistinguishable** from a key |
| 216 | 217 | the relay genuinely doesn't hold. A signed proof is cryptographic, |
| 217 | 218 | relay-attributable evidence of possession; serving it for a blind drive would |
| … | … | @@ -2195,7 +2195,7 @@ saveConfig(config) // Write to ~/.hiverelay/config.json |
| 2195 | 2195 | |
| 2196 | 2196 | ### Outbox Service Configuration |
| 2197 | 2197 | |
| 2198 | The OutboxLog service (signed, append-only outboxes for browser apps — see `docs/SERVICES.md`) is enabled explicitly through `config.plugins` / `services.json` (`outboxlog`), or on the Bare/appliance path with: | |
| 2198 | The OutboxLog service (signed, append-only outboxes for browser apps — see `docs/SERVICES.md`) is enabled explicitly through `config.plugins` / `services.json` (`outboxlog`), or as a Node/Bare packaging default with: | |
| 2199 | 2199 | |
| 2200 | 2200 | ```bash |
| 2201 | 2201 | export HIVERELAY_OUTBOXLOG=1 |
| … | … | @@ -2204,7 +2204,7 @@ export HIVERELAY_OUTBOXLOG=1 |
| 2204 | 2204 | **Namespace registration (required — deployment footgun):** the outbox is app-neutral, and every signed record carries a namespace (Peerit signs `'peerit'`). The relay rejects any record whose namespace is not registered with `unknown namespace` (HTTP `400`) — so `HIVERELAY_OUTBOXLOG=1` alone is not enough; a freshly ENV-provisioned box `400`s every publish until you register the app's namespace: |
| 2205 | 2205 | |
| 2206 | 2206 | ```bash |
| 2207 | # env (Bare/appliance) | |
| 2207 | # env (Node/Bare first-boot default) | |
| 2208 | 2208 | export HIVERELAY_OUTBOXLOG_NAMESPACE=peerit |
| 2209 | 2209 | ``` |
| 2210 | 2210 |
| … | … | @@ -77,10 +77,17 @@ rewrites — on all 13 relays. Its own README says "do not expose port 3458"; th |
| 77 | 77 | Per the audit's verdict: **G1 in code, G4-T in code.** It moves G2-S, G2-W, G3 and G4-I not at all, |
| 78 | 78 | and is **net-negative on G0** because of C-1 below. |
| 79 | 79 | |
| 80 | Both headline packaging claims are **inert**. All nine `HIVERELAY_*` service and Tor variables have | |
| 81 | **zero readers** on the Node runtime that Docker, systemd and Umbrel all launch — six are Bare-only, | |
| 82 | three are read nowhere in the repo. "Tor on by default" and "a fresh appliance is no longer an empty | |
| 83 | seeder" are true of the *expressed* configuration and false of the *running* one. | |
| 80 | At the audited RC6 baseline, both headline packaging claims were **inert**. All nine | |
| 81 | `HIVERELAY_*` service and Tor variables had **zero readers** on the Node runtime | |
| 82 | that Docker, systemd and Umbrel all launch — six were Bare-only and three were | |
| 83 | read nowhere in the repo. "Tor on by default" and "a fresh appliance is no | |
| 84 | longer an empty seeder" were true of the *expressed* configuration and false of | |
| 85 | the *running* one. | |
| 86 | ||
| 87 | > Post-RC7 repair (2026-07-29): the Node CLI now translates those variables as | |
| 88 | > first-boot defaults with persisted config and `services.json` precedence; | |
| 89 | > packaged OutboxLog is bounded and shard-store remains opt-in. Deployment and | |
| 90 | > signed runtime evidence are still pending. | |
| 84 | 91 | |
| 85 | 92 | --- |
| 86 | 93 |
| … | … | @@ -19,7 +19,10 @@ |
| 19 | 19 | |
| 20 | 20 | ## Enablement note |
| 21 | 21 | |
| 22 | Fleet CLI enables Tor via **`config.transports.tor`**, not only `HIVERELAY_TOR=1`. Packaging env alone is not sufficient on these boxes. | |
| 22 | The Node CLI now translates `HIVERELAY_TOR=1` and the bounded `HIVERELAY_TOR_*` | |
| 23 | host/port/key/min-version/roster fields into first-boot defaults. Persisted | |
| 24 | `config.transports.tor` and `config.tor.*` still win, so the fleet cohort keeps | |
| 25 | its explicit config and verifies readiness from the signed capability document. | |
| 23 | 26 | |
| 24 | 27 | ## Not in cohort |
| 25 | 28 |
| … | … | @@ -168,7 +168,7 @@ Phase 3 **must** land before the blind store-format authority and protocol hashe |
| 168 | 168 | - public app served via gateway while custody entries hard-403; same relay reachable over onion (gateway × tor); |
| 169 | 169 | - two apps sharing one relay's outboxlog under distinct blind namespaces, with takedown (namespace × blind); |
| 170 | 170 | - wake hints over the onion control path into namespaced outboxes (namespace × tor/nym). |
| 171 | 3. **Adversarial/privacy gates** — fail-closed downgrade suites ✓, redaction audit sweeps on public payloads ✓, orphan-pin (403 UNAUTHORIZED_PIN) ✓ and cross-namespace replay rejection ✓ verified 2026-07-18. **Gap: negative-probe on private onion endpoints is NOT implemented** — the health machine verifies positive reachability (HS_DESC uploads + SOCKS self-probe) but never verifies an intentionally-unauthorized client is refused; an all-invalid roster silently degrades to fail-open at the tor daemon level. Required before RC: a health-gate negative probe on restricted-discovery services. | |
| 171 | 3. **Adversarial/privacy gates** — fail-closed downgrade suites ✓, redaction audit sweeps on public payloads ✓, orphan-pin (403 UNAUTHORIZED_PIN) ✓ and cross-namespace replay rejection ✓ verified 2026-07-18. **Restricted-onion negative probe implemented locally 2026-07-29** — an unauthenticated SOCKS rejection is required for readiness; an accepted connection degrades immediately; ambiguous failures never pass; and the signed capability entry is omitted unless `negativeProbe === true`. Fleet/live Tor evidence remains part of Ship 8. | |
| 172 | 172 | 4. **Release-gate holdouts** — Linux Phase-0 rerun (durability/throughput; harness portable at `00-core/wal-phase0-evidence-2026-07-13/`), app-origin gateway G7–G13 operator/fleet evidence, 100 MB bulk-over-onion measurement. Recorded follow-ups (not gates): P1×P2 atomic-staging lock narrowing (measured ~35 vs ~126 PUTs/s under 64-way concurrency), storage-startup-rollback corestore-7 teardown drain (merge-surfaced), pre-existing v1-tip failures (blind-protocol-vectors, vnext-program-state, vnext-protocol-remediation, blind-client-late-app). |
| 173 | 173 | |
| 174 | 174 | ## 8. Sources of truth |
| … | … | @@ -275,7 +275,7 @@ footer{border-top:1px solid var(--card-border);padding:2.5rem 0;margin-top:3rem; |
| 275 | 275 | <div class="card h-accent"> |
| 276 | 276 | <div class="ico">🧅</div> |
| 277 | 277 | <h3>Tor as a packaging default</h3> |
| 278 | <p>Docker Compose, the systemd unit, and the Umbrel appliance <em>express</em> a Tor default as <code>HIVERELAY_TOR=1</code>, with full SOCKS / control / key-file wiring. <strong>The Node runtime those packagings launch does not read that variable</strong> — enabling Tor is <code>config.transports.tor</code> or <code>--tor</code>. Verified: with every <code>HIVERELAY_*</code> var exported, <code>loadConfig()</code> still returns <code>transports.tor = false</code>. Live endpoints additionally need host Tor ≥ 0.4.9 with cookie control.</p> | |
| 278 | <p>Docker Compose, the systemd unit, and the Umbrel appliance set <code>HIVERELAY_TOR=1</code> with SOCKS, control, persistent-key, and minimum-daemon wiring. The Node CLI translates those values into first-boot config defaults; explicit CLI flags and persisted <code>config.transports.tor</code> / <code>config.tor.*</code> win. A configured default is not proof of a live onion: signed advertisement still requires daemon readiness, descriptor health, and the restricted-endpoint negative probe.</p> | |
| 279 | 279 | <div class="unlocks"> |
| 280 | 280 | <span class="lbl">Capability path</span> |
| 281 | 281 | <span class="rung">G4-T route separation</span> |
| … | … | @@ -285,7 +285,7 @@ footer{border-top:1px solid var(--card-border);padding:2.5rem 0;margin-top:3rem; |
| 285 | 285 | <div class="card h-green"> |
| 286 | 286 | <div class="ico">⚙️</div> |
| 287 | 287 | <h3>Utility services, declared in packaging</h3> |
| 288 | <p>Outboxlog, notify, storage-proof, VRF, witnesslog and repairticket are <em>declared</em> in the unit / compose path as <code>HIVERELAY_*</code> env vars. Those are read by the <strong>Bare/Pear runtime only</strong>; the unit, compose and Umbrel paths all launch the Node CLI, which takes services from <code><storage>/services.json</code> or <code>config.plugins</code>. A fresh appliance on the Node runtime still starts as an empty seeder — verified: with all eight vars exported, <code>loadConfig()</code> returns <code>enableServices: false, plugins: []</code>. <code>shard-store</code> stays opt-in for disks that can carry it.</p> | |
| 288 | <p>Outboxlog, notify, storage-proof, VRF, witnesslog and repairticket can be selected by <code>HIVERELAY_*</code> first-boot defaults on both Node and Bare paths. Persisted <code>config.json</code> wins over environment defaults, and <code><storage>/services.json</code> is the final authority, including an explicit disable. Packaged OutboxLog uses a finite partitioned-Hypercore journal; notify remains non-live until an operator supplies a private APNs/FCM/WebPush descriptor; <code>shard-store</code> stays opt-in for disks that can carry it.</p> | |
| 289 | 289 | <ul class="tags"><li>outboxlog</li><li>notify</li><li>vrf</li><li>witnesslog</li><li>repairticket</li><li>storage-proof</li></ul> |
| 290 | 290 | </div> |
| 291 | 291 |
| … | … | @@ -1,6 +1,6 @@ |
| 1 | 1 | # Ladder ship map (chronological) |
| 2 | 2 | |
| 3 | **Date:** 2026-07-26 · **Status:** living operator/product plan · **Baseline:** monorepo canary `v0.25.0-rc.1` · stable `v0.24.3` (see [community update](./COMMUNITY-UPDATE-0.25.0-rc.1.md)) | |
| 3 | **Date:** 2026-07-29 · **Status:** living operator/product plan · **Baseline:** fleet canary `v0.25.0-rc.7` · stable `v0.24.3` · post-RC7 repair candidate in `codex/hiverelay-release-fleet-20260729` | |
| 4 | 4 | |
| 5 | 5 | > Two parallel tracks, one honesty ladder. **Version tags and claim badges stay separate until a deliberate merge.** Do not call the blind public-test pilot “fleet v0.25.0,” and do not claim rungs the current ship has not evidenced. |
| 6 | 6 | |
| … | … | @@ -54,17 +54,37 @@ Marketing one-liner when both are live: **“Fleet v0.25.0 + Blind public-test o |
| 54 | 54 | |
| 55 | 55 | | Item | State | |
| 56 | 56 | |------|--------| |
| 57 | | Fleet channel | **canary `v0.25.0-rc.1`** · **stable `v0.24.3`** (GitHub `main` `channels.json`) | | |
| 58 | | Tag / GH prerelease | [`v0.25.0-rc.1`](https://github.com/bigdestiny2/P2P-Hiverelay/releases/tag/v0.25.0-rc.1) signed + published | | |
| 59 | | Packaging in RC | Tor + utility **defaults** in Docker/systemd/Umbrel | | |
| 60 | | Service wiring (Track A content) | VRF/notify HTTP, client helpers, dashboard panels, outboxlog durability — in the RC tag | | |
| 61 | | Live fleet (as of 2026-07-26 probe) | Most boxes still **0.24.3** until canary updater tick; `transports.tor` often null until host Tor | | |
| 62 | | Dubai pattern | Utility services live via **`services.json`** (runtime authority for plugins), not config alone | | |
| 57 | | Fleet channel | **canary `v0.25.0-rc.7`** · **stable `v0.24.3`** · held `v0.24.3` | | |
| 58 | | Repair candidate | Post-RC7 tree: full Node unit **3528/3528**, integration **106/106**, lint and workspace audit green; requires a new prerelease identity before deployment | | |
| 59 | | Scoped live health (2026-07-29) | 9/10 permitted relays reachable; eight are healthy at their assigned target; `utah-8gb` is healthy but drifted at RC4 with its stable updater disabled; Bern is unreachable | | |
| 60 | | Wake runtime | `notify` loaded 9/9 reachable, but signed live egress **0/9** and exact-lane wake **0/9**; seven stable relays still carry the old false `notify-v1` feature claim | | |
| 61 | | Agent mailbox | `outboxlog` loaded and signed-advertised on **7/9** reachable permitted relays; absent on canary Utah and drifted Utah-8GB | | |
| 62 | | Privacy runtime | two signed Tor endpoints, both restricted without a negative proof on their deployed versions; post-RC7 code suppresses that claim until the anonymous rejection probe succeeds | | |
| 63 | | One-hop forward | advertised on one permitted relay; remains a bounded compatibility route, not split-transport/OHTTP evidence | | |
| 64 | | Packaging truth | Node/Bare now translate bounded service/Tor env into first-boot defaults; persisted config and `services.json` remain authoritative, while `/api/v1/services` plus the signed capability doc are runtime proof | | |
| 63 | 65 | | Track B | Blind public-test pilot (syd/dal) — **parallel** version line, not this fleet tag | |
| 64 | 66 | | Boot-restore | Worktree only — **not on main** until Track C | |
| 65 | 67 | | Blind packages on main | **Absent** (`blind-daemon` / edge only in vNext worktrees) | |
| 66 | 68 | |
| 67 | **Badge today (honest):** G0 + G1 + **partial G2-S/G3** (shard-store code exists; not fleet-wide) + **partial G4-T** (Tor code + packaging default; host deploy uneven). | |
| 69 | **Badge today (honest):** G0 + **partial G1** (mailbox present; push wake not live) + **partial G2-S/G3** (shard-store code exists; not fleet-wide) + **partial G4-T** (two deployed onions do not yet carry the new restricted-endpoint negative proof). | |
| 70 | ||
| 71 | ### Retiring or becoming redundant | |
| 72 | ||
| 73 | | Surface | Action / status | Why | | |
| 74 | |---------|-----------------|-----| | |
| 75 | | `v0.25.0-rc.5` | Retired; never deploy | Its updater install path cannot complete. | | |
| 76 | | RC4 / RC6 fleet targets | Retired by RC7 and the next repair prerelease | They lack the complete updater/security/health corrections. | | |
| 77 | | Stable `v0.24.3` `notify-v1` claim | Must be retired by a health-gated stable promotion | Seven live relays advertise wake while using a non-live memory sink. | | |
| 78 | | `notify-feed-head` | Compatibility-only; deprecated in signed capabilities when exact lanes are wired | Global sender-head changes can wake unrelated recipients; `notify-outbox-lane` is precise and opaque. | | |
| 79 | | Config-derived fleet service reports | Replaced by runtime `/api/v1/services` plus signed capability evidence | Config can be overridden by `services.json` and previously reported services that never loaded. | | |
| 80 | | Grep-and-strip package version parsing | Replaced by exact package JSON parsing | It changed `v0.25.0-rc.7` into the misleading `v0.25.0.7`. | | |
| 81 | | Memory notify provider | Test/development only; never an advertised production capability | It records attempts but deliberately has `live:false`. | | |
| 82 | | Legacy one-hop byte bridge | Compatibility path pending replacement, not G2-W | It does not provide OHTTP role separation or BlindForward flow-control semantics. | | |
| 83 | | Nym full-data-path claims | Parked; bounded control-message candidate only | Coverage/cost do not support a general relay data path today. | | |
| 84 | ||
| 85 | Split transport itself is **not redundant**: its vNext wire codecs are useful, but the runtime, | |
| 86 | `BlindTransportDescriptorV1`, OHTTP ingress/gateway, browser client, padding reconciliation, and | |
| 87 | main-tree merge remain Ship 9 work. No current fleet claim should imply G2-W. | |
| 68 | 88 | |
| 69 | 89 | ### Physical co-location (Sydney pattern; same on Dallas) |
| 70 | 90 | |
| … | … | @@ -100,6 +120,9 @@ Qualification order for Track B: **syd-1 full accept → then Dallas (or Bern) o |
| 100 | 120 | |
| 101 | 121 | ### Ship 1 — `v0.25.0-rc.1` · Fleet canary (Track A only) |
| 102 | 122 | |
| 123 | **Historical status:** superseded through RC7. The live canary is RC7; post-RC7 | |
| 124 | repairs require a new prerelease rather than mutating the meaning of that tag. | |
| 125 | ||
| 103 | 126 | **When:** after 0.1 committed + green tests |
| 104 | 127 | **Channel:** `canary` only (utah, utah-0.5gb) |
| 105 | 128 | **Not:** syd Blind stack, not stable, not Dallas Blind |
| … | … | @@ -127,6 +150,12 @@ Qualification order for Track B: **syd-1 full accept → then Dallas (or Bern) o |
| 127 | 150 | |
| 128 | 151 | ### Ship 2 — `v0.25.0` · Fleet stable (Track A) |
| 129 | 152 | |
| 153 | **Current status:** not ready. First deploy the post-RC7 candidate to canary, | |
| 154 | configure and prove at least one real push egress if wake is in the claim, land | |
| 155 | restricted-Tor negative-probe and 100 MB onion evidence, observe 24–48 hours, | |
| 156 | then promote stable. Held and independently owned lanes remain excluded until | |
| 157 | their owners release them. | |
| 158 | ||
| 130 | 159 | **When:** Ship 1 green |
| 131 | 160 | **Channel:** bump `stable` (+ keep canary on same or next RC) |
| 132 | 161 | |
| … | … | @@ -227,7 +256,7 @@ Still not fleet-wide enable. |
| 227 | 256 | | Syd/dal promote from public-test digests → release digests **or** re-pin | Continuous line | |
| 228 | 257 | | Capability-doc honesty: advertise only evidenced routes | Badges | |
| 229 | 258 | |
| 230 | **Gate:** Linux Phase-0-class durability/throughput (giga holdouts), negative-probe on restricted Tor if claiming private onion, boot-restore after wipe test. | |
| 259 | **Gate:** Linux Phase-0-class durability/throughput (giga holdouts), fleet/live evidence for the locally implemented restricted-Tor negative probe, boot-restore after wipe test. | |
| 231 | 260 | |
| 232 | 261 | --- |
| 233 | 262 | |
| … | … | @@ -269,7 +298,7 @@ Not: every relay is blind; not G2-W/G4-I/G5. |
| 269 | 298 | | Contents | Ladder | |
| 270 | 299 | |----------|--------| |
| 271 | 300 | | Persistent onion keys + backup | Production Tor | |
| 272 | | Negative-probe on restricted discovery (giga holdout) | No fail-open roster | | |
| 301 | | Negative-probe on restricted discovery (implemented locally; fleet evidence pending) | No fail-open roster | | |
| 273 | 302 | | 100 MB bulk-over-onion measurement | Capacity honesty | |
| 274 | 303 | | Optional: onion peer vport + control-plane health ads | G4-T operational | |
| 275 | 304 | |
| … | … | @@ -409,9 +438,18 @@ You can call the **product ladder complete for honest marketing** when: |
| 409 | 438 | |
| 410 | 439 | ## 9. Immediate next three moves (in order) |
| 411 | 440 | |
| 412 | 1. **Branch + tag path for Track A:** `feat/service-http-wiring` → `v0.25.0-rc.1` canary → `v0.25.0` stable. | |
| 413 | 2. **Unblock Track B:** DNS `relay-syd.p2phiverelay.xyz` (live) → deploy/qualify syd-1 → Dallas as 2nd FD. | |
| 414 | 3. **Only then** merge boot-restore + tag **0.26** for “blind-capable product.” | |
| 441 | 1. **Name and ship the repair candidate:** finish release gates, cut a new signed | |
| 442 | post-RC7 prerelease, and deploy it to the permitted canary only. Correct | |
| 443 | Utah-8GB's disabled stable updater separately; do not fold a downgrade into | |
| 444 | a canary promotion. | |
| 445 | 2. **Prove the claims on live canary:** configure operator-owned push egress if | |
| 446 | wake is a release claim; verify exact-lane wake, outboxlog persistence, | |
| 447 | restricted-Tor negative probe, and the 100 MB onion transfer. Observe for | |
| 448 | 24–48 hours. | |
| 449 | 3. **Promote stable, then continue the later ladder:** health-gated stable rollout | |
| 450 | retires the false v0.24.3 notify advertisement. Ship 9 split transport remains | |
| 451 | after its vNext owner lane produces the runtime/descriptor; Track B ownership | |
| 452 | and Sydney/Dallas are deliberately untouched by this sequence. | |
| 415 | 453 | |
| 416 | 454 | --- |
| 417 | 455 |
| … | … | @@ -201,12 +201,12 @@ adds stronger catalog tooling: |
| 201 | 201 | |
| 202 | 202 | ## Compatibility note for this workspace |
| 203 | 203 | |
| 204 | There can be active version skew across the ecosystem. On 2026-07-07, the | |
| 204 | There can be active version skew across the ecosystem. On 2026-07-29, the | |
| 205 | 205 | current Hiverelay workspace packages are: |
| 206 | 206 | |
| 207 | - `p2p-hiverelay` `0.24.1` | |
| 208 | - `p2p-hiverelay-client` `0.24.1` | |
| 209 | - `p2p-hiverelay-verifier` `0.24.1` | |
| 207 | - `p2p-hiverelay` `0.25.0-rc.7` | |
| 208 | - `p2p-hiverelay-client` `0.25.0-rc.7` | |
| 209 | - `p2p-hiverelay-verifier` `0.25.0-rc.7` | |
| 210 | 210 | |
| 211 | 211 | PearBrowser desktop is the bundled consumer that must be kept in lockstep: |
| 212 | 212 | `scripts/check-hiverelay-layout.mjs`, `package-lock.json`, and desktop CI must |
| … | … | @@ -219,12 +219,12 @@ documenting and testing stable wire contracts |
| 219 | 219 | `/v1/hyper/...`, signed catalog bees, signed DHT relay records, and `indexRoom`) |
| 220 | 220 | instead of relying on package-version parity alone. |
| 221 | 221 | |
| 222 | p2pbuilders now points at the local Hiverelay `0.24.1` workspace packages and | |
| 222 | p2pbuilders now points at the local Hiverelay `0.25.0-rc.7` workspace packages and | |
| 223 | 223 | uses the split `p2p-hiverelay-client` SDK, with a local migration guard that |
| 224 | 224 | rejects the removed `p2p-hiverelay/client` subpath. Opengit's optional |
| 225 | `--use-hiverelay` bridge also points at the local `0.24.1` core/client packages | |
| 225 | `--use-hiverelay` bridge also points at the local `0.25.0-rc.7` core/client packages | |
| 226 | 226 | and dynamically imports the split ESM client while preserving the native relay |
| 227 | default. PearPaste and anonGPT now also point at the local `0.24.1` Hiverelay | |
| 227 | default. PearPaste and anonGPT now also point at the local `0.25.0-rc.7` Hiverelay | |
| 228 | 228 | workspace packages by default, because they are live customer-facing relay |
| 229 | 229 | consumers. The main PearBrowser bundle is still the release-critical bundled |
| 230 | 230 | consumer, but whole-ecosystem package-parity claims require these |
| … | … | @@ -205,21 +205,21 @@ node scripts/publish-catalog-bee.js <catalog.json> --storage <dir> --serve |
| 205 | 205 | |
| 206 | 206 | That gives you a stable `hyperbee://` catalog plus relay pinning. |
| 207 | 207 | |
| 208 | ## Current 0.24.1 ecosystem alignment | |
| 208 | ## Current 0.25.0-rc.7 ecosystem alignment | |
| 209 | 209 | |
| 210 | The Hiverelay workspace packages are now `0.24.1`. Pear Browser desktop is the | |
| 210 | The Hiverelay workspace packages are now `0.25.0-rc.7`. Pear Browser desktop is the | |
| 211 | 211 | main bundled consumer. The desktop alignment pass moved the following surfaces |
| 212 | 212 | to the newest Hiverelay line: |
| 213 | 213 | |
| 214 | 1. `scripts/check-hiverelay-layout.mjs` now expects `0.24.1`. | |
| 214 | 1. `scripts/check-hiverelay-layout.mjs` now expects `0.25.0-rc.7`. | |
| 215 | 215 | 2. `package-lock.json` resolves `p2p-hiverelay`, `p2p-hiverelay-client`, and |
| 216 | `p2p-hiverelay-verifier` to `0.24.1`. | |
| 216 | `p2p-hiverelay-verifier` to `0.25.0-rc.7`. | |
| 217 | 217 | 3. `.github/workflows/desktop-ci.yml` checks out |
| 218 | `bigdestiny2/P2P-Hiverelay@v0.24.1` and guards `0.24.1`. | |
| 218 | `bigdestiny2/P2P-Hiverelay@v0.25.0-rc.7` and guards `0.25.0-rc.7`. | |
| 219 | 219 | 4. Desktop README, release-readiness docs, and the release-packaging test now |
| 220 | name the local `0.24.1` workspace packages. | |
| 220 | name the local `0.25.0-rc.7` workspace packages. | |
| 221 | 221 | 5. Desktop local install state has been refreshed so `npm ls` resolves the |
| 222 | three `p2p-hiverelay*` packages to the `0.24.1` workspace links. | |
| 222 | three `p2p-hiverelay*` packages to the `0.25.0-rc.7` workspace links. | |
| 223 | 223 | 6. Desktop and mobile relay clients now use explicit HTTP/HTTPS transport |
| 224 | 224 | selection for public relay gateway fetches. This HTTPS relay transport path |
| 225 | 225 | is backed by `bare-https`, which is declared directly in both browser |
| … | … | @@ -236,7 +236,7 @@ when mobile wants parity with the `0.20.x` signed discovery model. |
| 236 | 236 | Public ecosystem copy should also be refreshed in the same pass: |
| 237 | 237 | |
| 238 | 238 | - `hyper-fetch` live-smoke evidence that names relay `v0.20.0` should be |
| 239 | refreshed after a real public relay advertises `v0.24.1`. | |
| 239 | refreshed after a real public relay advertises `v0.25.0-rc.7`. | |
| 240 | 240 | |
| 241 | 241 | ### Non-bundled direct consumers |
| 242 | 242 | |
| … | … | @@ -246,15 +246,15 @@ but they are not all release-critical for the browser: |
| 246 | 246 | - `02-apps/pear-pos`, `02-apps/pear-tickets`, and |
| 247 | 247 | `04-experiments/hiverelay-test` already point at the local Hiverelay |
| 248 | 248 | workspace packages. |
| 249 | - `02-apps/pearpaste` now points at the local Hiverelay `0.24.1` core/client | |
| 249 | - `02-apps/pearpaste` now points at the local Hiverelay `0.25.0-rc.7` core/client | |
| 250 | 250 | workspace packages for encrypted availability, custody, and relay pinning. |
| 251 | 251 | - `03-sites/pearbrowser-publishers/src/p2pbuilders` now points at the local |
| 252 | Hiverelay `0.24.1` workspace packages and imports the split | |
| 252 | Hiverelay `0.25.0-rc.7` workspace packages and imports the split | |
| 253 | 253 | `p2p-hiverelay-client` SDK. |
| 254 | 254 | - `04-experiments/Opengit/packages/opengit-relay` now points its optional |
| 255 | `--use-hiverelay` bridge at the local Hiverelay `0.24.1` workspace packages | |
| 255 | `--use-hiverelay` bridge at the local Hiverelay `0.25.0-rc.7` workspace packages | |
| 256 | 256 | and dynamically imports the split ESM client. |
| 257 | - `04-experiments/anongpt-native` now points at the local Hiverelay `0.24.1` | |
| 257 | - `04-experiments/anongpt-native` now points at the local Hiverelay `0.25.0-rc.7` | |
| 258 | 258 | core workspace package for customer relay/onion AI transport and directory |
| 259 | 259 | discovery. |
| 260 | 260 | - `00-core/hr-acct` and `00-core/hr-fleet` are snapshot/worktree-style relay |
| … | … | @@ -290,9 +290,9 @@ The per-app source/import migration notes live in |
| 290 | 290 | This workspace is not pinned to one HiveRelay version line at all times. On |
| 291 | 291 | 2026-07-07 the authoritative Hiverelay package line is: |
| 292 | 292 | |
| 293 | - `p2p-hiverelay` `0.24.1` | |
| 294 | - `p2p-hiverelay-client` `0.24.1` | |
| 295 | - `p2p-hiverelay-verifier` `0.24.1` | |
| 293 | - `p2p-hiverelay` `0.25.0-rc.7` | |
| 294 | - `p2p-hiverelay-client` `0.25.0-rc.7` | |
| 295 | - `p2p-hiverelay-verifier` `0.25.0-rc.7` | |
| 296 | 296 | |
| 297 | 297 | That means compatibility work should bias toward: |
| 298 | 298 |
| … | … | @@ -147,6 +147,9 @@ Merge these from the local Core3 README draft: |
| 147 | 147 | service manager, restart/status, measured stored/served counters. |
| 148 | 148 | - Accept policy contract: `review`, `allowlist`, `open`, `closed`, with |
| 149 | 149 | `HIVERELAY_ACCEPT_MODE=review` as the first-boot home-server default. |
| 150 | - Preserved the first-boot-only Umbrel storage-cap behavior: | |
| 151 | `HIVERELAY_MAX_STORAGE=10GB` applies until the operator persists an explicit | |
| 152 | `maxStorageBytes` value. | |
| 150 | 153 | - Management API hardening: bearer auth, strict JSON content type, bounded |
| 151 | 154 | object-only body handling, live `/api/manage/services` disable/restart |
| 152 | 155 | actions, and durable service disable persistence before unregister; |
| … | … | @@ -1,5 +1,5 @@ |
| 1 | 1 | > [!NOTE] |
| 2 | > Refreshed for the next relay update: the core builtin services remain opt-in, with new app-facing service providers for encrypted wake notifications (`notify`) and single-writer signed outboxes (`outboxlog`). Services are opt-in (`enableServices` / selected `plugins`); `vrf` is production-ready (RFC 9381, validated against the spec's own test vectors), while `ai`/`zk`/`sla`/`arbitration`, `notify`, and `outboxlog` remain experimental. Notify and outboxlog now have local JSON state, but production provider adapters, billing gates, and operational hardening are still incomplete. See the [CHANGELOG](../CHANGELOG.md) for the authoritative architecture. | |
| 2 | > Refreshed for the next relay update: the core builtin services remain opt-in, with app-facing providers for encrypted wake notifications (`notify`) and single-writer signed outboxes (`outboxlog`). Services are opt-in (`enableServices` / selected `plugins`); `vrf` is production-ready (RFC 9381, validated against the spec's own test vectors), while `ai`/`zk`/`sla`/`arbitration`, `notify`, and `outboxlog` remain experimental. Notify has real APNs, FCM, and Web Push adapters but advertises wake only when an operator configures one; outboxlog uses bounded Hypercore persistence on storage-authority relays. See the [CHANGELOG](../CHANGELOG.md) for the authoritative architecture. | |
| 3 | 3 | |
| 4 | 4 | # HiveRelay Services Layer |
| 5 | 5 | |
| … | … | @@ -216,16 +216,26 @@ model. |
| 216 | 216 | meter attempts without leaking wake payloads. |
| 217 | 217 | - Watch mode stores opaque Hypercore/feed heads only; app-defined filtering and |
| 218 | 218 | message truth stay in the application. |
| 219 | - `notify-outbox-lane` is the preferred mailbox trigger: it observes only the | |
| 220 | sender-owned opaque recipient lane. `notify-feed-head` remains accepted for | |
| 221 | older clients, but is marked deprecated and omitted from preferred signed | |
| 222 | watch sources once exact-lane wiring is present because a global head can | |
| 223 | wake unrelated recipients sharing the same outbox. | |
| 219 | 224 | |
| 220 | 225 | **Current implementation status:** the v0.1.0 provider has signed object |
| 221 | 226 | verification, Node relay JSON persistence under `<storage>/notify-service-state.json`, |
| 222 | 227 | quota/replay/dedupe guards, redacted signed delivery events, capability-doc |
| 223 | advertising, a lightweight HTTP facade, Bare-safe client signing helpers, and a | |
| 224 | memory push provider for tests. Production APNs/FCM/Web Push adapters and | |
| 225 | pricing/billing enforcement are follow-up gates. | |
| 226 | ||
| 227 | Enable it explicitly through `config.plugins` / `services.json` (`notify`) or, | |
| 228 | on the Bare/appliance path, with `HIVERELAY_NOTIFY=1`. | |
| 228 | advertising, a lightweight HTTP facade, Bare-safe client signing helpers, real | |
| 229 | APNs/FCM/Web Push adapters, and a non-live memory provider for tests. Provider | |
| 230 | credentials and pricing/billing enforcement are operator deployment gates; a | |
| 231 | relay using the memory provider is deliberately omitted from the signed | |
| 232 | `notify-v1` capability surface. | |
| 233 | ||
| 234 | Enable it explicitly through `config.plugins` / `services.json` (`notify`) or | |
| 235 | with the first-boot default `HIVERELAY_NOTIFY=1` on Node and Bare runtimes. | |
| 236 | Real egress additionally needs `config.notify.push`, or an owner-readable JSON | |
| 237 | descriptor selected by `HIVERELAY_NOTIFY_PUSH_CONFIG`; without that descriptor | |
| 238 | the memory provider stays non-live and is not signed-advertised. | |
| 229 | 239 | |
| 230 | 240 | See [PUSH-NOTIFICATION-SERVICE-SPEC.md](./PUSH-NOTIFICATION-SERVICE-SPEC.md). |
| 231 | 241 | |
| … | … | @@ -251,7 +261,11 @@ keeps entries opaque, and exposes a token-gated HTTP/SSE bridge for sync. |
| 251 | 261 | with `OUTBOXLOG_BOUNDED_PERSISTENCE_REQUIRED` until migrated or disabled. |
| 252 | 262 | |
| 253 | 263 | Enable it explicitly through `config.plugins` / `services.json` (`outboxlog`) |
| 254 | or, on the Bare/appliance path, with `HIVERELAY_OUTBOXLOG=1`. | |
| 264 | or with the first-boot default `HIVERELAY_OUTBOXLOG=1` on Node and Bare | |
| 265 | runtimes. On storage-authority relays, packaged installs also set | |
| 266 | `HIVERELAY_OUTBOXLOG_JOURNAL=hypercore-outboxes` and a finite | |
| 267 | `HIVERELAY_OUTBOXLOG_MAX_JOURNAL_STORAGE`; both are required for bounded | |
| 268 | persistence. | |
| 255 | 269 | |
| 256 | 270 | For a Hypercore-backed journal, configure a finite aggregate local bound with |
| 257 | 271 | `config.outboxlog.maxJournalStorageBytes`. This single authority commitment |
| … | … | @@ -277,7 +291,7 @@ namespace to admit. Register it one of two ways: |
| 277 | 291 | |
| 278 | 292 | - config: `config.outboxlog.namespace = "peerit"` (or a `config.outboxlog.namespaces` |
| 279 | 293 | map to register several at once) |
| 280 | - env (Bare/appliance): `HIVERELAY_OUTBOXLOG_NAMESPACE=peerit` | |
| 294 | - env (Node/Bare packaging default): `HIVERELAY_OUTBOXLOG_NAMESPACE=peerit` | |
| 281 | 295 | |
| 282 | 296 | The env var is a **default, not an override**: a `outboxlog.namespace` persisted |
| 283 | 297 | in `config.json` wins over it (matching `HIVERELAY_ACCEPT_MODE`). When neither is |
| … | … | @@ -372,7 +386,7 @@ only the exact token succeeds (constant-time compare). Configure the credential |
| 372 | 386 | with: |
| 373 | 387 | |
| 374 | 388 | - config: `config.outboxlog.adminKey = "<operator-secret>"` |
| 375 | - env (Bare/appliance): `HIVERELAY_OUTBOXLOG_ADMIN_KEY=<operator-secret>` | |
| 389 | - env (Node/Bare): `HIVERELAY_OUTBOXLOG_ADMIN_KEY=<operator-secret>` | |
| 376 | 390 | |
| 377 | 391 | There is no default — provision this only on relays where you intend to enable |
| 378 | 392 | takedown. Outboxlog claims exactly these three `/api/admin/*` routes and does not |
| … | … | @@ -391,8 +405,8 @@ Signed challenge-response proof that this relay genuinely holds a seeded app. A |
| 391 | 405 | **Access:** route `storage-proof.prove` is policy `public` (anonymous-callable) in `packages/core/core/router/index.js`. |
| 392 | 406 | |
| 393 | 407 | **Opt-in (default OFF):** a stock node does not run it. |
| 394 | - Node runtime: select `storage-proof` via `config.plugins` / `services.json` (Services tab; persisted under `<storage>/services.json`). | |
| 395 | - Bare/appliance runtime: select `storage-proof` via `config.services` (or `config.plugins`), or set env `HIVERELAY_STORAGE_PROOF=1`. | |
| 408 | - Node runtime: select `storage-proof` via `config.plugins` / `services.json` (Services tab; persisted under `<storage>/services.json`), or use the first-boot default `HIVERELAY_STORAGE_PROOF=1`. | |
| 409 | - Bare runtime: select `storage-proof` via `config.services` (or `config.plugins`), or set `HIVERELAY_STORAGE_PROOF=1`. | |
| 396 | 410 | |
| 397 | 411 | **What it proves (v1):** the drive **metadata** core only — the head the client learns from `open()`, so the `minLength` pin lines up. Blobs-core proofs are a follow-up. |
| 398 | 412 | |
| … | … | @@ -444,7 +458,7 @@ const weather = await protocol.request(relayPubkey, 'weather', 'current', { loca |
| 444 | 458 | |
| 445 | 459 | ## Configuration |
| 446 | 460 | |
| 447 | Services are **opt-in**, not automatic. A relay starts with none: `config/default.js` ships `enableServices: false` and `plugins: []`, and `relay-node/index.js` only constructs the registry when `enableServices` is set. To run services, either set `enableServices` and list `plugins` in the config, or write `<storage>/services.json` — which is the **runtime authority** and overrides config. A plugin name that does not match a registered builtin fails to load silently, so verify with `GET /api/v1/services` (or `node scripts/probe-fleet-services.mjs`) rather than trusting the config. The SLA service requires proof-of-relay to be active for automated enforcement. | |
| 461 | Services are **opt-in**, not automatic. The library default is none: `config/default.js` ships `enableServices: false` and `plugins: []`, and `relay-node/index.js` only constructs the registry when `enableServices` is set. Packaged Node/Bare deployments may express first-boot selections with `HIVERELAY_ENABLE_SERVICES` plus individual `HIVERELAY_*` service flags. Persisted `config.json` values win over those defaults, and `<storage>/services.json` is the **final runtime authority**, including an explicit `enabled: false`. A plugin name that does not match a registered builtin fails to load silently, so verify with `GET /api/v1/services` (or `node scripts/probe-fleet-services.mjs`) rather than trusting configuration. The SLA service requires proof-of-relay to be active for automated enforcement. | |
| 448 | 462 | |
| 449 | 463 | Service lifecycle is controlled by: |
| 450 | 464 |
| … | … | @@ -212,7 +212,7 @@ framing loses to any adversary who knows the literature. |
| 212 | 212 | |---|---|---| |
| 213 | 213 | | Authored state immutability | `packages/core/core/delegation.js` (signed certs), Hyperdrive append-only | ✅ shipped | |
| 214 | 214 | | Hypercore append-only Merkle | upstream `hypercore` v10 | ✅ shipped | |
| 215 | | Replica diversity at infrastructure | `packages/core/core/federation.js` (per-relay catalog mirroring) + `packages/core/core/quorum-selector.js` | ✅ shipped; client `selectQuorum` now diversity-ranks by region and operator, and warns when either floor is not met | | |
| 215 | | Replica diversity at infrastructure | `packages/core/core/federation.js` (per-relay catalog mirroring) + `packages/core/core/quorum-selector.js` | ✅ shipped; client `selectQuorum` now spreads by region and failure domain while counting operator independence separately, and warns when either floor is not met | | |
| 216 | 216 | | Quorum selection UX | `packages/client/index.js` (`HiveRelayClient`) | ✅ shipped v0.6.0 (`selectQuorum`, `queryQuorumWithComparison`) | |
| 217 | 217 | | Fork-detection during local replication | `client.open()` attaches listeners to `drive.core.on('truncate' / 'verification-error')`; auto-reports to `ForkDetector` | ✅ shipped v0.6.0 (catches local fork-detected events, NOT silent multi-replica equivocation) | |
| 218 | 218 | | Fork-proof gossip across federation | `packages/core/core/federation.js` (pulls `/api/forks/proofs` per cycle) | ✅ shipped v0.6.0 | |
| … | … | @@ -99,16 +99,8 @@ Load-bearing semantics (each verified live against tor 0.4.9.6 in the M0 spike): |
| 99 | 99 | - `Flags=V3Auth` is **mandatory** with `ClientAuthV3`; the daemon never holds client secrets — only public keys. |
| 100 | 100 | - **No runtime roster add exists**: enrollment rebuilds the service in place (same address; brief intro-point churn). Roster persists in `rosterFile` (operator-private) and re-applies at start. |
| 101 | 101 | - Client credential install is one control-port command with `Flags=Permanent`; no hand-written files. |
| 102 | - Filesystem deployments (`HiddenServiceDir` + `authorized_clients/*.auth` + RELOAD) are the PoW-capable shape; the valid `.auth` line is exactly `descriptor:x25519:<pub>` (3 fields), and an all-invalid roster runs **fail-open**. | |
| 103 | ||
| 104 | > **No negative probe exists.** This sentence previously claimed one "in the health gate"; | |
| 105 | > it is unbuilt and is tracked as Ship 8 (`LADDER-SHIP-MAP.md:230`), consistent with | |
| 106 | > `GIGA-RELEASE-ARCHITECTURE.md:171` ("NOT implemented … Required before RC"). | |
| 107 | > Note also that this fail-open scenario is **not reachable in this runtime**: the transport | |
| 108 | > uses control-port `ADD_ONION` with ClientAuthV3, where invalid entries throw | |
| 109 | > (`auth-keys.js:245-246`) and an empty roster installs an unreachable guard credential | |
| 110 | > (`tor/index.js:709-717`). The filesystem shape described above is the one that would | |
| 111 | > fail open, and we do not deploy it. | |
| 102 | - Filesystem deployments (`HiddenServiceDir` + `authorized_clients/*.auth` + RELOAD) are the PoW-capable shape; the valid `.auth` line is exactly `descriptor:x25519:<pub>` (3 fields), and an all-invalid filesystem roster runs **fail-open**. That filesystem scenario is not reachable in this runtime: control-port `ADD_ONION` rejects invalid `ClientAuthV3` entries and an empty roster installs an unreachable guard credential. | |
| 103 | - Restricted-discovery health now makes a credential-free SOCKS connection attempt. An expected Tor authorization rejection is the successful negative proof; an accepted connection degrades immediately, while ambiguous failures never pass the gate. The signed `privacyTransports` entry is omitted unless `negativeProbe === true`. | |
| 112 | 104 | |
| 113 | 105 | ## 5. Health-gated advertisement |
| 114 | 106 | |
| … | … | @@ -118,15 +110,15 @@ stateDiagram-v2 |
| 118 | 110 | disabled --> tor_starting: start() |
| 119 | 111 | tor_starting --> key_loaded: ADD_ONION ok |
| 120 | 112 | key_loaded --> descriptor_uploaded: ≥2 HS_DESC UPLOADED |
| 121 | descriptor_uploaded --> ready: self-probe ok (or no probe vport) | |
| 113 | descriptor_uploaded --> ready: positive or negative probe passes | |
| 122 | 114 | ready --> degraded: probe fails ×3 |
| 123 | 115 | degraded --> ready: probe recovers |
| 124 | 116 | ready --> disabled: stop() |
| 125 | 117 | ``` |
| 126 | 118 | |
| 127 | The signed capability doc carries `privacyTransports` **only while `ready`**: version floor checked, key restored and address matched, ≥2 descriptor uploads observed, and a SOCKS self-probe through the network back to the read-plane vport. Degraded health removes the entry — clients never see a dead endpoint (`ONION-INV-003`). The field is omitted entirely when empty, so relays without Tor produce byte-identical docs to older builds (fixture-stable signing). | |
| 119 | The signed capability doc carries `privacyTransports` **only while `ready`**: version floor checked, key restored and address matched, ≥2 descriptor uploads observed, and a SOCKS probe through the network to an automatically selected exposed vport. Public onions require a successful connection. Restricted onions probe without a client credential and require an authorization rejection; `negativeProbe: true` is included in the signed entry. `health.probeVport: false` is an explicit descriptor-only compatibility mode, but restricted endpoints in that mode are never advertised. Degraded health removes the entry (`ONION-INV-003`). The field is omitted entirely when empty, so relays without Tor produce byte-identical docs to older builds (fixture-stable signing). | |
| 128 | 120 | |
| 129 | The advertisement entry: `{id: tor-v3-onion-v1, addresses: [{address, keyId, notBefore, notAfter, priority}], vports, vportRoles {readPlane, peer}, auth {mode, enrollment}, pow, exposure, relayLocation: 'hidden-onion', supports[]}` — signed by the stable relay identity, with rotation overlap handled by a fresh `keyId`. | |
| 121 | The advertisement entry: `{id: tor-v3-onion-v1, addresses: [{address, keyId, notBefore, notAfter, priority}], vports, vportRoles {readPlane, peer}, auth {mode, enrollment}, negativeProbe?, pow, exposure, relayLocation: 'hidden-onion', supports[]}` — signed by the stable relay identity, with rotation overlap handled by a fresh `keyId`. | |
| 130 | 122 | |
| 131 | 123 | ## 6. Policy resolution (client side) |
| 132 | 124 | |
| … | … | @@ -159,7 +151,7 @@ Public surfaces get coarse health only (`running`, `activeConnections`) — the |
| 159 | 151 | 2. Long-lived services accumulate guard-discovery exposure (PoPETs 2022; arXiv 2602.23560 intro-circuit intersection attack, Feb 2026) — mitigated by vanguards-lite, endpoint rotation, `hidden-only` exposure. |
| 160 | 152 | 3. `dual` exposure links the public and onion identities via shared uptime — operators choose: hidden-only identity, or accept the link. |
| 161 | 153 | 4. Operator-side deanonymization (payment rails, hosting accounts, cross-signing) is out of transport scope — see the operator checklist in `docs/tor-transport.md`. |
| 162 | 5. Remaining follow-up: 100 MB/10 min bulk gate on realistic uplinks (5 MB live gate passed at 1.19 Mbps). Shipped since (2026-07-18): pairing-channel enrollment hookup (`transports/tor/enrollment.js` — the envelope rides `RelayNode.pairDevice` extras; verify → roster rebuild-in-place → rosterFile persist → signed receipt; 4 tests in `tor-enrollment.test.js`) and peer-vport listener binding (`transports/tor/peer-listener.js` — loopback Noise XK endpoint the onion peer vport 19737 forwards to, folded into the transport's vport mapping and lifecycle; 7 tests in `tor-peer-listener.test.js`). | |
| 154 | 5. Remaining follow-up: 100 MB/10 min bulk gate on realistic uplinks (5 MB live gate passed at 1.19 Mbps). Implemented on the release-repair branch (2026-07-29): automatic positive/negative probing and fail-closed signed admission for restricted discovery. Shipped since (2026-07-18): pairing-channel enrollment hookup (`transports/tor/enrollment.js` — the envelope rides `RelayNode.pairDevice` extras; verify → roster rebuild-in-place → rosterFile persist → signed receipt; 4 tests in `tor-enrollment.test.js`) and peer-vport listener binding (`transports/tor/peer-listener.js` — loopback Noise XK endpoint the onion peer vport 19737 forwards to, folded into the transport's vport mapping and lifecycle; 7 tests in `tor-peer-listener.test.js`). | |
| 163 | 155 | |
| 164 | 156 | ## 10. Test evidence |
| 165 | 157 |
| … | … | @@ -62,13 +62,14 @@ |
| 62 | 62 | <li><code>Flags=V3Auth</code> is <strong>mandatory</strong> with <code>ClientAuthV3</code>; the daemon never holds client secrets — only public keys.</li> |
| 63 | 63 | <li><strong>No runtime roster add exists</strong>: enrollment rebuilds the service in place (same address; brief intro-point churn). Roster persists in <code>rosterFile</code> (operator-private) and re-applies at start.</li> |
| 64 | 64 | <li>Client credential install is one control-port command with <code>Flags=Permanent</code>; no hand-written files.</li> |
| 65 | <li>Filesystem deployments (<code>HiddenServiceDir</code> + <code>authorized_clients/*.auth</code> + RELOAD) are the PoW-capable shape; the valid <code>.auth</code> line is exactly <code>descriptor:x25519:<pub></code> (3 fields), and an all-invalid roster runs <strong>fail-open</strong> — hence the negative probe in the health gate.</li> | |
| 65 | <li>Filesystem deployments (<code>HiddenServiceDir</code> + <code>authorized_clients/*.auth</code> + RELOAD) are the PoW-capable shape; the valid <code>.auth</code> line is exactly <code>descriptor:x25519:<pub></code> (3 fields), and an all-invalid filesystem roster runs <strong>fail-open</strong>. That filesystem scenario is not reachable in this runtime.</li> | |
| 66 | <li>Restricted-discovery health makes a credential-free SOCKS connection attempt. An expected authorization rejection is the successful negative proof; an accepted connection degrades immediately, while ambiguous failures never pass the gate. Signed advertisement requires <code>negativeProbe === true</code>.</li> | |
| 66 | 67 | </ul> |
| 67 | 68 | <h2>5. Health-gated advertisement</h2> |
| 68 | 69 | <div class="mermaid-figure"><svg id="my-svg" width="100%" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" class="statediagram" style="max-width: 292.695px; background-color: transparent;" viewBox="0 0 292.6953125 714" role="graphics-document document" aria-roledescription="stateDiagram"><style>#my-svg{font-family:-apple-system,BlinkMacSystemFont,Segoe UI,Roboto,Helvetica,Arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#my-svg .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#my-svg .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#my-svg .error-icon{fill:#552222;}#my-svg .error-text{fill:#552222;stroke:#552222;}#my-svg .edge-thickness-normal{stroke-width:1px;}#my-svg .edge-thickness-thick{stroke-width:3.5px;}#my-svg .edge-pattern-solid{stroke-dasharray:0;}#my-svg .edge-thickness-invisible{stroke-width:0;fill:none;}#my-svg .edge-pattern-dashed{stroke-dasharray:3;}#my-svg .edge-pattern-dotted{stroke-dasharray:2;}#my-svg .marker{fill:#5a7196;stroke:#5a7196;}#my-svg .marker.cross{stroke:#5a7196;}#my-svg svg{font-family:-apple-system,BlinkMacSystemFont,Segoe UI,Roboto,Helvetica,Arial,sans-serif;font-size:16px;}#my-svg p{margin:0;}#my-svg defs [id$="-barbEnd"]{fill:#333333;stroke:#333333;}#my-svg g.stateGroup text{fill:#9370DB;stroke:none;font-size:10px;}#my-svg g.stateGroup text{fill:#333;stroke:none;font-size:10px;}#my-svg g.stateGroup .state-title{font-weight:bolder;fill:#131300;}#my-svg g.stateGroup rect{fill:#ECECFF;stroke:#9370DB;}#my-svg g.stateGroup line{stroke:#5a7196;stroke-width:1;}#my-svg .transition{stroke:#333333;stroke-width:1;fill:none;}#my-svg .stateGroup .composit{fill:white;border-bottom:1px;}#my-svg .stateGroup .alt-composit{fill:#e0e0e0;border-bottom:1px;}#my-svg .state-note{stroke:#aaaa33;fill:#fff5ad;}#my-svg .state-note text{fill:black;stroke:none;font-size:10px;}#my-svg .stateLabel .box{stroke:none;stroke-width:0;fill:#ECECFF;opacity:0.5;}#my-svg .edgeLabel .label rect{fill:#ECECFF;opacity:0.5;}#my-svg .edgeLabel{background-color:#ffffff;text-align:center;}#my-svg .edgeLabel p{background-color:#ffffff;}#my-svg .edgeLabel rect{opacity:0.5;background-color:#ffffff;fill:#ffffff;}#my-svg .edgeLabel .label text{fill:#333;}#my-svg .label div .edgeLabel{color:#333;}#my-svg .stateLabel text{fill:#131300;font-size:10px;font-weight:bold;}#my-svg .node circle.state-start{fill:#5a7196;stroke:#5a7196;}#my-svg .node .fork-join{fill:#5a7196;stroke:#5a7196;}#my-svg .node circle.state-end{fill:#9370DB;stroke:white;stroke-width:1.5;}#my-svg .end-state-inner{fill:white;stroke-width:1.5;}#my-svg .node rect{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#my-svg .node polygon{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#my-svg [id$="-barbEnd"]{fill:#5a7196;}#my-svg .statediagram-cluster rect{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#my-svg .cluster-label,#my-svg .nodeLabel{color:#131300;}#my-svg .statediagram-cluster rect.outer{rx:5px;ry:5px;}#my-svg .statediagram-state .divider{stroke:#9370DB;}#my-svg .statediagram-state .title-state{rx:5px;ry:5px;}#my-svg .statediagram-cluster.statediagram-cluster .inner{fill:white;}#my-svg .statediagram-cluster.statediagram-cluster-alt .inner{fill:#f0f0f0;}#my-svg .statediagram-cluster .inner{rx:0;ry:0;}#my-svg .statediagram-state rect.basic{rx:5px;ry:5px;}#my-svg .statediagram-state rect.divider{stroke-dasharray:10,10;fill:#f0f0f0;}#my-svg .note-edge{stroke-dasharray:5;}#my-svg .statediagram-note rect{fill:#fff5ad;stroke:#aaaa33;stroke-width:1px;rx:0;ry:0;}#my-svg .statediagram-note rect{fill:#fff5ad;stroke:#aaaa33;stroke-width:1px;rx:0;ry:0;}#my-svg .statediagram-note text{fill:black;}#my-svg .statediagram-note .nodeLabel{color:black;}#my-svg .statediagram .edgeLabel{color:red;}#my-svg [id$="-dependencyStart"],#my-svg [id$="-dependencyEnd"]{fill:#5a7196;stroke:#5a7196;stroke-width:1;}#my-svg .statediagramTitleText{text-anchor:middle;font-size:18px;fill:#333;}#my-svg [data-look="neo"].statediagram-cluster rect{fill:#ECECFF;stroke:#9370DB;stroke-width:1;}#my-svg [data-look="neo"].statediagram-cluster rect.outer{rx:5px;ry:5px;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#my-svg .node .neo-node{stroke:#9370DB;}#my-svg [data-look="neo"].node rect,#my-svg [data-look="neo"].cluster rect,#my-svg [data-look="neo"].node polygon{stroke:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#my-svg [data-look="neo"].swimlane.cluster rect{filter:none;}#my-svg [data-look="neo"].node path{stroke:#9370DB;stroke-width:1px;}#my-svg [data-look="neo"].node .outer-path{filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#my-svg [data-look="neo"].node .neo-line path{stroke:#9370DB;filter:none;}#my-svg [data-look="neo"].node circle{stroke:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#my-svg [data-look="neo"].node circle .state-start{fill:#000000;}#my-svg [data-look="neo"].icon-shape .icon{fill:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#my-svg [data-look="neo"].icon-shape .icon-neo path{stroke:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#my-svg :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;}</style><g><defs><marker id="my-svg_stateDiagram-barbEnd" refX="19" refY="7" markerWidth="20" markerHeight="14" markerUnits="userSpaceOnUse" orient="auto"><path d="M 19,7 L9,13 L14,7 L9,1 Z"/></marker></defs><g class="root"><g class="clusters"/><g class="edgePaths"><path d="M168,22L168,26.167C168,30.333,168,38.667,168,47C168,55.333,168,63.667,168,67.833L168,72" id="my-svg-edge0" class="edge-thickness-normal edge-pattern-solid transition" style="fill:none;;;fill:none" data-edge="true" data-et="edge" data-id="edge0" data-points="W3sieCI6MTY4LCJ5IjoyMn0seyJ4IjoxNjgsInkiOjQ3fSx7IngiOjE2OCwieSI6NzJ9XQ==" data-look="classic" marker-end="url(#my-svg_stateDiagram-barbEnd)"/><path d="M146.947,112L140.456,118.167C133.965,124.333,120.982,136.667,114.491,149C108,161.333,108,173.667,108,179.833L108,186" id="my-svg-edge1" class="edge-thickness-normal edge-pattern-solid transition" style="fill:none;;;fill:none" data-edge="true" data-et="edge" data-id="edge1" data-points="W3sieCI6MTQ2Ljk0NzM2ODQyMTA1MjYzLCJ5IjoxMTJ9LHsieCI6MTA4LCJ5IjoxNDl9LHsieCI6MTA4LCJ5IjoxODZ9XQ==" data-look="classic" marker-end="url(#my-svg_stateDiagram-barbEnd)"/><path d="M108,226L108,232.167C108,238.333,108,250.667,108,263C108,275.333,108,287.667,108,293.833L108,300" id="my-svg-edge2" class="edge-thickness-normal edge-pattern-solid transition" style="fill:none;;;fill:none" data-edge="true" data-et="edge" data-id="edge2" data-points="W3sieCI6MTA4LCJ5IjoyMjZ9LHsieCI6MTA4LCJ5IjoyNjN9LHsieCI6MTA4LCJ5IjozMDB9XQ==" data-look="classic" marker-end="url(#my-svg_stateDiagram-barbEnd)"/><path d="M108,340L108,346.167C108,352.333,108,364.667,108,377C108,389.333,108,401.667,108,407.833L108,414" id="my-svg-edge3" class="edge-thickness-normal edge-pattern-solid transition" style="fill:none;;;fill:none" data-edge="true" data-et="edge" data-id="edge3" data-points="W3sieCI6MTA4LCJ5IjozNDB9LHsieCI6MTA4LCJ5IjozNzd9LHsieCI6MTA4LCJ5Ijo0MTR9XQ==" data-look="classic" marker-end="url(#my-svg_stateDiagram-barbEnd)"/><path d="M108,454L108,462.167C108,470.333,108,486.667,115.101,503C122.203,519.333,136.406,535.667,143.507,543.833L150.609,552" id="my-svg-edge4" class="edge-thickness-normal edge-pattern-solid transition" style="fill:none;;;fill:none" data-edge="true" data-et="edge" data-id="edge4" data-points="W3sieCI6MTA4LCJ5Ijo0NTR9LHsieCI6MTA4LCJ5Ijo1MDN9LHsieCI6MTUwLjYwODY5NTY1MjE3MzksInkiOjU1Mn1d" data-look="classic" marker-end="url(#my-svg_stateDiagram-barbEnd)"/><path d="M146.191,592L139.466,598.167C132.742,604.333,119.293,616.667,119.293,629C119.293,641.333,132.742,653.667,139.466,659.833L146.191,666" id="my-svg-edge5" class="edge-thickness-normal edge-pattern-solid transition" style="fill:none;;;fill:none" data-edge="true" data-et="edge" data-id="edge5" data-points="W3sieCI6MTQ2LjE5MDc4OTQ3MzY4NDIyLCJ5Ijo1OTJ9LHsieCI6MTA1Ljg0Mzc1LCJ5Ijo2Mjl9LHsieCI6MTQ2LjE5MDc4OTQ3MzY4NDIyLCJ5Ijo2NjZ9XQ==" data-look="classic" marker-end="url(#my-svg_stateDiagram-barbEnd)"/><path d="M189.809,666L196.534,659.833C203.258,653.667,216.707,641.333,216.707,629C216.707,616.667,203.258,604.333,196.534,598.167L189.809,592" id="my-svg-edge6" class="edge-thickness-normal edge-pattern-solid transition" style="fill:none;;;fill:none" data-edge="true" data-et="edge" data-id="edge6" data-points="W3sieCI6MTg5LjgwOTIxMDUyNjMxNTc4LCJ5Ijo2NjZ9LHsieCI6MjMwLjE1NjI1LCJ5Ijo2Mjl9LHsieCI6MTg5LjgwOTIxMDUyNjMxNTc4LCJ5Ijo1OTJ9XQ==" data-look="classic" marker-end="url(#my-svg_stateDiagram-barbEnd)"/><path d="M185.391,552L192.493,543.833C199.594,535.667,213.797,519.333,220.899,499.667C228,480,228,457,228,436C228,415,228,396,228,377C228,358,228,339,228,320C228,301,228,282,228,263C228,244,228,225,228,206C228,187,228,168,221.509,152.333C215.018,136.667,202.035,124.333,195.544,118.167L189.053,112" id="my-svg-edge7" class="edge-thickness-normal edge-pattern-solid transition" style="fill:none;;;fill:none" data-edge="true" data-et="edge" data-id="edge7" data-points="W3sieCI6MTg1LjM5MTMwNDM0NzgyNjEsInkiOjU1Mn0seyJ4IjoyMjgsInkiOjUwM30seyJ4IjoyMjgsInkiOjQzNH0seyJ4IjoyMjgsInkiOjM3N30seyJ4IjoyMjgsInkiOjMyMH0seyJ4IjoyMjgsInkiOjI2M30seyJ4IjoyMjgsInkiOjIwNn0seyJ4IjoyMjgsInkiOjE0OX0seyJ4IjoxODkuMDUyNjMxNTc4OTQ3MzcsInkiOjExMn1d" data-look="classic" marker-end="url(#my-svg_stateDiagram-barbEnd)"/></g><g class="edgeLabels"><g class="edgeLabel"><g class="label" data-id="edge0" transform="translate(0, 0)"><foreignObject width="0" height="0"><div xmlns="http://www.w3.org/1999/xhtml" class="labelBkg" style="display: table-cell; white-space: nowrap; line-height: 1.5; max-width: 200px; text-align: center;"><span class="edgeLabel"></span></div> |
| 69 | <p></foreignObject></g></g><g class="edgeLabel" transform="translate(108, 149)"><g class="label" data-id="edge1" transform="translate(-22.515625, -12)"><foreignObject width="45.03125" height="24"><div xmlns="http://www.w3.org/1999/xhtml" class="labelBkg" style="display: table-cell; white-space: nowrap; line-height: 1.5; max-width: 200px; text-align: center;"><span class="edgeLabel"><p>start()</p></span></div></foreignObject></g></g><g class="edgeLabel" transform="translate(108, 263)"><g class="label" data-id="edge2" transform="translate(-56.7890625, -12)"><foreignObject width="113.578125" height="24"><div xmlns="http://www.w3.org/1999/xhtml" class="labelBkg" style="display: table-cell; white-space: nowrap; line-height: 1.5; max-width: 200px; text-align: center;"><span class="edgeLabel"><p>ADD_ONION ok</p></span></div></foreignObject></g></g><g class="edgeLabel" transform="translate(108, 377)"><g class="label" data-id="edge3" transform="translate(-91.4765625, -12)"><foreignObject width="182.953125" height="24"><div xmlns="http://www.w3.org/1999/xhtml" class="labelBkg" style="display: table-cell; white-space: nowrap; line-height: 1.5; max-width: 200px; text-align: center;"><span class="edgeLabel"><p>≥2 HS_DESC UPLOADED</p></span></div></foreignObject></g></g><g class="edgeLabel" transform="translate(108, 503)"><g class="label" data-id="edge4" transform="translate(-100, -24)"><foreignObject width="200" height="48"><div xmlns="http://www.w3.org/1999/xhtml" class="labelBkg" style="display: table; white-space: break-spaces; line-height: 1.5; max-width: 200px; text-align: center; width: 200px;"><span class="edgeLabel"><p>self-probe ok (or no probe vport)</p></span></div></foreignObject></g></g><g class="edgeLabel" transform="translate(105.84375, 629)"><g class="label" data-id="edge5" transform="translate(-49.7734375, -12)"><foreignObject width="99.546875" height="24"><div xmlns="http://www.w3.org/1999/xhtml" class="labelBkg" style="display: table-cell; white-space: nowrap; line-height: 1.5; max-width: 200px; text-align: center;"><span class="edgeLabel"><p>probe fails ×3</p></span></div></foreignObject></g></g><g class="edgeLabel" transform="translate(230.15625, 629)"><g class="label" data-id="edge6" transform="translate(-54.5390625, -12)"><foreignObject width="109.078125" height="24"><div xmlns="http://www.w3.org/1999/xhtml" class="labelBkg" style="display: table-cell; white-space: nowrap; line-height: 1.5; max-width: 200px; text-align: center;"><span class="edgeLabel"><p>probe recovers</p></span></div></foreignObject></g></g><g class="edgeLabel" transform="translate(228, 320)"><g class="label" data-id="edge7" transform="translate(-21.78125, -12)"><foreignObject width="43.5625" height="24"><div xmlns="http://www.w3.org/1999/xhtml" class="labelBkg" style="display: table-cell; white-space: nowrap; line-height: 1.5; max-width: 200px; text-align: center;"><span class="edgeLabel"><p>stop()</p></span></div></foreignObject></g></g></g><g class="nodes"><g class="node default" id="my-svg-state-root_start-0" data-look="classic" transform="translate(168, 15)"><circle class="state-start" r="7" width="14" height="14"/></g><g class="node statediagram-state" id="my-svg-state-disabled-7" data-look="classic" transform="translate(168, 92)"><rect class="basic label-container" style="" rx="5" ry="5" x="-38.6640625" y="-20" width="77.328125" height="40"/><g class="label" style="" transform="translate(-30.6640625, -12)"><rect/><foreignObject width="61.328125" height="24"><div xmlns="http://www.w3.org/1999/xhtml" style="display: table-cell; white-space: nowrap; line-height: 1.5; max-width: 200px; text-align: center;"><span class="nodeLabel markdown-node-label"><p>disabled</p></span></div></foreignObject></g></g><g class="node statediagram-state" id="my-svg-state-tor_starting-2" data-look="classic" transform="translate(108, 206)"><rect class="basic label-container" style="" rx="5" ry="5" x="-50.0859375" y="-20" width="100.171875" height="40"/><g class="label" style="" transform="translate(-42.0859375, -12)"><rect/><foreignObject width="84.171875" height="24"><div xmlns="http://www.w3.org/1999/xhtml" style="display: table-cell; white-space: nowrap; line-height: 1.5; max-width: 200px; text-align: center;"><span class="nodeLabel markdown-node-label"><p>tor_starting</p></span></div></foreignObject></g></g><g class="node statediagram-state" id="my-svg-state-key_loaded-3" data-look="classic" transform="translate(108, 320)"><rect class="basic label-container" style="" rx="5" ry="5" x="-49.03125" y="-20" width="98.0625" height="40"/><g class="label" style="" transform="translate(-41.03125, -12)"><rect/><foreignObject width="82.0625" height="24"><div xmlns="http://www.w3.org/1999/xhtml" style="display: table-cell; white-space: nowrap; line-height: 1.5; max-width: 200px; text-align: center;"><span class="nodeLabel markdown-node-label"><p>key_loaded</p></span></div></foreignObject></g></g><g class="node statediagram-state" id="my-svg-state-descriptor_uploaded-4" data-look="classic" transform="translate(108, 434)"><rect class="basic label-container" style="" rx="5" ry="5" x="-83.0625" y="-20" width="166.125" height="40"/><g class="label" style="" transform="translate(-75.0625, -12)"><rect/><foreignObject width="150.125" height="24"><div xmlns="http://www.w3.org/1999/xhtml" style="display: table-cell; white-space: nowrap; line-height: 1.5; max-width: 200px; text-align: center;"><span class="nodeLabel markdown-node-label"><p>descriptor_uploaded</p></span></div></foreignObject></g></g><g class="node statediagram-state" id="my-svg-state-ready-7" data-look="classic" transform="translate(168, 572)"><rect class="basic label-container" style="" rx="5" ry="5" x="-28.390625" y="-20" width="56.78125" height="40"/><g class="label" style="" transform="translate(-20.390625, -12)"><rect/><foreignObject width="40.78125" height="24"><div xmlns="http://www.w3.org/1999/xhtml" style="display: table-cell; white-space: nowrap; line-height: 1.5; max-width: 200px; text-align: center;"><span class="nodeLabel markdown-node-label"><p>ready</p></span></div></foreignObject></g></g><g class="node statediagram-state" id="my-svg-state-degraded-6" data-look="classic" transform="translate(168, 686)"><rect class="basic label-container" style="" rx="5" ry="5" x="-42.96875" y="-20" width="85.9375" height="40"/><g class="label" style="" transform="translate(-34.96875, -12)"><rect/><foreignObject width="69.9375" height="24"><div xmlns="http://www.w3.org/1999/xhtml" style="display: table-cell; white-space: nowrap; line-height: 1.5; max-width: 200px; text-align: center;"><span class="nodeLabel markdown-node-label"><p>degraded</p></span></div></foreignObject></g></g></g></g></g><defs><filter id="my-svg-drop-shadow" height="130%" width="130%"><feDropShadow dx="4" dy="4" stdDeviation="0" flood-opacity="0.06" flood-color="#000000"/></filter></defs><defs><filter id="my-svg-drop-shadow-small" height="150%" width="150%"><feDropShadow dx="2" dy="2" stdDeviation="0" flood-opacity="0.06" flood-color="#000000"/></filter></defs></svg></div></p> | |
| 70 | <p>The signed capability doc carries <code>privacyTransports</code> <strong>only while <code>ready</code></strong>: version floor checked, key restored and address matched, ≥2 descriptor uploads observed, and a SOCKS self-probe through the network back to the read-plane vport. Degraded health removes the entry — clients never see a dead endpoint (<code>ONION-INV-003</code>). The field is omitted entirely when empty, so relays without Tor produce byte-identical docs to older builds (fixture-stable signing).</p> | |
| 71 | <p>The advertisement entry: <code>{id: tor-v3-onion-v1, addresses: [{address, keyId, notBefore, notAfter, priority}], vports, vportRoles {readPlane, peer}, auth {mode, enrollment}, pow, exposure, relayLocation: 'hidden-onion', supports[]}</code> — signed by the stable relay identity, with rotation overlap handled by a fresh <code>keyId</code>.</p> | |
| 70 | <p></foreignObject></g></g><g class="edgeLabel" transform="translate(108, 149)"><g class="label" data-id="edge1" transform="translate(-22.515625, -12)"><foreignObject width="45.03125" height="24"><div xmlns="http://www.w3.org/1999/xhtml" class="labelBkg" style="display: table-cell; white-space: nowrap; line-height: 1.5; max-width: 200px; text-align: center;"><span class="edgeLabel"><p>start()</p></span></div></foreignObject></g></g><g class="edgeLabel" transform="translate(108, 263)"><g class="label" data-id="edge2" transform="translate(-56.7890625, -12)"><foreignObject width="113.578125" height="24"><div xmlns="http://www.w3.org/1999/xhtml" class="labelBkg" style="display: table-cell; white-space: nowrap; line-height: 1.5; max-width: 200px; text-align: center;"><span class="edgeLabel"><p>ADD_ONION ok</p></span></div></foreignObject></g></g><g class="edgeLabel" transform="translate(108, 377)"><g class="label" data-id="edge3" transform="translate(-91.4765625, -12)"><foreignObject width="182.953125" height="24"><div xmlns="http://www.w3.org/1999/xhtml" class="labelBkg" style="display: table-cell; white-space: nowrap; line-height: 1.5; max-width: 200px; text-align: center;"><span class="edgeLabel"><p>≥2 HS_DESC UPLOADED</p></span></div></foreignObject></g></g><g class="edgeLabel" transform="translate(108, 503)"><g class="label" data-id="edge4" transform="translate(-100, -24)"><foreignObject width="200" height="48"><div xmlns="http://www.w3.org/1999/xhtml" class="labelBkg" style="display: table; white-space: break-spaces; line-height: 1.5; max-width: 200px; text-align: center; width: 200px;"><span class="edgeLabel"><p>positive or negative probe passes</p></span></div></foreignObject></g></g><g class="edgeLabel" transform="translate(105.84375, 629)"><g class="label" data-id="edge5" transform="translate(-49.7734375, -12)"><foreignObject width="99.546875" height="24"><div xmlns="http://www.w3.org/1999/xhtml" class="labelBkg" style="display: table-cell; white-space: nowrap; line-height: 1.5; max-width: 200px; text-align: center;"><span class="edgeLabel"><p>probe fails ×3</p></span></div></foreignObject></g></g><g class="edgeLabel" transform="translate(230.15625, 629)"><g class="label" data-id="edge6" transform="translate(-54.5390625, -12)"><foreignObject width="109.078125" height="24"><div xmlns="http://www.w3.org/1999/xhtml" class="labelBkg" style="display: table-cell; white-space: nowrap; line-height: 1.5; max-width: 200px; text-align: center;"><span class="edgeLabel"><p>probe recovers</p></span></div></foreignObject></g></g><g class="edgeLabel" transform="translate(228, 320)"><g class="label" data-id="edge7" transform="translate(-21.78125, -12)"><foreignObject width="43.5625" height="24"><div xmlns="http://www.w3.org/1999/xhtml" class="labelBkg" style="display: table-cell; white-space: nowrap; line-height: 1.5; max-width: 200px; text-align: center;"><span class="edgeLabel"><p>stop()</p></span></div></foreignObject></g></g></g><g class="nodes"><g class="node default" id="my-svg-state-root_start-0" data-look="classic" transform="translate(168, 15)"><circle class="state-start" r="7" width="14" height="14"/></g><g class="node statediagram-state" id="my-svg-state-disabled-7" data-look="classic" transform="translate(168, 92)"><rect class="basic label-container" style="" rx="5" ry="5" x="-38.6640625" y="-20" width="77.328125" height="40"/><g class="label" style="" transform="translate(-30.6640625, -12)"><rect/><foreignObject width="61.328125" height="24"><div xmlns="http://www.w3.org/1999/xhtml" style="display: table-cell; white-space: nowrap; line-height: 1.5; max-width: 200px; text-align: center;"><span class="nodeLabel markdown-node-label"><p>disabled</p></span></div></foreignObject></g></g><g class="node statediagram-state" id="my-svg-state-tor_starting-2" data-look="classic" transform="translate(108, 206)"><rect class="basic label-container" style="" rx="5" ry="5" x="-50.0859375" y="-20" width="100.171875" height="40"/><g class="label" style="" transform="translate(-42.0859375, -12)"><rect/><foreignObject width="84.171875" height="24"><div xmlns="http://www.w3.org/1999/xhtml" style="display: table-cell; white-space: nowrap; line-height: 1.5; max-width: 200px; text-align: center;"><span class="nodeLabel markdown-node-label"><p>tor_starting</p></span></div></foreignObject></g></g><g class="node statediagram-state" id="my-svg-state-key_loaded-3" data-look="classic" transform="translate(108, 320)"><rect class="basic label-container" style="" rx="5" ry="5" x="-49.03125" y="-20" width="98.0625" height="40"/><g class="label" style="" transform="translate(-41.03125, -12)"><rect/><foreignObject width="82.0625" height="24"><div xmlns="http://www.w3.org/1999/xhtml" style="display: table-cell; white-space: nowrap; line-height: 1.5; max-width: 200px; text-align: center;"><span class="nodeLabel markdown-node-label"><p>key_loaded</p></span></div></foreignObject></g></g><g class="node statediagram-state" id="my-svg-state-descriptor_uploaded-4" data-look="classic" transform="translate(108, 434)"><rect class="basic label-container" style="" rx="5" ry="5" x="-83.0625" y="-20" width="166.125" height="40"/><g class="label" style="" transform="translate(-75.0625, -12)"><rect/><foreignObject width="150.125" height="24"><div xmlns="http://www.w3.org/1999/xhtml" style="display: table-cell; white-space: nowrap; line-height: 1.5; max-width: 200px; text-align: center;"><span class="nodeLabel markdown-node-label"><p>descriptor_uploaded</p></span></div></foreignObject></g></g><g class="node statediagram-state" id="my-svg-state-ready-7" data-look="classic" transform="translate(168, 572)"><rect class="basic label-container" style="" rx="5" ry="5" x="-28.390625" y="-20" width="56.78125" height="40"/><g class="label" style="" transform="translate(-20.390625, -12)"><rect/><foreignObject width="40.78125" height="24"><div xmlns="http://www.w3.org/1999/xhtml" style="display: table-cell; white-space: nowrap; line-height: 1.5; max-width: 200px; text-align: center;"><span class="nodeLabel markdown-node-label"><p>ready</p></span></div></foreignObject></g></g><g class="node statediagram-state" id="my-svg-state-degraded-6" data-look="classic" transform="translate(168, 686)"><rect class="basic label-container" style="" rx="5" ry="5" x="-42.96875" y="-20" width="85.9375" height="40"/><g class="label" style="" transform="translate(-34.96875, -12)"><rect/><foreignObject width="69.9375" height="24"><div xmlns="http://www.w3.org/1999/xhtml" style="display: table-cell; white-space: nowrap; line-height: 1.5; max-width: 200px; text-align: center;"><span class="nodeLabel markdown-node-label"><p>degraded</p></span></div></foreignObject></g></g></g></g></g><defs><filter id="my-svg-drop-shadow" height="130%" width="130%"><feDropShadow dx="4" dy="4" stdDeviation="0" flood-opacity="0.06" flood-color="#000000"/></filter></defs><defs><filter id="my-svg-drop-shadow-small" height="150%" width="150%"><feDropShadow dx="2" dy="2" stdDeviation="0" flood-opacity="0.06" flood-color="#000000"/></filter></defs></svg></div></p> | |
| 71 | <p>The signed capability doc carries <code>privacyTransports</code> <strong>only while <code>ready</code></strong>: version floor checked, key restored and address matched, ≥2 descriptor uploads observed, and a SOCKS probe through the network to an automatically selected exposed vport. Public onions require a successful connection. Restricted onions require an authorization rejection and carry signed <code>negativeProbe: true</code>. Explicit descriptor-only compatibility never advertises a restricted endpoint. Degraded health removes the entry (<code>ONION-INV-003</code>).</p> | |
| 72 | <p>The advertisement entry: <code>{id: tor-v3-onion-v1, addresses: [{address, keyId, notBefore, notAfter, priority}], vports, vportRoles {readPlane, peer}, auth {mode, enrollment}, negativeProbe?, pow, exposure, relayLocation: 'hidden-onion', supports[]}</code> — signed by the stable relay identity, with rotation overlap handled by a fresh <code>keyId</code>.</p> | |
| 72 | 73 | <h2>6. Policy resolution (client side)</h2> |
| 73 | 74 | <div class="mermaid-figure"><svg id="my-svg" width="100%" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" class="flowchart" style="max-width: 730.016px; background-color: transparent;" viewBox="0 0 730.015625 1082.5625" role="graphics-document document" aria-roledescription="flowchart-v2"><style>#my-svg{font-family:-apple-system,BlinkMacSystemFont,Segoe UI,Roboto,Helvetica,Arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#my-svg .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#my-svg .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#my-svg .error-icon{fill:#552222;}#my-svg .error-text{fill:#552222;stroke:#552222;}#my-svg .edge-thickness-normal{stroke-width:1px;}#my-svg .edge-thickness-thick{stroke-width:3.5px;}#my-svg .edge-pattern-solid{stroke-dasharray:0;}#my-svg .edge-thickness-invisible{stroke-width:0;fill:none;}#my-svg .edge-pattern-dashed{stroke-dasharray:3;}#my-svg .edge-pattern-dotted{stroke-dasharray:2;}#my-svg .marker{fill:#5a7196;stroke:#5a7196;}#my-svg .marker.cross{stroke:#5a7196;}#my-svg svg{font-family:-apple-system,BlinkMacSystemFont,Segoe UI,Roboto,Helvetica,Arial,sans-serif;font-size:16px;}#my-svg p{margin:0;}#my-svg .label{font-family:-apple-system,BlinkMacSystemFont,Segoe UI,Roboto,Helvetica,Arial,sans-serif;color:#333;}#my-svg .cluster-label text{fill:#333;}#my-svg .cluster-label span{color:#333;}#my-svg .cluster-label span p{background-color:transparent;}#my-svg .label text,#my-svg span{fill:#333;color:#333;}#my-svg .node rect,#my-svg .node circle,#my-svg .node ellipse,#my-svg .node polygon,#my-svg .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#my-svg .rough-node .label text,#my-svg .node .label text,#my-svg .image-shape .label,#my-svg .icon-shape .label{text-anchor:middle;}#my-svg .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#my-svg .rough-node .label,#my-svg .node .label,#my-svg .image-shape .label,#my-svg .icon-shape .label{text-align:center;}#my-svg .node.clickable{cursor:pointer;}#my-svg .root .anchor path{fill:#5a7196!important;stroke-width:0;stroke:#5a7196;}#my-svg .arrowheadPath{fill:#333333;}#my-svg .edgePath .path{stroke:#5a7196;stroke-width:1px;}#my-svg .flowchart-link{stroke:#5a7196;fill:none;}#my-svg .edgeLabel{background-color:#ffffff;text-align:center;}#my-svg .edgeLabel p{background-color:#ffffff;}#my-svg .edgeLabel rect{opacity:0.5;background-color:#ffffff;fill:#ffffff;}#my-svg .labelBkg{background-color:rgba(255, 255, 255, 0.5);}#my-svg .cluster rect{fill:#f6f8fc;stroke:#aaaa33;stroke-width:1px;}#my-svg .cluster text{fill:#333;}#my-svg .cluster span{color:#333;}#my-svg div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:-apple-system,BlinkMacSystemFont,Segoe UI,Roboto,Helvetica,Arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#my-svg .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#my-svg rect.text{fill:none;stroke-width:0;}#my-svg .icon-shape,#my-svg .image-shape{background-color:#ffffff;text-align:center;}#my-svg .icon-shape p,#my-svg .image-shape p{background-color:#ffffff;padding:2px;}#my-svg .icon-shape .label rect,#my-svg .image-shape .label rect{opacity:0.5;background-color:#ffffff;fill:#ffffff;}#my-svg .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#my-svg .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#my-svg .node .neo-node{stroke:#9370DB;}#my-svg [data-look="neo"].node rect,#my-svg [data-look="neo"].cluster rect,#my-svg [data-look="neo"].node polygon{stroke:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#my-svg [data-look="neo"].swimlane.cluster rect{filter:none;}#my-svg [data-look="neo"].node path{stroke:#9370DB;stroke-width:1px;}#my-svg [data-look="neo"].node .outer-path{filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#my-svg [data-look="neo"].node .neo-line path{stroke:#9370DB;filter:none;}#my-svg [data-look="neo"].node circle{stroke:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#my-svg [data-look="neo"].node circle .state-start{fill:#000000;}#my-svg [data-look="neo"].icon-shape .icon{fill:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#my-svg [data-look="neo"].icon-shape .icon-neo path{stroke:#9370DB;filter:drop-shadow(1px 2px 2px rgba(185, 185, 185, 1));}#my-svg :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;}</style><g><marker id="my-svg_flowchart-v2-pointEnd" class="marker flowchart-v2" viewBox="0 0 10 10" refX="5" refY="5" markerUnits="userSpaceOnUse" markerWidth="8" markerHeight="8" orient="auto"><path d="M 0 0 L 10 5 L 0 10 z" class="arrowMarkerPath" style="stroke-width: 1; stroke-dasharray: 1, 0;"/></marker><marker id="my-svg_flowchart-v2-pointStart" class="marker flowchart-v2" viewBox="0 0 10 10" refX="4.5" refY="5" markerUnits="userSpaceOnUse" markerWidth="8" markerHeight="8" orient="auto"><path d="M 0 5 L 10 10 L 10 0 z" class="arrowMarkerPath" style="stroke-width: 1; stroke-dasharray: 1, 0;"/></marker><marker id="my-svg_flowchart-v2-pointEnd-margin" class="marker flowchart-v2" viewBox="0 0 11.5 14" refX="11.5" refY="7" markerUnits="userSpaceOnUse" markerWidth="10.5" markerHeight="14" orient="auto"><path d="M 0 0 L 11.5 7 L 0 14 z" class="arrowMarkerPath" style="stroke-width: 0; stroke-dasharray: 1, 0;"/></marker><marker id="my-svg_flowchart-v2-pointStart-margin" class="marker flowchart-v2" viewBox="0 0 11.5 14" refX="1" refY="7" markerUnits="userSpaceOnUse" markerWidth="11.5" markerHeight="14" orient="auto"><polygon points="0,7 11.5,14 11.5,0" class="arrowMarkerPath" style="stroke-width: 0; stroke-dasharray: 1, 0;"/></marker><marker id="my-svg_flowchart-v2-circleEnd" class="marker flowchart-v2" viewBox="0 0 10 10" refX="11" refY="5" markerUnits="userSpaceOnUse" markerWidth="11" markerHeight="11" orient="auto"><circle cx="5" cy="5" r="5" class="arrowMarkerPath" style="stroke-width: 1; stroke-dasharray: 1, 0;"/></marker><marker id="my-svg_flowchart-v2-circleStart" class="marker flowchart-v2" viewBox="0 0 10 10" refX="-1" refY="5" markerUnits="userSpaceOnUse" markerWidth="11" markerHeight="11" orient="auto"><circle cx="5" cy="5" r="5" class="arrowMarkerPath" style="stroke-width: 1; stroke-dasharray: 1, 0;"/></marker><marker id="my-svg_flowchart-v2-circleEnd-margin" class="marker flowchart-v2" viewBox="0 0 10 10" refY="5" refX="12.25" markerUnits="userSpaceOnUse" markerWidth="14" markerHeight="14" orient="auto"><circle cx="5" cy="5" r="5" class="arrowMarkerPath" style="stroke-width: 0; stroke-dasharray: 1, 0;"/></marker><marker id="my-svg_flowchart-v2-circleStart-margin" class="marker flowchart-v2" viewBox="0 0 10 10" refX="-2" refY="5" markerUnits="userSpaceOnUse" markerWidth="14" markerHeight="14" orient="auto"><circle cx="5" cy="5" r="5" class="arrowMarkerPath" style="stroke-width: 0; stroke-dasharray: 1, 0;"/></marker><marker id="my-svg_flowchart-v2-crossEnd" class="marker cross flowchart-v2" viewBox="0 0 11 11" refX="12" refY="5.2" markerUnits="userSpaceOnUse" markerWidth="11" markerHeight="11" orient="auto"><path d="M 1,1 l 9,9 M 10,1 l -9,9" class="arrowMarkerPath" style="stroke-width: 2; stroke-dasharray: 1, 0;"/></marker><marker id="my-svg_flowchart-v2-crossStart" class="marker cross flowchart-v2" viewBox="0 0 11 11" refX="-1" refY="5.2" markerUnits="userSpaceOnUse" markerWidth="11" markerHeight="11" orient="auto"><path d="M 1,1 l 9,9 M 10,1 l -9,9" class="arrowMarkerPath" style="stroke-width: 2; stroke-dasharray: 1, 0;"/></marker><marker id="my-svg_flowchart-v2-crossEnd-margin" class="marker cross flowchart-v2" viewBox="0 0 15 15" refX="17.7" refY="7.5" markerUnits="userSpaceOnUse" markerWidth="12" markerHeight="12" orient="auto"><path d="M 1,1 L 14,14 M 1,14 L 14,1" class="arrowMarkerPath" style="stroke-width: 2.5;"/></marker><marker id="my-svg_flowchart-v2-crossStart-margin" class="marker cross flowchart-v2" viewBox="0 0 15 15" refX="-3.5" refY="7.5" markerUnits="userSpaceOnUse" markerWidth="12" markerHeight="12" orient="auto"><path d="M 1,1 L 14,14 M 1,14 L 14,1" class="arrowMarkerPath" style="stroke-width: 2.5; stroke-dasharray: 1, 0;"/></marker><g class="root"><g class="clusters"/><g class="edgePaths"><path d="M138,182L138,186.167C138,190.333,138,198.667,151.391,207.122C164.782,215.577,191.565,224.154,204.956,228.442L218.347,232.731" id="my-svg-L_I_R_0" class="edge-thickness-normal edge-pattern-solid edge-thickness-normal edge-pattern-solid flowchart-link" style=";" data-edge="true" data-et="edge" data-id="L_I_R_0" data-points="W3sieCI6MTM4LCJ5IjoxODJ9LHsieCI6MTM4LCJ5IjoyMDd9LHsieCI6MjIyLjE1NjI1LCJ5IjoyMzMuOTUwNzMxMzMxNzkzN31d" data-look="classic" marker-end="url(#my-svg_flowchart-v2-pointEnd)"/><path d="M462.75,146L462.75,156.167C462.75,166.333,462.75,186.667,449.359,201.122C435.968,215.577,409.185,224.154,395.794,228.442L382.403,232.731" id="my-svg-L_C_R_0" class="edge-thickness-normal edge-pattern-solid edge-thickness-normal edge-pattern-solid flowchart-link" style=";" data-edge="true" data-et="edge" data-id="L_C_R_0" data-points="W3sieCI6NDYyLjc1LCJ5IjoxNDZ9LHsieCI6NDYyLjc1LCJ5IjoyMDd9LHsieCI6Mzc4LjU5Mzc1LCJ5IjoyMzMuOTUwNzMxMzMxNzkzN31d" data-look="classic" marker-end="url(#my-svg_flowchart-v2-pointEnd)"/><path d="M300.375,286L300.375,290.167C300.375,294.333,300.375,302.667,300.375,310.333C300.375,318,300.375,325,300.375,328.5L300.375,332" id="my-svg-L_R_E_0" class="edge-thickness-normal edge-pattern-solid edge-thickness-normal edge-pattern-solid flowchart-link" style=";" data-edge="true" data-et="edge" data-id="L_R_E_0" data-points="W3sieCI6MzAwLjM3NSwieSI6Mjg2fSx7IngiOjMwMC4zNzUsInkiOjMxMX0seyJ4IjozMDAuMzc1LCJ5IjozMzZ9XQ==" data-look="classic" marker-end="url(#my-svg_flowchart-v2-pointEnd)"/><path d="M239.625,553.25L226.976,569.541C214.328,585.833,189.031,618.417,176.383,649.255C163.734,680.094,163.734,709.188,163.734,723.734L163.734,738.281" id="my-svg-L_E_S_0" class="edge-thickness-normal edge-pattern-solid edge-thickness-normal edge-pattern-solid flowchart-link" style=";" data-edge="true" data-et="edge" data-id="L_E_S_0" data-points="W3sieCI6MjM5LjYyNDU4NzY4NTU0MTUsInkiOjU1My4yNDk1ODc2ODU1NDE1fSx7IngiOjE2My43MzQzNzUsInkiOjY1MX0seyJ4IjoxNjMuNzM0Mzc1LCJ5Ijo3NDIuMjgxMjV9XQ==" data-look="classic" marker-end="url(#my-svg_flowchart-v2-pointEnd)"/><path d="M361.125,553.25L373.774,569.541C386.422,585.833,411.719,618.417,424.367,640.208C437.016,662,437.016,673,437.016,678.5L437.016,684" id="my-svg-L_E_D_0" class="edge-thickness-normal edge-pattern-solid edge-thickness-normal edge-pattern-solid flowchart-link" style=";" data-edge="true" data-et="edge" data-id="L_E_D_0" data-points="W3sieCI6MzYxLjEyNTQxMjMxNDQ1ODUsInkiOjU1My4yNDk1ODc2ODU1NDE1fSx7IngiOjQzNy4wMTU2MjUsInkiOjY1MX0seyJ4Ijo0MzcuMDE1NjI1LCJ5Ijo2ODh9XQ==" data-look="classic" marker-end="url(#my-svg_flowchart-v2-pointEnd)"/><path d="M386.334,823.881L368.947,838.494C351.561,853.108,316.788,882.335,299.402,904.449C282.016,926.563,282.016,941.563,282.016,949.063L282.016,956.563" id="my-svg-L_D_F_0" class="edge-thickness-normal edge-pattern-solid edge-thickness-normal edge-pattern-solid flowchart-link" style=";" data-edge="true" data-et="edge" data-id="L_D_F_0" data-points="W3sieCI6Mzg2LjMzMzczMjEzMTEyMDYsInkiOjgyMy44ODA2MDcxMzExMjA3fSx7IngiOjI4Mi4wMTU2MjUsInkiOjkxMS41NjI1fSx7IngiOjI4Mi4wMTU2MjUsInkiOjk2MC41NjI1fV0=" data-look="classic" marker-end="url(#my-svg_flowchart-v2-pointEnd)"/><path d="M487.698,823.881L505.084,838.494C522.47,853.108,557.243,882.335,574.629,902.449C592.016,922.563,592.016,933.563,592.016,939.063L592.016,944.563" id="my-svg-L_D_O_0" class="edge-thickness-normal edge-pattern-solid edge-thickness-normal edge-pattern-solid flowchart-link" style=";" data-edge="true" data-et="edge" data-id="L_D_O_0" data-points="W3sieCI6NDg3LjY5NzUxNzg2ODg3OTQsInkiOjgyMy44ODA2MDcxMzExMjA3fSx7IngiOjU5Mi4wMTU2MjUsInkiOjkxMS41NjI1fSx7IngiOjU5Mi4wMTU2MjUsInkiOjk0OC41NjI1fV0=" data-look="classic" marker-end="url(#my-svg_flowchart-v2-pointEnd)"/></g><g class="edgeLabels"><g class="edgeLabel"><g class="label" data-id="L_I_R_0" transform="translate(0, 0)"><foreignObject width="0" height="0"><div xmlns="http://www.w3.org/1999/xhtml" class="labelBkg" style="display: table-cell; white-space: nowrap; line-height: 1.5; max-width: 200px; text-align: center;"><span class="edgeLabel"></span></div> |
| 74 | 75 | <p></foreignObject></g></g><g class="edgeLabel"><g class="label" data-id="L_C_R_0" transform="translate(0, 0)"><foreignObject width="0" height="0"><div xmlns="http://www.w3.org/1999/xhtml" class="labelBkg" style="display: table-cell; white-space: nowrap; line-height: 1.5; max-width: 200px; text-align: center;"><span class="edgeLabel"></span></div></foreignObject></g></g><g class="edgeLabel"><g class="label" data-id="L_R_E_0" transform="translate(0, 0)"><foreignObject width="0" height="0"><div xmlns="http://www.w3.org/1999/xhtml" class="labelBkg" style="display: table-cell; white-space: nowrap; line-height: 1.5; max-width: 200px; text-align: center;"><span class="edgeLabel"></span></div></foreignObject></g></g><g class="edgeLabel" transform="translate(163.734375, 651)"><g class="label" data-id="L_E_S_0" transform="translate(-12.4765625, -12)"><foreignObject width="24.953125" height="24"><div xmlns="http://www.w3.org/1999/xhtml" class="labelBkg" style="display: table-cell; white-space: nowrap; line-height: 1.5; max-width: 200px; text-align: center;"><span class="edgeLabel"><p>yes</p></span></div></foreignObject></g></g><g class="edgeLabel" transform="translate(437.015625, 651)"><g class="label" data-id="L_E_D_0" transform="translate(-9.0859375, -12)"><foreignObject width="18.171875" height="24"><div xmlns="http://www.w3.org/1999/xhtml" class="labelBkg" style="display: table-cell; white-space: nowrap; line-height: 1.5; max-width: 200px; text-align: center;"><span class="edgeLabel"><p>no</p></span></div></foreignObject></g></g><g class="edgeLabel" transform="translate(282.015625, 911.5625)"><g class="label" data-id="L_D_F_0" transform="translate(-17.71875, -12)"><foreignObject width="35.4375" height="24"><div xmlns="http://www.w3.org/1999/xhtml" class="labelBkg" style="display: table-cell; white-space: nowrap; line-height: 1.5; max-width: 200px; text-align: center;"><span class="edgeLabel"><p>deny</p></span></div></foreignObject></g></g><g class="edgeLabel" transform="translate(592.015625, 911.5625)"><g class="label" data-id="L_D_O_0" transform="translate(-46.8828125, -12)"><foreignObject width="93.765625" height="24"><div xmlns="http://www.w3.org/1999/xhtml" class="labelBkg" style="display: table-cell; white-space: nowrap; line-height: 1.5; max-width: 200px; text-align: center;"><span class="edgeLabel"><p>[ordered ids]</p></span></div></foreignObject></g></g></g><g class="nodes"><g class="node default" id="my-svg-flowchart-I-0" data-look="classic" transform="translate(138, 95)"><rect class="basic label-container" style="" x="-130" y="-87" width="260" height="174"/><g class="label" style="" transform="translate(-100, -72)"><rect/><foreignObject width="200" height="144"><div xmlns="http://www.w3.org/1999/xhtml" style="display: table; white-space: break-spaces; line-height: 1.5; max-width: 200px; text-align: center; width: 200px;"><span class="nodeLabel"><p>privacyIntent<br />{transportPrivacy, relayLocation,<br />metadataShaping, pathCoverage, downgradePolicy}</p></span></div></foreignObject></g></g><g class="node default" id="my-svg-flowchart-R-1" data-look="classic" transform="translate(300.375, 259)"><rect class="basic label-container" style="" x="-78.21875" y="-27" width="156.4375" height="54"/><g class="label" style="" transform="translate(-48.21875, -12)"><rect/><foreignObject width="96.4375" height="24"><div xmlns="http://www.w3.org/1999/xhtml" style="display: table-cell; white-space: nowrap; line-height: 1.5; max-width: 200px; text-align: center;"><span class="nodeLabel"><p>resolvePath()</p></span></div></foreignObject></g></g><g class="node default" id="my-svg-flowchart-C-2" data-look="classic" transform="translate(462.75, 95)"><rect class="basic label-container" style="" x="-144.75" y="-51" width="289.5" height="102"/><g class="label" style="" transform="translate(-114.75, -36)"><rect/><foreignObject width="229.5" height="72"><div xmlns="http://www.w3.org/1999/xhtml" style="display: table; white-space: break-spaces; line-height: 1.5; max-width: 200px; text-align: center; width: 200px;"><span class="nodeLabel"><p>candidatesFromCapabilityDoc()<br />direct · forward-relay · onion · nym</p></span></div></foreignObject></g></g><g class="node default" id="my-svg-flowchart-E-5" data-look="classic" transform="translate(300.375, 475)"><polygon points="139,0 278,-139 139,-278 0,-139" class="label-container" transform="translate(-138.5, 139)"/><g class="label" style="" transform="translate(-100, -24)"><rect/><foreignObject width="200" height="48"><div xmlns="http://www.w3.org/1999/xhtml" style="display: table; white-space: break-spaces; line-height: 1.5; max-width: 200px; text-align: center; width: 200px;"><span class="nodeLabel"><p>any candidate fully satisfies?</p></span></div></foreignObject></g></g><g class="node default" id="my-svg-flowchart-S-7" data-look="classic" transform="translate(163.734375, 781.28125)"><rect class="basic label-container" style="" x="-130" y="-39" width="260" height="78"/><g class="label" style="" transform="translate(-100, -24)"><rect/><foreignObject width="200" height="48"><div xmlns="http://www.w3.org/1999/xhtml" style="display: table; white-space: break-spaces; line-height: 1.5; max-width: 200px; text-align: center; width: 200px;"><span class="nodeLabel"><p>selectedTransport + satisfied[] + coverage</p></span></div></foreignObject></g></g><g class="node default" id="my-svg-flowchart-D-9" data-look="classic" transform="translate(437.015625, 781.28125)"><polygon points="93.28125,0 186.5625,-93.28125 93.28125,-186.5625 0,-93.28125" class="label-container" transform="translate(-92.78125, 93.28125)"/><g class="label" style="" transform="translate(-66.28125, -12)"><rect/><foreignObject width="132.5625" height="24"><div xmlns="http://www.w3.org/1999/xhtml" style="display: table-cell; white-space: nowrap; line-height: 1.5; max-width: 200px; text-align: center;"><span class="nodeLabel"><p>downgradePolicy?</p></span></div></foreignObject></g></g><g class="node default" id="my-svg-flowchart-F-11" data-look="classic" transform="translate(282.015625, 1011.5625)"><rect class="basic label-container" style="" x="-130" y="-51" width="260" height="102"/><g class="label" style="" transform="translate(-100, -36)"><rect/><foreignObject width="200" height="72"><div xmlns="http://www.w3.org/1999/xhtml" style="display: table; white-space: break-spaces; line-height: 1.5; max-width: 200px; text-align: center; width: 200px;"><span class="nodeLabel"><p>FAIL CLOSED<br />(no send, evidence: unsatisfied[])</p></span></div></foreignObject></g></g><g class="node default" id="my-svg-flowchart-O-13" data-look="classic" transform="translate(592.015625, 1011.5625)"><rect class="basic label-container" style="" x="-130" y="-63" width="260" height="126"/><g class="label" style="" transform="translate(-100, -48)"><rect/><foreignObject width="200" height="96"><div xmlns="http://www.w3.org/1999/xhtml" style="display: table; white-space: break-spaces; line-height: 1.5; max-width: 200px; text-align: center; width: 200px;"><span class="nodeLabel"><p>first available declared fallback<br />downgraded: true, never silent</p></span></div></foreignObject></g></g></g></g></g><defs><filter id="my-svg-drop-shadow" height="130%" width="130%"><feDropShadow dx="4" dy="4" stdDeviation="0" flood-opacity="0.06" flood-color="#000000"/></filter></defs><defs><filter id="my-svg-drop-shadow-small" height="150%" width="150%"><feDropShadow dx="2" dy="2" stdDeviation="0" flood-opacity="0.06" flood-color="#000000"/></filter></defs></svg></div></p> |
| … | … | @@ -115,8 +115,13 @@ with the relay's stable pubkey (from the signed capability doc) as |
| 115 | 115 | |
| 116 | 116 | The onion endpoint appears in the signed capability doc |
| 117 | 117 | (`privacyTransports`) only while the transport reports `ready` |
| 118 | (descriptor uploads observed + optional SOCKS self-probe). Degraded health | |
| 119 | removes the entry — clients never see a dead endpoint. Health states: | |
| 118 | (descriptor uploads observed + an automatically selected SOCKS probe). | |
| 119 | Public endpoints require a successful self-connect. Restricted-discovery | |
| 120 | endpoints require an unauthenticated connection attempt to be rejected and | |
| 121 | carry signed `negativeProbe: true`; without that proof they are omitted. | |
| 122 | `health.probeVport: false` explicitly selects descriptor-only compatibility | |
| 123 | and therefore cannot advertise a restricted endpoint. Degraded health removes | |
| 124 | the entry. Health states: | |
| 120 | 125 | `tor-starting → key-loaded → descriptor-uploaded → ready`, plus `degraded` |
| 121 | 126 | and `disabled`. |
| 122 | 127 |
| … | … | @@ -1,9 +1,11 @@ |
| 1 | 1 | # Fleet management |
| 2 | 2 | |
| 3 | 3 | Tooling for the **raw systemd fleet** — the relay boxes we operate |
| 4 | directly (utah, utah-us, utah-2gb-a, utah-0.5gb, utah-8gb, sing-1, sing-2, bern, dubai). The Umbrel and StartOS | |
| 5 | packaged relays auto-update through their own registries and are *not* | |
| 6 | managed here. | |
| 4 | directly. [`relays.json`](relays.json), rather than a duplicated list in this | |
| 5 | document, is the authoritative public inventory. A private | |
| 6 | `relays.local.json` may add operator-only connection details or isolated lanes. | |
| 7 | The Umbrel and StartOS packaged relays auto-update through their own registries | |
| 8 | and are *not* managed here. | |
| 7 | 9 | |
| 8 | 10 | It exists because hand-SSHing five boxes from one laptop is fragile: a |
| 9 | 11 | stale `known_hosts` entry looks identical to "we lost a server" (it cost |
| … | … | @@ -20,8 +22,8 @@ for boxes we don't control. |
| 20 | 22 | |
| 21 | 23 | ``` |
| 22 | 24 | fleet/channels.json ──(raw.githubusercontent)──▶ each box's updater |
| 23 | stable: v0.20.1 reads its channel, | |
| 24 | canary: v0.20.1 checks out the tag, | |
| 25 | stable: <signed tag> reads its channel, | |
| 26 | canary: <signed tag> checks out the tag, | |
| 25 | 27 | health-gates, and |
| 26 | 28 | rolls back on failure |
| 27 | 29 | ``` |
| … | … | @@ -86,6 +88,20 @@ For a quick table without waiting for convergence, run: |
| 86 | 88 | bash fleet/fleet-status.sh |
| 87 | 89 | ``` |
| 88 | 90 | |
| 91 | Scope a scan before it opens SSH connections with comma-separated relay-name | |
| 92 | allow/deny lists. This is required when another operator owns a lane: | |
| 93 | ||
| 94 | ```bash | |
| 95 | HIVERELAY_FLEET_INVENTORY=fleet/relays.local.json \ | |
| 96 | HIVERELAY_FLEET_INCLUDE=utah,utah-us,bern \ | |
| 97 | bash fleet/fleet-status.sh | |
| 98 | ``` | |
| 99 | ||
| 100 | `HIVERELAY_FLEET_EXCLUDE` uses the same syntax and is applied after the | |
| 101 | allowlist. Invalid list syntax fails closed before any SSH connection. The | |
| 102 | runtime service-catalogue probe accepts the same environment variables, or | |
| 103 | equivalent `--include` / `--exclude` options. | |
| 104 | ||
| 89 | 105 | To **hold** a box, point its channel at the version it already runs. To |
| 90 | 106 | **roll the whole channel back**, set the tag back — boxes check out the |
| 91 | 107 | older tag exactly like a forward update (health-gated the same way). |
| … | … | @@ -141,11 +157,10 @@ name automatically. |
| 141 | 157 | ```bash |
| 142 | 158 | # On each box (repo already cloned at ~/hiverelay): |
| 143 | 159 | sudo bash fleet/tailscale-enroll.sh |
| 144 | sudo bash fleet/install-updater.sh canary # utah | |
| 145 | sudo bash fleet/install-updater.sh stable # the other four | |
| 160 | sudo bash fleet/install-updater.sh canary # only boxes assigned canary in relays.json | |
| 161 | sudo bash fleet/install-updater.sh stable # only boxes assigned stable in relays.json | |
| 146 | 162 | hiverelay-updater --dry-run # confirm the decision, no changes |
| 147 | 163 | ``` |
| 148 | 164 | |
| 149 | `channels.json` ships with `stable: v0.20.1` and `canary: v0.20.1`, so | |
| 150 | installing the agent on a box already at the current release is a | |
| 151 | **no-op** until you promote the channel to a newer tag. | |
| 165 | Installing the agent on a box already at its channel's current signed tag is a | |
| 166 | **no-op** until that channel is promoted or rolled back. |
| … | … | @@ -7,6 +7,12 @@ |
| 7 | 7 | # expose it; we shell in and query locally). |
| 8 | 8 | # |
| 9 | 9 | # bash fleet/fleet-status.sh |
| 10 | # HIVERELAY_FLEET_INCLUDE=utah,bern bash fleet/fleet-status.sh | |
| 11 | # | |
| 12 | # HIVERELAY_FLEET_INCLUDE and HIVERELAY_FLEET_EXCLUDE are comma-separated | |
| 13 | # relay-name lists. They are applied before any SSH connection is opened, so an | |
| 14 | # operator can prove a maintenance scan cannot touch an independently owned or | |
| 15 | # held lane. | |
| 10 | 16 | # |
| 11 | 17 | # StrictHostKeyChecking=accept-new is deliberate so a rotated host key never |
| 12 | 18 | # masquerades as "server down" (the 2026-06-16 bern false alarm). |
| … | … | @@ -18,6 +24,8 @@ HERE="$(cd "$(dirname "$0")" && pwd)" |
| 18 | 24 | # at a private copy (e.g. fleet/relays.local.json, gitignored) to supply them. |
| 19 | 25 | RELAYS="${HIVERELAY_FLEET_INVENTORY:-$HERE/relays.json}" |
| 20 | 26 | [ -f "$RELAYS" ] || { echo "fleet inventory not found: $RELAYS"; exit 1; } |
| 27 | INCLUDE="${HIVERELAY_FLEET_INCLUDE:-}" | |
| 28 | EXCLUDE="${HIVERELAY_FLEET_EXCLUDE:-}" | |
| 21 | 29 | |
| 22 | 30 | CH="$(curl -fsS --max-time 15 https://raw.githubusercontent.com/bigdestiny2/P2P-Hiverelay/main/fleet/channels.json 2>/dev/null || cat "$HERE/channels.json")" |
| 23 | 31 | |
| … | … | @@ -50,9 +58,17 @@ valid_key_path() { |
| 50 | 58 | ! printf '%s' "$value" | LC_ALL=C grep -q '[[:cntrl:]]' |
| 51 | 59 | } |
| 52 | 60 | |
| 61 | valid_name_list() { | |
| 62 | local value="${1:-}" | |
| 63 | [ -z "$value" ] || [[ "$value" =~ ^[A-Za-z0-9._-]+(,[A-Za-z0-9._-]+)*$ ]] | |
| 64 | } | |
| 65 | ||
| 66 | valid_name_list "$INCLUDE" || { echo "invalid HIVERELAY_FLEET_INCLUDE relay list" >&2; exit 1; } | |
| 67 | valid_name_list "$EXCLUDE" || { echo "invalid HIVERELAY_FLEET_EXCLUDE relay list" >&2; exit 1; } | |
| 68 | ||
| 53 | 69 | # Remote probe, delivered verbatim over stdin (bash -s) — no quote escaping. |
| 54 | 70 | read -r -d '' REMOTE <<'REMOTE_EOF' || true |
| 55 | V="v$(grep -m1 '"version"' "$HOME/hiverelay/package.json" | tr -dc '0-9.')" | |
| 71 | V="v$(node -p 'require(process.env.HOME + "/hiverelay/package.json").version' 2>/dev/null || printf '?')" | |
| 56 | 72 | read_api_key() { |
| 57 | 73 | local key |
| 58 | 74 | key="$(systemctl show hiverelay -p Environment 2>/dev/null | awk 'BEGIN{RS=" "} /^HIVERELAY_API_KEY=/{sub(/^HIVERELAY_API_KEY=/,""); print; exit}' || true)" |
| … | … | @@ -87,27 +103,34 @@ curl_with_optional_key() { |
| 87 | 103 | } |
| 88 | 104 | K="$(read_api_key)" |
| 89 | 105 | H=$(curl_with_optional_key "$K" -fsS --max-time 10 http://127.0.0.1:9100/health 2>/dev/null || true) |
| 90 | R=$(printf '%s' "$H" | python3 -c 'import sys,json;d=json.load(sys.stdin);print(d.get("running"),d.get("seededApps","?"),d.get("connections","?"))' 2>/dev/null || echo "? ? ?") | |
| 106 | S=$(curl_with_optional_key "$K" -fsS --max-time 10 http://127.0.0.1:9100/status 2>/dev/null || true) | |
| 107 | R=$(HEALTH="$H" STATUS="$S" python3 -c 'import os,json;h=json.loads(os.environ["HEALTH"]);s=json.loads(os.environ["STATUS"]);print(h.get("running"),s.get("seededApps","?"),s.get("connections","?"))' 2>/dev/null || echo "? ? ?") | |
| 91 | 108 | D=$(df -h / | awk 'NR==2{print $5}') |
| 92 | echo "$V|$R|$D" | |
| 109 | U_ENABLED="$(systemctl is-enabled hiverelay-updater.timer 2>/dev/null || true)" | |
| 110 | [ -n "$U_ENABLED" ] || U_ENABLED=missing | |
| 111 | U_ACTIVE="$(systemctl is-active hiverelay-updater.timer 2>/dev/null || true)" | |
| 112 | [ -n "$U_ACTIVE" ] || U_ACTIVE=inactive | |
| 113 | C=$(awk -F= '/^[[:space:]]*CHANNEL[[:space:]]*=/ { sub(/^[^=]*=/,""); sub(/^[[:space:]]*/,""); sub(/[[:space:]]*$/,""); print; exit }' /etc/hiverelay-updater.conf 2>/dev/null || true) | |
| 114 | [ -n "$C" ] || C=? | |
| 115 | echo "$V|$R|$D|$U_ENABLED/$U_ACTIVE|$C" | |
| 93 | 116 | REMOTE_EOF |
| 94 | 117 | |
| 95 | printf '%-11s %-9s %-8s %-6s %-6s %-6s %s\n' RELAY VERSION RUNNING APPS CONNS DISK TARGET | |
| 96 | printf '%-11s %-9s %-8s %-6s %-6s %-6s %s\n' ───── ─────── ─────── ──── ───── ──── ────── | |
| 118 | printf '%-12s %-14s %-8s %-6s %-6s %-6s %-14s %-10s %s\n' RELAY VERSION RUNNING APPS CONNS DISK UPDATER CONFIGURED ASSIGNED | |
| 119 | printf '%-12s %-14s %-8s %-6s %-6s %-6s %-14s %-10s %s\n' ───── ─────── ─────── ──── ───── ──── ─────── ────────── ──────── | |
| 97 | 120 | |
| 98 | 121 | while IFS='|' read -r name host keyspec channel; do |
| 99 | 122 | safe_name="$(clean_field "$name")" |
| 100 | 123 | safe_channel="$(clean_field "$channel")" |
| 101 | 124 | if ! valid_host "$host"; then |
| 102 | printf '%-9s %-9s %-8s %-6s %-6s %-6s %s\n' "$safe_name" BADHOST - - - - "$safe_channel" | |
| 125 | printf '%-12s %-14s %-8s %-6s %-6s %-6s %-14s %-10s %s\n' "$safe_name" BADHOST - - - - - - "$safe_channel" | |
| 103 | 126 | continue |
| 104 | 127 | fi |
| 105 | 128 | if ! valid_channel "$channel"; then |
| 106 | printf '%-9s %-9s %-8s %-6s %-6s %-6s %s\n' "$safe_name" BADCHAN - - - - "$safe_channel" | |
| 129 | printf '%-12s %-14s %-8s %-6s %-6s %-6s %-14s %-10s %s\n' "$safe_name" BADCHAN - - - - - - "$safe_channel" | |
| 107 | 130 | continue |
| 108 | 131 | fi |
| 109 | 132 | if ! valid_key_path "$keyspec"; then |
| 110 | printf '%-9s %-9s %-8s %-6s %-6s %-6s %s\n' "$safe_name" BADKEY - - - - "$safe_channel" | |
| 133 | printf '%-12s %-14s %-8s %-6s %-6s %-6s %-14s %-10s %s\n' "$safe_name" BADKEY - - - - - - "$safe_channel" | |
| 111 | 134 | continue |
| 112 | 135 | fi |
| 113 | 136 | keyarg=(); [ -n "$keyspec" ] && keyarg=(-i "${keyspec/#\~/$HOME}") |
| … | … | @@ -117,11 +140,11 @@ while IFS='|' read -r name host keyspec channel; do |
| 117 | 140 | # on bash <4.4 (macOS ships 3.2) with "unbound variable". |
| 118 | 141 | out="$(ssh ${keyarg[@]+"${keyarg[@]}"} -o StrictHostKeyChecking=accept-new -o ConnectTimeout=10 -o BatchMode=yes "root@$host" bash -s <<<"$REMOTE" 2>/dev/null || true)" |
| 119 | 142 | if [ -z "$out" ]; then |
| 120 | printf '%-9s %-9s %-8s %-6s %-6s %-6s %s\n' "$safe_name" UNREACH - - - - "$safe_target" | |
| 143 | printf '%-12s %-14s %-8s %-6s %-6s %-6s %-14s %-10s %s\n' "$safe_name" UNREACH - - - - - - "$safe_target" | |
| 121 | 144 | continue |
| 122 | 145 | fi |
| 123 | 146 | out="$(printf '%s\n' "$out" | tail -n 1)" |
| 124 | IFS='|' read -r ver rest disk <<<"$out" | |
| 147 | IFS='|' read -r ver rest disk updater configured <<<"$out" | |
| 125 | 148 | read -r run apps conns <<<"${rest:-? ? ?}" |
| 126 | 149 | # Flag disk pressure so a filling box is caught BEFORE it wedges (a full disk |
| 127 | 150 | # hangs the relay → public 502). ! at >=75%, !! at >=90%. |
| … | … | @@ -131,17 +154,25 @@ while IFS='|' read -r name host keyspec channel; do |
| 131 | 154 | if [ "$disk_pct" -ge 90 ] 2>/dev/null; then disk_display="${disk}!!" |
| 132 | 155 | elif [ "$disk_pct" -ge 75 ] 2>/dev/null; then disk_display="${disk}!"; fi |
| 133 | 156 | fi |
| 134 | printf '%-9s %-9s %-8s %-6s %-6s %-6s %s\n' \ | |
| 157 | printf '%-12s %-14s %-8s %-6s %-6s %-6s %-14s %-10s %s\n' \ | |
| 135 | 158 | "$safe_name" \ |
| 136 | 159 | "$(clean_field "${ver:-?}")" \ |
| 137 | 160 | "$(clean_field "${run:-?}")" \ |
| 138 | 161 | "$(clean_field "${apps:-?}")" \ |
| 139 | 162 | "$(clean_field "${conns:-?}")" \ |
| 140 | 163 | "$(clean_field "${disk_display}")" \ |
| 164 | "$(clean_field "${updater:-?}")" \ | |
| 165 | "$(clean_field "${configured:-?}")" \ | |
| 141 | 166 | "$safe_target" |
| 142 | done < <(python3 -c ' | |
| 143 | import json,sys | |
| 167 | done < <(INCLUDE="$INCLUDE" EXCLUDE="$EXCLUDE" python3 -c ' | |
| 168 | import json,os,sys | |
| 169 | include = set(filter(None, os.environ.get("INCLUDE", "").split(","))) | |
| 170 | exclude = set(filter(None, os.environ.get("EXCLUDE", "").split(","))) | |
| 144 | 171 | for r in json.load(open(sys.argv[1]))["relays"]: |
| 172 | if include and r["name"] not in include: | |
| 173 | continue | |
| 174 | if r["name"] in exclude: | |
| 175 | continue | |
| 145 | 176 | host = r["tailnet"] or r["publicIp"] |
| 146 | 177 | key = "" if r.get("sshKey") in (None,"default") else r["sshKey"] |
| 147 | 178 | print("{}|{}|{}|{}".format(r["name"], host, key, r.get("channel", "stable"))) |
| … | … | @@ -21,7 +21,11 @@ Environment=HIVERELAY_LOG_LEVEL=info |
| 21 | 21 | # These make the relay useful to all apps on the substrate. |
| 22 | 22 | Environment=HIVERELAY_ENABLE_SERVICES=1 |
| 23 | 23 | Environment=HIVERELAY_OUTBOXLOG=1 |
| 24 | Environment=HIVERELAY_OUTBOXLOG_JOURNAL=hypercore-outboxes | |
| 25 | Environment=HIVERELAY_OUTBOXLOG_MAX_JOURNAL_STORAGE=512MB | |
| 24 | 26 | Environment=HIVERELAY_NOTIFY=1 |
| 27 | # Real APNs/FCM/WebPush egress: mount an owner-readable JSON descriptor, then | |
| 28 | # set Environment=HIVERELAY_NOTIFY_PUSH_CONFIG=/etc/hiverelay/notify-push.json | |
| 25 | 29 | Environment=HIVERELAY_STORAGE_PROOF=1 |
| 26 | 30 | Environment=HIVERELAY_VRF=1 |
| 27 | 31 | Environment=HIVERELAY_WITNESSLOG=1 |
| … | … | @@ -38,7 +42,9 @@ Environment=HIVERELAY_TOR_SOCKS_PORT=9050 |
| 38 | 42 | Environment=HIVERELAY_TOR_CONTROL_HOST=127.0.0.1 |
| 39 | 43 | Environment=HIVERELAY_TOR_CONTROL_PORT=9051 |
| 40 | 44 | Environment=HIVERELAY_TOR_KEY_FILE=/var/lib/hiverelay/tor-onion-key.blob |
| 41 | Environment=HIVERELAY_TOR_CLIENT_AUTH_DIR=/var/lib/hiverelay/tor-client-auth | |
| 45 | Environment=HIVERELAY_TOR_MIN_DAEMON_VERSION=0.4.9.5 | |
| 46 | # Restricted discovery: set HIVERELAY_TOR_ROSTER_FILE to an owner-readable | |
| 47 | # roster containing one base32 x25519 client public key per line. | |
| 42 | 48 | |
| 43 | 49 | # Restart policy — RestartSec=15 prevents ELOCKED crash loops |
| 44 | 50 | # (random-access-file needs time to release primary-key lock) |
| … | … | @@ -791,8 +791,12 @@ export class HiveRelayClient extends EventEmitter { |
| 791 | 791 | |
| 792 | 792 | // Wire replication for all connections |
| 793 | 793 | this._connectionHandler = (conn, info) => { |
| 794 | if (this.store) this.store.replicate(conn) | |
| 794 | // Attach the peer error sink before Corestore/Protomux can schedule any | |
| 795 | // stream work. NoiseSecretStream may emit a late ECONNRESET while a | |
| 796 | // swarm is being destroyed; attaching only after protocol setup leaves | |
| 797 | // a race that can become an unhandled EventEmitter error. | |
| 795 | 798 | this._onConnection(conn, info) |
| 799 | if (this.store) this.store.replicate(conn) | |
| 796 | 800 | } |
| 797 | 801 | this.swarm.on('connection', this._connectionHandler) |
| 798 | 802 | |
| … | … | @@ -4588,10 +4592,21 @@ export class HiveRelayClient extends EventEmitter { |
| 4588 | 4592 | // ─── Internal ──────────────────────────────────────────────────── |
| 4589 | 4593 | |
| 4590 | 4594 | _onConnection (conn, info) { |
| 4591 | const pubkeyHex = info.publicKey | |
| 4595 | const pubkeyHex = info && info.publicKey | |
| 4592 | 4596 | ? b4a.toString(info.publicKey, 'hex') |
| 4593 | 4597 | : null |
| 4594 | 4598 | |
| 4599 | // This listener is deliberately connection-scoped and installed before | |
| 4600 | // any early return or protocol setup. It therefore also covers inbound | |
| 4601 | // streams whose identity metadata is incomplete and late shutdown errors | |
| 4602 | // after channel state has already been torn down. | |
| 4603 | conn.on('error', () => { | |
| 4604 | if (!pubkeyHex) return | |
| 4605 | this.relays.delete(pubkeyHex) | |
| 4606 | const errorScores = this._relayScores.get(pubkeyHex) | |
| 4607 | if (errorScores) errorScores.failures++ | |
| 4608 | }) | |
| 4609 | ||
| 4595 | 4610 | if (!pubkeyHex) return |
| 4596 | 4611 | |
| 4597 | 4612 | const mux = Protomux.from(conn) |
| … | … | @@ -4798,12 +4813,6 @@ export class HiveRelayClient extends EventEmitter { |
| 4798 | 4813 | } |
| 4799 | 4814 | }) |
| 4800 | 4815 | |
| 4801 | conn.on('error', () => { | |
| 4802 | this.relays.delete(pubkeyHex) | |
| 4803 | const errorScores = this._relayScores.get(pubkeyHex) | |
| 4804 | if (errorScores) errorScores.failures++ | |
| 4805 | }) | |
| 4806 | ||
| 4807 | 4816 | this._resetReconnect() |
| 4808 | 4817 | this.emit('relay-connected', { pubkey: pubkeyHex }) |
| 4809 | 4818 | } |
| … | … | @@ -27,6 +27,7 @@ import { |
| 27 | 27 | STORAGE_DIR, |
| 28 | 28 | deriveTokenFromSeed, |
| 29 | 29 | applyOutboxlogNamespaceEnv, |
| 30 | applyRuntimeEnvDefaults, | |
| 30 | 31 | getStorageCapProvenance, |
| 31 | 32 | markStorageCapExplicit, |
| 32 | 33 | resolveStorageCap |
| … | … | @@ -489,19 +490,15 @@ async function start () { |
| 489 | 490 | hasPersistedOutboxlogNamespace() |
| 490 | 491 | ) |
| 491 | 492 | |
| 492 | // Atomic-writer deployment controls. Legacy create/append stay enabled by | |
| 493 | // default for wire compatibility; production Peerit relays set this to 0 so | |
| 494 | // replayable signed rows can only enter through CAS-protected /sync/commit. | |
| 495 | const legacyWrites = process.env.HIVERELAY_OUTBOXLOG_LEGACY_WRITES | |
| 496 | if (legacyWrites === '0' || legacyWrites === 'false' || legacyWrites === '1' || legacyWrites === 'true') { | |
| 497 | if (!cliOverrides.outboxlog || typeof cliOverrides.outboxlog !== 'object' || Array.isArray(cliOverrides.outboxlog)) cliOverrides.outboxlog = {} | |
| 498 | cliOverrides.outboxlog.legacyWrites = legacyWrites === '1' || legacyWrites === 'true' | |
| 499 | } | |
| 500 | // A durable commit is acknowledged only with a synchronous journal. This | |
| 501 | // container-friendly path selects the fsynced JSONL journal implementation. | |
| 502 | if (process.env.HIVERELAY_OUTBOXLOG_JOURNAL_PATH) { | |
| 503 | if (!cliOverrides.outboxlog || typeof cliOverrides.outboxlog !== 'object' || Array.isArray(cliOverrides.outboxlog)) cliOverrides.outboxlog = {} | |
| 504 | cliOverrides.outboxlog.journalPath = process.env.HIVERELAY_OUTBOXLOG_JOURNAL_PATH | |
| 493 | // Docker/systemd/appliance environment is a first-boot default only. CLI, | |
| 494 | // config.json and the storage-local services.json remain authoritative. | |
| 495 | // Also loads an optional push-provider descriptor from a file path so real | |
| 496 | // credentials never need to appear in compose/systemd source or argv. | |
| 497 | try { | |
| 498 | applyRuntimeEnvDefaults(cliOverrides, process.env) | |
| 499 | } catch (err) { | |
| 500 | console.error(err && err.message ? err.message : 'Invalid HiveRelay runtime environment') | |
| 501 | process.exit(1) | |
| 505 | 502 | } |
| 506 | 503 | |
| 507 | 504 | const config = loadConfig(cliOverrides) |
| … | … | @@ -1656,6 +1653,15 @@ Environment: |
| 1656 | 1653 | HIVERELAY_ACCEPT_MODE Catalog mode: open, review, allowlist, or closed |
| 1657 | 1654 | HIVERELAY_STORAGE Storage path used when --storage is absent |
| 1658 | 1655 | HIVERELAY_MAX_STORAGE Storage cap until maxStorageBytes is persisted |
| 1656 | HIVERELAY_ENABLE_SERVICES First-boot service default: 1/0 or true/false | |
| 1657 | HIVERELAY_{SERVICE}=1 Select VRF, NOTIFY, OUTBOXLOG, STORAGE_PROOF, | |
| 1658 | WITNESSLOG, REPAIRTICKET, or SHARD_STORE | |
| 1659 | HIVERELAY_OUTBOXLOG_JOURNAL Use hypercore or hypercore-outboxes persistence | |
| 1660 | HIVERELAY_OUTBOXLOG_MAX_JOURNAL_STORAGE | |
| 1661 | Required finite journal bound (for example 512MB) | |
| 1662 | HIVERELAY_NOTIFY_PUSH_CONFIG Owner-readable APNs/FCM/WebPush descriptor path | |
| 1663 | HIVERELAY_TOR First-boot Tor default: 1/0 or true/false | |
| 1664 | HIVERELAY_TOR_* SOCKS/control/key/min-version/roster defaults | |
| 1659 | 1665 | |
| 1660 | 1666 | Examples: |
| 1661 | 1667 | npx p2p-hiverelay setup # Interactive setup wizard |
| … | … | @@ -339,7 +339,10 @@ export default { |
| 339 | 339 | // PoW defense is daemon-wide SETCONF and requires a pow-enabled tor build |
| 340 | 340 | // plus a HiddenServiceDir-configured service (M0 finding, 2026-07-17). |
| 341 | 341 | pow: { enabled: false, queueRate: 250, queueBurst: 1000 }, |
| 342 | // Health-gated advertisement: descriptor uploads + optional SOCKS self-probe. | |
| 342 | // Health-gated advertisement: null auto-selects an exposed vport for a | |
| 343 | // SOCKS self-probe; false is an explicit descriptor-only compatibility | |
| 344 | // opt-out. Restricted discovery is never advertised without a successful | |
| 345 | // unauthenticated negative probe. | |
| 343 | 346 | health: { probeIntervalMs: 900000, probeFailLimit: 3, minDescriptorUploads: 2, probeVport: null } |
| 344 | 347 | }, |
| 345 | 348 |
| … | … | @@ -81,20 +81,173 @@ export function applyOutboxlogNamespaceEnv (cliOverrides = {}, rawEnvValue, hasP |
| 81 | 81 | return cliOverrides |
| 82 | 82 | } |
| 83 | 83 | |
| 84 | const PACKAGED_SERVICE_ENV = Object.freeze({ | |
| 85 | HIVERELAY_VRF: 'vrf', | |
| 86 | HIVERELAY_NOTIFY: 'notify', | |
| 87 | HIVERELAY_OUTBOXLOG: 'outboxlog', | |
| 88 | HIVERELAY_STORAGE_PROOF: 'storage-proof', | |
| 89 | HIVERELAY_WITNESSLOG: 'witnesslog', | |
| 90 | HIVERELAY_REPAIRTICKET: 'repairticket', | |
| 91 | HIVERELAY_SHARD_STORE: 'shard-store' | |
| 92 | }) | |
| 93 | ||
| 84 | 94 | /** |
| 85 | * Load config: defaults < config.json < CLI overrides | |
| 95 | * Translate the declarative environment shipped by Docker, systemd and | |
| 96 | * appliance packages into Node-runtime config defaults. | |
| 97 | * | |
| 98 | * Precedence is deliberately conservative: explicit CLI values and matching | |
| 99 | * config.json fields win. The storage-local services.json remains the final | |
| 100 | * services authority because RelayNode loads it immediately before plugins. | |
| 101 | * This makes package env useful on first boot without trapping an operator in | |
| 102 | * a permanent environment override. | |
| 86 | 103 | */ |
| 87 | export function loadConfig (cliOverrides = {}) { | |
| 88 | let fileConfig = {} | |
| 104 | export function applyRuntimeEnvDefaults ( | |
| 105 | cliOverrides = {}, | |
| 106 | env = process.env, | |
| 107 | persistedConfig = readPersistedConfig(), | |
| 108 | { readTextFile = (path) => readFileSync(path, 'utf8') } = {} | |
| 109 | ) { | |
| 110 | const persisted = isPlainObject(persistedConfig) ? persistedConfig : {} | |
| 111 | ||
| 112 | const globalServices = parseOptionalBoolean(env.HIVERELAY_ENABLE_SERVICES, 'HIVERELAY_ENABLE_SERVICES') | |
| 113 | const persistedPlugins = hasOwn(persisted, 'plugins') | |
| 114 | const cliPlugins = hasOwn(cliOverrides, 'plugins') | |
| 115 | const selectedPlugins = [] | |
| 116 | for (const [name, plugin] of Object.entries(PACKAGED_SERVICE_ENV)) { | |
| 117 | if (parseOptionalBoolean(env[name], name) === true) selectedPlugins.push(plugin) | |
| 118 | } | |
| 119 | if (!persistedPlugins && !cliPlugins && selectedPlugins.length > 0) { | |
| 120 | cliOverrides.plugins = selectedPlugins | |
| 121 | } | |
| 122 | if (!hasOwn(persisted, 'enableServices') && !hasOwn(cliOverrides, 'enableServices')) { | |
| 123 | if (globalServices !== undefined) cliOverrides.enableServices = globalServices | |
| 124 | else if (selectedPlugins.length > 0) cliOverrides.enableServices = true | |
| 125 | } | |
| 126 | ||
| 127 | const torEnabled = parseOptionalBoolean(env.HIVERELAY_TOR, 'HIVERELAY_TOR') | |
| 128 | if (torEnabled !== undefined && | |
| 129 | !hasNestedOwn(persisted, 'transports', 'tor') && | |
| 130 | !hasNestedOwn(cliOverrides, 'transports', 'tor')) { | |
| 131 | nestedObject(cliOverrides, 'transports').tor = torEnabled | |
| 132 | } | |
| 133 | applyStringEnvDefault(cliOverrides, persisted, env, 'HIVERELAY_TOR_SOCKS_HOST', 'tor', 'socksHost') | |
| 134 | applyPortEnvDefault(cliOverrides, persisted, env, 'HIVERELAY_TOR_SOCKS_PORT', 'tor', 'socksPort') | |
| 135 | applyStringEnvDefault(cliOverrides, persisted, env, 'HIVERELAY_TOR_CONTROL_HOST', 'tor', 'controlHost') | |
| 136 | applyPortEnvDefault(cliOverrides, persisted, env, 'HIVERELAY_TOR_CONTROL_PORT', 'tor', 'controlPort') | |
| 137 | applyStringEnvDefault(cliOverrides, persisted, env, 'HIVERELAY_TOR_COOKIE_AUTH_FILE', 'tor', 'cookieAuthFile') | |
| 138 | applyStringEnvDefault(cliOverrides, persisted, env, 'HIVERELAY_TOR_KEY_FILE', 'tor', 'keyFile') | |
| 139 | applyStringEnvDefault(cliOverrides, persisted, env, 'HIVERELAY_TOR_MIN_DAEMON_VERSION', 'tor', 'minDaemonVersion') | |
| 140 | applyStringEnvDefault(cliOverrides, persisted, env, 'HIVERELAY_TOR_ROSTER_FILE', 'tor', 'rosterFile') | |
| 141 | ||
| 142 | applyStringEnvDefault(cliOverrides, persisted, env, 'HIVERELAY_OUTBOXLOG_JOURNAL', 'outboxlog', 'journal') | |
| 143 | applyStringEnvDefault(cliOverrides, persisted, env, 'HIVERELAY_OUTBOXLOG_JOURNAL_PATH', 'outboxlog', 'journalPath') | |
| 144 | applyBooleanEnvDefault(cliOverrides, persisted, env, 'HIVERELAY_OUTBOXLOG_LEGACY_WRITES', 'outboxlog', 'legacyWrites') | |
| 145 | applyBytesEnvDefault(cliOverrides, persisted, env, 'HIVERELAY_OUTBOXLOG_MAX_JOURNAL_STORAGE', 'outboxlog', 'maxJournalStorageBytes') | |
| 89 | 146 | |
| 90 | if (existsSync(CONFIG_PATH)) { | |
| 147 | const notifyPushPath = stringEnv(env.HIVERELAY_NOTIFY_PUSH_CONFIG) | |
| 148 | if (notifyPushPath && | |
| 149 | !hasNestedOwn(persisted, 'notify', 'push') && | |
| 150 | !hasNestedOwn(cliOverrides, 'notify', 'push')) { | |
| 151 | let descriptor | |
| 91 | 152 | try { |
| 92 | fileConfig = JSON.parse(readFileSync(CONFIG_PATH, 'utf8')) | |
| 93 | } catch { | |
| 94 | // Ignore malformed config file | |
| 153 | descriptor = JSON.parse(readTextFile(notifyPushPath)) | |
| 154 | } catch (err) { | |
| 155 | throw new Error(`Invalid HIVERELAY_NOTIFY_PUSH_CONFIG: ${err && err.message ? err.message : 'unreadable JSON'}`) | |
| 156 | } | |
| 157 | if (!isPlainObject(descriptor)) { | |
| 158 | throw new Error('Invalid HIVERELAY_NOTIFY_PUSH_CONFIG: expected a JSON object') | |
| 95 | 159 | } |
| 160 | nestedObject(cliOverrides, 'notify').push = descriptor | |
| 96 | 161 | } |
| 97 | 162 | |
| 163 | return cliOverrides | |
| 164 | } | |
| 165 | ||
| 166 | function readPersistedConfig () { | |
| 167 | if (!existsSync(CONFIG_PATH)) return {} | |
| 168 | try { | |
| 169 | const parsed = JSON.parse(readFileSync(CONFIG_PATH, 'utf8')) | |
| 170 | return isPlainObject(parsed) ? parsed : {} | |
| 171 | } catch { | |
| 172 | return {} | |
| 173 | } | |
| 174 | } | |
| 175 | ||
| 176 | function parseOptionalBoolean (value, name) { | |
| 177 | if (value === undefined || value === null || value === '') return undefined | |
| 178 | const normalized = String(value).trim().toLowerCase() | |
| 179 | if (normalized === '1' || normalized === 'true') return true | |
| 180 | if (normalized === '0' || normalized === 'false') return false | |
| 181 | throw new Error(`Invalid ${name}: expected 1, 0, true, or false`) | |
| 182 | } | |
| 183 | ||
| 184 | function parsePositiveBytes (value, name) { | |
| 185 | const units = { B: 1, KB: 1024, MB: 1024 ** 2, GB: 1024 ** 3, TB: 1024 ** 4 } | |
| 186 | const match = String(value || '').trim().match(/^(\d+(?:\.\d+)?)\s*(TB|GB|MB|KB|B)?$/i) | |
| 187 | const bytes = match ? Math.floor(Number(match[1]) * units[(match[2] || 'B').toUpperCase()]) : NaN | |
| 188 | if (!Number.isSafeInteger(bytes) || bytes <= 0) { | |
| 189 | throw new Error(`Invalid ${name}: expected a positive size such as 512MB or 1GB`) | |
| 190 | } | |
| 191 | return bytes | |
| 192 | } | |
| 193 | ||
| 194 | function isPlainObject (value) { | |
| 195 | return !!value && typeof value === 'object' && !Array.isArray(value) | |
| 196 | } | |
| 197 | ||
| 198 | function hasOwn (value, key) { | |
| 199 | return isPlainObject(value) && Object.prototype.hasOwnProperty.call(value, key) | |
| 200 | } | |
| 201 | ||
| 202 | function hasNestedOwn (value, parent, key) { | |
| 203 | return hasOwn(value, parent) && hasOwn(value[parent], key) | |
| 204 | } | |
| 205 | ||
| 206 | function nestedObject (value, key) { | |
| 207 | if (!isPlainObject(value[key])) value[key] = {} | |
| 208 | return value[key] | |
| 209 | } | |
| 210 | ||
| 211 | function stringEnv (value) { | |
| 212 | if (value === undefined || value === null) return null | |
| 213 | const normalized = String(value).trim() | |
| 214 | return normalized || null | |
| 215 | } | |
| 216 | ||
| 217 | function applyStringEnvDefault (cli, persisted, env, envName, parent, key) { | |
| 218 | const value = stringEnv(env[envName]) | |
| 219 | if (!value || hasNestedOwn(persisted, parent, key) || hasNestedOwn(cli, parent, key)) return | |
| 220 | nestedObject(cli, parent)[key] = value | |
| 221 | } | |
| 222 | ||
| 223 | function applyPortEnvDefault (cli, persisted, env, envName, parent, key) { | |
| 224 | const value = stringEnv(env[envName]) | |
| 225 | if (!value || hasNestedOwn(persisted, parent, key) || hasNestedOwn(cli, parent, key)) return | |
| 226 | const port = Number(value) | |
| 227 | if (!Number.isInteger(port) || port < 1 || port > 65535) { | |
| 228 | throw new Error(`Invalid ${envName}: expected an integer from 1 to 65535`) | |
| 229 | } | |
| 230 | nestedObject(cli, parent)[key] = port | |
| 231 | } | |
| 232 | ||
| 233 | function applyBooleanEnvDefault (cli, persisted, env, envName, parent, key) { | |
| 234 | const value = parseOptionalBoolean(env[envName], envName) | |
| 235 | if (value === undefined || hasNestedOwn(persisted, parent, key) || hasNestedOwn(cli, parent, key)) return | |
| 236 | nestedObject(cli, parent)[key] = value | |
| 237 | } | |
| 238 | ||
| 239 | function applyBytesEnvDefault (cli, persisted, env, envName, parent, key) { | |
| 240 | const value = stringEnv(env[envName]) | |
| 241 | if (!value || hasNestedOwn(persisted, parent, key) || hasNestedOwn(cli, parent, key)) return | |
| 242 | nestedObject(cli, parent)[key] = parsePositiveBytes(value, envName) | |
| 243 | } | |
| 244 | ||
| 245 | /** | |
| 246 | * Load config: defaults < config.json < CLI overrides | |
| 247 | */ | |
| 248 | export function loadConfig (cliOverrides = {}) { | |
| 249 | const fileConfig = readPersistedConfig() | |
| 250 | ||
| 98 | 251 | // Deep merge: defaults < file < CLI (preserves nested object keys) |
| 99 | 252 | const config = deepMerge(deepMerge(defaults, fileConfig), cliOverrides) |
| 100 | 253 |
| … | … | @@ -47,10 +47,17 @@ function buildPrivacyTransports ({ relay, config, custodyEnabled, advertisedAtMs |
| 47 | 47 | if (tt.health !== 'ready') return [] |
| 48 | 48 | |
| 49 | 49 | const torCfg = (config && config.tor) || {} |
| 50 | const notBefore = tt.startedAtMs || advertisedAtMs | |
| 51 | 50 | const restrictedDiscovery = typeof tt.isRestrictedDiscoveryActive === 'function' |
| 52 | 51 | ? tt.isRestrictedDiscoveryActive() |
| 53 | 52 | : !!((tt.clientAuthKeys && tt.clientAuthKeys.length > 0) || tt.rosterFile) |
| 53 | // A private onion endpoint is only safe to advertise after a live SOCKS | |
| 54 | // probe made *without* a client credential has been rejected by Tor. The | |
| 55 | // transport can still report descriptor-only readiness when an operator | |
| 56 | // explicitly disables probing, but that weaker compatibility mode must | |
| 57 | // never create a signed restricted-discovery claim. | |
| 58 | if (restrictedDiscovery && tt.negativeProbe !== true) return [] | |
| 59 | ||
| 60 | const notBefore = tt.startedAtMs || advertisedAtMs | |
| 54 | 61 | const authMode = restrictedDiscovery ? 'client-auth-v3' : 'none' |
| 55 | 62 | const vports = typeof tt._effectiveVports === 'function' ? tt._effectiveVports() : [] |
| 56 | 63 | const hasPeerListener = !!tt.peerListener |
| … | … | @@ -94,6 +101,7 @@ function buildPrivacyTransports ({ relay, config, custodyEnabled, advertisedAtMs |
| 94 | 101 | vports: vports.map((v) => v.vport), |
| 95 | 102 | vportRoles, |
| 96 | 103 | auth: { mode: authMode, enrollment: authMode === 'client-auth-v3' ? ['pairing-channel'] : [] }, |
| 104 | ...(restrictedDiscovery ? { negativeProbe: true } : {}), | |
| 97 | 105 | pow: { enabled: !!(tt.pow && tt.pow.enabled) }, |
| 98 | 106 | exposure: torCfg.exposure || 'dual', |
| 99 | 107 | relayLocation: 'hidden-onion', |
| … | … | @@ -542,12 +550,21 @@ function buildServicesProtocolProfile (relay, servicesEnabled) { |
| 542 | 550 | // the doc cannot claim a vocabulary or quota the service doesn't enforce. |
| 543 | 551 | // Literals remain only as the fallback for a service without limits(). |
| 544 | 552 | const notifyLimits = serviceLimits(services.get('notify')) |
| 553 | const availableWatchSources = Array.isArray(notifyLimits.watchSources) ? notifyLimits.watchSources : [] | |
| 554 | const hasExactLaneWake = availableWatchSources.includes('notify-outbox-lane') | |
| 555 | const watchSources = hasExactLaneWake | |
| 556 | ? availableWatchSources.filter(source => source !== 'notify-feed-head') | |
| 557 | : availableWatchSources | |
| 558 | const deprecatedWatchSources = hasExactLaneWake && availableWatchSources.includes('notify-feed-head') | |
| 559 | ? ['notify-feed-head'] | |
| 560 | : [] | |
| 545 | 561 | profile.notify = { |
| 546 | 562 | version: serviceVersion(services.get('notify')) || '0.1.0', |
| 547 | 563 | providers: notifyLimits.providers || ['runtime', 'apns', 'fcm', 'webpush'], |
| 548 | 564 | credential_modes: notifyLimits.credentialModes || ['runtime-owned', 'app-owned'], |
| 549 | 565 | modes: notifyLimits.modes || ['direct', 'watch', 'presence-fallback'], |
| 550 | watch_sources: notifyLimits.watchSources || [], | |
| 566 | watch_sources: watchSources, | |
| 567 | ...(deprecatedWatchSources.length ? { deprecated_watch_sources: deprecatedWatchSources } : {}), | |
| 551 | 568 | payload: { |
| 552 | 569 | max_ciphertext_bytes: notifyLimits.maxCiphertextBytes || 3072, |
| 553 | 570 | plaintext_allowed: false, |
| … | … | @@ -12,6 +12,7 @@ export const BUILTIN_SERVICE_PLUGINS = Object.freeze([ |
| 12 | 12 | 'witnesslog', |
| 13 | 13 | 'repairticket', |
| 14 | 14 | 'notify', |
| 15 | 'storage-proof', | |
| 15 | 16 | 'shard-store' |
| 16 | 17 | ]) |
| 17 | 18 |
| … | … | @@ -1854,7 +1854,7 @@ export class RelayAPI extends EventEmitter { |
| 1854 | 1854 | body, |
| 1855 | 1855 | config: this.node.config, |
| 1856 | 1856 | registry: this.node.serviceRegistry, |
| 1857 | persistConfig: () => this._persistConfig(), | |
| 1857 | persistConfig: () => this._persistServicesConfig(), | |
| 1858 | 1858 | serviceConfigPayload: () => serviceConfigPayload(this.node.config, this.node.serviceRegistry) |
| 1859 | 1859 | }) |
| 1860 | 1860 | if (!result.ok && result.kind === 'config-persist') return this._persistFailureResponse(res, result) |
| … | … | @@ -1869,7 +1869,7 @@ export class RelayAPI extends EventEmitter { |
| 1869 | 1869 | config: this.node.config, |
| 1870 | 1870 | node: this.node, |
| 1871 | 1871 | store: this.node.store, |
| 1872 | persistConfig: () => this._persistConfig(), | |
| 1872 | persistConfig: () => this._persistServicesConfig(), | |
| 1873 | 1873 | serviceConfigPayload: () => serviceConfigPayload(this.node.config, this.node.serviceRegistry) |
| 1874 | 1874 | }) |
| 1875 | 1875 | if (!result.ok && result.kind === 'config-persist') return this._persistFailureResponse(res, result) |
| … | … | @@ -2202,6 +2202,14 @@ export class RelayAPI extends EventEmitter { |
| 2202 | 2202 | } |
| 2203 | 2203 | } |
| 2204 | 2204 | |
| 2205 | async _persistServicesConfig () { | |
| 2206 | if (typeof this.node.setServicesConfig !== 'function') return this._persistConfig() | |
| 2207 | return this.node.setServicesConfig({ | |
| 2208 | enabled: this.node.config.enableServices === true, | |
| 2209 | plugins: Array.isArray(this.node.config.plugins) ? this.node.config.plugins : [] | |
| 2210 | }) | |
| 2211 | } | |
| 2212 | ||
| 2205 | 2213 | async _persistConfigOrRespond (res, rollback = null) { |
| 2206 | 2214 | try { |
| 2207 | 2215 | await this._persistConfig() |
| … | … | @@ -4,7 +4,7 @@ import sodium from 'sodium-universal' |
| 4 | 4 | import { EventEmitter } from 'events' |
| 5 | 5 | import { readFile } from 'fs/promises' |
| 6 | 6 | import { join } from 'path' |
| 7 | import { updateWithTimeout, downloadWithTimeout, getDriveSize } from './cancellable-drive-update.js' | |
| 7 | import { updateWithTimeout, getDriveSize } from './cancellable-drive-update.js' | |
| 8 | 8 | import { isAbortError } from './lifecycle-scope.js' |
| 9 | 9 | import { verifyShareBundleForRelay } from '../pvss.js' |
| 10 | 10 | import { STORAGE_SHARE_BUNDLE_MAX_BYTES } from '../../config/storage-admission-authority.js' |
| … | … | @@ -17,6 +17,10 @@ import { |
| 17 | 17 | normalizeStorageClass |
| 18 | 18 | } from '../constants.js' |
| 19 | 19 | |
| 20 | // Capability marker used only by durable registry recovery. A Symbol keeps | |
| 21 | // external callers from forging the serve-only path through JSON/options. | |
| 22 | const STORAGE_RECOVERY_INGRESS = Symbol('storage-recovery-ingress') | |
| 23 | ||
| 20 | 24 | function seedAdmissionTimeoutError (timeoutMs) { |
| 21 | 25 | const err = new Error(`Seed admission timed out after ${timeoutMs}ms`) |
| 22 | 26 | err.code = 'STORAGE_SEED_ADMISSION_TIMEOUT' |
| … | … | @@ -30,6 +34,24 @@ function seedAdmissionAbortError () { |
| 30 | 34 | return err |
| 31 | 35 | } |
| 32 | 36 | |
| 37 | function stableCoreProofState (core, attempts = 4) { | |
| 38 | if (!core) return null | |
| 39 | for (let attempt = 0; attempt < attempts; attempt++) { | |
| 40 | const forkBefore = Number(core.fork ?? 0) | |
| 41 | const lengthBefore = Number(core.length) | |
| 42 | const byteLengthBefore = Number(core.byteLength) | |
| 43 | const lengthAfter = Number(core.length) | |
| 44 | const byteLengthAfter = Number(core.byteLength) | |
| 45 | const forkAfter = Number(core.fork ?? 0) | |
| 46 | if (Number.isSafeInteger(forkBefore) && forkBefore >= 0 && forkBefore === forkAfter && | |
| 47 | Number.isSafeInteger(lengthBefore) && lengthBefore >= 0 && lengthBefore === lengthAfter && | |
| 48 | Number.isSafeInteger(byteLengthBefore) && byteLengthBefore >= 0 && byteLengthBefore === byteLengthAfter) { | |
| 49 | return { fork: forkAfter, length: lengthAfter, byteLength: byteLengthAfter } | |
| 50 | } | |
| 51 | } | |
| 52 | return null | |
| 53 | } | |
| 54 | ||
| 33 | 55 | /** |
| 34 | 56 | * AppLifecycle — owns seeding, unseeding, and manifest indexing for a RelayNode. |
| 35 | 57 | * |
| … | … | @@ -69,6 +91,16 @@ export class AppLifecycle extends EventEmitter { |
| 69 | 91 | return this.node.appRegistry.apps |
| 70 | 92 | } |
| 71 | 93 | |
| 94 | _joinDriveDiscovery (appKeyHex, discoveryKey, handles = null) { | |
| 95 | const handle = this.node.swarm.join(discoveryKey, { server: true, client: true }) | |
| 96 | const entry = this.node.appRegistry.get(appKeyHex) | |
| 97 | const owned = entry && entry.discoveryHandles instanceof Set | |
| 98 | ? entry.discoveryHandles | |
| 99 | : (handles instanceof Set ? handles : null) | |
| 100 | if (owned) owned.add(handle) | |
| 101 | return handle | |
| 102 | } | |
| 103 | ||
| 72 | 104 | /** |
| 73 | 105 | * Serialize a seed-admission closure behind every earlier one. |
| 74 | 106 | * |
| … | … | @@ -189,6 +221,7 @@ export class AppLifecycle extends EventEmitter { |
| 189 | 221 | if (!entry || !entry.appKey) continue |
| 190 | 222 | try { |
| 191 | 223 | await this.seedApp(entry.appKey, { |
| 224 | [STORAGE_RECOVERY_INGRESS]: true, | |
| 192 | 225 | appId: entry.appId || null, |
| 193 | 226 | type: entry.type || 'app', |
| 194 | 227 | parentKey: entry.parentKey || null, |
| … | … | @@ -266,6 +299,7 @@ export class AppLifecycle extends EventEmitter { |
| 266 | 299 | if (!appKey) continue |
| 267 | 300 | try { |
| 268 | 301 | await this.seedApp(appKey, { |
| 302 | [STORAGE_RECOVERY_INGRESS]: true, | |
| 269 | 303 | appId: entry.appId || null, |
| 270 | 304 | type: entry.type || 'app', |
| 271 | 305 | parentKey: entry.parentKey || null, |
| … | … | @@ -291,7 +325,7 @@ export class AppLifecycle extends EventEmitter { |
| 291 | 325 | // Checked before every other gate (including "seeding not enabled") so |
| 292 | 326 | // the caller gets the specific, retryable cause rather than a generic |
| 293 | 327 | // refusal that hides the fact that startup simply hasn't finished. |
| 294 | if (node._storageIngressReady === false) { | |
| 328 | if (node._storageIngressReady === false && opts[STORAGE_RECOVERY_INGRESS] !== true) { | |
| 295 | 329 | const err = new Error('storage recovery inventory has not sealed') |
| 296 | 330 | err.code = 'STORAGE_RECOVERY_INVENTORY_PENDING' |
| 297 | 331 | throw err |
| … | … | @@ -439,7 +473,8 @@ export class AppLifecycle extends EventEmitter { |
| 439 | 473 | // reservation first — each of those is a resource we would then have to |
| 440 | 474 | // unwind, and the reservation error (STORAGE_ADMISSION_BLOCKED) would |
| 441 | 475 | // mask the real cause. |
| 442 | if (node.appRegistry && typeof node.appRegistry.assertDurableWritesAvailable === 'function') { | |
| 476 | if (normalizedOpts[STORAGE_RECOVERY_INGRESS] !== true && | |
| 477 | node.appRegistry && typeof node.appRegistry.assertDurableWritesAvailable === 'function') { | |
| 443 | 478 | node.appRegistry.assertDurableWritesAvailable() |
| 444 | 479 | } |
| 445 | 480 | |
| … | … | @@ -468,6 +503,7 @@ export class AppLifecycle extends EventEmitter { |
| 468 | 503 | err.repin = reconcile |
| 469 | 504 | throw err |
| 470 | 505 | } |
| 506 | this._recordCustodyReceiptOnRepin(appKeyHex, existing, normalizedOpts) | |
| 471 | 507 | return { discoveryKey: dkHex, alreadySeeded: true } |
| 472 | 508 | } |
| 473 | 509 | // else: placeholder entry from load() — fall through to seed properly. |
| … | … | @@ -488,6 +524,11 @@ export class AppLifecycle extends EventEmitter { |
| 488 | 524 | |
| 489 | 525 | async _seedAppInner (appKeyHex, opts, contentType, parentKey, mountPath, privacyTier) { |
| 490 | 526 | const node = this.node |
| 527 | const recoveringEntry = this.seededApps.get(appKeyHex) || null | |
| 528 | const recoveringExisting = recoveringEntry !== null | |
| 529 | const readOnlyRecovery = recoveringExisting && | |
| 530 | opts[STORAGE_RECOVERY_INGRESS] === true && | |
| 531 | node.appRegistry.physicalReadOnly === true | |
| 491 | 532 | |
| 492 | 533 | // Re-check after acquiring mutex — another call may have seeded it. |
| 493 | 534 | // Same null-discoveryKey guard + v0.8.12 opts reconcile as the |
| … | … | @@ -505,6 +546,7 @@ export class AppLifecycle extends EventEmitter { |
| 505 | 546 | err.repin = reconcile |
| 506 | 547 | throw err |
| 507 | 548 | } |
| 549 | this._recordCustodyReceiptOnRepin(appKeyHex, existing, opts) | |
| 508 | 550 | return { discoveryKey: dkHex, alreadySeeded: true } |
| 509 | 551 | } |
| 510 | 552 | // else: placeholder entry from load() — fall through. |
| … | … | @@ -516,8 +558,13 @@ export class AppLifecycle extends EventEmitter { |
| 516 | 558 | const admissionCap = Number.isFinite(opts.maxStorage) && opts.maxStorage > 0 |
| 517 | 559 | ? Math.floor(opts.maxStorage) |
| 518 | 560 | : null |
| 561 | if (!recoveringExisting && admissionCap === null) { | |
| 562 | const err = new Error('positive safe-integer maxStorage is required') | |
| 563 | err.code = 'STORAGE_BOUND_REQUIRED' | |
| 564 | throw err | |
| 565 | } | |
| 519 | 566 | let admissionReservation = null |
| 520 | if (node.storageAdmission && typeof node.storageAdmission.reserve === 'function' && admissionCap !== null) { | |
| 567 | if (!readOnlyRecovery && node.storageAdmission && typeof node.storageAdmission.reserve === 'function' && admissionCap !== null) { | |
| 521 | 568 | const reservation = node.storageAdmission.reserve(`drive:${appKeyHex}`, admissionCap) |
| 522 | 569 | if (!reservation || reservation.allowed !== true) { |
| 523 | 570 | const err = new Error('Storage admission blocked: ' + (reservation && reservation.reason)) |
| … | … | @@ -537,6 +584,7 @@ export class AppLifecycle extends EventEmitter { |
| 537 | 584 | } |
| 538 | 585 | |
| 539 | 586 | let drive = null |
| 587 | const discoveryHandles = new Set() | |
| 540 | 588 | try { |
| 541 | 589 | // Evict oldest app if storage capacity would be exceeded |
| 542 | 590 | if (node.config.enableEviction !== false && node.seeder && node.seeder.totalBytesStored >= node.config.maxStorageBytes && this.seededApps.size > 0) { |
| … | … | @@ -600,7 +648,7 @@ export class AppLifecycle extends EventEmitter { |
| 600 | 648 | try { |
| 601 | 649 | await Promise.race([ |
| 602 | 650 | drive.ready(), |
| 603 | new Promise((_, reject) => { | |
| 651 | new Promise((_resolve, reject) => { | |
| 604 | 652 | readyTimer = setTimeout( |
| 605 | 653 | () => reject(new Error('DRIVE_READY_TIMEOUT after ' + READY_TIMEOUT_MS + 'ms')), |
| 606 | 654 | READY_TIMEOUT_MS |
| … | … | @@ -616,16 +664,32 @@ export class AppLifecycle extends EventEmitter { |
| 616 | 664 | |
| 617 | 665 | // Signal that we're looking for peers for this drive's cores |
| 618 | 666 | const done = drive.findingPeers ? drive.findingPeers() : null |
| 619 | node.swarm.join(discoveryKey, { server: true, client: true }) | |
| 620 | node.swarm.flush().then(() => { if (done) done() }).catch(() => { if (done) done() }) | |
| 621 | ||
| 622 | // Eagerly replicate drive content. Extracted to a method in v0.8.12 | |
| 623 | // so the alreadySeeded re-pin path can call it too — see | |
| 624 | // _reconcileSeedOptsOnRepin. | |
| 625 | // | |
| 626 | // Tracked in the LifecycleScope so stop() drains the loop before | |
| 627 | // tearing down the corestore (vector A1 in STALE-REF-INVENTORY.md). | |
| 628 | this._trackEagerReplicate(appKeyHex, drive, opts, { source: 'fresh-seed' }) | |
| 667 | this._joinDriveDiscovery(appKeyHex, discoveryKey, discoveryHandles) | |
| 668 | try { | |
| 669 | await node.swarm.flush() | |
| 670 | if (admissionReservation) { | |
| 671 | // A fresh remote Hyperdrive does not know its blob-core key until | |
| 672 | // the metadata feed has synchronized. Asking getBlobs() first can | |
| 673 | // therefore time out even while the publisher is correctly serving | |
| 674 | // the drive (the PVSS dealer→relay path exercises this exact race). | |
| 675 | await updateWithTimeout(drive, { timeoutMs: 10_000 }) | |
| 676 | const measured = await getDriveSize(drive, { timeoutMs: 10_000, requireAuthoritative: true }) | |
| 677 | if (!Number.isSafeInteger(measured.totalBytes) || measured.totalBytes < 0) { | |
| 678 | const err = new Error('DRIVE_SIZE_UNRESOLVED') | |
| 679 | err.code = 'STORAGE_SIZE_PROOF_UNAVAILABLE' | |
| 680 | throw err | |
| 681 | } | |
| 682 | if (measured.totalBytes > admissionCap) { | |
| 683 | const err = new Error('maxStorage is below the authoritative drive size') | |
| 684 | err.code = 'STORAGE_BOUND_BELOW_ACTUAL' | |
| 685 | err.actualBytes = measured.totalBytes | |
| 686 | err.boundBytes = admissionCap | |
| 687 | throw err | |
| 688 | } | |
| 689 | } | |
| 690 | } finally { | |
| 691 | if (done) done() | |
| 692 | } | |
| 629 | 693 | |
| 630 | 694 | // Revocability commitments — recorded at seed time, derived from the |
| 631 | 695 | // signed seed-request payload (committed by publisher signature, so |
| … | … | @@ -661,10 +725,10 @@ export class AppLifecycle extends EventEmitter { |
| 661 | 725 | // exact pre-seed view (usually the null-discoveryKey placeholder that |
| 662 | 726 | // load() created), rather than leaving a half-seeded entry behind. |
| 663 | 727 | const previousEntry = node.appRegistry.get(appKeyHex) || null |
| 664 | ||
| 665 | node.appRegistry.set(appKeyHex, { | |
| 728 | const nextEntry = { | |
| 666 | 729 | drive, |
| 667 | 730 | discoveryKey, |
| 731 | discoveryHandles, | |
| 668 | 732 | startedAt: Date.now(), |
| 669 | 733 | bytesServed: 0, |
| 670 | 734 | type: contentType, |
| … | … | @@ -691,7 +755,21 @@ export class AppLifecycle extends EventEmitter { |
| 691 | 755 | unseedFreezeMs, |
| 692 | 756 | durability, |
| 693 | 757 | maxStorage |
| 694 | }, { persist: false }) | |
| 758 | } | |
| 759 | if (readOnlyRecovery) { | |
| 760 | if (typeof node.appRegistry.attachRuntime !== 'function' || | |
| 761 | node.appRegistry.attachRuntime(appKeyHex, { | |
| 762 | drive, | |
| 763 | discoveryKey, | |
| 764 | discoveryHandles, | |
| 765 | bytesServed: 0, | |
| 766 | retiring: false | |
| 767 | }) !== true) { | |
| 768 | throw new Error('APP_REGISTRY_RUNTIME_ATTACH_FAILED') | |
| 769 | } | |
| 770 | } else { | |
| 771 | node.appRegistry.set(appKeyHex, nextEntry, { persist: false }) | |
| 772 | } | |
| 695 | 773 | |
| 696 | 774 | // Durability gate: persist EXPLICITLY and await it before reporting the |
| 697 | 775 | // seed as successful. AppRegistry.set() persists fire-and-forget, so a |
| … | … | @@ -699,7 +777,7 @@ export class AppLifecycle extends EventEmitter { |
| 699 | 777 | // discoveryKey, the drive served traffic, and the entry simply was not |
| 700 | 778 | // there after the next restart. On failure, restore the pre-seed view |
| 701 | 779 | // and rethrow; the enclosing catch closes the drive. |
| 702 | if (typeof node.appRegistry.persistEntry === 'function') { | |
| 780 | if (!readOnlyRecovery && typeof node.appRegistry.persistEntry === 'function') { | |
| 703 | 781 | try { |
| 704 | 782 | await node.appRegistry.persistEntry(appKeyHex, { throwOnError: true }) |
| 705 | 783 | } catch (err) { |
| … | … | @@ -724,6 +802,12 @@ export class AppLifecycle extends EventEmitter { |
| 724 | 802 | admissionReservation = null |
| 725 | 803 | } |
| 726 | 804 | |
| 805 | // Start exact-state replication only after both durable registry | |
| 806 | // persistence and storage-authority commitment are visible. | |
| 807 | if (!readOnlyRecovery) { | |
| 808 | this._trackEagerReplicate(appKeyHex, drive, opts, { source: 'fresh-seed' }) | |
| 809 | } | |
| 810 | ||
| 727 | 811 | // 2026-05-23: register persistent download ranges on the drive's |
| 728 | 812 | // cores so they actively pull missing blocks from any peer that |
| 729 | 813 | // has them — not only during the one-shot _eagerReplicate + |
| … | … | @@ -743,15 +827,17 @@ export class AppLifecycle extends EventEmitter { |
| 743 | 827 | // for this seed cycle. _eagerReplicate's drive.download('/') will |
| 744 | 828 | // populate the blob core on next pass, then a subsequent |
| 745 | 829 | // _registerPersistentDownloads call picks up the slack. |
| 746 | this._registerPersistentDownloads(appKeyHex, drive).catch((err) => { | |
| 747 | this.emit('persistent-download-error', { | |
| 748 | appKey: appKeyHex, | |
| 749 | error: err.message || String(err) | |
| 830 | if (!readOnlyRecovery) { | |
| 831 | this._registerPersistentDownloads(appKeyHex, drive).catch((err) => { | |
| 832 | this.emit('persistent-download-error', { | |
| 833 | appKey: appKeyHex, | |
| 834 | error: err.message || String(err) | |
| 835 | }) | |
| 750 | 836 | }) |
| 751 | }) | |
| 837 | } | |
| 752 | 838 | |
| 753 | 839 | if (node.distributedDriveBridge) { |
| 754 | node.distributedDriveBridge.registerDrive(appKeyHex, drive) | |
| 840 | try { node.distributedDriveBridge.registerDrive(appKeyHex, drive) } catch (_) {} | |
| 755 | 841 | } |
| 756 | 842 | |
| 757 | 843 | this.emit('seeding', { appKey: appKeyHex, discoveryKey: b4a.toString(discoveryKey, 'hex') }) |
| … | … | @@ -761,6 +847,9 @@ export class AppLifecycle extends EventEmitter { |
| 761 | 847 | // Hyperdrive constructor both throw with `drive` still null), then undo |
| 762 | 848 | // the admission reservation so a failed seed leaves the ledger exactly as |
| 763 | 849 | // it found it. |
| 850 | for (const handle of discoveryHandles) { | |
| 851 | try { if (handle && typeof handle.destroy === 'function') await handle.destroy() } catch (_) {} | |
| 852 | } | |
| 764 | 853 | if (drive) { try { await drive.close() } catch (_) {} } |
| 765 | 854 | this._rollbackSeedAdmission(admissionReservation) |
| 766 | 855 | throw err |
| … | … | @@ -825,6 +914,173 @@ export class AppLifecycle extends EventEmitter { |
| 825 | 914 | return promise |
| 826 | 915 | } |
| 827 | 916 | |
| 917 | async _downloadProvedDriveRanges (drive, proof, opts = {}) { | |
| 918 | const timeoutMs = Number.isFinite(opts.timeoutMs) ? opts.timeoutMs : 120_000 | |
| 919 | const signal = opts.signal || null | |
| 920 | const metaCore = proof?.metaCoreSnapshot | |
| 921 | const blobCore = proof?.blobCoreSnapshot | |
| 922 | if (!metaCore || !blobCore || !Number.isSafeInteger(proof?.metaLength) || | |
| 923 | !Number.isSafeInteger(proof?.blobLength)) { | |
| 924 | throw new Error('authoritative drive range proof unavailable') | |
| 925 | } | |
| 926 | const trackers = [] | |
| 927 | const add = (core, end) => { | |
| 928 | if (end <= 0) return | |
| 929 | trackers.push(core.download({ start: 0, end })) | |
| 930 | } | |
| 931 | add(metaCore, proof.metaLength) | |
| 932 | add(blobCore, proof.blobLength) | |
| 933 | let timer = null | |
| 934 | let abortHandler = null | |
| 935 | try { | |
| 936 | await new Promise((resolve, reject) => { | |
| 937 | timer = setTimeout(() => reject(new Error('download timeout')), timeoutMs) | |
| 938 | if (signal) { | |
| 939 | if (signal.aborted) { | |
| 940 | const err = new Error('Aborted') | |
| 941 | err.name = 'AbortError' | |
| 942 | reject(err) | |
| 943 | return | |
| 944 | } | |
| 945 | abortHandler = () => { | |
| 946 | const err = new Error('Aborted') | |
| 947 | err.name = 'AbortError' | |
| 948 | reject(err) | |
| 949 | } | |
| 950 | signal.addEventListener('abort', abortHandler) | |
| 951 | } | |
| 952 | Promise.all(trackers.map((tracker) => { | |
| 953 | if (typeof tracker.done === 'function') return tracker.done() | |
| 954 | if (typeof tracker.downloaded === 'function') return tracker.downloaded() | |
| 955 | throw new Error('drive range tracker unavailable') | |
| 956 | })).then(resolve, reject) | |
| 957 | }) | |
| 958 | } finally { | |
| 959 | if (timer) clearTimeout(timer) | |
| 960 | if (signal && abortHandler) signal.removeEventListener('abort', abortHandler) | |
| 961 | for (const tracker of trackers) { | |
| 962 | try { tracker.destroy() } catch {} | |
| 963 | } | |
| 964 | } | |
| 965 | } | |
| 966 | ||
| 967 | _validAnchoredProof (proof) { | |
| 968 | return Number.isSafeInteger(proof?.driveVersion) && proof.driveVersion > 0 && | |
| 969 | Number.isSafeInteger(proof.metaLength) && proof.metaLength >= 0 && | |
| 970 | Number.isSafeInteger(proof.blobLength) && proof.blobLength >= 0 && | |
| 971 | Number.isSafeInteger(proof.totalBytes) && proof.totalBytes >= 0 && | |
| 972 | Number.isSafeInteger(proof.metaFork) && proof.metaFork >= 0 && | |
| 973 | Number.isSafeInteger(proof.blobFork) && proof.blobFork >= 0 | |
| 974 | } | |
| 975 | ||
| 976 | async _closeDriveProof (proof) { | |
| 977 | for (const core of [proof?.metaCoreSnapshot, proof?.blobCoreSnapshot]) { | |
| 978 | if (core && typeof core.close === 'function') { | |
| 979 | try { await core.close() } catch {} | |
| 980 | } | |
| 981 | } | |
| 982 | if (proof) { | |
| 983 | proof.metaCoreSnapshot = null | |
| 984 | proof.blobCoreSnapshot = null | |
| 985 | } | |
| 986 | } | |
| 987 | ||
| 988 | async _installPersistentDownloadProof (appKeyHex, drive, proof) { | |
| 989 | const entry = this.node.appRegistry && this.node.appRegistry.get(appKeyHex) | |
| 990 | if (!entry || entry.drive !== drive || !this._validAnchoredProof(proof) || | |
| 991 | !proof.metaCoreSnapshot || !proof.blobCoreSnapshot) return false | |
| 992 | ||
| 993 | const metaState = stableCoreProofState(proof.metaCoreSnapshot) | |
| 994 | const blobState = stableCoreProofState(proof.blobCoreSnapshot) | |
| 995 | if (!metaState || !blobState || | |
| 996 | metaState.length !== proof.metaLength || metaState.fork !== proof.metaFork || | |
| 997 | blobState.length !== proof.blobLength || blobState.fork !== proof.blobFork || | |
| 998 | metaState.byteLength + blobState.byteLength !== proof.totalBytes) return false | |
| 999 | ||
| 1000 | const nextRanges = [] | |
| 1001 | try { | |
| 1002 | nextRanges.push(proof.metaCoreSnapshot.download({ start: 0, end: proof.metaLength })) | |
| 1003 | nextRanges.push(proof.blobCoreSnapshot.download({ start: 0, end: proof.blobLength })) | |
| 1004 | } catch (err) { | |
| 1005 | for (const range of nextRanges) { | |
| 1006 | try { range.destroy() } catch {} | |
| 1007 | } | |
| 1008 | throw err | |
| 1009 | } | |
| 1010 | ||
| 1011 | const previousRanges = Array.isArray(entry.downloadRanges) ? entry.downloadRanges : [] | |
| 1012 | const previousCores = Array.isArray(entry.downloadSnapshotCores) ? entry.downloadSnapshotCores : [] | |
| 1013 | entry.downloadRanges = nextRanges | |
| 1014 | entry.downloadSnapshotCores = [proof.metaCoreSnapshot, proof.blobCoreSnapshot] | |
| 1015 | proof.metaCoreSnapshot = null | |
| 1016 | proof.blobCoreSnapshot = null | |
| 1017 | ||
| 1018 | for (const range of previousRanges) { | |
| 1019 | try { if (range && typeof range.destroy === 'function') range.destroy() } catch {} | |
| 1020 | } | |
| 1021 | for (const core of previousCores) { | |
| 1022 | try { if (core && typeof core.close === 'function') await core.close() } catch {} | |
| 1023 | } | |
| 1024 | return true | |
| 1025 | } | |
| 1026 | ||
| 1027 | async _commitAnchoredProof (appKeyHex, drive, proof, tracked = false) { | |
| 1028 | const node = this.node | |
| 1029 | if (!tracked) { | |
| 1030 | return this._withDriveLease(appKeyHex, () => this._commitAnchoredProof(appKeyHex, drive, proof, true)) | |
| 1031 | } | |
| 1032 | if (!this._validAnchoredProof(proof)) return false | |
| 1033 | if (!node.storageAdmission?.canAcknowledge(`drive:${appKeyHex}`)) return false | |
| 1034 | const entry = node.appRegistry.get(appKeyHex) | |
| 1035 | if (!entry || entry.drive !== drive || drive.closed || drive.closing) return false | |
| 1036 | const previous = { | |
| 1037 | anchored: entry.anchored, | |
| 1038 | anchoredAt: entry.anchoredAt, | |
| 1039 | anchoredLength: entry.anchoredLength, | |
| 1040 | lastAnchorCheck: entry.lastAnchorCheck, | |
| 1041 | storageProvedDriveVersion: entry.storageProvedDriveVersion, | |
| 1042 | storageProvedMetaLength: entry.storageProvedMetaLength, | |
| 1043 | storageProvedBlobLength: entry.storageProvedBlobLength, | |
| 1044 | storageProvedTotalBytes: entry.storageProvedTotalBytes, | |
| 1045 | storageProvedMetaFork: entry.storageProvedMetaFork, | |
| 1046 | storageProvedBlobFork: entry.storageProvedBlobFork | |
| 1047 | } | |
| 1048 | const now = Date.now() | |
| 1049 | node.appRegistry.update(appKeyHex, { | |
| 1050 | anchored: true, | |
| 1051 | anchoredAt: entry.anchoredAt || now, | |
| 1052 | anchoredLength: proof.driveVersion, | |
| 1053 | lastAnchorCheck: now, | |
| 1054 | storageProvedDriveVersion: proof.driveVersion, | |
| 1055 | storageProvedMetaLength: proof.metaLength, | |
| 1056 | storageProvedBlobLength: proof.blobLength, | |
| 1057 | storageProvedTotalBytes: proof.totalBytes, | |
| 1058 | storageProvedMetaFork: proof.metaFork, | |
| 1059 | storageProvedBlobFork: proof.blobFork | |
| 1060 | }, { persist: false }) | |
| 1061 | try { | |
| 1062 | if (typeof node.appRegistry.persistEntry === 'function') { | |
| 1063 | await node.appRegistry.persistEntry(appKeyHex, { throwOnError: true }) | |
| 1064 | } else { | |
| 1065 | await node.appRegistry.flush({ throwOnError: true }) | |
| 1066 | } | |
| 1067 | } catch (err) { | |
| 1068 | if (node.appRegistry.get(appKeyHex) === entry) { | |
| 1069 | node.appRegistry.update(appKeyHex, previous, { persist: false }) | |
| 1070 | } | |
| 1071 | throw err | |
| 1072 | } | |
| 1073 | try { | |
| 1074 | await this._installPersistentDownloadProof(appKeyHex, drive, proof) | |
| 1075 | } catch (err) { | |
| 1076 | this.emit('persistent-download-error', { | |
| 1077 | appKey: appKeyHex, | |
| 1078 | error: err && err.message ? err.message : String(err) | |
| 1079 | }) | |
| 1080 | } | |
| 1081 | return true | |
| 1082 | } | |
| 1083 | ||
| 828 | 1084 | /** |
| 829 | 1085 | * Eagerly replicate a drive with the v0.8.11 size-check. Called from: |
| 830 | 1086 | * 1. _seedAppInner — fresh seed, after the drive is created. |
| … | … | @@ -857,6 +1113,9 @@ export class AppLifecycle extends EventEmitter { |
| 857 | 1113 | if (!drive || drive.closed || drive.closing) return |
| 858 | 1114 | const discoveryKey = drive.discoveryKey |
| 859 | 1115 | const source = meta.source || 'fresh-seed' |
| 1116 | const initialBound = node.appRegistry && node.appRegistry.get(appKeyHex)?.maxStorage | |
| 1117 | const initialCap = positiveStorageBound(initialBound ?? opts.maxStorage) | |
| 1118 | if (initialCap === null) return | |
| 860 | 1119 | |
| 861 | 1120 | const MAX_RETRIES = 6 |
| 862 | 1121 | // Tightened tail — 120s was wasteful when the repair monitor takes |
| … | … | @@ -871,8 +1130,9 @@ export class AppLifecycle extends EventEmitter { |
| 871 | 1130 | // was aborted (stop() / self-heal restart in progress). |
| 872 | 1131 | if (aborted() || drive.closed || drive.closing) return |
| 873 | 1132 | |
| 1133 | let pinnedProof = null | |
| 874 | 1134 | try { |
| 875 | node.swarm.join(discoveryKey, { server: true, client: true }) | |
| 1135 | this._joinDriveDiscovery(appKeyHex, discoveryKey) | |
| 876 | 1136 | await raceOr(node.swarm.flush()) |
| 877 | 1137 | |
| 878 | 1138 | if (aborted() || drive.closed || drive.closing) return |
| … | … | @@ -898,32 +1158,34 @@ export class AppLifecycle extends EventEmitter { |
| 898 | 1158 | // locally if the actual size exceeds the cap the publisher |
| 899 | 1159 | // declared. Publisher sees the event (via SDK) at pin time |
| 900 | 1160 | // instead of discovering it via end-user reports. |
| 901 | const cap = Number.isFinite(opts.maxStorage) && opts.maxStorage > 0 | |
| 902 | ? opts.maxStorage | |
| 903 | : null | |
| 904 | if (cap !== null) { | |
| 905 | const { totalBytes, metaBytes, blobBytes } = await raceOr( | |
| 906 | getDriveSize(drive, { timeoutMs: 10_000 }) | |
| 907 | ) | |
| 908 | if (totalBytes > cap) { | |
| 909 | const recommendedCap = Math.ceil(totalBytes * 1.25) | |
| 910 | this.emit('seed-aborted', { | |
| 911 | appKey: appKeyHex, | |
| 912 | reason: 'maxStorage-too-small', | |
| 913 | recoverable: false, | |
| 914 | driveBytes: totalBytes, | |
| 915 | metaBytes, | |
| 916 | blobBytes, | |
| 917 | cap, | |
| 918 | recommendedCap, | |
| 919 | source, | |
| 920 | hint: 'drive is ' + totalBytes + ' bytes; publisher should re-seed with maxStorage ≥ ' + recommendedCap | |
| 921 | }) | |
| 922 | // Clean up — we won't accumulate partial bytes for a drive | |
| 923 | // we can never anchor. | |
| 924 | try { await this.unseedApp(appKeyHex) } catch (_) {} | |
| 925 | return | |
| 926 | } | |
| 1161 | const liveBound = node.appRegistry && node.appRegistry.get(appKeyHex)?.maxStorage | |
| 1162 | const cap = positiveStorageBound(liveBound ?? opts.maxStorage) | |
| 1163 | // The admission contract must remain finite for the entire pin. | |
| 1164 | // If a concurrent registry mutation removes/corrupts the bound, | |
| 1165 | // stop without anchoring and let reconciliation repair policy. | |
| 1166 | if (cap === null) return | |
| 1167 | pinnedProof = await raceOr( | |
| 1168 | getDriveSize(drive, { timeoutMs: 10_000, requireAuthoritative: true, pinSnapshots: true }) | |
| 1169 | ) | |
| 1170 | const { totalBytes, metaBytes, blobBytes } = pinnedProof | |
| 1171 | if (!Number.isSafeInteger(totalBytes) || totalBytes < 0 || totalBytes > cap) { | |
| 1172 | const recommendedCap = Number.isSafeInteger(totalBytes) ? Math.ceil(totalBytes * 1.25) : null | |
| 1173 | this.emit('seed-aborted', { | |
| 1174 | appKey: appKeyHex, | |
| 1175 | reason: 'maxStorage-too-small', | |
| 1176 | recoverable: false, | |
| 1177 | driveBytes: totalBytes, | |
| 1178 | metaBytes, | |
| 1179 | blobBytes, | |
| 1180 | cap, | |
| 1181 | recommendedCap, | |
| 1182 | source, | |
| 1183 | hint: 'drive is ' + totalBytes + ' bytes; publisher should re-seed with maxStorage ≥ ' + recommendedCap | |
| 1184 | }) | |
| 1185 | // Clean up — we won't accumulate partial bytes for a drive | |
| 1186 | // we can never anchor. | |
| 1187 | try { await this.unseedApp(appKeyHex) } catch (_) {} | |
| 1188 | return | |
| 927 | 1189 | } |
| 928 | 1190 | |
| 929 | 1191 | // Cancellable download — destroys the download tracker on |
| … | … | @@ -939,7 +1201,10 @@ export class AppLifecycle extends EventEmitter { |
| 939 | 1201 | // unanchored so runRepairPass keeps re-queuing them. |
| 940 | 1202 | let downloadComplete = true |
| 941 | 1203 | try { |
| 942 | await raceOr(downloadWithTimeout(drive, '/', { timeoutMs: 120_000 })) | |
| 1204 | await raceOr(this._downloadProvedDriveRanges(drive, pinnedProof, { | |
| 1205 | timeoutMs: 120_000, | |
| 1206 | signal: scope ? scope.signal : null | |
| 1207 | })) | |
| 943 | 1208 | } catch (err) { |
| 944 | 1209 | if (isAbortError(err)) return |
| 945 | 1210 | downloadComplete = false |
| … | … | @@ -955,12 +1220,13 @@ export class AppLifecycle extends EventEmitter { |
| 955 | 1220 | // (every blob block present), not just that metadata synced. |
| 956 | 1221 | // If we don't have all blocks yet, record an anchor check and |
| 957 | 1222 | // let the retry loop / periodic repair monitor keep pulling. |
| 958 | const fullyReplicated = downloadComplete && await this._isDriveFullyReplicated(drive) | |
| 959 | if (fullyReplicated && node.appRegistry && typeof node.appRegistry.setAnchored === 'function') { | |
| 960 | node.appRegistry.setAnchored(appKeyHex, drive.version) | |
| 961 | await this._recordCustodyReceipt(appKeyHex, opts, drive.version) | |
| 962 | this.emit('anchored', { appKey: appKeyHex, version: drive.version, source }) | |
| 963 | this.emit('reseeded', { appKey: appKeyHex, version: drive.version, source }) | |
| 1223 | const fullyReplicated = downloadComplete && this._validAnchoredProof(pinnedProof) | |
| 1224 | const anchored = fullyReplicated && await this._commitAnchoredProof(appKeyHex, drive, pinnedProof) | |
| 1225 | if (anchored) { | |
| 1226 | const anchoredVersion = pinnedProof.driveVersion | |
| 1227 | await this._recordCustodyReceipt(appKeyHex, opts, anchoredVersion) | |
| 1228 | this.emit('anchored', { appKey: appKeyHex, version: anchoredVersion, source }) | |
| 1229 | this.emit('reseeded', { appKey: appKeyHex, version: anchoredVersion, source }) | |
| 964 | 1230 | return |
| 965 | 1231 | } |
| 966 | 1232 | |
| … | … | @@ -976,6 +1242,8 @@ export class AppLifecycle extends EventEmitter { |
| 976 | 1242 | // SESSION_CLOSED / timeout / drive closed mid-call → fall through |
| 977 | 1243 | // to the retry delay below if the drive is still considered open. |
| 978 | 1244 | if (drive.closed || drive.closing) return |
| 1245 | } finally { | |
| 1246 | await this._closeDriveProof(pinnedProof) | |
| 979 | 1247 | } |
| 980 | 1248 | |
| 981 | 1249 | if (attempt < MAX_RETRIES - 1) { |
| … | … | @@ -1054,15 +1322,21 @@ export class AppLifecycle extends EventEmitter { |
| 1054 | 1322 | |
| 1055 | 1323 | if (newCap === null || newCap === oldCap) { |
| 1056 | 1324 | return { |
| 1057 | ok: true, changed: false, | |
| 1325 | ok: true, | |
| 1326 | changed: false, | |
| 1058 | 1327 | reason: newCap === null ? 'cap-omitted-on-recovery' : 'cap-unchanged', |
| 1059 | oldCap, newCap, effectiveCap: oldCap | |
| 1328 | oldCap, | |
| 1329 | newCap, | |
| 1330 | effectiveCap: oldCap | |
| 1060 | 1331 | } |
| 1061 | 1332 | } |
| 1062 | 1333 | |
| 1063 | 1334 | if (oldCap !== null && newCap < oldCap) { |
| 1064 | 1335 | const warning = { |
| 1065 | appKey: appKeyHex, reason: 'cap-lowered-on-repin', oldCap, newCap, | |
| 1336 | appKey: appKeyHex, | |
| 1337 | reason: 'cap-lowered-on-repin', | |
| 1338 | oldCap, | |
| 1339 | newCap, | |
| 1066 | 1340 | hint: 'Lowering maxStorage on a recovered entry is not honored; relay keeps the higher persisted cap.' |
| 1067 | 1341 | } |
| 1068 | 1342 | this._safeEmit('seed-cap-warning', warning) |
| … | … | @@ -1071,8 +1345,12 @@ export class AppLifecycle extends EventEmitter { |
| 1071 | 1345 | |
| 1072 | 1346 | if (oldCap === null) { |
| 1073 | 1347 | const warning = { |
| 1074 | appKey: appKeyHex, reason: 'cap-baseline-unknown-on-repin', oldCap, newCap, | |
| 1075 | incrementalBytes: null, effectiveCap: oldCap, | |
| 1348 | appKey: appKeyHex, | |
| 1349 | reason: 'cap-baseline-unknown-on-repin', | |
| 1350 | oldCap, | |
| 1351 | newCap, | |
| 1352 | incrementalBytes: null, | |
| 1353 | effectiveCap: oldCap, | |
| 1076 | 1354 | hint: 'Cannot safely calculate incremental storage for a previously-uncapped recovered entry. Let recovery finish, unseed it, then seed fresh with maxStorage.' |
| 1077 | 1355 | } |
| 1078 | 1356 | this._safeEmit('seed-cap-warning', warning) |
| … | … | @@ -1207,6 +1485,7 @@ export class AppLifecycle extends EventEmitter { |
| 1207 | 1485 | |
| 1208 | 1486 | return { ok: true, changed: true, incrementalBytes, replicationStarted: true } |
| 1209 | 1487 | } |
| 1488 | ||
| 1210 | 1489 | /** |
| 1211 | 1490 | * Read /manifest.json for indexing. |
| 1212 | 1491 | * |
| … | … | @@ -1378,6 +1657,8 @@ export class AppLifecycle extends EventEmitter { |
| 1378 | 1657 | const entry = node.appRegistry.get(appKeyHex) |
| 1379 | 1658 | if (!entry || !entry.drive) return false |
| 1380 | 1659 | if (entry.anchored === true) return true // already anchored, nothing to do |
| 1660 | const cap = positiveStorageBound(entry.maxStorage) | |
| 1661 | if (cap === null) return false | |
| 1381 | 1662 | |
| 1382 | 1663 | const drive = entry.drive |
| 1383 | 1664 | if (drive.closed || drive.closing) return false |
| … | … | @@ -1396,7 +1677,7 @@ export class AppLifecycle extends EventEmitter { |
| 1396 | 1677 | |
| 1397 | 1678 | // Re-announce on the discovery topic in case the swarm dropped us |
| 1398 | 1679 | try { |
| 1399 | node.swarm.join(drive.discoveryKey, { server: true, client: true }) | |
| 1680 | this._joinDriveDiscovery(appKeyHex, drive.discoveryKey) | |
| 1400 | 1681 | await raceOr(Promise.race([ |
| 1401 | 1682 | node.swarm.flush().catch(() => {}), |
| 1402 | 1683 | new Promise(resolve => { |
| … | … | @@ -1430,6 +1711,34 @@ export class AppLifecycle extends EventEmitter { |
| 1430 | 1711 | return false |
| 1431 | 1712 | } |
| 1432 | 1713 | |
| 1714 | // Bind repair to one signed metadata/blob snapshot before opening any | |
| 1715 | // storage-producing range. This keeps repair from racing eager replication | |
| 1716 | // and publishing the retired metadata-only `anchored=true` state. | |
| 1717 | let sizeProof = null | |
| 1718 | try { | |
| 1719 | sizeProof = await raceOr(getDriveSize(drive, { | |
| 1720 | timeoutMs: 10_000, | |
| 1721 | requireAuthoritative: true, | |
| 1722 | pinSnapshots: true | |
| 1723 | })) | |
| 1724 | if (!Number.isSafeInteger(sizeProof.totalBytes) || sizeProof.totalBytes < 0 || | |
| 1725 | sizeProof.totalBytes > cap) { | |
| 1726 | this.emit('seed-aborted', { | |
| 1727 | appKey: appKeyHex, | |
| 1728 | reason: 'maxStorage-too-small', | |
| 1729 | driveBytes: sizeProof.totalBytes, | |
| 1730 | cap, | |
| 1731 | source: 'repair' | |
| 1732 | }) | |
| 1733 | await this._closeDriveProof(sizeProof) | |
| 1734 | return false | |
| 1735 | } | |
| 1736 | } catch (err) { | |
| 1737 | await this._closeDriveProof(sizeProof) | |
| 1738 | if (isAbortError(err)) return false | |
| 1739 | return false | |
| 1740 | } | |
| 1741 | ||
| 1433 | 1742 | // We have metadata; pull blob content (cancellable on timeout). |
| 1434 | 1743 | // |
| 1435 | 1744 | // 2026-05-22: same fix as _eagerReplicate — don't mark anchored on a |
| … | … | @@ -1447,7 +1756,10 @@ export class AppLifecycle extends EventEmitter { |
| 1447 | 1756 | // as success — flag downloadComplete=false so the partial-pin |
| 1448 | 1757 | // gate below keeps the entry unanchored and runRepairPass |
| 1449 | 1758 | // re-queues it on the next tick. |
| 1450 | await raceOr(downloadWithTimeout(drive, '/', { timeoutMs: downloadTimeout })) | |
| 1759 | await raceOr(this._downloadProvedDriveRanges(drive, sizeProof, { | |
| 1760 | timeoutMs: downloadTimeout, | |
| 1761 | signal: scope ? scope.signal : null | |
| 1762 | })) | |
| 1451 | 1763 | } catch (err) { |
| 1452 | 1764 | if (isAbortError(err)) return false |
| 1453 | 1765 | downloadComplete = false |
| … | … | @@ -1455,10 +1767,10 @@ export class AppLifecycle extends EventEmitter { |
| 1455 | 1767 | |
| 1456 | 1768 | if ((scope && scope.aborted) || drive.closed || drive.closing) return false |
| 1457 | 1769 | |
| 1458 | if (drive.version > 0 && downloadComplete && await this._isDriveFullyReplicated(drive)) { | |
| 1459 | node.appRegistry.setAnchored(appKeyHex, drive.version) | |
| 1460 | await this._recordCustodyReceipt(appKeyHex, entry, drive.version) | |
| 1461 | this.emit('anchored', { appKey: appKeyHex, version: drive.version, source: 'repair' }) | |
| 1770 | if (downloadComplete && this._validAnchoredProof(sizeProof) && | |
| 1771 | await this._commitAnchoredProof(appKeyHex, drive, sizeProof)) { | |
| 1772 | await this._recordCustodyReceipt(appKeyHex, entry, sizeProof.driveVersion) | |
| 1773 | this.emit('anchored', { appKey: appKeyHex, version: sizeProof.driveVersion, source: 'repair' }) | |
| 1462 | 1774 | return true |
| 1463 | 1775 | } |
| 1464 | 1776 | |
| … | … | @@ -1471,6 +1783,8 @@ export class AppLifecycle extends EventEmitter { |
| 1471 | 1783 | } catch (err) { |
| 1472 | 1784 | if (isAbortError(err)) return false |
| 1473 | 1785 | this.emit('repair-download-failed', { appKey: appKeyHex, error: err.message }) |
| 1786 | } finally { | |
| 1787 | await this._closeDriveProof(sizeProof) | |
| 1474 | 1788 | } |
| 1475 | 1789 | return false |
| 1476 | 1790 | } |
| … | … | @@ -1778,7 +2092,7 @@ export class AppLifecycle extends EventEmitter { |
| 1778 | 2092 | if (tracker && typeof tracker.done === 'function') { |
| 1779 | 2093 | await Promise.race([ |
| 1780 | 2094 | tracker.done(), |
| 1781 | new Promise((_, reject) => { const tmr = setTimeout(() => reject(new Error('share bundle range timeout')), timeoutMs); if (typeof tmr.unref === 'function') tmr.unref() }) | |
| 2095 | new Promise((_resolve, reject) => { const tmr = setTimeout(() => reject(new Error('share bundle range timeout')), timeoutMs); if (typeof tmr.unref === 'function') tmr.unref() }) | |
| 1782 | 2096 | ]) |
| 1783 | 2097 | } |
| 1784 | 2098 | } |
| … | … | @@ -1808,6 +2122,33 @@ export class AppLifecycle extends EventEmitter { |
| 1808 | 2122 | } |
| 1809 | 2123 | } |
| 1810 | 2124 | |
| 2125 | /** | |
| 2126 | * An already-seeded drive never re-enters the async anchor path. A custody | |
| 2127 | * re-pin still has to verify and record its share receipt, otherwise the | |
| 2128 | * dealer waits forever for a quorum that this relay silently ignored. | |
| 2129 | * Keep this asynchronous to match fresh-seed behaviour: /seed acknowledges | |
| 2130 | * durable drive ownership while the publisher polls the receipt registry. | |
| 2131 | */ | |
| 2132 | _recordCustodyReceiptOnRepin (appKeyHex, existing, opts) { | |
| 2133 | if (!opts || opts.blind !== true || !opts.custodyIntentId) return | |
| 2134 | ||
| 2135 | const version = Number.isFinite(opts.contentVersion) | |
| 2136 | ? opts.contentVersion | |
| 2137 | : (existing && Number.isFinite(existing.version) ? existing.version : 0) | |
| 2138 | const promise = this._recordCustodyReceipt(appKeyHex, opts, version) | |
| 2139 | .catch((err) => { | |
| 2140 | this._safeEmit('custody-receipt-error', { | |
| 2141 | appKey: appKeyHex, | |
| 2142 | intentId: opts.custodyIntentId, | |
| 2143 | error: (err && err.message) || String(err) | |
| 2144 | }) | |
| 2145 | return null | |
| 2146 | }) | |
| 2147 | ||
| 2148 | const scope = this.node && this.node._scope | |
| 2149 | if (scope && typeof scope.tracked === 'function') scope.tracked(promise) | |
| 2150 | } | |
| 2151 | ||
| 1811 | 2152 | async _recordCustodyReceipt (appKeyHex, opts = {}, contentVersion = 0) { |
| 1812 | 2153 | const node = this.node |
| 1813 | 2154 | if (!opts.blind || !opts.custodyIntentId || !node.seedingRegistry || !node.swarm?.keyPair) return null |
| … | … | @@ -1838,7 +2179,7 @@ export class AppLifecycle extends EventEmitter { |
| 1838 | 2179 | } |
| 1839 | 2180 | const relayPubkey = b4a.toString(node.swarm.keyPair.publicKey, 'hex') |
| 1840 | 2181 | const bundle = typeof this._readShareBundle === 'function' |
| 1841 | ? await this._readShareBundle(intent.shareBundleKey) | |
| 2182 | ? await this._readShareBundle(intent.shareBundleKey, { appKey: appKeyHex }) | |
| 1842 | 2183 | : null |
| 1843 | 2184 | const result = verifyShareBundleForRelay(intent, bundle, relayPubkey) |
| 1844 | 2185 | if (!result.ok) { |
| … | … | @@ -2064,39 +2405,51 @@ export class AppLifecycle extends EventEmitter { |
| 2064 | 2405 | * @param {Hyperdrive} drive |
| 2065 | 2406 | * @returns {Promise<void>} |
| 2066 | 2407 | */ |
| 2067 | async _registerPersistentDownloads (appKeyHex, drive) { | |
| 2408 | async _registerPersistentDownloads (appKeyHex, drive, tracked = false) { | |
| 2068 | 2409 | const node = this.node |
| 2410 | if (!tracked && node.storageAdmission && typeof node.storageAdmission.runKeyMutation === 'function') { | |
| 2411 | return node.storageAdmission.runKeyMutation(`drive:${appKeyHex}`, () => | |
| 2412 | this._registerPersistentDownloads(appKeyHex, drive, true)) | |
| 2413 | } | |
| 2069 | 2414 | const entry = node.appRegistry && node.appRegistry.get(appKeyHex) |
| 2070 | 2415 | if (!entry) return |
| 2071 | 2416 | if (!drive || drive.closed || drive.closing) return |
| 2072 | ||
| 2073 | // Replace any prior ranges (idempotent — see retry path above). | |
| 2074 | if (Array.isArray(entry.downloadRanges)) { | |
| 2075 | for (const dl of entry.downloadRanges) { | |
| 2076 | try { if (dl && typeof dl.destroy === 'function') dl.destroy() } catch (_) {} | |
| 2417 | const cap = positiveStorageBound(entry.maxStorage) | |
| 2418 | if (cap === null) return | |
| 2419 | ||
| 2420 | // A live `end: -1` range keeps expanding as metadata advances and can | |
| 2421 | // materialize bytes that were never measured or admitted. Prove one | |
| 2422 | // immutable metadata/blob snapshot, ensure its aggregate footprint fits | |
| 2423 | // the durable cap, then register only those finite ranges. | |
| 2424 | let settleRegistration = null | |
| 2425 | const token = { | |
| 2426 | settled: new Promise(resolve => { settleRegistration = resolve }) | |
| 2427 | } | |
| 2428 | entry.downloadRegistration = token | |
| 2429 | let proof = null | |
| 2430 | try { | |
| 2431 | proof = await getDriveSize(drive, { | |
| 2432 | timeoutMs: 10_000, | |
| 2433 | requireAuthoritative: true, | |
| 2434 | pinSnapshots: true | |
| 2435 | }) | |
| 2436 | if (node.appRegistry.get(appKeyHex) !== entry || entry.drive !== drive || | |
| 2437 | entry.downloadRegistration !== token || drive.closed || drive.closing) return | |
| 2438 | if (!Number.isSafeInteger(proof && proof.totalBytes) || proof.totalBytes < 0 || proof.totalBytes > cap) { | |
| 2439 | this._safeEmit('persistent-download-error', { | |
| 2440 | appKey: appKeyHex, | |
| 2441 | error: 'drive footprint exceeds durable maxStorage bound' | |
| 2442 | }) | |
| 2443 | return | |
| 2077 | 2444 | } |
| 2078 | } | |
| 2079 | entry.downloadRanges = [] | |
| 2080 | ||
| 2081 | // Metadata core — always present after drive.ready(). | |
| 2082 | if (drive.db && drive.db.core && typeof drive.db.core.download === 'function') { | |
| 2083 | try { | |
| 2084 | const metaDl = drive.db.core.download({ start: 0, end: -1 }) | |
| 2085 | entry.downloadRanges.push(metaDl) | |
| 2086 | } catch (_) { /* core may already be closing — skip */ } | |
| 2087 | } | |
| 2088 | ||
| 2089 | // Blob core — lazy. getBlobs() lazily creates / opens it. May fail | |
| 2090 | // if the drive is mid-close; that's fine, skip + retry next cycle. | |
| 2091 | let blobs = drive.blobs | |
| 2092 | if (!blobs && typeof drive.getBlobs === 'function') { | |
| 2093 | blobs = await drive.getBlobs().catch(() => null) | |
| 2094 | } | |
| 2095 | if (blobs && blobs.core && typeof blobs.core.download === 'function') { | |
| 2445 | await this._installPersistentDownloadProof(appKeyHex, drive, proof) | |
| 2446 | } finally { | |
| 2096 | 2447 | try { |
| 2097 | const blobDl = blobs.core.download({ start: 0, end: -1 }) | |
| 2098 | entry.downloadRanges.push(blobDl) | |
| 2099 | } catch (_) { /* skip */ } | |
| 2448 | await this._closeDriveProof(proof) | |
| 2449 | } finally { | |
| 2450 | if (entry.downloadRegistration === token) entry.downloadRegistration = null | |
| 2451 | settleRegistration() | |
| 2452 | } | |
| 2100 | 2453 | } |
| 2101 | 2454 | } |
| 2102 | 2455 |
| … | … | @@ -37,7 +37,7 @@ |
| 37 | 37 | */ |
| 38 | 38 | |
| 39 | 39 | import Hyperswarm from 'hyperswarm' |
| 40 | import Corestore from 'corestore' | |
| 40 | import { openCorestore } from '../persistence/storage-root-restore.js' | |
| 41 | 41 | import b4a from 'b4a' |
| 42 | 42 | import sodium from 'sodium-universal' |
| 43 | 43 | // Use Node-shaped names. Under Bare/Pear they get remapped via the |
| … | … | @@ -359,7 +359,7 @@ export class BareRelay extends EventEmitter { |
| 359 | 359 | await this.storageAdmission.activatePhysicalEnforcement({ purpose: 'bare-relay-startup' }) |
| 360 | 360 | |
| 361 | 361 | // 1. Corestore — persistent hypercore storage |
| 362 | this.store = new Corestore(this.config.storage) | |
| 362 | this.store = openCorestore(this.config.storage) | |
| 363 | 363 | await this.store.ready() |
| 364 | 364 | |
| 365 | 365 | // 2. App registry — tracks what we're seeding. |
| … | … | @@ -204,14 +204,36 @@ export const DOWNLOAD_TIMEOUT_MS = DEFAULT_DOWNLOAD_TIMEOUT_MS |
| 204 | 204 | */ |
| 205 | 205 | export async function getDriveSize (drive, opts = {}) { |
| 206 | 206 | const timeoutMs = Number.isFinite(opts.timeoutMs) ? opts.timeoutMs : 10_000 |
| 207 | const requireAuthoritative = opts.requireAuthoritative === true | |
| 208 | const pinSnapshots = opts.pinSnapshots === true | |
| 209 | const readDriveVersion = () => { | |
| 210 | const version = drive?.version | |
| 211 | return Number.isSafeInteger(version) ? version : null | |
| 212 | } | |
| 207 | 213 | |
| 208 | 214 | const metaCore = drive && drive.db && drive.db.core |
| 209 | const blobCore = drive && drive.blobs && drive.blobs.core | |
| 215 | let blobs = drive && drive.blobs | |
| 216 | if (!blobs && drive && typeof drive.getBlobs === 'function') { | |
| 217 | let blobsTimer = null | |
| 218 | try { | |
| 219 | blobs = await Promise.race([ | |
| 220 | drive.getBlobs(), | |
| 221 | new Promise((_resolve, reject) => { | |
| 222 | blobsTimer = setTimeout(() => reject(new Error('get-blobs timeout')), timeoutMs) | |
| 223 | }) | |
| 224 | ]) | |
| 225 | } catch (err) { | |
| 226 | if (requireAuthoritative) throw err | |
| 227 | } finally { | |
| 228 | if (blobsTimer) clearTimeout(blobsTimer) | |
| 229 | } | |
| 230 | } | |
| 231 | const blobCore = blobs && blobs.core | |
| 210 | 232 | |
| 211 | 233 | // Helper to update a raw hypercore with a cancellable timeout. |
| 212 | 234 | // Mirrors updateWithTimeout but for a core (not a drive). |
| 213 | 235 | async function updateCore (core) { |
| 214 | if (!core || typeof core.update !== 'function') return | |
| 236 | if (!core || typeof core.update !== 'function') return false | |
| 215 | 237 | const activeRequests = [] |
| 216 | 238 | let timer = null |
| 217 | 239 | try { |
| … | … | @@ -226,9 +248,11 @@ export async function getDriveSize (drive, opts = {}) { |
| 226 | 248 | () => { clearTimeout(timer); resolve() }, |
| 227 | 249 | (err) => { clearTimeout(timer); reject(err) } |
| 228 | 250 | ) |
| 229 | }).catch(() => { | |
| 230 | // Best-effort: partial size info is fine for the caller's decision. | |
| 231 | 251 | }) |
| 252 | return true | |
| 253 | } catch (err) { | |
| 254 | if (requireAuthoritative) throw err | |
| 255 | return false | |
| 232 | 256 | } finally { |
| 233 | 257 | if (timer) clearTimeout(timer) |
| 234 | 258 | if (core.replicator && typeof core.replicator.clearRequests === 'function' && activeRequests.length > 0) { |
| … | … | @@ -239,12 +263,12 @@ export async function getDriveSize (drive, opts = {}) { |
| 239 | 263 | |
| 240 | 264 | // Metadata core size — usually already known after the initial drive.update, |
| 241 | 265 | // but we re-update just in case the caller hasn't done one yet. |
| 242 | await updateCore(metaCore) | |
| 266 | const metaUpdated = await updateCore(metaCore) | |
| 243 | 267 | |
| 244 | 268 | // Blob core size — separate update required; drive.update doesn't touch it. |
| 245 | await updateCore(blobCore) | |
| 269 | const blobUpdated = await updateCore(blobCore) | |
| 246 | 270 | |
| 247 | if (!opts.requireAuthoritative) { | |
| 271 | if (!requireAuthoritative) { | |
| 248 | 272 | const metaBytes = (metaCore && metaCore.byteLength) || 0 |
| 249 | 273 | const blobBytes = (blobCore && blobCore.byteLength) || 0 |
| 250 | 274 | return { totalBytes: metaBytes + blobBytes, metaBytes, blobBytes } |
| … | … | @@ -257,46 +281,67 @@ export async function getDriveSize (drive, opts = {}) { |
| 257 | 281 | // length (or vice versa). Sizing a cap or signing a proof off that tuple |
| 258 | 282 | // under-counts the drive. Read length → byteLength → length again and retry |
| 259 | 283 | // until the bracketing reads agree. |
| 284 | const driveVersionBeforeProof = readDriveVersion() | |
| 260 | 285 | const metaState = readStableCoreState(metaCore) |
| 261 | 286 | const blobState = readStableCoreState(blobCore) |
| 287 | const totalBytes = metaState.byteLength + blobState.byteLength | |
| 288 | if (!metaUpdated || !blobUpdated || driveVersionBeforeProof === null || driveVersionBeforeProof <= 0 || | |
| 289 | !Number.isSafeInteger(totalBytes)) { | |
| 290 | const err = new Error('DRIVE_SIZE_UNRESOLVED') | |
| 291 | err.code = 'STORAGE_SIZE_PROOF_UNAVAILABLE' | |
| 292 | throw err | |
| 293 | } | |
| 262 | 294 | |
| 263 | 295 | const result = { |
| 264 | 296 | metaBytes: metaState.byteLength, |
| 265 | 297 | blobBytes: blobState.byteLength, |
| 266 | totalBytes: metaState.byteLength + blobState.byteLength, | |
| 298 | totalBytes, | |
| 267 | 299 | metaLength: metaState.length, |
| 268 | 300 | blobLength: blobState.length, |
| 269 | 301 | metaFork: metaState.fork, |
| 270 | blobFork: blobState.fork | |
| 302 | blobFork: blobState.fork, | |
| 303 | driveVersion: driveVersionBeforeProof, | |
| 304 | authoritative: true, | |
| 305 | metaCoreSnapshot: null, | |
| 306 | blobCoreSnapshot: null | |
| 271 | 307 | } |
| 272 | 308 | |
| 273 | if (!opts.pinSnapshots) return result | |
| 309 | if (!pinSnapshots) { | |
| 310 | const versionAfter = readDriveVersion() | |
| 311 | if (versionAfter !== driveVersionBeforeProof) { | |
| 312 | const err = new Error(`DRIVE_VERSION_CHANGED_DURING_PROOF: ${driveVersionBeforeProof} -> ${versionAfter}`) | |
| 313 | err.code = 'DRIVE_VERSION_CHANGED_DURING_PROOF' | |
| 314 | throw err | |
| 315 | } | |
| 316 | return result | |
| 317 | } | |
| 274 | 318 | |
| 275 | 319 | // Pinned proof: capture a snapshot session per core so the caller can read |
| 276 | 320 | // the exact bytes the proof covers. drive.version must not move across the |
| 277 | 321 | // capture — if it does, the two snapshots belong to different drive states |
| 278 | 322 | // and the proof would attest to a version that never existed. |
| 279 | const versionBefore = drive ? drive.version : undefined | |
| 280 | 323 | const snapshots = [] |
| 281 | 324 | try { |
| 282 | 325 | for (const core of [metaCore, blobCore]) { |
| 283 | if (core && typeof core.snapshot === 'function') { | |
| 284 | const snap = core.snapshot() | |
| 285 | snapshots.push(snap) | |
| 286 | if (snap && typeof snap.ready === 'function') await snap.ready() | |
| 287 | } else { | |
| 288 | snapshots.push(null) | |
| 289 | } | |
| 326 | if (!core || typeof core.snapshot !== 'function') throw new Error('DRIVE_SNAPSHOT_UNAVAILABLE') | |
| 327 | const snap = core.snapshot({ wait: false }) | |
| 328 | snapshots.push(snap) | |
| 329 | if (snap && typeof snap.ready === 'function') await snap.ready() | |
| 330 | } | |
| 331 | const pinnedMeta = readStableCoreState(snapshots[0]) | |
| 332 | const pinnedBlob = readStableCoreState(snapshots[1]) | |
| 333 | if (pinnedMeta.length !== metaState.length || pinnedMeta.byteLength !== metaState.byteLength || pinnedMeta.fork !== metaState.fork || | |
| 334 | pinnedBlob.length !== blobState.length || pinnedBlob.byteLength !== blobState.byteLength || pinnedBlob.fork !== blobState.fork) { | |
| 335 | throw new Error('DRIVE_SIZE_CHANGED_BEFORE_SNAPSHOT') | |
| 290 | 336 | } |
| 291 | const versionAfter = drive ? drive.version : undefined | |
| 292 | if (versionBefore !== versionAfter) { | |
| 337 | const versionAfter = readDriveVersion() | |
| 338 | if (versionAfter !== driveVersionBeforeProof) { | |
| 293 | 339 | const err = new Error( |
| 294 | `DRIVE_VERSION_CHANGED_DURING_PROOF: ${versionBefore} -> ${versionAfter}` | |
| 340 | `DRIVE_VERSION_CHANGED_DURING_PROOF: ${driveVersionBeforeProof} -> ${versionAfter}` | |
| 295 | 341 | ) |
| 296 | 342 | err.code = 'DRIVE_VERSION_CHANGED_DURING_PROOF' |
| 297 | 343 | throw err |
| 298 | 344 | } |
| 299 | result.driveVersion = versionAfter | |
| 300 | 345 | result.metaCoreSnapshot = snapshots[0] |
| 301 | 346 | result.blobCoreSnapshot = snapshots[1] |
| 302 | 347 | return result |
| … | … | @@ -322,13 +367,18 @@ const STABLE_READ_ATTEMPTS = 8 |
| 322 | 367 | * instead of spinning. |
| 323 | 368 | */ |
| 324 | 369 | function readStableCoreState (core) { |
| 325 | if (!core) return { length: 0, byteLength: 0, fork: 0 } | |
| 370 | if (!core) throw new Error('proved core unavailable') | |
| 326 | 371 | for (let attempt = 0; attempt < STABLE_READ_ATTEMPTS; attempt++) { |
| 327 | const lengthBefore = core.length || 0 | |
| 328 | const byteLength = core.byteLength || 0 | |
| 329 | const lengthAfter = core.length || 0 | |
| 330 | if (lengthBefore === lengthAfter) { | |
| 331 | return { length: lengthAfter, byteLength, fork: core.fork || 0 } | |
| 372 | const forkBefore = Number(core.fork ?? 0) | |
| 373 | const lengthBefore = Number(core.length) | |
| 374 | const byteLengthBefore = Number(core.byteLength) | |
| 375 | const lengthAfter = Number(core.length) | |
| 376 | const byteLengthAfter = Number(core.byteLength) | |
| 377 | const forkAfter = Number(core.fork ?? 0) | |
| 378 | if (Number.isSafeInteger(forkBefore) && forkBefore >= 0 && forkBefore === forkAfter && | |
| 379 | Number.isSafeInteger(lengthBefore) && lengthBefore >= 0 && lengthBefore === lengthAfter && | |
| 380 | Number.isSafeInteger(byteLengthBefore) && byteLengthBefore >= 0 && byteLengthBefore === byteLengthAfter) { | |
| 381 | return { length: lengthAfter, byteLength: byteLengthAfter, fork: forkAfter } | |
| 332 | 382 | } |
| 333 | 383 | } |
| 334 | 384 | const err = new Error('CORE_STATE_UNSTABLE: core kept appending across size reads') |
| … | … | @@ -997,8 +997,8 @@ export class RelayNode extends EventEmitter { |
| 997 | 997 | if (!path) return |
| 998 | 998 | try { |
| 999 | 999 | const data = JSON.parse(await readFile(path, 'utf8')) |
| 1000 | if (data && data.enabled === true && Array.isArray(data.plugins)) { | |
| 1001 | this.config.enableServices = true | |
| 1000 | if (data && typeof data.enabled === 'boolean' && Array.isArray(data.plugins)) { | |
| 1001 | this.config.enableServices = data.enabled | |
| 1002 | 1002 | // expandServiceDeps pulls in bundle support services (e.g. poker -> vrf+ |
| 1003 | 1003 | // arbitration+zk) and filters to builtins, in case services.json was |
| 1004 | 1004 | // hand-edited to list a bundle key without its deps. |
| … | … | @@ -1040,6 +1040,7 @@ export class RelayNode extends EventEmitter { |
| 1040 | 1040 | } catch (err) { |
| 1041 | 1041 | try { await unlink(tmp) } catch (_) {} |
| 1042 | 1042 | this.emit('services-config-persist-error', err) |
| 1043 | throw err | |
| 1043 | 1044 | } |
| 1044 | 1045 | } |
| 1045 | 1046 | // Reflect in live config; the registry itself (re)starts on restart. |
| … | … | @@ -25,6 +25,10 @@ |
| 25 | 25 | "./*": "./*" |
| 26 | 26 | }, |
| 27 | 27 | "imports": { |
| 28 | "fs": { | |
| 29 | "bare": "bare-fs", | |
| 30 | "default": "fs" | |
| 31 | }, | |
| 28 | 32 | "events": { |
| 29 | 33 | "bare": "bare-events", |
| 30 | 34 | "default": "events" |
| … | … | @@ -49,6 +49,7 @@ const DEFAULT_CONTROL_HOST = '127.0.0.1' |
| 49 | 49 | const DEFAULT_CONTROL_PORT = 9051 |
| 50 | 50 | const TOR_CHECK_TIMEOUT = 5000 |
| 51 | 51 | const KEY_BLOB_RE = /^ED25519-V3:[A-Za-z0-9+/=]+$/ |
| 52 | const RESTRICTED_PROBE_REJECTION_RE = /(?:HostUnreachable|NotAllowed|onion service descriptor (?:can not|cannot) be found)/i | |
| 52 | 53 | |
| 53 | 54 | export const TorHealth = Object.freeze({ |
| 54 | 55 | DISABLED: 'disabled', |
| … | … | @@ -215,6 +216,7 @@ export class TorTransport extends EventEmitter { |
| 215 | 216 | this._probeTimer = null |
| 216 | 217 | this._probeFails = 0 |
| 217 | 218 | this._descriptorUploads = 0 |
| 219 | this.negativeProbe = null | |
| 218 | 220 | this._controlFactory = opts._controlFactory || null // test injection |
| 219 | 221 | this._probeConnectionFactory = opts._probeConnectionFactory || |
| 220 | 222 | ((options) => SocksClient.createConnection(options)) |
| … | … | @@ -298,6 +300,7 @@ export class TorTransport extends EventEmitter { |
| 298 | 300 | } |
| 299 | 301 | this._descriptorUploads = 0 |
| 300 | 302 | this._probeFails = 0 |
| 303 | this.negativeProbe = null | |
| 301 | 304 | this._setHealth(TorHealth.DISABLED) |
| 302 | 305 | throw err |
| 303 | 306 | } |
| … | … | @@ -317,6 +320,8 @@ export class TorTransport extends EventEmitter { |
| 317 | 320 | if (this._control) { this._control.destroy(); this._control = null } |
| 318 | 321 | if (this.peerListener) await this.peerListener.stop() |
| 319 | 322 | this._descriptorUploads = 0 |
| 323 | this._probeFails = 0 | |
| 324 | this.negativeProbe = null | |
| 320 | 325 | this.emit('stopped') |
| 321 | 326 | } |
| 322 | 327 | |
| … | … | @@ -363,6 +368,8 @@ export class TorTransport extends EventEmitter { |
| 363 | 368 | let deleted = false |
| 364 | 369 | let added = false |
| 365 | 370 | this._descriptorUploads = 0 |
| 371 | this._probeFails = 0 | |
| 372 | this.negativeProbe = null | |
| 366 | 373 | try { |
| 367 | 374 | await this._control.cmd('DEL_ONION ' + this.serviceId) |
| 368 | 375 | deleted = true |
| … | … | @@ -386,6 +393,8 @@ export class TorTransport extends EventEmitter { |
| 386 | 393 | throw new Error('onion address changed while restoring the previous roster') |
| 387 | 394 | } |
| 388 | 395 | this._descriptorUploads = 0 |
| 396 | this._probeFails = 0 | |
| 397 | this.negativeProbe = null | |
| 389 | 398 | this._setHealth(TorHealth.KEY_LOADED) |
| 390 | 399 | } catch (rollbackErr) { |
| 391 | 400 | err.rollbackError = rollbackErr |
| … | … | @@ -567,6 +576,9 @@ export class TorTransport extends EventEmitter { |
| 567 | 576 | this._serviceActive = false |
| 568 | 577 | const blob = await this._loadOrThrowKey() |
| 569 | 578 | await this._addOnion(blob, nextKeys) |
| 579 | this._descriptorUploads = 0 | |
| 580 | this._probeFails = 0 | |
| 581 | this.negativeProbe = null | |
| 570 | 582 | this._setHealth(TorHealth.KEY_LOADED) |
| 571 | 583 | } catch (err) { |
| 572 | 584 | this._setHealth(TorHealth.DEGRADED) |
| … | … | @@ -613,11 +625,33 @@ export class TorTransport extends EventEmitter { |
| 613 | 625 | this._probeTimer.unref && this._probeTimer.unref() |
| 614 | 626 | } |
| 615 | 627 | |
| 616 | /** Self-probe: SOCKS-connect back to our own onion through the network. */ | |
| 628 | _healthProbeVport () { | |
| 629 | if (this.healthOpts.probeVport === false) return null | |
| 630 | const configured = Number(this.healthOpts.probeVport) | |
| 631 | if (Number.isSafeInteger(configured) && configured > 0 && configured <= 65535) return configured | |
| 632 | const effective = this._effectiveVports() | |
| 633 | const preferred = [this.readVport, this.peerVport] | |
| 634 | .find((port) => Number.isSafeInteger(port) && effective.some((entry) => entry.vport === port)) | |
| 635 | if (preferred) return preferred | |
| 636 | const fallback = effective.find((entry) => Number.isSafeInteger(entry.vport) && entry.vport > 0 && entry.vport <= 65535) | |
| 637 | return fallback ? fallback.vport : null | |
| 638 | } | |
| 639 | ||
| 640 | /** | |
| 641 | * Probe our own onion through the Tor network. | |
| 642 | * | |
| 643 | * Public services require a successful connection. Restricted services | |
| 644 | * deliberately probe without a client credential: a Tor authorization | |
| 645 | * rejection is success, while an accepted connection proves that the | |
| 646 | * descriptor became public and degrades the endpoint immediately. | |
| 647 | */ | |
| 617 | 648 | async _probeNow () { |
| 618 | const vport = this.healthOpts.probeVport | |
| 649 | const vport = this._healthProbeVport() | |
| 619 | 650 | if (!vport || !this.onionAddress) { |
| 620 | // No probe surface configured: descriptor uploads are the readiness signal. | |
| 651 | // Explicit probe opt-out or no exposed port: descriptor uploads are the | |
| 652 | // only available signal. Restricted endpoints never claim a negative | |
| 653 | // authorization probe in this mode. | |
| 654 | this.negativeProbe = null | |
| 621 | 655 | if (this.health === TorHealth.DESCRIPTOR_UPLOADED) this._setHealth(TorHealth.READY) |
| 622 | 656 | return |
| 623 | 657 | } |
| … | … | @@ -627,6 +661,7 @@ export class TorTransport extends EventEmitter { |
| 627 | 661 | this.health !== TorHealth.READY && |
| 628 | 662 | this.health !== TorHealth.DEGRADED |
| 629 | 663 | ) return |
| 664 | const restricted = this.isRestrictedDiscoveryActive() | |
| 630 | 665 | try { |
| 631 | 666 | const { socket } = await this._probeConnectionFactory({ |
| 632 | 667 | proxy: { host: this.socksHost, port: this.socksPort, type: 5 }, |
| … | … | @@ -635,9 +670,23 @@ export class TorTransport extends EventEmitter { |
| 635 | 670 | timeout: 60000 |
| 636 | 671 | }) |
| 637 | 672 | socket.destroy() |
| 673 | if (restricted) { | |
| 674 | this.negativeProbe = false | |
| 675 | this._probeFails = this.healthOpts.probeFailLimit | |
| 676 | this._setHealth(TorHealth.DEGRADED) | |
| 677 | return | |
| 678 | } | |
| 679 | this.negativeProbe = null | |
| 638 | 680 | this._probeFails = 0 |
| 639 | 681 | this._setHealth(TorHealth.READY) |
| 640 | } catch { | |
| 682 | } catch (err) { | |
| 683 | if (restricted && RESTRICTED_PROBE_REJECTION_RE.test(String(err && err.message ? err.message : err))) { | |
| 684 | this.negativeProbe = true | |
| 685 | this._probeFails = 0 | |
| 686 | this._setHealth(TorHealth.READY) | |
| 687 | return | |
| 688 | } | |
| 689 | if (restricted) this.negativeProbe = null | |
| 641 | 690 | this._probeFails++ |
| 642 | 691 | if (this._probeFails >= this.healthOpts.probeFailLimit) this._setHealth(TorHealth.DEGRADED) |
| 643 | 692 | } |
| … | … | @@ -810,6 +859,7 @@ export class TorTransport extends EventEmitter { |
| 810 | 859 | pow: this.pow ? !!this.pow.enabled : false, |
| 811 | 860 | persistent: !!this.keyFile, |
| 812 | 861 | descriptorUploads: this._descriptorUploads, |
| 862 | negativeProbe: this.negativeProbe, | |
| 813 | 863 | activeConnections: this._connections.size |
| 814 | 864 | } |
| 815 | 865 | } |
| … | … | @@ -181,13 +181,34 @@ class JournalStorageController { |
| 181 | 181 | for (const core of this.cores) { |
| 182 | 182 | if (!core || typeof core.info !== 'function') throw new Error('OutboxLog exact core storage info unavailable') |
| 183 | 183 | const info = await core.info({ storage: true }) |
| 184 | if (!info || !info.storage) throw new Error('OutboxLog exact core storage info unavailable') | |
| 185 | for (const value of Object.values(info.storage)) { | |
| 186 | const bytes = Number(value) | |
| 187 | if (!Number.isSafeInteger(bytes) || bytes < 0) throw new Error('OutboxLog exact core storage info invalid') | |
| 188 | actual += bytes | |
| 189 | if (!Number.isSafeInteger(actual)) throw new Error('OutboxLog exact core storage exceeds safe integer range') | |
| 184 | if (!info) throw new Error('OutboxLog exact core storage info unavailable') | |
| 185 | if (info.storage) { | |
| 186 | for (const value of Object.values(info.storage)) { | |
| 187 | const bytes = Number(value) | |
| 188 | if (!Number.isSafeInteger(bytes) || bytes < 0) throw new Error('OutboxLog exact core storage info invalid') | |
| 189 | actual += bytes | |
| 190 | if (!Number.isSafeInteger(actual)) throw new Error('OutboxLog exact core storage exceeds safe integer range') | |
| 191 | } | |
| 192 | continue | |
| 190 | 193 | } |
| 194 | ||
| 195 | // Corestore 7 multiplexes cores into one shared database, so Hypercore | |
| 196 | // cannot attribute allocated files per core and reports storage:null. | |
| 197 | // Use the signed logical byteLength for this core in that layout. This | |
| 198 | // never releases capacity: the journal's monotonic durable ledger keeps | |
| 199 | // the conservative per-operation debt, the full aggregate commitment | |
| 200 | // remains charged, and StorageAdmissionAuthority independently measures | |
| 201 | // and enforces the complete Corestore tree's allocated bytes. | |
| 202 | const before = Number(core.byteLength ?? info.byteLength) | |
| 203 | const lengthBefore = Number(core.length ?? info.length) | |
| 204 | const after = Number(core.byteLength ?? info.byteLength) | |
| 205 | const lengthAfter = Number(core.length ?? info.length) | |
| 206 | if (!Number.isSafeInteger(before) || before < 0 || before !== after || | |
| 207 | !Number.isSafeInteger(lengthBefore) || lengthBefore < 0 || lengthBefore !== lengthAfter) { | |
| 208 | throw new Error('OutboxLog exact core storage info invalid') | |
| 209 | } | |
| 210 | actual += after | |
| 211 | if (!Number.isSafeInteger(actual)) throw new Error('OutboxLog exact core storage exceeds safe integer range') | |
| 191 | 212 | } |
| 192 | 213 | return actual |
| 193 | 214 | } |
| … | … | @@ -769,6 +790,20 @@ async function readyCore (core) { |
| 769 | 790 | } |
| 770 | 791 | |
| 771 | 792 | async function tryOpenExistingCore (store, opts) { |
| 793 | // Corestore's `get({ createIfMissing: false })` still allocates a Hypercore | |
| 794 | // session before its async existence check settles. When the core is absent, | |
| 795 | // ready() and close() both reject with STORAGE_EMPTY and that rejected | |
| 796 | // session can make the root Corestore's later close reject as well. Consult | |
| 797 | // Corestore's durable alias/index first so a normal first boot creates no | |
| 798 | // doomed session. | |
| 799 | if (opts?.name && store?.storage && typeof store.storage.getAlias === 'function') { | |
| 800 | if (typeof store.ready === 'function') await store.ready() | |
| 801 | const discoveryKey = await store.storage.getAlias({ name: opts.name, namespace: store.ns }) | |
| 802 | if (!discoveryKey) return null | |
| 803 | } else if (opts?.discoveryKey && store?.storage && typeof store.storage.hasCore === 'function') { | |
| 804 | if (typeof store.ready === 'function') await store.ready() | |
| 805 | if (!await store.storage.hasCore(opts.discoveryKey)) return null | |
| 806 | } | |
| 772 | 807 | const core = store.get({ ...opts, createIfMissing: false }) |
| 773 | 808 | try { |
| 774 | 809 | await readyCore(core) |
| … | … | @@ -102,16 +102,24 @@ export class VRFService extends ServiceProvider { |
| 102 | 102 | // Allow an explicit override seed (mainly for tests / fixed deployments). |
| 103 | 103 | this._explicitSeed = opts.seed ? toBytes(opts.seed, 'seed') : null |
| 104 | 104 | |
| 105 | // Beacon is opt-in. opts.beacon: { enabled, intervalMs, domain, retain } | |
| 106 | const b = opts.beacon || {} | |
| 105 | // Beacon is opt-in. opts.beacon: { enabled, intervalMs, domain, retain }. | |
| 106 | // Node's generic PluginLoader constructs every provider before it has the | |
| 107 | // service context, so start() also accepts context.config.vrfBeacon when | |
| 108 | // no explicit constructor descriptor was supplied. | |
| 109 | this._beaconConfigExplicit = Object.prototype.hasOwnProperty.call(opts, 'beacon') | |
| 110 | this._configureBeacon(opts.beacon) | |
| 111 | this.beacon = null | |
| 112 | this._beaconTimer = null | |
| 113 | } | |
| 114 | ||
| 115 | _configureBeacon (value) { | |
| 116 | const b = value && typeof value === 'object' && !Array.isArray(value) ? value : {} | |
| 107 | 117 | this.beaconConfig = { |
| 108 | 118 | enabled: b.enabled === true, |
| 109 | 119 | intervalMs: Math.max(MIN_BEACON_INTERVAL_MS, b.intervalMs || DEFAULT_BEACON_INTERVAL_MS), |
| 110 | 120 | domain: b.domain || DEFAULT_DOMAIN, |
| 111 | 121 | retain: b.retain || DEFAULT_RETAIN |
| 112 | 122 | } |
| 113 | this.beacon = null | |
| 114 | this._beaconTimer = null | |
| 115 | 123 | } |
| 116 | 124 | |
| 117 | 125 | manifest () { |
| … | … | @@ -129,6 +137,9 @@ export class VRFService extends ServiceProvider { |
| 129 | 137 | |
| 130 | 138 | async start (context = {}) { |
| 131 | 139 | this.node = context.node || null |
| 140 | if (!this._beaconConfigExplicit && context.config && context.config.vrfBeacon) { | |
| 141 | this._configureBeacon(context.config.vrfBeacon) | |
| 142 | } | |
| 132 | 143 | |
| 133 | 144 | // Resolve the source key material, in priority order. |
| 134 | 145 | let nodeSeed = null |
| … | … | @@ -205,11 +205,14 @@ const fleetStatusScript = readText(hiverelayRoot, 'fleet', 'fleet-status.sh') |
| 205 | 205 | const fleetUpdaterScript = readText(hiverelayRoot, 'fleet', 'updater.sh') |
| 206 | 206 | const deployVpsScript = readText(hiverelayRoot, 'scripts', 'deploy-vps.sh') |
| 207 | 207 | const relayJanitorScript = readText(hiverelayRoot, 'scripts', 'relay-janitor.js') |
| 208 | const rootDockerCompose = readText(hiverelayRoot, 'docker-compose.yml') | |
| 209 | const systemdService = readText(hiverelayRoot, 'hiverelay.service') | |
| 208 | 210 | const cliIndex = readText(hiverelayRoot, 'packages', 'core', 'cli', 'index.js') |
| 209 | 211 | const cliCatalog = readText(hiverelayRoot, 'packages', 'core', 'cli', 'catalog.js') |
| 210 | 212 | const cliSetup = readText(hiverelayRoot, 'packages', 'core', 'cli', 'setup.js') |
| 211 | 213 | const cliManage = readText(hiverelayRoot, 'packages', 'core', 'cli', 'manage.js') |
| 212 | 214 | const defaultConfig = readText(hiverelayRoot, 'packages', 'core', 'config', 'default.js') |
| 215 | const configLoader = readText(hiverelayRoot, 'packages', 'core', 'config', 'loader.js') | |
| 213 | 216 | const constantsCore = readText(hiverelayRoot, 'packages', 'core', 'core', 'constants.js') |
| 214 | 217 | const blindsparkDashboard = readText(hiverelayRoot, 'dashboard', 'blindspark.html') |
| 215 | 218 | const fullDashboard = readText(hiverelayRoot, 'dashboard', 'index.html') |
| … | … | @@ -322,6 +325,7 @@ const pokerReadme = readText(hiverelayRoot, 'packages', 'services', 'builtin', ' |
| 322 | 325 | const servicesIndex = readText(hiverelayRoot, 'packages', 'services', 'index.js') |
| 323 | 326 | const servicesReadme = readText(hiverelayRoot, 'packages', 'services', 'README.md') |
| 324 | 327 | const notifyService = readText(hiverelayRoot, 'packages', 'services', 'builtin', 'notify-service.js') |
| 328 | const vrfService = readText(hiverelayRoot, 'packages', 'services', 'builtin', 'vrf-service.js') | |
| 325 | 329 | const outboxLogApp = readText(hiverelayRoot, 'packages', 'services', 'builtin', 'outboxlog', 'index.js') |
| 326 | 330 | const outboxLogBlindSeal = readText(hiverelayRoot, 'packages', 'services', 'builtin', 'outboxlog', 'blind-seal.js') |
| 327 | 331 | const outboxLogEngine = readText(hiverelayRoot, 'packages', 'services', 'builtin', 'outboxlog', 'outbox-log.js') |
| … | … | @@ -509,9 +513,11 @@ const standaloneGatewayServerTest = readText(hiverelayRoot, 'test', 'unit', 'gat |
| 509 | 513 | const pokerHttpAdapterTest = readText(hiverelayRoot, 'test', 'unit', 'poker-http-adapter.test.js') |
| 510 | 514 | const pokerWsAdapterTest = readText(hiverelayRoot, 'test', 'unit', 'poker-ws-adapter.test.js') |
| 511 | 515 | const outboxLogTest = readText(hiverelayRoot, 'test', 'unit', 'outboxlog.test.js') |
| 516 | const outboxLogAtomicCommitTest = readText(hiverelayRoot, 'test', 'unit', 'outboxlog-atomic-commit.test.js') | |
| 512 | 517 | const outboxLogBlindSealTest = readText(hiverelayRoot, 'test', 'unit', 'outboxlog-blind-seal.test.js') |
| 513 | 518 | const outboxLogHttpAdapterTest = readText(hiverelayRoot, 'test', 'unit', 'outboxlog-http-adapter.test.js') |
| 514 | 519 | const notifyServiceTest = readText(hiverelayRoot, 'test', 'unit', 'notify-service.test.js') |
| 520 | const apiVrfTest = readText(hiverelayRoot, 'test', 'unit', 'api-vrf.test.js') | |
| 515 | 521 | const docsDashboard = readText(hiverelayRoot, 'dashboard', 'docs.html') |
| 516 | 522 | const catalogDashboard = readText(hiverelayRoot, 'dashboard', 'catalog.html') |
| 517 | 523 | const networkDashboard = readText(hiverelayRoot, 'dashboard', 'network.html') |
| … | … | @@ -1464,7 +1470,7 @@ if ( |
| 1464 | 1470 | relayApiOverview.includes('unknown overview route') && |
| 1465 | 1471 | relayApiOverview.includes('export function overviewHealth') && |
| 1466 | 1472 | relayApiOverview.includes('node.getStats({ includeSecrets: authed === true })') && |
| 1467 | relayApiOverview.includes('sanitizeGatewayStats(gateway.getStats())') && | |
| 1473 | relayApiOverview.includes('sanitizeGatewayStats(gw.getStats())') && | |
| 1468 | 1474 | relayApiOverview.includes('export function overviewStorage') && |
| 1469 | 1475 | relayApiOverview.includes('export function overviewServed') && |
| 1470 | 1476 | relayApiOverview.includes('export function overviewRelay') && |
| … | … | @@ -1885,7 +1891,7 @@ if ( |
| 1885 | 1891 | outboxLogEngine.includes('const head = group.rows.get(headKey)') && |
| 1886 | 1892 | outboxLogApp.includes('export class OutboxLogApp extends ServiceProvider') && |
| 1887 | 1893 | outboxLogApp.includes("name: 'outboxlog'") && |
| 1888 | outboxLogApp.includes("capabilities: ['outboxlog.sync', 'outboxlog.directory', 'outboxlog.events', 'outboxlog.namespaces']") && | |
| 1894 | outboxLogApp.includes("capabilities: ['create', 'append', 'get', 'range', 'directory', 'outboxlog.sync', 'outboxlog.commit', 'outboxlog.directory', 'outboxlog.events', 'outboxlog.namespaces']") && | |
| 1889 | 1895 | outboxLogTest.includes('runWireConformance') && |
| 1890 | 1896 | outboxLogTest.includes("t.is(runWireConformance(log.sync, { label: 'outboxlog' }), 26)") && |
| 1891 | 1897 | outboxLogTest.includes('default verifier accepts same-writer opaque records') && |
| … | … | @@ -1906,7 +1912,10 @@ if ( |
| 1906 | 1912 | outboxLogEngine.includes('BLIND_FORBIDDEN_FIELDS') && |
| 1907 | 1913 | outboxLogEngine.includes('OUTBOXLOG_BLIND_SEAL_VERSION') && |
| 1908 | 1914 | outboxLogEngine.includes('createOutboxBlindSealedBody') && |
| 1909 | outboxLogEngine.includes('isOutboxBlindRecord(data) && !hasBlindForbiddenField(data)') && | |
| 1915 | outboxLogEngine.includes('function namespaceRecordAllowed (namespaceInfo, type, data)') && | |
| 1916 | outboxLogEngine.includes('if (hasBlindForbiddenField(data)) return false') && | |
| 1917 | outboxLogEngine.includes('if (isOutboxBlindRecord(data)) return true') && | |
| 1918 | outboxLogEngine.includes('return isBlindControlRecord(type, data)') && | |
| 1910 | 1919 | outboxLogEngine.includes("const BLIND_BODY_FIELDS = new Set(['sealed'])") && |
| 1911 | 1920 | outboxLogEngine.includes('function normalizeBlindSealedBody') && |
| 1912 | 1921 | outboxLogEngine.includes('function normalizeBlindSeal') && |
| … | … | @@ -1917,7 +1926,7 @@ if ( |
| 1917 | 1926 | outboxLogApp.includes('isOutboxBlindRecord') && |
| 1918 | 1927 | servicesIndex.includes('createOutboxBlindSealedBody') && |
| 1919 | 1928 | servicesIndex.includes('isOutboxBlindSealedBody') && |
| 1920 | outboxLogApp.includes("capabilities: ['outboxlog.sync', 'outboxlog.directory', 'outboxlog.events', 'outboxlog.namespaces']") && | |
| 1929 | outboxLogApp.includes("capabilities: ['create', 'append', 'get', 'range', 'directory', 'outboxlog.sync', 'outboxlog.commit', 'outboxlog.directory', 'outboxlog.events', 'outboxlog.namespaces']") && | |
| 1921 | 1930 | outboxLogApp.includes('namespaces ()') && |
| 1922 | 1931 | outboxLogHttpAdapter.includes('sync.create(body.body.appId, { namespace: body.body.namespace })') && |
| 1923 | 1932 | outboxLogTest.includes('namespace registry accepts non-Peerit app records and enforces caps') && |
| … | … | @@ -1926,6 +1935,7 @@ if ( |
| 1926 | 1935 | outboxLogTest.includes('opaque-but-unsealed') && |
| 1927 | 1936 | outboxLogTest.includes('version: 0') && |
| 1928 | 1937 | outboxLogTest.includes('namespace config admits Poked-style apps without a relay fork') && |
| 1938 | outboxLogAtomicCommitTest.includes('blind namespace admits only sealed data and exact control rows') && | |
| 1929 | 1939 | servicesReadme.includes('It is namespace-driven rather than Peerit-specific') && |
| 1930 | 1940 | servicesReadme.includes('`outboxlog.namespaces` such as `peerit`, `poked`, or `privchat`') && |
| 1931 | 1941 | servicesReadme.includes('`createOutboxBlindSealedBody()`') && |
| … | … | @@ -2089,7 +2099,10 @@ if ( |
| 2089 | 2099 | outboxLogHttpAdapter.includes("return respond(res, 500, { error: 'outboxlog route failed' })") && |
| 2090 | 2100 | outboxLogHttpAdapter.includes('sseMaxPerIp') && |
| 2091 | 2101 | outboxLogHttpAdapter.includes('sseMaxTotal') && |
| 2092 | outboxLogHttpAdapter.includes('overLimit(ip, ctx.rateLimit || DEFAULT_RATE_LIMIT, state)') && | |
| 2102 | outboxLogHttpAdapter.includes('const rate = consumeRateLimit(ip, rateLimit, state)') && | |
| 2103 | outboxLogHttpAdapter.includes('OUTBOXLOG_HTTP_MAX_RATE_BUCKETS') && | |
| 2104 | outboxLogHttpAdapter.includes('function claimSseSlot (ip, state, ctx)') && | |
| 2105 | outboxLogHttpAdapter.includes('function writeSseData (res, event, name = null)') && | |
| 2093 | 2106 | outboxLogHttpAdapter.includes("res.setHeader('Content-Type', 'text/event-stream')") && |
| 2094 | 2107 | outboxLogHttpAdapter.includes("res.write('data: ' + JSON.stringify(event) + '\\n\\n')") && |
| 2095 | 2108 | outboxLogSwarmHub.includes('export function createOutboxSwarmHub') && |
| … | … | @@ -2105,8 +2118,11 @@ if ( |
| 2105 | 2118 | outboxLogHttpAdapterTest.includes('serves Peerit sync routes and directory') && |
| 2106 | 2119 | outboxLogHttpAdapterTest.includes('hardens JSON body parsing before sync mutation') && |
| 2107 | 2120 | outboxLogHttpAdapterTest.includes('rate limits by client IP') && |
| 2121 | outboxLogHttpAdapterTest.includes('rate bucket cardinality stays bounded under unique-IP churn') && | |
| 2122 | outboxLogHttpAdapterTest.includes('rejected requests do not consume rate-limit budget') && | |
| 2108 | 2123 | outboxLogHttpAdapterTest.includes('tokenized swarm SSE receives peer messages') && |
| 2109 | outboxLogHttpAdapterTest.includes('caps open SSE streams per IP') | |
| 2124 | outboxLogHttpAdapterTest.includes('caps open SSE streams per IP') && | |
| 2125 | outboxLogHttpAdapterTest.includes('sync event stream honors backpressure and resumes on drain') | |
| 2110 | 2126 | ) { |
| 2111 | 2127 | pass('OutboxLog HTTP adapter covers Peerit token, sync, directory, and swarm SSE routes with hardened bounds before mount wiring') |
| 2112 | 2128 | } else { |
| … | … | @@ -2229,10 +2245,12 @@ if ( |
| 2229 | 2245 | outboxLogHypercoreJournal.includes("OUTBOXLOG_PARTITIONED_JOURNAL_INDEX_NAME = 'outboxlog/outboxes/index'") && |
| 2230 | 2246 | outboxLogHypercoreJournal.includes("OUTBOXLOG_OUTBOX_CORE_PREFIX = 'outboxlog/outboxes/'") && |
| 2231 | 2247 | outboxLogHypercoreJournal.includes('export async function createPartitionedHypercoreOutboxJournal') && |
| 2232 | outboxLogHypercoreJournal.includes('store.get({ name: meta.coreName })') && | |
| 2248 | outboxLogHypercoreJournal.includes('tryOpenExistingCore(store, { name: meta.coreName })') && | |
| 2249 | outboxLogHypercoreJournal.includes('this.store.get({ name: outbox.name })') && | |
| 2233 | 2250 | outboxLogHypercoreJournal.includes('coreNameForOutbox(entry.appId, this.coreNamePrefix)') && |
| 2234 | 2251 | outboxLogHypercoreJournal.includes('async seedCores') && |
| 2235 | outboxLogHypercoreJournal.includes('await seeder.seedCore(coreKey)') && | |
| 2252 | outboxLogHypercoreJournal.includes('await seeder.announceAuthorityOwnedCore(') && | |
| 2253 | outboxLogHypercoreJournal.includes('await seeder.withdrawAuthorityOwnedCore(coreKey, this.storageController.ownershipHandoff)') && | |
| 2236 | 2254 | outboxLogApp.includes("outboxlog.journal === 'hypercore-outboxes'") && |
| 2237 | 2255 | outboxLogApp.includes('seedPersistenceCores') && |
| 2238 | 2256 | outboxLogTest.includes('partitioned Corestore journal mirrors accepted rows into per-outbox cores') && |
| … | … | @@ -2249,10 +2267,12 @@ if ( |
| 2249 | 2267 | outboxLogSeederRuntimeTest.includes('outboxlog app runtime seeder pins partitioned Hypercore cores by key') && |
| 2250 | 2268 | outboxLogSeederRuntimeTest.includes("journal: 'hypercore-outboxes'") && |
| 2251 | 2269 | outboxLogSeederRuntimeTest.includes('app.seedPersistenceCores()') && |
| 2252 | outboxLogSeederRuntimeTest.includes('get ({ name, key })') && | |
| 2270 | outboxLogSeederRuntimeTest.includes('get ({ name, key, createIfMissing = true })') && | |
| 2253 | 2271 | outboxLogSeederRuntimeTest.includes('keyGets.push(keyHex)') && |
| 2254 | outboxLogSeederRuntimeTest.includes('seeder.cores.get(info.index.coreKey).core') && | |
| 2255 | outboxLogSeederRuntimeTest.includes("seeder.on('seeding-core'") && | |
| 2272 | outboxLogSeederRuntimeTest.includes("seeder.on('seeding-owned-core'") && | |
| 2273 | outboxLogSeederRuntimeTest.includes("t.alike(store.keyGets, [], 'writer-owned cores are never re-opened as remote seed pins')") && | |
| 2274 | outboxLogSeederRuntimeTest.includes('seeder._ownedAnnouncements.get(info.index.coreKey).sourceCore') && | |
| 2275 | outboxLogSeederRuntimeTest.includes('same-process stop and restart withdraws old sessions before rebinding a new handoff') && | |
| 2256 | 2276 | outboxLogSeederRuntimeTest.includes('swarm.joins.length') && |
| 2257 | 2277 | outboxLogSeederRuntimeTest.includes('OUTBOXLOG_PARTITIONED_JOURNAL_INDEX_NAME') && |
| 2258 | 2278 | outboxLogSeederRuntimeTest.includes('OUTBOXLOG_OUTBOX_CORE_PREFIX') |
| … | … | @@ -2834,8 +2854,9 @@ if ( |
| 2834 | 2854 | relayApi.includes('const cors = buildCorsDecision(this.corsOrigins, requestOrigin, requestPath)') && |
| 2835 | 2855 | relayApi.includes('if (cors.varyOrigin)') && |
| 2836 | 2856 | relayApi.includes("appendVaryHeader(res, 'Origin')") && |
| 2837 | relayApi.includes("res.setHeader('Access-Control-Allow-Origin', cors.allowedOrigin)") && | |
| 2838 | relayApi.includes('if (cors.preflightDenied)') && | |
| 2857 | relayApi.includes('const publicOutboxLog = isOutboxLogHttpRoute(requestPath)') && | |
| 2858 | relayApi.includes("res.setHeader('Access-Control-Allow-Origin', publicOutboxLog ? '*' : cors.allowedOrigin)") && | |
| 2859 | relayApi.includes('if (cors.preflightDenied && !publicOutboxLog)') && | |
| 2839 | 2860 | relayApi.includes('return getAllowedOrigin(this.corsOrigins, requestOrigin)') && |
| 2840 | 2861 | apiResponseTest.includes('appendVaryHeader preserves wildcard and avoids duplicates') && |
| 2841 | 2862 | apiCorsTest.includes('origin allowlist supports deny-by-default, strings, arrays, and wildcard') && |
| … | … | @@ -2843,7 +2864,8 @@ if ( |
| 2843 | 2864 | apiCorsTest.includes('public poker routes stay wildcard but usage telemetry stays managed') && |
| 2844 | 2865 | apiAuthTest.includes('configured CORS origins vary cached responses by Origin') && |
| 2845 | 2866 | apiAuthTest.includes('allowed dynamic CORS response varies by Origin') && |
| 2846 | apiAuthTest.includes('denied preflight also varies by Origin') | |
| 2867 | apiAuthTest.includes('denied preflight also varies by Origin') && | |
| 2868 | apiOutboxLogTest.includes('/api/outboxlog browser preflight exposes only the public blind-pipe headers') | |
| 2847 | 2869 | ) { |
| 2848 | 2870 | pass('management API CORS decisions are extracted and dynamic responses vary by Origin for cache safety') |
| 2849 | 2871 | } else { |
| … | … | @@ -3056,7 +3078,8 @@ if ( |
| 3056 | 3078 | } |
| 3057 | 3079 | |
| 3058 | 3080 | if ( |
| 3059 | relayApi.includes('import {\n checkApiRateLimit,') && | |
| 3081 | relayApi.includes('checkApiRateLimit,') && | |
| 3082 | relayApi.includes("} from './api-rate-limit.js'") && | |
| 3060 | 3083 | relayApi.includes('return checkApiRateLimit(this._rateLimits, ip)') && |
| 3061 | 3084 | relayApi.includes('return checkEndpointRateLimit(this._endpointRateLimits, ip, path)') && |
| 3062 | 3085 | relayApi.includes('return clientIpFromRequest(req, this.trustProxy)') && |
| … | … | @@ -3872,7 +3895,7 @@ if ( |
| 3872 | 3895 | relayApiConfigUpdate.includes("const result = validatePositiveNumber(body.maxRelayBandwidthMbps, 0.1, 100000, 'maxRelayBandwidthMbps')") && |
| 3873 | 3896 | relayApiModeTransport.includes("const result = validatePositiveInt(body.maxConnections, 0, 100000, 'maxConnections')") && |
| 3874 | 3897 | relayApiModeTransport.includes("const result = validatePositiveNumber(body.maxRelayBandwidthMbps, 0, 100000, 'maxRelayBandwidthMbps')") && |
| 3875 | relayApiModeTransport.includes("const result = validatePositiveInt(body.maxStorageBytes, 0, 10e12, 'maxStorageBytes')") && | |
| 3898 | relayApiModeTransport.includes("const result = validatePositiveInt(body.maxStorageBytes, 1, 10e12, 'maxStorageBytes')") && | |
| 3876 | 3899 | relayApi.includes('runDevicePairingManagementRouteAction') && |
| 3877 | 3900 | relayApi.includes('resolveDevicePairingManagementRoute,') && |
| 3878 | 3901 | relayApi.includes('const devicePairingRoute = resolveDevicePairingManagementRoute(req.method, path)') && |
| … | … | @@ -3906,9 +3929,9 @@ if ( |
| 3906 | 3929 | } |
| 3907 | 3930 | |
| 3908 | 3931 | if ( |
| 3909 | gatewayServer.includes('opts.gatewayPort ?? DEFAULT_GATEWAY_PORT') && | |
| 3932 | gatewayServer.includes('opts.gatewayPort ?? nodeConfig.gatewayPort ?? DEFAULT_GATEWAY_PORT') && | |
| 3910 | 3933 | gatewayServer.includes('_activeSockets = new Set()') && |
| 3911 | gatewayServer.includes("this.server.on('connection'") && | |
| 3934 | gatewayServer.includes("server.on('connection'") && | |
| 3912 | 3935 | gatewayServer.includes("import { buildGatewayCatalogPayload } from './api-catalog-read.js'") && |
| 3913 | 3936 | gatewayServer.includes("import { writeJson } from './api-response.js'") && |
| 3914 | 3937 | gatewayServer.includes('const result = buildGatewayCatalogPayload({ node: this.node, url })') && |
| … | … | @@ -3968,13 +3991,13 @@ if ( |
| 3968 | 3991 | gatewayServer.includes('closeAllConnections') && |
| 3969 | 3992 | gatewayServer.includes('socket.destroy()') && |
| 3970 | 3993 | gatewayServer.includes('const RATE_LIMIT_MAX_BUCKETS = 50_000') && |
| 3971 | gatewayServer.includes('this._maxRateLimitBuckets = positiveInteger(opts.maxRateLimitBuckets, RATE_LIMIT_MAX_BUCKETS)') && | |
| 3994 | gatewayServer.includes("this._maxRateLimitBuckets = strictPositiveInteger(opts.maxRateLimitBuckets, RATE_LIMIT_MAX_BUCKETS, 'maxRateLimitBuckets', 1_000_000)") && | |
| 3972 | 3995 | gatewayServer.includes('if (!entry && this._rateLimits.size >= this._maxRateLimitBuckets)') && |
| 3973 | 3996 | gatewayServer.includes('this._sweepRateLimits(now)') && |
| 3974 | 3997 | gatewayServer.includes('if (this._rateLimits.size >= this._maxRateLimitBuckets) return false') && |
| 3975 | 3998 | gatewayServer.includes('!Number.isFinite(entry.count)') && |
| 3976 | 3999 | gatewayServer.includes('!Number.isFinite(entry.resetAt)') && |
| 3977 | gatewayServer.includes('function positiveInteger') && | |
| 4000 | gatewayServer.includes('function strictPositiveInteger') && | |
| 3978 | 4001 | apiCatalogReadTest.includes('api catalog read: route helper maps exact public catalog reads') && |
| 3979 | 4002 | apiCatalogReadTest.includes('api catalog read: route payload helper dispatches catalog and legacy typed routes') && |
| 3980 | 4003 | apiCatalogReadTest.includes('unknown catalog read route') && |
| … | … | @@ -4015,6 +4038,11 @@ const gatewayStatsRouteEnd = relayApi.indexOf('// Index-layer query routes', gat |
| 4015 | 4038 | const gatewayStatsRouteBlock = gatewayStatsRouteStart === -1 |
| 4016 | 4039 | ? '' |
| 4017 | 4040 | : relayApi.slice(gatewayStatsRouteStart, gatewayStatsRouteEnd) |
| 4041 | const appLifecycleRetireDriveStart = appLifecycleCore.indexOf('async _retireDrive (owner)') | |
| 4042 | const appLifecycleRetireDriveEnd = appLifecycleCore.indexOf('async _registerPersistentDownloads', appLifecycleRetireDriveStart) | |
| 4043 | const appLifecycleRetireDriveBlock = appLifecycleRetireDriveStart === -1 | |
| 4044 | ? '' | |
| 4045 | : appLifecycleCore.slice(appLifecycleRetireDriveStart, appLifecycleRetireDriveEnd) | |
| 4018 | 4046 | if ( |
| 4019 | 4047 | relayApi.includes("import {\n buildGatewayStatsRoutePayload,\n resolveGatewayStatsRoute\n} from './api-gateway-stats.js'") && |
| 4020 | 4048 | relayApi.includes('const gatewayStatsRoute = resolveGatewayStatsRoute(req.method, path)') && |
| … | … | @@ -4024,7 +4052,7 @@ if ( |
| 4024 | 4052 | gatewayStatsRouteBlock.includes('route: gatewayStatsRoute') && |
| 4025 | 4053 | !gatewayStatsRouteBlock.includes('this._gateway.getStats()') && |
| 4026 | 4054 | relayApiOverview.includes("import { sanitizeGatewayStats } from './api-gateway-stats.js'") && |
| 4027 | relayApiOverview.includes("gateway: gateway && typeof gateway.getStats === 'function' ? sanitizeGatewayStats(gateway.getStats()) : null") && | |
| 4055 | relayApiOverview.includes("const gatewayStats = gw && typeof gw.getStats === 'function'\n ? sanitizeGatewayStats(gw.getStats())\n : null") && | |
| 4028 | 4056 | relayApiGatewayStats.includes('export function resolveGatewayStatsRoute') && |
| 4029 | 4057 | relayApiGatewayStats.includes("'GET /api/gateway'") && |
| 4030 | 4058 | relayApiGatewayStats.includes('export function buildGatewayStatsRoutePayload') && |
| … | … | @@ -4549,19 +4577,31 @@ if ( |
| 4549 | 4577 | appRegistryCore.includes('const throwOnError = opts.throwOnError === true') && |
| 4550 | 4578 | appRegistryCore.includes('this._beeWriteTail = op.catch(() => {})') && |
| 4551 | 4579 | appRegistryCore.includes('throw err') && |
| 4552 | appLifecycleCore.includes('node.appRegistry.set(appKeyHex, {') && | |
| 4580 | ( | |
| 4581 | appLifecycleCore.includes('node.appRegistry.set(appKeyHex, {') || | |
| 4582 | appLifecycleCore.includes('node.appRegistry.set(appKeyHex, nextEntry, { persist: false })') | |
| 4583 | ) && | |
| 4553 | 4584 | appLifecycleCore.includes('}, { persist: false })') && |
| 4554 | 4585 | appLifecycleCore.includes('await node.appRegistry.persistEntry(appKeyHex, { throwOnError: true })') && |
| 4555 | appLifecycleCore.includes('node.appRegistry.restoreSnapshot(registrySnapshot)') && | |
| 4586 | appLifecycleCore.includes('const previousEntry = node.appRegistry.get(appKeyHex) || null') && | |
| 4587 | appLifecycleCore.includes('if (previousEntry) node.appRegistry.set(appKeyHex, previousEntry, { persist: false })') && | |
| 4588 | appLifecycleCore.includes('else node.appRegistry.delete(appKeyHex, { persist: false })') && | |
| 4556 | 4589 | appLifecycleCore.includes('node.appRegistry.delete(appKeyHex, { persist: false })') && |
| 4557 | 4590 | appLifecycleCore.includes('await node.appRegistry.persistDelete(appKeyHex, { throwOnError: true })') && |
| 4558 | appLifecycleCore.indexOf('await node.appRegistry.persistDelete(appKeyHex, { throwOnError: true })') < | |
| 4559 | appLifecycleCore.indexOf('Destroy persistent download ranges before tearing down the drive') && | |
| 4591 | appLifecycleRetireDriveBlock.includes('Phase 1: Settle live resources, inverse of acquisition') && | |
| 4592 | appLifecycleRetireDriveBlock.includes('Phase 2: Durable delete') && | |
| 4593 | appLifecycleRetireDriveBlock.includes('Phase 3: Complete') && | |
| 4594 | appLifecycleCore.includes('this._retiringDrives.set(appKeyHex, owner)') && | |
| 4595 | appLifecycleRetireDriveBlock.indexOf('Phase 1: Settle live resources, inverse of acquisition') < | |
| 4596 | appLifecycleRetireDriveBlock.indexOf('await node.appRegistry.persistDelete(appKeyHex, { throwOnError: true })') && | |
| 4597 | appLifecycleRetireDriveBlock.indexOf('await node.appRegistry.persistDelete(appKeyHex, { throwOnError: true })') < | |
| 4598 | appLifecycleRetireDriveBlock.indexOf('Phase 3: Complete') && | |
| 4560 | 4599 | appLifecycleCore.indexOf('await node.appRegistry.persistEntry(appKeyHex, { throwOnError: true })') < |
| 4561 | appLifecycleCore.indexOf('node.swarm.join(discoveryKey, { server: true, client: true })') && | |
| 4600 | appLifecycleCore.indexOf("this.emit('seeding', { appKey: appKeyHex") && | |
| 4562 | 4601 | appRegistryTest.includes('snapshot restore preserves map identity and indexes') && |
| 4563 | 4602 | appRegistryTest.includes('explicit JSON persistence rejects write failures') && |
| 4564 | appLifecyclePersistenceTest.includes('failed unseed registry delete keeps live resources intact') && | |
| 4603 | appLifecyclePersistenceTest.includes('failed durable unseed retains retry debt after ordered live teardown') && | |
| 4604 | appLifecyclePersistenceTest.includes('closed retiring entry is restored in memory for retry') && | |
| 4565 | 4605 | appLifecyclePersistenceTest.includes('seedApp rolls back registry when explicit registry persist fails') |
| 4566 | 4606 | ) { |
| 4567 | 4607 | pass('app seed/unseed lifecycle persists registry changes before API-visible success and rolls back failed durable writes') |
| … | … | @@ -4935,7 +4975,7 @@ if ( |
| 4935 | 4975 | networkDiscoveryCore.includes('MAX_META_BYTES = 2048') && |
| 4936 | 4976 | networkDiscoveryCore.includes('HOLESAIL_Z32_KEY') && |
| 4937 | 4977 | networkDiscoveryCore.includes('normalizeHolesailKey (key)') && |
| 4938 | networkDiscoveryCore.includes('_handleMetadataFrame (pubkey, buf)') && | |
| 4978 | networkDiscoveryCore.includes('_handleMetadataFrame (pubkey, buf, conn = null, epoch = this._lifecycleEpoch)') && | |
| 4939 | 4979 | networkDiscoveryCore.includes('buf.byteLength > MAX_META_BYTES') && |
| 4940 | 4980 | networkDiscoveryCore.includes('contentLength > MAX_API_OVERVIEW_BYTES') && |
| 4941 | 4981 | networkDiscoveryCore.includes("Buffer.byteLength(chunk, 'utf8')") && |
| … | … | @@ -5065,7 +5105,8 @@ if ( |
| 5065 | 5105 | relayApiSeedCore.includes("typeof body.appKey === 'string'") && |
| 5066 | 5106 | relayApiSeedCore.includes('isValidHexKey(coreKey, 64)') && |
| 5067 | 5107 | relayApiSeedCore.includes("return { ok: false, status: 503, payload: { error: 'seeder not available' } }") && |
| 5068 | relayApiSeedCore.includes('node.seeder.seedCore(coreKey)') && | |
| 5108 | relayApiSeedCore.includes('node.seeder.seedCore(coreKey, { maxStorageBytes })') && | |
| 5109 | relayApiSeedCore.includes('positiveStorageBound(body.maxStorageBytes)') && | |
| 5069 | 5110 | relayApiSeedCore.includes("body.catalog === true && typeof node.setCatalogBeeKey === 'function'") && |
| 5070 | 5111 | relayApiSeedCore.includes("return { ok: false, kind: 'seed-error', error: err }") && |
| 5071 | 5112 | apiSeedCoreTest.includes('seed-core helper: route resolver maps only the exact operator seed-core route') && |
| … | … | @@ -5765,7 +5806,10 @@ if ( |
| 5765 | 5806 | if ( |
| 5766 | 5807 | quorumSelector.includes('const DEFAULT_MIN_OPERATORS = 3') && |
| 5767 | 5808 | quorumSelector.includes('adds the most missing dimensions') && |
| 5768 | quorumSelector.includes('gain = (seenRegions.has(region) ? 0 : 1) + (seenOperators.has(op) ? 0 : 1)') && | |
| 5809 | quorumSelector.includes('function failureDomainOf') && | |
| 5810 | quorumSelector.includes('function operatorOf') && | |
| 5811 | quorumSelector.includes('seenDomains.has(failureDomainOf(r))') && | |
| 5812 | quorumSelector.includes("return r.operator || '__undeclared__'") && | |
| 5769 | 5813 | quorumSelector.includes('function normalizedScore') && |
| 5770 | 5814 | quorumSelector.includes('Number.isFinite(score) && score >= 0 && score <= 1') && |
| 5771 | 5815 | quorumSelector.includes('function normalizedLatency') && |
| … | … | @@ -5777,7 +5821,9 @@ if ( |
| 5777 | 5821 | quorumSelectorTest.includes('warns when operator diversity is below the floor') && |
| 5778 | 5822 | quorumSelectorTest.includes('ignores malformed or unnormalized score claims') && |
| 5779 | 5823 | quorumSelectorTest.includes('ignores malformed latency claims when scores tie') && |
| 5780 | threatModelDoc.includes('client `selectQuorum` now diversity-ranks by region and operator') | |
| 5824 | quorumSelectorTest.includes('one owner across many hosts counts as ONE operator') && | |
| 5825 | quorumSelectorTest.includes('an undeclared operator is unknown, not independent') && | |
| 5826 | threatModelDoc.includes('client `selectQuorum` now spreads by region and failure domain while counting operator independence separately') | |
| 5781 | 5827 | ) { |
| 5782 | 5828 | pass('client quorum selection prioritizes operator/region diversity and ignores malformed ranking signals') |
| 5783 | 5829 | } else { |
| … | … | @@ -7140,6 +7186,43 @@ if ( |
| 7140 | 7186 | |
| 7141 | 7187 | const umbrelAppDataDirExpression = '$' + '{APP_DATA_DIR}' |
| 7142 | 7188 | const umbrelAppSeedExpression = '$' + '{APP_SEED}' |
| 7189 | if ( | |
| 7190 | vrfService.includes('context.config.vrfBeacon') && | |
| 7191 | vrfService.includes('this._configureBeacon(context.config.vrfBeacon)') && | |
| 7192 | apiVrfTest.includes('VRF service consumes Node PluginLoader context.config.vrfBeacon') | |
| 7193 | ) { | |
| 7194 | pass('Node PluginLoader context activates the configured VRF beacon instead of leaving its HTTP surface permanently disabled') | |
| 7195 | } else { | |
| 7196 | fail('Node VRF beacon config can drift back into an unreachable constructor-only option') | |
| 7197 | } | |
| 7198 | ||
| 7199 | if ( | |
| 7200 | configLoader.includes('export function applyRuntimeEnvDefaults') && | |
| 7201 | configLoader.includes("HIVERELAY_OUTBOXLOG: 'outboxlog'") && | |
| 7202 | configLoader.includes("HIVERELAY_STORAGE_PROOF: 'storage-proof'") && | |
| 7203 | configLoader.includes("'HIVERELAY_TOR_MIN_DAEMON_VERSION', 'tor', 'minDaemonVersion'") && | |
| 7204 | configLoader.includes("'HIVERELAY_TOR_ROSTER_FILE', 'tor', 'rosterFile'") && | |
| 7205 | configLoader.includes("'HIVERELAY_OUTBOXLOG_MAX_JOURNAL_STORAGE', 'outboxlog', 'maxJournalStorageBytes'") && | |
| 7206 | configLoader.includes('HIVERELAY_NOTIFY_PUSH_CONFIG') && | |
| 7207 | cliIndex.includes('applyRuntimeEnvDefaults(cliOverrides, process.env)') && | |
| 7208 | relayNode.includes("typeof data.enabled === 'boolean'") && | |
| 7209 | configLoaderTest.includes('runtime env defaults: Node packaging enables bounded services and Tor') && | |
| 7210 | configLoaderTest.includes('runtime env defaults: persisted fields and CLI flags remain authoritative') && | |
| 7211 | relayNodeTest.includes('persisted services disable overrides package defaults') && | |
| 7212 | rootDockerCompose.includes('HIVERELAY_OUTBOXLOG_JOURNAL=hypercore-outboxes') && | |
| 7213 | rootDockerCompose.includes('HIVERELAY_OUTBOXLOG_MAX_JOURNAL_STORAGE=512MB') && | |
| 7214 | systemdService.includes('Environment=HIVERELAY_OUTBOXLOG_JOURNAL=hypercore-outboxes') && | |
| 7215 | systemdService.includes('Environment=HIVERELAY_OUTBOXLOG_MAX_JOURNAL_STORAGE=512MB') && | |
| 7216 | umbrelCompose.includes('HIVERELAY_OUTBOXLOG_JOURNAL: hypercore-outboxes') && | |
| 7217 | umbrelCompose.includes('HIVERELAY_OUTBOXLOG_MAX_JOURNAL_STORAGE: 512MB') && | |
| 7218 | !/^\s+- HIVERELAY_SHARD_STORE=1\s*$/m.test(rootDockerCompose) && | |
| 7219 | !/^\s+HIVERELAY_SHARD_STORE:\s*["']?true["']?\s*$/m.test(umbrelCompose) | |
| 7220 | ) { | |
| 7221 | pass('Node packaging env maps to operator-overridable services, bounded OutboxLog persistence, private wake descriptors, and Tor defaults') | |
| 7222 | } else { | |
| 7223 | fail('Node packaging can drift back to inert or unbounded service/Tor environment claims') | |
| 7224 | } | |
| 7225 | ||
| 7143 | 7226 | if ( |
| 7144 | 7227 | // The official-submission compose must pin an immutable, digest-qualified GHCR |
| 7145 | 7228 | // image (real 64-hex digest, never a floating tag) so Umbrel reviewers get a |
| … | … | @@ -7164,11 +7247,11 @@ if ( |
| 7164 | 7247 | cliIndex.includes('else if (process.env.HIVERELAY_STORAGE) cliOverrides.storage = process.env.HIVERELAY_STORAGE') && |
| 7165 | 7248 | cliIndex.includes('HIVERELAY_MAX_STORAGE') && |
| 7166 | 7249 | cliIndex.includes("parseBytesOrExit(process.env.HIVERELAY_MAX_STORAGE, 'HIVERELAY_MAX_STORAGE')") && |
| 7167 | cliIndex.includes('!hasPersistedConfig()') && | |
| 7168 | cliIndex.includes('function hasPersistedConfig') && | |
| 7250 | cliIndex.includes('!hasPersistedMaxStorage()') && | |
| 7251 | cliIndex.includes('function hasPersistedMaxStorage') && | |
| 7169 | 7252 | cliIndex.includes('Invalid $' + '{label}: expected a positive size') && |
| 7170 | 7253 | configLoaderTest.includes('cli start rejects invalid HIVERELAY_MAX_STORAGE before boot') && |
| 7171 | configLoaderTest.includes('cli start uses HIVERELAY_MAX_STORAGE only before saved operator config exists') && | |
| 7254 | configLoaderTest.includes('cli start keeps HIVERELAY_MAX_STORAGE explicit until a cap is persisted') && | |
| 7172 | 7255 | configLoaderTest.includes('cli start uses HIVERELAY_STORAGE when --storage is absent') && |
| 7173 | 7256 | configLoaderTest.includes('cli start --storage overrides HIVERELAY_STORAGE') && |
| 7174 | 7257 | configLoaderTest.includes('Max Store: 50.0 GB') && |
| … | … | @@ -9094,7 +9177,10 @@ const updaterRollbackHealthGate = 'if healthy "' + '$' + '{CUR_VER#v}"; then' |
| 9094 | 9177 | if ( |
| 9095 | 9178 | fleetUpdaterScript.includes('rollback_to_previous') && |
| 9096 | 9179 | fleetUpdaterScript.includes('deps_if_changed "$CUR_SHA" "$TARGET_SHA" || rollback_to_previous') && |
| 9097 | fleetUpdaterScript.includes('if ! git checkout --quiet "$CUR_SHA"; then') && | |
| 9180 | // Rollback must force away only updater-created dirt. The updater already | |
| 9181 | // refuses to start on a dirty tree, so this is both safe and necessary when | |
| 9182 | // a failed dependency install has modified the target checkout. | |
| 9183 | fleetUpdaterScript.includes('if ! git checkout --quiet --force "$CUR_SHA"; then') && | |
| 9098 | 9184 | fleetUpdaterScript.includes('if ! deps_if_changed "$TARGET_SHA" "$CUR_SHA"; then') && |
| 9099 | 9185 | fleetUpdaterScript.includes(updaterExpectedVersionArg) && |
| 9100 | 9186 | fleetUpdaterScript.includes(updaterTargetHealthGate) && |
| … | … | @@ -14,9 +14,8 @@ |
| 14 | 14 | */ |
| 15 | 15 | import http from 'node:http' |
| 16 | 16 | import { readFileSync, existsSync } from 'node:fs' |
| 17 | import { join, extname } from 'node:path' | |
| 17 | import { join, extname, dirname } from 'node:path' | |
| 18 | 18 | import { fileURLToPath } from 'node:url' |
| 19 | import { dirname } from 'node:path' | |
| 20 | 19 | |
| 21 | 20 | const __dirname = dirname(fileURLToPath(import.meta.url)) |
| 22 | 21 | const ROOT = join(__dirname, '..') |
| … | … | @@ -20,6 +20,8 @@ |
| 20 | 20 | * node scripts/probe-fleet-services.mjs |
| 21 | 21 | * node scripts/probe-fleet-services.mjs --json > fleet-services.json |
| 22 | 22 | * node scripts/probe-fleet-services.mjs --relays fleet/relays.local.json |
| 23 | * node scripts/probe-fleet-services.mjs --include utah,bern | |
| 24 | * node scripts/probe-fleet-services.mjs --exclude sydney,dallas | |
| 23 | 25 | * node scripts/probe-fleet-services.mjs --no-ssh # HTTP only |
| 24 | 26 | */ |
| 25 | 27 | |
| … | … | @@ -36,6 +38,24 @@ const args = process.argv.slice(2) |
| 36 | 38 | const asJson = args.includes('--json') |
| 37 | 39 | const allowSsh = !args.includes('--no-ssh') |
| 38 | 40 | const relaysArg = args[args.indexOf('--relays') + 1] |
| 41 | const include = parseNameList(optionValue('--include') || process.env.HIVERELAY_FLEET_INCLUDE || '', '--include') | |
| 42 | const exclude = parseNameList(optionValue('--exclude') || process.env.HIVERELAY_FLEET_EXCLUDE || '', '--exclude') | |
| 43 | ||
| 44 | function optionValue (name) { | |
| 45 | const at = args.indexOf(name) | |
| 46 | if (at === -1) return '' | |
| 47 | const value = args[at + 1] | |
| 48 | if (!value || value.startsWith('--')) throw new Error(`${name} requires a value`) | |
| 49 | return value | |
| 50 | } | |
| 51 | ||
| 52 | function parseNameList (value, label) { | |
| 53 | if (!value) return new Set() | |
| 54 | if (!/^[A-Za-z0-9._-]+(?:,[A-Za-z0-9._-]+)*$/.test(value)) { | |
| 55 | throw new Error(`${label} must be a comma-separated relay-name list`) | |
| 56 | } | |
| 57 | return new Set(value.split(',')) | |
| 58 | } | |
| 39 | 59 | |
| 40 | 60 | function inventoryPath () { |
| 41 | 61 | if (relaysArg && args.includes('--relays')) return relaysArg |
| … | … | @@ -56,24 +76,43 @@ function expandHome (p) { |
| 56 | 76 | } |
| 57 | 77 | |
| 58 | 78 | async function probeHttp (relay) { |
| 59 | const url = `http://${relay.publicIp}:9100/api/v1/services` | |
| 60 | const res = await fetch(url, { signal: AbortSignal.timeout(6000) }) | |
| 61 | return { body: await res.json(), status: res.status, via: 'http' } | |
| 79 | const base = `http://${relay.publicIp}:9100` | |
| 80 | const [services, capability] = await Promise.all([ | |
| 81 | fetch(`${base}/api/v1/services`, { signal: AbortSignal.timeout(6000) }), | |
| 82 | fetch(`${base}/.well-known/hiverelay.json`, { signal: AbortSignal.timeout(6000) }) | |
| 83 | ]) | |
| 84 | return { | |
| 85 | body: await services.json(), | |
| 86 | capability: capability.ok ? await capability.json() : null, | |
| 87 | status: services.status, | |
| 88 | via: 'http' | |
| 89 | } | |
| 62 | 90 | } |
| 63 | 91 | |
| 64 | 92 | async function probeSsh (relay) { |
| 65 | 93 | if (!isSafeHost(relay.publicIp)) throw new Error('unsafe host in inventory') |
| 66 | 94 | const key = expandHome(relay.sshKey) |
| 67 | 95 | const keyArgs = key && key !== 'default' ? ['-i', key] : [] |
| 96 | const remote = [ | |
| 97 | 'S=$(curl -fsS --max-time 6 http://127.0.0.1:9100/api/v1/services) || exit $?', | |
| 98 | 'C=$(curl -fsS --max-time 6 http://127.0.0.1:9100/.well-known/hiverelay.json 2>/dev/null || printf \'{}\')', | |
| 99 | 'printf \'%s\\n__HIVERELAY_CAP__\\n%s\\n\' "$S" "$C"' | |
| 100 | ].join('\n') | |
| 68 | 101 | const { stdout } = await execFileAsync('ssh', [ |
| 69 | 102 | ...keyArgs, |
| 70 | 103 | '-o', 'BatchMode=yes', |
| 71 | 104 | '-o', 'ConnectTimeout=8', |
| 72 | 105 | '-o', 'StrictHostKeyChecking=accept-new', |
| 73 | 106 | `root@${relay.publicIp}`, |
| 74 | 'curl -fsS --max-time 6 http://127.0.0.1:9100/api/v1/services' | |
| 107 | remote | |
| 75 | 108 | ], { timeout: 25000 }) |
| 76 | return { body: JSON.parse(stdout), status: 200, via: 'ssh' } | |
| 109 | const parts = stdout.split('\n__HIVERELAY_CAP__\n') | |
| 110 | return { | |
| 111 | body: JSON.parse(parts[0]), | |
| 112 | capability: parts[1] ? JSON.parse(parts[1]) : null, | |
| 113 | status: 200, | |
| 114 | via: 'ssh' | |
| 115 | } | |
| 77 | 116 | } |
| 78 | 117 | |
| 79 | 118 | async function probe (relay) { |
| … | … | @@ -83,7 +122,7 @@ async function probe (relay) { |
| 83 | 122 | } |
| 84 | 123 | for (const attempt of allowSsh ? [probeHttp, probeSsh] : [probeHttp]) { |
| 85 | 124 | try { |
| 86 | const { body, via } = await attempt(relay) | |
| 125 | const { body, capability, via } = await attempt(relay) | |
| 87 | 126 | // 503 {"error":"Services not enabled"} is a real answer, not a failure. |
| 88 | 127 | if (body && body.error) { |
| 89 | 128 | return { ...base, reachable: true, via, servicesEnabled: false, services: [], note: body.error } |
| … | … | @@ -91,32 +130,87 @@ async function probe (relay) { |
| 91 | 130 | const services = Array.isArray(body?.services) |
| 92 | 131 | ? body.services.map(s => (typeof s === 'string' ? s : s?.name)).filter(Boolean).sort() |
| 93 | 132 | : [] |
| 94 | return { ...base, reachable: true, via, servicesEnabled: true, services, total: body?.total ?? services.length } | |
| 133 | const serviceProfile = capability?.protocol_profile?.services || {} | |
| 134 | const notifyProfile = serviceProfile.notify || null | |
| 135 | const features = Array.isArray(capability?.features) ? capability.features : [] | |
| 136 | const supportedTransports = Array.isArray(capability?.supported_transports) ? capability.supported_transports : [] | |
| 137 | const privacyTransports = Array.isArray(capability?.privacyTransports) ? capability.privacyTransports : [] | |
| 138 | const torPrivacy = privacyTransports.find(entry => entry?.network === 'tor') || null | |
| 139 | const notifyWatchSources = Array.isArray(notifyProfile?.watch_sources) ? notifyProfile.watch_sources : [] | |
| 140 | return { | |
| 141 | ...base, | |
| 142 | reachable: true, | |
| 143 | via, | |
| 144 | servicesEnabled: true, | |
| 145 | services, | |
| 146 | total: body?.total ?? services.length, | |
| 147 | notifyLoaded: services.includes('notify'), | |
| 148 | notifyFeatureAdvertised: features.includes('notify-v1'), | |
| 149 | notifyEgressLive: notifyProfile?.egress?.live === true, | |
| 150 | notifyWatchSources, | |
| 151 | exactLaneWake: notifyWatchSources.includes('notify-outbox-lane'), | |
| 152 | outboxlogLoaded: services.includes('outboxlog'), | |
| 153 | outboxlogAdvertised: !!serviceProfile.outboxlog, | |
| 154 | torRuntime: supportedTransports.includes('tor'), | |
| 155 | torSignedReady: !!torPrivacy, | |
| 156 | torRestricted: torPrivacy?.auth?.mode === 'client-auth-v3', | |
| 157 | torNegativeProbe: torPrivacy?.negativeProbe === true, | |
| 158 | forwardRelayAdvertised: supportedTransports.includes('hiverelay-forward') | |
| 159 | } | |
| 95 | 160 | } catch (err) { |
| 96 | base.error = err?.message?.slice(0, 120) || String(err) | |
| 161 | base.error = safeProbeError(err) | |
| 97 | 162 | } |
| 98 | 163 | } |
| 99 | 164 | return { ...base, reachable: false, services: [] } |
| 100 | 165 | } |
| 101 | 166 | |
| 167 | function safeProbeError (err) { | |
| 168 | if (err?.name === 'TimeoutError' || err?.code === 'ETIMEDOUT') return 'probe timeout' | |
| 169 | if (err?.code === 'ECONNREFUSED') return 'connection refused' | |
| 170 | if (err?.code === 'EHOSTUNREACH' || err?.code === 'ENETUNREACH') return 'host unreachable' | |
| 171 | if (err?.code === 'ENOENT') return 'probe command unavailable' | |
| 172 | return 'probe failed' | |
| 173 | } | |
| 174 | ||
| 102 | 175 | const inv = inventoryPath() |
| 103 | const relays = JSON.parse(readFileSync(inv, 'utf8')).relays | |
| 176 | const inventoryLabel = inv === join(REPO, 'fleet', 'relays.json') ? 'fleet/relays.json' : 'operator inventory' | |
| 177 | const inventoryRelays = JSON.parse(readFileSync(inv, 'utf8')).relays | |
| 178 | const knownNames = new Set(inventoryRelays.map(relay => relay.name)) | |
| 179 | for (const name of include) { | |
| 180 | if (!knownNames.has(name)) throw new Error(`--include names unknown relay ${name}`) | |
| 181 | } | |
| 182 | const relays = inventoryRelays.filter(relay => { | |
| 183 | if (include.size > 0 && !include.has(relay.name)) return false | |
| 184 | return !exclude.has(relay.name) | |
| 185 | }) | |
| 186 | if (relays.length === 0) throw new Error('relay scope selected no inventory entries') | |
| 104 | 187 | const results = await Promise.all(relays.map(probe)) |
| 105 | 188 | |
| 106 | 189 | if (asJson) { |
| 107 | 190 | console.log(JSON.stringify({ |
| 108 | 191 | probedAt: new Date().toISOString(), |
| 109 | inventory: inv, | |
| 192 | inventory: inventoryLabel, | |
| 110 | 193 | source: 'runtime GET /api/v1/services (not config)', |
| 111 | 194 | relays: results |
| 112 | 195 | }, null, 2)) |
| 113 | 196 | } else { |
| 114 | console.log(`\nRuntime service catalogue — ${relays.length} relays from ${inv}`) | |
| 197 | console.log(`\nRuntime service catalogue — ${relays.length} relays from ${inventoryLabel}`) | |
| 115 | 198 | console.log('Source: GET /api/v1/services (what is LOADED, not what is configured)\n') |
| 116 | 199 | const w = Math.max(...results.map(r => r.name.length), 6) |
| 117 | 200 | for (const r of results) { |
| 118 | 201 | const status = !r.reachable ? 'unreachable' : r.servicesEnabled === false ? 'services off' : `${r.services.length} loaded` |
| 119 | const detail = !r.reachable ? (r.error || '') : r.services.join(', ') || (r.note || '') | |
| 202 | const wake = !r.notifyLoaded | |
| 203 | ? 'wake=off' | |
| 204 | : r.notifyEgressLive | |
| 205 | ? (r.exactLaneWake ? 'wake=live+exact-lane' : 'wake=live+direct') | |
| 206 | : r.notifyFeatureAdvertised | |
| 207 | ? 'wake=misadvertised-no-egress' | |
| 208 | : 'wake=loaded-not-live' | |
| 209 | const mailbox = r.outboxlogLoaded | |
| 210 | ? (r.outboxlogAdvertised ? 'mailbox=advertised' : 'mailbox=loaded-not-advertised') | |
| 211 | : 'mailbox=off' | |
| 212 | const privacy = privacyLabel(r) | |
| 213 | const detail = !r.reachable ? (r.error || '') : `${wake} ${mailbox} ${privacy}; ${r.services.join(', ') || (r.note || '')}` | |
| 120 | 214 | console.log(` ${r.name.padEnd(w)} ${String(r.region ?? '-').padEnd(5)} ${status.padEnd(13)} ${detail}`) |
| 121 | 215 | } |
| 122 | 216 | |
| … | … | @@ -131,5 +225,19 @@ if (asJson) { |
| 131 | 225 | for (const r of reachable) for (const s of r.services) counts.set(s, (counts.get(s) || 0) + 1) |
| 132 | 226 | const singletons = [...counts].filter(([, n]) => n === 1).map(([s]) => s).sort() |
| 133 | 227 | if (singletons.length) console.log(` running on exactly ONE relay (SPOF): ${singletons.join(', ')}`) |
| 228 | console.log(` signed live wake egress: ${reachable.filter(r => r.notifyEgressLive).length}/${reachable.length}`) | |
| 229 | console.log(` signed exact-lane wake: ${reachable.filter(r => r.exactLaneWake).length}/${reachable.length}`) | |
| 230 | console.log(` unsafe legacy wake advertisements: ${reachable.filter(r => r.notifyFeatureAdvertised && !r.notifyEgressLive).length}/${reachable.length}`) | |
| 231 | console.log(` advertised outbox mailbox: ${reachable.filter(r => r.outboxlogAdvertised).length}/${reachable.length}`) | |
| 232 | console.log(` signed ready Tor endpoints: ${reachable.filter(r => r.torSignedReady).length}/${reachable.length}`) | |
| 233 | console.log(` restricted Tor endpoints with negative proof: ${reachable.filter(r => r.torRestricted && r.torNegativeProbe).length}/${reachable.length}`) | |
| 234 | console.log(` advertised one-hop forward relays: ${reachable.filter(r => r.forwardRelayAdvertised).length}/${reachable.length}`) | |
| 134 | 235 | console.log() |
| 135 | 236 | } |
| 237 | ||
| 238 | function privacyLabel (relay) { | |
| 239 | if (!relay.torRuntime) return relay.forwardRelayAdvertised ? 'privacy=forward-only' : 'privacy=clearnet' | |
| 240 | if (!relay.torSignedReady) return 'privacy=tor-loaded-not-ready' | |
| 241 | if (!relay.torRestricted) return 'privacy=tor-public-ready' | |
| 242 | return relay.torNegativeProbe ? 'privacy=tor-restricted-proved' : 'privacy=tor-restricted-unproved' | |
| 243 | } |
| … | … | @@ -50,7 +50,7 @@ const testnet = await createTestnet(3) |
| 50 | 50 | for (let i = 0; i < N; i++) { |
| 51 | 51 | const apiKey = 'cohort-' + randomBytes(6).toString('hex') |
| 52 | 52 | const node = new RelayNode({ |
| 53 | storage: join(BASE, 'relay-' + i), // persistent → stable relay pubkey across restarts | |
| 53 | storage: join(BASE, 'relay-' + i), // persistent → stable relay pubkey across restarts | |
| 54 | 54 | bootstrapNodes: testnet.bootstrap, |
| 55 | 55 | enableAPI: true, |
| 56 | 56 | apiPort: 0, |
| … | … | @@ -80,6 +80,6 @@ await writeFile(ROSTER, JSON.stringify(roster, null, 2) + '\n') |
| 80 | 80 | console.log(`\n▶ ${N}-relay shard-store cohort LIVE — /api/v1/shard mounted on each.`) |
| 81 | 81 | console.log(` roster written → ${ROSTER}`) |
| 82 | 82 | console.log(` smoke it: node scripts/blind-dispersal-live.mjs ${ROSTER}`) |
| 83 | console.log(` (same-operator cohort → mechanism proof, not the security property. Ctrl-C to stop.)\n`) | |
| 83 | console.log(' (same-operator cohort → mechanism proof, not the security property. Ctrl-C to stop.)\n') | |
| 84 | 84 | |
| 85 | 85 | await new Promise(() => {}) // stay alive |
| … | … | @@ -21,7 +21,7 @@ export const manifest = setupManifest({ |
| 21 | 21 | volumes: ['main'], |
| 22 | 22 | images: { |
| 23 | 23 | blindspark: { |
| 24 | source: { dockerTag: 'ghcr.io/bigdestiny2/p2p-hiverelay:0.24.3' }, | |
| 24 | source: { dockerTag: 'ghcr.io/bigdestiny2/p2p-hiverelay:0.25.0-rc.7' }, | |
| 25 | 25 | arch: ['x86_64', 'aarch64'], |
| 26 | 26 | }, |
| 27 | 27 | }, |
| … | … | @@ -6,7 +6,7 @@ import { VersionInfo, IMPOSSIBLE } from '@start9labs/start-sdk' |
| 6 | 6 | // (the 0.3.x YAML package is a separate distribution channel), so `up` is a |
| 7 | 7 | // no-op and `down` is impossible. |
| 8 | 8 | export const current = VersionInfo.of({ |
| 9 | version: '0.24.3:0', | |
| 9 | version: '0.25.0-rc.7:0', | |
| 10 | 10 | releaseNotes: { |
| 11 | 11 | en_US: |
| 12 | 12 | 'Initial StartOS 0.4 (start-sdk 2.x) build of Blindspark — the HiveRelay ' + |
| … | … | @@ -13,7 +13,7 @@ |
| 13 | 13 | |
| 14 | 14 | id: blindspark |
| 15 | 15 | title: "Blindspark" |
| 16 | version: 0.25.0-rc.4 | |
| 16 | version: 0.25.0-rc.7 | |
| 17 | 17 | release-notes: | |
| 18 | 18 | HiveRelay v0.25.0-rc.4: complete canary fix on the wiring branch. Gateway |
| 19 | 19 | modules, loader fix, ETIMEDOUT recoverable. Prefer over rc.1–rc.3. |
| … | … | @@ -163,7 +163,10 @@ test('Reliability v2: multi-cycle start/stop with seeded apps is clean', async ( |
| 163 | 163 | await publisher.start() |
| 164 | 164 | const seededKeys = [] |
| 165 | 165 | for (let i = 0; i < 3; i++) { |
| 166 | const drive = new Hyperdrive(publisher.store.namespace('reliability-source-' + i).session()) | |
| 166 | // namespace() already returns an owning Corestore session. Wrapping it in | |
| 167 | // another session drops the namespace in Corestore 7 and makes the second | |
| 168 | // writable Hyperdrive contend for the first drive's deterministic key. | |
| 169 | const drive = new Hyperdrive(publisher.store.namespace('reliability-source-' + i)) | |
| 167 | 170 | await drive.ready() |
| 168 | 171 | await drive.put('/initial.bin', randomBytes(64 * 1024)) |
| 169 | 172 | sourceDrives.push(drive) |
| … | … | @@ -99,7 +99,9 @@ async function startPublisher (baseDir, bootstrap, count, label) { |
| 99 | 99 | const keys = [] |
| 100 | 100 | await publisher.start() |
| 101 | 101 | for (let i = 0; i < count; i++) { |
| 102 | const drive = new Hyperdrive(publisher.store.namespace(`${label}-${i}`).session()) | |
| 102 | // namespace() is itself a Corestore session; a nested session loses the | |
| 103 | // namespace under Corestore 7 and deadlocks the second writable drive. | |
| 104 | const drive = new Hyperdrive(publisher.store.namespace(`${label}-${i}`)) | |
| 103 | 105 | await drive.ready() |
| 104 | 106 | await drive.put('/index.bin', randomBytes(64 * 1024)) |
| 105 | 107 | drives.push(drive) |
| … | … | @@ -1,29 +0,0 @@ |
| 1 | /** | |
| 2 | * Force-exit guard for the integration suite. | |
| 3 | * | |
| 4 | * The integration tests all pass (assertion-wise) but the process hangs | |
| 5 | * for several minutes afterwards before exiting because some | |
| 6 | * Hyperswarm / Hypercore / DHT resource is held open across the file | |
| 7 | * boundary. Brittle has no global afterAll hook and no --exit flag. | |
| 8 | * | |
| 9 | * Two-stage workaround: | |
| 10 | * 1. Schedule a 5-second `.unref()` timer that calls process.exit(0) | |
| 11 | * if the Node event loop is still alive after this last test. | |
| 12 | * 2. The .unref() ensures we don't artificially block a clean exit: | |
| 13 | * if the loop drains naturally, the timer is collected and we | |
| 14 | * exit normally. If something leaks, the timer fires. | |
| 15 | * | |
| 16 | * Filename `zz-finalize` ensures it runs last alphabetically — brittle's | |
| 17 | * glob expansion respects sort order, and every other integration test | |
| 18 | * file starts with a letter before 'z'. | |
| 19 | * | |
| 20 | * If this guard ever stops firing, that means whatever leak this is | |
| 21 | * masking has been fixed and this file can be removed. | |
| 22 | */ | |
| 23 | ||
| 24 | import { test } from 'brittle' | |
| 25 | ||
| 26 | test('integration suite: schedule post-suite force-exit', async (t) => { | |
| 27 | setTimeout(() => process.exit(0), 5000).unref() | |
| 28 | t.pass('force-exit timer armed (5s, .unref())') | |
| 29 | }) |
| … | … | @@ -191,6 +191,22 @@ test('/api/token redacts adapter load failures and emits internals', async (t) = |
| 191 | 191 | t.ok(events[0].error.message.includes('/data/hiverelay/private')) |
| 192 | 192 | }) |
| 193 | 193 | |
| 194 | test('/api/outboxlog browser preflight exposes only the public blind-pipe headers', async (t) => { | |
| 195 | const { port } = await serverWithApi(t, mockNode({ registry: null })) | |
| 196 | const res = await request(port, 'OPTIONS', '/api/sync/create', null, { | |
| 197 | Origin: 'https://peerit.example', | |
| 198 | 'Access-Control-Request-Method': 'POST', | |
| 199 | 'Access-Control-Request-Headers': 'Content-Type, X-Pear-Token' | |
| 200 | }) | |
| 201 | ||
| 202 | t.is(res.statusCode, 204) | |
| 203 | t.is(res.headers['access-control-allow-origin'], '*') | |
| 204 | t.is(res.headers['access-control-allow-methods'], 'GET, POST, OPTIONS') | |
| 205 | t.is(res.headers['access-control-allow-headers'], 'Content-Type, X-Pear-Token, X-Pear-Admin-Token') | |
| 206 | t.is(res.headers['access-control-expose-headers'], 'Retry-After') | |
| 207 | t.is(res.headers.vary, 'Origin', 'the outer API cache key remains origin-safe before adapter handoff') | |
| 208 | }) | |
| 209 | ||
| 194 | 210 | test('/api/token explicit OutboxLog config retains a coarse fail-safe when adapter loading fails', async (t) => { |
| 195 | 211 | const app = fakeOutboxLogApp() |
| 196 | 212 | const node = mockNode({ |
| … | … | @@ -23,11 +23,11 @@ test('api service config helpers: route helper maps exact service config update' |
| 23 | 23 | }) |
| 24 | 24 | |
| 25 | 25 | test('api service config helpers: normalize builtins and expand bundles', (t) => { |
| 26 | const normalized = normalizeManageServicePlugins([' Poker ', 'AI', 'vrf', 'outboxlog', 'notify', '']) | |
| 26 | const normalized = normalizeManageServicePlugins([' Poker ', 'AI', 'vrf', 'outboxlog', 'notify', 'storage-proof', '']) | |
| 27 | 27 | |
| 28 | 28 | t.alike(normalized, { |
| 29 | 29 | ok: true, |
| 30 | plugins: ['poker', 'vrf', 'arbitration', 'zk', 'ai', 'outboxlog', 'notify'] | |
| 30 | plugins: ['poker', 'vrf', 'arbitration', 'zk', 'ai', 'outboxlog', 'notify', 'storage-proof'] | |
| 31 | 31 | }) |
| 32 | 32 | |
| 33 | 33 | t.alike(normalizeManageServicePlugins(null), { ok: true, plugins: [] }) |
| … | … | @@ -111,10 +111,25 @@ test('service config api: catalog is authenticated and reports configured servic |
| 111 | 111 | t.ok(res.body.available.includes('poker')) |
| 112 | 112 | t.ok(res.body.available.includes('outboxlog')) |
| 113 | 113 | t.ok(res.body.available.includes('notify')) |
| 114 | t.ok(res.body.available.includes('storage-proof')) | |
| 114 | 115 | t.alike(res.body.bundles.poker, ['poker', 'vrf', 'arbitration', 'zk']) |
| 115 | 116 | t.alike(res.body.active, ['identity']) |
| 116 | 117 | }) |
| 117 | 118 | |
| 119 | test('service config api: persists through services.json authority when node supports it', async (t) => { | |
| 120 | const { node, port, auth } = await makeServer(t) | |
| 121 | const calls = [] | |
| 122 | node.setServicesConfig = async (value) => { calls.push(value); return value } | |
| 123 | ||
| 124 | const res = await request(port, 'POST', '/api/manage/services/config', { | |
| 125 | enabled: true, | |
| 126 | plugins: ['storage-proof'] | |
| 127 | }, auth) | |
| 128 | ||
| 129 | t.is(res.statusCode, 200) | |
| 130 | t.alike(calls, [{ enabled: true, plugins: ['storage-proof'] }]) | |
| 131 | }) | |
| 132 | ||
| 118 | 133 | test('service config api: saves builtins and expands poker bundle', async (t) => { |
| 119 | 134 | const { node, port, auth, persisted } = await makeServer(t) |
| 120 | 135 |
| … | … | @@ -150,9 +150,11 @@ test('POST /api/manage/services/config: persists opt-in, restartRequired', async |
| 150 | 150 | t.is(res.body.restartRequired, true) |
| 151 | 151 | t.is(res.body.config.enabled, true) |
| 152 | 152 | t.alike(res.body.config.plugins, ['ai']) |
| 153 | // The opt-in is persisted by mutating node.config (then api._persistConfig()). | |
| 153 | // The opt-in is persisted through node.setServicesConfig(), whose runtime | |
| 154 | // authority is <storage>/services.json rather than config.json. | |
| 154 | 155 | t.is(node.config.enableServices, true) |
| 155 | 156 | t.alike(node.config.plugins, ['ai']) |
| 157 | t.alike(node._servicesConfigCalls, [{ enabled: true, plugins: ['ai'] }]) | |
| 156 | 158 | }) |
| 157 | 159 | |
| 158 | 160 | test('config endpoint requires auth', async (t) => { |
| … | … | @@ -93,6 +93,21 @@ test('vrf is a builtin service provider', (t) => { |
| 93 | 93 | t.ok(BUILTIN_SERVICE_NAMES.includes('vrf')) |
| 94 | 94 | }) |
| 95 | 95 | |
| 96 | test('VRF service consumes Node PluginLoader context.config.vrfBeacon', async (t) => { | |
| 97 | const service = new VRFService({ seed: FIXED_SEED }) | |
| 98 | await service.start({ | |
| 99 | config: { | |
| 100 | vrfBeacon: { enabled: true, intervalMs: 12_345, retain: 7, domain: 'context-beacon' } | |
| 101 | } | |
| 102 | }) | |
| 103 | t.teardown(async () => { await service.stop() }) | |
| 104 | ||
| 105 | const info = await service['beacon-info']() | |
| 106 | t.is(info.intervalMs, 12_345) | |
| 107 | t.is(info.retain, 7) | |
| 108 | t.ok(service.beacon, 'Node runtime config enables the beacon') | |
| 109 | }) | |
| 110 | ||
| 96 | 111 | test('isVrfHttpRoute and resolveVrfRoute match the documented surface', (t) => { |
| 97 | 112 | t.ok(isVrfHttpRoute('/api/v1/vrf/beacon-info')) |
| 98 | 113 | t.ok(isVrfHttpRoute('/api/v1/vrf/beacon-latest')) |
| … | … | @@ -3,6 +3,7 @@ import { tmpdir } from 'os' |
| 3 | 3 | import { join } from 'path' |
| 4 | 4 | import { randomBytes } from 'crypto' |
| 5 | 5 | import { mkdir, rm } from 'fs/promises' |
| 6 | import Hyperdrive from 'hyperdrive' | |
| 6 | 7 | import { AppRegistry } from 'p2p-hiverelay/core/app-registry.js' |
| 7 | 8 | import { AppLifecycle } from 'p2p-hiverelay/core/relay-node/app-lifecycle.js' |
| 8 | 9 | import { LifecycleScope } from 'p2p-hiverelay/core/relay-node/lifecycle-scope.js' |
| … | … | @@ -102,6 +103,69 @@ test('AppLifecycle: failed durable unseed retains retry debt after ordered live |
| 102 | 103 | t.absent(lifecycle._retiringDrives.has(appKey)) |
| 103 | 104 | }) |
| 104 | 105 | |
| 106 | test('AppLifecycle: persistent drive pulls are finite ranges on one proved snapshot', async (t) => { | |
| 107 | const appKey = 'b'.repeat(64) | |
| 108 | const registry = new AppRegistry(null) | |
| 109 | const downloads = [] | |
| 110 | let mutationKey = null | |
| 111 | ||
| 112 | const makeCore = (name, length, byteLength, fork) => ({ | |
| 113 | length, | |
| 114 | byteLength, | |
| 115 | fork, | |
| 116 | async update () { return true }, | |
| 117 | snapshot () { | |
| 118 | return { | |
| 119 | length, | |
| 120 | byteLength, | |
| 121 | fork, | |
| 122 | async ready () {}, | |
| 123 | download (range) { | |
| 124 | downloads.push({ name, range }) | |
| 125 | return { destroy () {} } | |
| 126 | }, | |
| 127 | async close () {} | |
| 128 | } | |
| 129 | } | |
| 130 | }) | |
| 131 | const metaCore = makeCore('meta', 7, 700, 2) | |
| 132 | const blobCore = makeCore('blob', 11, 1100, 3) | |
| 133 | const drive = { | |
| 134 | version: 5, | |
| 135 | closed: false, | |
| 136 | closing: false, | |
| 137 | db: { core: metaCore }, | |
| 138 | blobs: { core: blobCore } | |
| 139 | } | |
| 140 | registry.set(appKey, { | |
| 141 | type: 'app', | |
| 142 | maxStorage: 4096, | |
| 143 | discoveryKey: Buffer.alloc(32, 2), | |
| 144 | drive | |
| 145 | }, { persist: false }) | |
| 146 | ||
| 147 | const lifecycle = new AppLifecycle({ | |
| 148 | appRegistry: registry, | |
| 149 | storageAdmission: { | |
| 150 | runKeyMutation (key, operation) { | |
| 151 | mutationKey = key | |
| 152 | return operation() | |
| 153 | } | |
| 154 | } | |
| 155 | }) | |
| 156 | await lifecycle._registerPersistentDownloads(appKey, drive) | |
| 157 | ||
| 158 | const entry = registry.get(appKey) | |
| 159 | t.is(mutationKey, `drive:${appKey}`, 'registration serializes on the drive storage key') | |
| 160 | t.alike(downloads, [ | |
| 161 | { name: 'meta', range: { start: 0, end: 7 } }, | |
| 162 | { name: 'blob', range: { start: 0, end: 11 } } | |
| 163 | ], 'only the proved metadata/blob lengths become persistent wants') | |
| 164 | t.is(entry.downloadRanges.length, 2, 'the finite range owners stay reachable for retirement') | |
| 165 | t.is(entry.downloadSnapshotCores.length, 2, 'the exact snapshot owners stay reachable for retirement') | |
| 166 | t.absent(entry.downloadRegistration, 'the registration settlement barrier is released') | |
| 167 | }) | |
| 168 | ||
| 105 | 169 | test('RelayNode: seedApp rolls back registry when explicit registry persist fails', async (t) => { |
| 106 | 170 | const storage = tmpStorage() |
| 107 | 171 | await mkdir(storage, { recursive: true }) |
| … | … | @@ -119,7 +183,11 @@ test('RelayNode: seedApp rolls back registry when explicit registry persist fail |
| 119 | 183 | }) |
| 120 | 184 | await node.start() |
| 121 | 185 | |
| 122 | const appKey = randomBytes(32).toString('hex') | |
| 186 | const writer = new Hyperdrive(node.store.session()) | |
| 187 | await writer.ready() | |
| 188 | await writer.put('/persist-fixture.txt', Buffer.from('durable rollback fixture')) | |
| 189 | const appKey = writer.key.toString('hex') | |
| 190 | await writer.close() | |
| 123 | 191 | const placeholder = { discoveryKey: null, maxStorage: 1024 * 1024, type: 'app' } |
| 124 | 192 | node.appRegistry.set(appKey, placeholder, { persist: false }) |
| 125 | 193 | const previous = node.appRegistry.get(appKey) |
| … | … | @@ -432,9 +500,18 @@ async function admissionNode (t) { |
| 432 | 500 | return node |
| 433 | 501 | } |
| 434 | 502 | |
| 503 | async function authorAdmissionDrive (node, label) { | |
| 504 | const writer = new Hyperdrive(node.store.session()) | |
| 505 | await writer.ready() | |
| 506 | await writer.put('/fixture.txt', Buffer.from(label)) | |
| 507 | const appKey = writer.key.toString('hex') | |
| 508 | await writer.close() | |
| 509 | return appKey | |
| 510 | } | |
| 511 | ||
| 435 | 512 | test('AppLifecycle: successful seed commits its storage admission reservation', async (t) => { |
| 436 | 513 | const node = await admissionNode(t) |
| 437 | const appKey = randomBytes(32).toString('hex') | |
| 514 | const appKey = await authorAdmissionDrive(node, 'successful admission') | |
| 438 | 515 | const cap = 1024 * 1024 |
| 439 | 516 | |
| 440 | 517 | await node.seedApp(appKey, { maxStorage: cap }) |
| … | … | @@ -458,7 +535,7 @@ test('AppLifecycle: successful seed commits its storage admission reservation', |
| 458 | 535 | |
| 459 | 536 | test('AppLifecycle: failed seed rolls its reservation back to the adopted commitment', async (t) => { |
| 460 | 537 | const node = await admissionNode(t) |
| 461 | const appKey = randomBytes(32).toString('hex') | |
| 538 | const appKey = await authorAdmissionDrive(node, 'failed admission rollback') | |
| 462 | 539 | const cap = 1024 * 1024 |
| 463 | 540 | |
| 464 | 541 | node.appRegistry.set(appKey, { discoveryKey: null, maxStorage: cap, type: 'app' }, { persist: false }) |
| … | … | @@ -483,7 +560,7 @@ test('AppLifecycle: failed seed rolls its reservation back to the adopted commit |
| 483 | 560 | |
| 484 | 561 | test('AppLifecycle: seed failure before the drive exists still rolls the reservation back', async (t) => { |
| 485 | 562 | const node = await admissionNode(t) |
| 486 | const appKey = randomBytes(32).toString('hex') | |
| 563 | const appKey = await authorAdmissionDrive(node, 'constructor failure rollback') | |
| 487 | 564 | |
| 488 | 565 | // Throws from the Hyperdrive construction — earlier than the drive-close |
| 489 | 566 | // catch that used to be _seedAppInner's only failure handler. |
| … | … | @@ -174,11 +174,28 @@ test('capability doc advertises notify and outboxlog service profiles when runni |
| 174 | 174 | t.ok(doc.features.includes('outboxlog-v1')) |
| 175 | 175 | t.is(doc.protocol_profile.services.notify.version, '0.1.0') |
| 176 | 176 | t.is(doc.protocol_profile.services.notify.payload.plaintext_allowed, false) |
| 177 | t.alike(doc.protocol_profile.services.notify.watch_sources, ['notify-feed-head', 'notify-outbox-lane']) | |
| 177 | t.alike(doc.protocol_profile.services.notify.watch_sources, ['notify-outbox-lane']) | |
| 178 | t.alike(doc.protocol_profile.services.notify.deprecated_watch_sources, ['notify-feed-head']) | |
| 178 | 179 | t.is(doc.protocol_profile.services.notify.limits.max_watches_per_receive_cap, 64) |
| 179 | 180 | t.is(doc.protocol_profile.services.outboxlog.model, 'single-writer-signed-outbox') |
| 180 | 181 | }) |
| 181 | 182 | |
| 183 | test('capability doc keeps legacy notify feed-head visible until exact lanes are wired', async (t) => { | |
| 184 | const relay = { | |
| 185 | config: { enableServices: true }, | |
| 186 | serviceRegistry: { | |
| 187 | services: new Map([ | |
| 188 | ['notify', { status: 'running', provider: { limits: () => ({ watchSources: ['notify-feed-head'], egress: { live: true } }) } }] | |
| 189 | ]), | |
| 190 | catalog: () => [{ name: 'notify', version: '0.1.0' }] | |
| 191 | } | |
| 192 | } | |
| 193 | ||
| 194 | const doc = buildCapabilityDoc({ relay }) | |
| 195 | t.alike(doc.protocol_profile.services.notify.watch_sources, ['notify-feed-head']) | |
| 196 | t.absent(doc.protocol_profile.services.notify.deprecated_watch_sources) | |
| 197 | }) | |
| 198 | ||
| 182 | 199 | test('relaykernel capability doc advertises only active kernel-compatible surfaces', async (t) => { |
| 183 | 200 | const kp = makeKeyPair() |
| 184 | 201 | const relay = { |
| … | … | @@ -622,6 +639,7 @@ function readyTor (overrides = {}) { |
| 622 | 639 | endpointKeyId: null, |
| 623 | 640 | rosterFile: null, |
| 624 | 641 | clientAuthKeys: [], |
| 642 | negativeProbe: null, | |
| 625 | 643 | pow: null, |
| 626 | 644 | localPort: 9200, |
| 627 | 645 | readVport: 80, |
| … | … | @@ -676,15 +694,16 @@ test('privacyTransports — ready onion entry shape and labels', async (t) => { |
| 676 | 694 | }) |
| 677 | 695 | |
| 678 | 696 | test('privacyTransports — client-auth mode from roster/keys; custody off drops custody kinds', async (t) => { |
| 679 | const withRoster = buildCapabilityDoc({ relay: relayWithTor(readyTor({ rosterFile: '/x/roster.json' })) }) | |
| 697 | const withRoster = buildCapabilityDoc({ relay: relayWithTor(readyTor({ rosterFile: '/x/roster.json', negativeProbe: true })) }) | |
| 680 | 698 | t.is(withRoster.privacyTransports[0].auth.mode, 'client-auth-v3') |
| 681 | 699 | t.alike(withRoster.privacyTransports[0].auth.enrollment, ['pairing-channel']) |
| 700 | t.is(withRoster.privacyTransports[0].negativeProbe, true) | |
| 682 | 701 | |
| 683 | const withKeys = buildCapabilityDoc({ relay: relayWithTor(readyTor({ clientAuthKeys: ['a'.repeat(52)] })) }) | |
| 702 | const withKeys = buildCapabilityDoc({ relay: relayWithTor(readyTor({ clientAuthKeys: ['a'.repeat(52)], negativeProbe: true })) }) | |
| 684 | 703 | t.is(withKeys.privacyTransports[0].auth.mode, 'client-auth-v3') |
| 685 | 704 | |
| 686 | 705 | const emptyButRestricted = buildCapabilityDoc({ |
| 687 | relay: relayWithTor(readyTor({ isRestrictedDiscoveryActive: () => true })) | |
| 706 | relay: relayWithTor(readyTor({ isRestrictedDiscoveryActive: () => true, negativeProbe: true })) | |
| 688 | 707 | }) |
| 689 | 708 | t.is(emptyButRestricted.privacyTransports[0].auth.mode, 'client-auth-v3') |
| 690 | 709 | t.alike(emptyButRestricted.privacyTransports[0].auth.enrollment, ['pairing-channel']) |
| … | … | @@ -693,6 +712,27 @@ test('privacyTransports — client-auth mode from roster/keys; custody off drops |
| 693 | 712 | t.absent(noCustody.privacyTransports[0].supports.includes('custody.commit')) |
| 694 | 713 | }) |
| 695 | 714 | |
| 715 | test('privacyTransports — restricted discovery requires a successful negative probe', async (t) => { | |
| 716 | for (const negativeProbe of [null, false, undefined]) { | |
| 717 | const doc = buildCapabilityDoc({ | |
| 718 | relay: relayWithTor(readyTor({ | |
| 719 | rosterFile: '/x/roster.json', | |
| 720 | negativeProbe | |
| 721 | })) | |
| 722 | }) | |
| 723 | t.absent('privacyTransports' in doc, `negativeProbe=${negativeProbe} must not advertise`) | |
| 724 | t.is(doc.onionGatewayUrl, null) | |
| 725 | } | |
| 726 | ||
| 727 | const proved = buildCapabilityDoc({ | |
| 728 | relay: relayWithTor(readyTor({ | |
| 729 | rosterFile: '/x/roster.json', | |
| 730 | negativeProbe: true | |
| 731 | })) | |
| 732 | }) | |
| 733 | t.is(proved.privacyTransports[0].negativeProbe, true) | |
| 734 | }) | |
| 735 | ||
| 696 | 736 | test('privacyTransports — dual vports map to roles; pow reported', async (t) => { |
| 697 | 737 | const tt = readyTor({ |
| 698 | 738 | pow: { enabled: true }, |
| … | … | @@ -198,6 +198,25 @@ test('HiveRelayClient - advanced mode: autoDiscover false', async (t) => { |
| 198 | 198 | t.is(client._started, true, 'still started') |
| 199 | 199 | }) |
| 200 | 200 | |
| 201 | test('HiveRelayClient - connection error guard precedes replication and identity parsing', async (t) => { | |
| 202 | const swarm = mockSwarm() | |
| 203 | const client = new HiveRelayClient({ swarm }) | |
| 204 | t.teardown(async () => { await client.destroy() }) | |
| 205 | await client.start() | |
| 206 | ||
| 207 | const conn = new EventEmitter() | |
| 208 | client.store = { | |
| 209 | replicate (stream) { | |
| 210 | const err = new Error('connection reset by peer') | |
| 211 | err.code = 'ECONNRESET' | |
| 212 | stream.emit('error', err) | |
| 213 | } | |
| 214 | } | |
| 215 | ||
| 216 | t.execution(() => swarm.emit('connection', conn, {}), 'early Noise errors are contained even before peer identity is available') | |
| 217 | client.store = null | |
| 218 | }) | |
| 219 | ||
| 201 | 220 | test('HiveRelayClient - advanced mode: getRelays empty initially', (t) => { |
| 202 | 221 | const swarm = mockSwarm() |
| 203 | 222 | const client = new HiveRelayClient({ swarm }) |
| … | … | @@ -356,6 +356,94 @@ test('applyOutboxlogNamespaceEnv: a persisted config.json namespace wins over en |
| 356 | 356 | t.is(config.outboxlog.namespace, 'saved-ns', 'persisted namespace wins over the env default') |
| 357 | 357 | }) |
| 358 | 358 | |
| 359 | test('runtime env defaults: Node packaging enables bounded services and Tor', async (t) => { | |
| 360 | const home = await mkdtemp(path.join(tmpdir(), 'hiverelay-runtime-env-')) | |
| 361 | t.teardown(async () => { | |
| 362 | await rm(home, { recursive: true, force: true }) | |
| 363 | }) | |
| 364 | const { applyRuntimeEnvDefaults, loadConfig } = await importLoaderWithHome(home) | |
| 365 | const overrides = {} | |
| 366 | ||
| 367 | applyRuntimeEnvDefaults(overrides, { | |
| 368 | HIVERELAY_ENABLE_SERVICES: 'true', | |
| 369 | HIVERELAY_OUTBOXLOG: '1', | |
| 370 | HIVERELAY_NOTIFY: 'true', | |
| 371 | HIVERELAY_STORAGE_PROOF: '1', | |
| 372 | HIVERELAY_TOR: '1', | |
| 373 | HIVERELAY_TOR_SOCKS_HOST: 'tor-sidecar', | |
| 374 | HIVERELAY_TOR_SOCKS_PORT: '19050', | |
| 375 | HIVERELAY_TOR_CONTROL_PORT: '19051', | |
| 376 | HIVERELAY_TOR_KEY_FILE: '/private/tor-key.blob', | |
| 377 | HIVERELAY_TOR_MIN_DAEMON_VERSION: '0.4.9.5', | |
| 378 | HIVERELAY_OUTBOXLOG_JOURNAL: 'hypercore-outboxes', | |
| 379 | HIVERELAY_OUTBOXLOG_MAX_JOURNAL_STORAGE: '512MB' | |
| 380 | }, {}) | |
| 381 | ||
| 382 | const config = loadConfig(overrides) | |
| 383 | t.is(config.enableServices, true) | |
| 384 | t.alike(config.plugins, ['notify', 'outboxlog', 'storage-proof']) | |
| 385 | t.is(config.transports.tor, true) | |
| 386 | t.is(config.tor.socksHost, 'tor-sidecar') | |
| 387 | t.is(config.tor.socksPort, 19050) | |
| 388 | t.is(config.tor.controlPort, 19051) | |
| 389 | t.is(config.tor.keyFile, '/private/tor-key.blob') | |
| 390 | t.is(config.tor.minDaemonVersion, '0.4.9.5') | |
| 391 | t.is(config.outboxlog.journal, 'hypercore-outboxes') | |
| 392 | t.is(config.outboxlog.maxJournalStorageBytes, 512 * 1024 * 1024) | |
| 393 | }) | |
| 394 | ||
| 395 | test('runtime env defaults: persisted fields and CLI flags remain authoritative', async (t) => { | |
| 396 | const home = await mkdtemp(path.join(tmpdir(), 'hiverelay-runtime-precedence-')) | |
| 397 | t.teardown(async () => { | |
| 398 | await rm(home, { recursive: true, force: true }) | |
| 399 | }) | |
| 400 | const { applyRuntimeEnvDefaults } = await importLoaderWithHome(home) | |
| 401 | const overrides = { transports: { tor: true }, tor: { socksPort: 29050 } } | |
| 402 | ||
| 403 | applyRuntimeEnvDefaults(overrides, { | |
| 404 | HIVERELAY_ENABLE_SERVICES: '1', | |
| 405 | HIVERELAY_NOTIFY: '1', | |
| 406 | HIVERELAY_TOR: '0', | |
| 407 | HIVERELAY_TOR_SOCKS_HOST: 'env-host', | |
| 408 | HIVERELAY_TOR_SOCKS_PORT: '39050', | |
| 409 | HIVERELAY_OUTBOXLOG_JOURNAL: 'hypercore-outboxes' | |
| 410 | }, { | |
| 411 | enableServices: false, | |
| 412 | plugins: ['identity'], | |
| 413 | tor: { socksHost: 'persisted-host' }, | |
| 414 | outboxlog: { journal: 'hypercore' } | |
| 415 | }) | |
| 416 | ||
| 417 | t.absent(Object.prototype.hasOwnProperty.call(overrides, 'enableServices'), 'persisted service toggle is not shadowed') | |
| 418 | t.absent(Object.prototype.hasOwnProperty.call(overrides, 'plugins'), 'persisted plugin selection is not shadowed') | |
| 419 | t.is(overrides.transports.tor, true, 'CLI Tor flag wins') | |
| 420 | t.absent(overrides.tor.socksHost, 'persisted Tor host is not shadowed') | |
| 421 | t.is(overrides.tor.socksPort, 29050, 'CLI Tor port wins') | |
| 422 | t.absent(overrides.outboxlog, 'persisted journal is not shadowed') | |
| 423 | }) | |
| 424 | ||
| 425 | test('runtime env defaults: push credentials load from a private JSON file path', async (t) => { | |
| 426 | const home = await mkdtemp(path.join(tmpdir(), 'hiverelay-notify-env-')) | |
| 427 | t.teardown(async () => { | |
| 428 | await rm(home, { recursive: true, force: true }) | |
| 429 | }) | |
| 430 | const { applyRuntimeEnvDefaults } = await importLoaderWithHome(home) | |
| 431 | const descriptor = { kind: 'webpush', credentials: { subject: 'mailto:ops@example.test' } } | |
| 432 | const overrides = {} | |
| 433 | applyRuntimeEnvDefaults(overrides, { | |
| 434 | HIVERELAY_NOTIFY_PUSH_CONFIG: '/run/secrets/hiverelay-notify.json' | |
| 435 | }, {}, { | |
| 436 | readTextFile: (requested) => { | |
| 437 | t.is(requested, '/run/secrets/hiverelay-notify.json') | |
| 438 | return JSON.stringify(descriptor) | |
| 439 | } | |
| 440 | }) | |
| 441 | t.alike(overrides.notify.push, descriptor) | |
| 442 | ||
| 443 | t.exception(() => applyRuntimeEnvDefaults({}, { HIVERELAY_TOR_SOCKS_PORT: 'nope' }, {}), /Invalid HIVERELAY_TOR_SOCKS_PORT/) | |
| 444 | t.exception(() => applyRuntimeEnvDefaults({}, { HIVERELAY_NOTIFY: 'sometimes' }, {}), /Invalid HIVERELAY_NOTIFY/) | |
| 445 | }) | |
| 446 | ||
| 359 | 447 | function execCli (argv, env) { |
| 360 | 448 | return new Promise((resolve, reject) => { |
| 361 | 449 | execFile(process.execPath, ['packages/core/cli/index.js', ...argv], { |
| … | … | @@ -234,7 +234,7 @@ test('operator pulse renders management overview and escapes namespace names', ( |
| 234 | 234 | ' suppress: document.getElementById("statSuppress").textContent,', |
| 235 | 235 | ' writers: document.getElementById("statOutboxWriters").textContent,', |
| 236 | 236 | ' privacy: document.getElementById("statPrivacy").textContent,', |
| 237 | ' nsAssigned: document.getElementById("namespacesBody").innerHTMLAssignments[0] || "",', | |
| 237 | ' nsHtml: document.getElementById("namespacesBody").innerHTML,', | |
| 238 | 238 | ' nsDisplay: document.getElementById("namespacesPanel").style.display,', |
| 239 | 239 | ' compDisplay: document.getElementById("compliancePanel").style.display,', |
| 240 | 240 | ' walDisplay: document.getElementById("walPanel").style.display,', |
| … | … | @@ -244,7 +244,7 @@ test('operator pulse renders management overview and escapes namespace names', ( |
| 244 | 244 | ' shardDisplay: document.getElementById("shardPanel").style.display,', |
| 245 | 245 | ' shardPut: document.getElementById("shardPut").textContent,', |
| 246 | 246 | ' armed: document.getElementById("compAdminArmed").textContent,', |
| 247 | ' attentionAssigned: (document.getElementById("attentionList").innerHTMLAssignments || []).join("")', | |
| 247 | ' attentionHtml: document.getElementById("attentionList").innerHTML', | |
| 248 | 248 | '})' |
| 249 | 249 | ].join('\n'))) |
| 250 | 250 | |
| … | … | @@ -263,11 +263,10 @@ test('operator pulse renders management overview and escapes namespace names', ( |
| 263 | 263 | t.is(out.shardDisplay, '') |
| 264 | 264 | t.is(out.shardPut, '10') |
| 265 | 265 | t.is(out.armed, 'ARMED') |
| 266 | // Use the raw assignment (FakeElement re-escapes on get). | |
| 267 | t.ok(out.nsAssigned.includes('<img src=x onerror=alert(1)>')) | |
| 268 | t.ok(out.nsAssigned.includes('blind')) | |
| 269 | t.absent(out.nsAssigned.includes('<img')) | |
| 270 | t.absent(out.attentionAssigned.includes('T2 custody with gateway'), 'gateway off → no co-location warning') | |
| 266 | t.ok(out.nsHtml.includes('<img src=x onerror=alert(1)>')) | |
| 267 | t.ok(out.nsHtml.includes('blind')) | |
| 268 | t.absent(out.nsHtml.includes('<img')) | |
| 269 | t.absent(out.attentionHtml.includes('T2 custody with gateway'), 'gateway off → no co-location warning') | |
| 271 | 270 | }) |
| 272 | 271 | |
| 273 | 272 | test('operator pulse public overview hides management panels', (t) => { |
| … | … | @@ -394,10 +393,13 @@ function renderDomWith (body) { |
| 394 | 393 | function renderOperatorPulseWith (body) { |
| 395 | 394 | const elements = {} |
| 396 | 395 | const source = [ |
| 397 | extractFunction('escapeHtml'), | |
| 396 | extractFunction('clearNode'), | |
| 397 | extractFunction('appendEl'), | |
| 398 | 398 | extractFunction('formatBytes'), |
| 399 | 399 | extractFunction('formatCount'), |
| 400 | 400 | extractFunction('setText'), |
| 401 | extractFunction('renderNamespaceRows'), | |
| 402 | extractFunction('renderAttentionList'), | |
| 401 | 403 | extractFunction('updateOperatorPulse'), |
| 402 | 404 | body |
| 403 | 405 | ].join('\n') |
| … | … | @@ -408,6 +410,12 @@ function renderOperatorPulseWith (body) { |
| 408 | 410 | Math, |
| 409 | 411 | isNaN, |
| 410 | 412 | document: { |
| 413 | createElement (tag) { | |
| 414 | return new FakeElement(tag) | |
| 415 | }, | |
| 416 | createTextNode (text) { | |
| 417 | return new FakeText(text) | |
| 418 | }, | |
| 411 | 419 | getElementById (id) { |
| 412 | 420 | if (!elements[id]) { |
| 413 | 421 | const el = new FakeElement('div') |
| … | … | @@ -6,6 +6,7 @@ const watchdog = readFileSync('fleet/health-watchdog.sh', 'utf8') |
| 6 | 6 | const fleetStatus = readFileSync('fleet/fleet-status.sh', 'utf8') |
| 7 | 7 | const deployVps = readFileSync('scripts/deploy-vps.sh', 'utf8') |
| 8 | 8 | const rolloutCheck = readFileSync('scripts/check-fleet-rollout.mjs', 'utf8') |
| 9 | const fleetServiceProbe = readFileSync('scripts/probe-fleet-services.mjs', 'utf8') | |
| 9 | 10 | const relayJanitor = readFileSync('scripts/relay-janitor.js', 'utf8') |
| 10 | 11 | const cli = readFileSync('packages/core/cli/index.js', 'utf8') |
| 11 | 12 | |
| … | … | @@ -80,6 +81,19 @@ test('fleet health tooling discovers API keys from root-only env files', (t) => |
| 80 | 81 | t.ok(relayJanitor.includes('root-only env file first')) |
| 81 | 82 | }) |
| 82 | 83 | |
| 84 | test('fleet status preserves prerelease versions and reads counters from status', (t) => { | |
| 85 | t.ok(fleetStatus.includes('require(process.env.HOME + "/hiverelay/package.json").version')) | |
| 86 | t.absent(fleetStatus.includes('tr -dc \'0-9.\'')) | |
| 87 | t.ok(fleetStatus.includes('http://127.0.0.1:9100/status')) | |
| 88 | t.ok(fleetStatus.includes('HEALTH="$H" STATUS="$S" python3 -c')) | |
| 89 | t.ok(fleetStatus.includes('s.get("seededApps","?")')) | |
| 90 | t.ok(fleetStatus.includes('s.get("connections","?")')) | |
| 91 | t.ok(fleetStatus.includes('systemctl is-enabled hiverelay-updater.timer')) | |
| 92 | t.ok(fleetStatus.includes('systemctl is-active hiverelay-updater.timer')) | |
| 93 | t.ok(fleetStatus.includes('/etc/hiverelay-updater.conf')) | |
| 94 | t.ok(fleetStatus.includes('UPDATER CONFIGURED ASSIGNED')) | |
| 95 | }) | |
| 96 | ||
| 83 | 97 | test('relay janitor keeps remote API keys out of local ssh argv', (t) => { |
| 84 | 98 | t.ok(relayJanitor.includes('remoteHasApiKey')) |
| 85 | 99 | t.ok(relayJanitor.includes('remoteUnseed')) |
| … | … | @@ -114,6 +128,12 @@ test('fleet status sanitizes terminal output and validates inventory before ssh' |
| 114 | 128 | t.ok(fleetStatus.includes('valid_host()')) |
| 115 | 129 | t.ok(fleetStatus.includes('valid_channel()')) |
| 116 | 130 | t.ok(fleetStatus.includes('valid_key_path()')) |
| 131 | t.ok(fleetStatus.includes('valid_name_list()')) | |
| 132 | t.ok(fleetStatus.includes('HIVERELAY_FLEET_INCLUDE')) | |
| 133 | t.ok(fleetStatus.includes('HIVERELAY_FLEET_EXCLUDE')) | |
| 134 | t.ok(fleetStatus.includes('INCLUDE="$INCLUDE" EXCLUDE="$EXCLUDE" python3 -c')) | |
| 135 | t.ok(fleetStatus.includes('if include and r["name"] not in include:')) | |
| 136 | t.ok(fleetStatus.includes('if r["name"] in exclude:')) | |
| 117 | 137 | t.ok(fleetStatus.includes('BADHOST')) |
| 118 | 138 | t.ok(fleetStatus.includes('BADCHAN')) |
| 119 | 139 | t.ok(fleetStatus.includes('BADKEY')) |
| … | … | @@ -126,6 +146,30 @@ test('fleet status sanitizes terminal output and validates inventory before ssh' |
| 126 | 146 | )) |
| 127 | 147 | }) |
| 128 | 148 | |
| 149 | test('fleet service probe scopes relay ownership before opening connections', (t) => { | |
| 150 | t.ok(fleetServiceProbe.includes("optionValue('--include')")) | |
| 151 | t.ok(fleetServiceProbe.includes("optionValue('--exclude')")) | |
| 152 | t.ok(fleetServiceProbe.includes('HIVERELAY_FLEET_INCLUDE')) | |
| 153 | t.ok(fleetServiceProbe.includes('HIVERELAY_FLEET_EXCLUDE')) | |
| 154 | t.ok(fleetServiceProbe.indexOf('const relays = inventoryRelays.filter') < fleetServiceProbe.indexOf('Promise.all(relays.map(probe))')) | |
| 155 | t.ok(fleetServiceProbe.includes('if (!knownNames.has(name)) throw new Error')) | |
| 156 | t.ok(fleetServiceProbe.includes("throw new Error('relay scope selected no inventory entries')")) | |
| 157 | t.ok(fleetServiceProbe.includes('base.error = safeProbeError(err)')) | |
| 158 | t.ok(fleetServiceProbe.includes("return 'probe failed'")) | |
| 159 | t.absent(fleetServiceProbe.includes('err?.message?.slice')) | |
| 160 | t.ok(fleetServiceProbe.includes("inventoryLabel = inv === join(REPO, 'fleet', 'relays.json') ? 'fleet/relays.json' : 'operator inventory'")) | |
| 161 | t.ok(fleetServiceProbe.includes('/.well-known/hiverelay.json')) | |
| 162 | t.ok(fleetServiceProbe.includes("notifyWatchSources.includes('notify-outbox-lane')")) | |
| 163 | t.ok(fleetServiceProbe.includes('notifyProfile?.egress?.live === true')) | |
| 164 | t.ok(fleetServiceProbe.includes("features.includes('notify-v1')")) | |
| 165 | t.ok(fleetServiceProbe.includes('unsafe legacy wake advertisements:')) | |
| 166 | t.ok(fleetServiceProbe.includes('signed exact-lane wake:')) | |
| 167 | t.ok(fleetServiceProbe.includes('advertised outbox mailbox:')) | |
| 168 | t.ok(fleetServiceProbe.includes("privacyTransports.find(entry => entry?.network === 'tor')")) | |
| 169 | t.ok(fleetServiceProbe.includes('restricted Tor endpoints with negative proof:')) | |
| 170 | t.ok(fleetServiceProbe.includes('advertised one-hop forward relays:')) | |
| 171 | }) | |
| 172 | ||
| 129 | 173 | test('relay CLI keeps high-frequency status output off service logs', (t) => { |
| 130 | 174 | t.ok(cli.includes('const interactiveStatus = process.stdout.isTTY === true')) |
| 131 | 175 | t.ok(cli.includes('const statusIntervalMs = interactiveStatus ? 5000 : 60000')) |
| … | … | @@ -91,8 +91,8 @@ test('dashboard notify panel fetches /api/manage/notify and maps real status fie |
| 91 | 91 | const fetchFn = html.slice(html.indexOf('async function fetchNotifyPanel')) |
| 92 | 92 | const end = fetchFn.indexOf('async function fetchVrfPanel') |
| 93 | 93 | const body = end > 0 ? fetchFn.slice(0, end) : fetchFn |
| 94 | t.absent(body.includes("/api/v1/notify/status"), 'does not GET public status') | |
| 95 | t.absent(body.includes("/api/v1/notify/capabilities"), 'does not fall back to capabilities') | |
| 94 | t.absent(body.includes('/api/v1/notify/status'), 'does not GET public status') | |
| 95 | t.absent(body.includes('/api/v1/notify/capabilities'), 'does not fall back to capabilities') | |
| 96 | 96 | |
| 97 | 97 | // applyNotifyPanel maps counts.deliveryEvents / limits.egress / flat fields. |
| 98 | 98 | t.ok(html.includes('counts.deliveryEvents') || html.includes('status.deliveryAttempts') || html.includes('deliveryAttempts'), |
| … | … | @@ -284,6 +284,29 @@ test('RelayNode - relaykernel locks persisted services overrides off', async (t) |
| 284 | 284 | t.is(persisted.lockReason, 'relaykernel-profile') |
| 285 | 285 | }) |
| 286 | 286 | |
| 287 | test('RelayNode - persisted services disable overrides package defaults', async (t) => { | |
| 288 | const storage = tmpStorage() | |
| 289 | await writeFile(path.join(storage, 'services.json'), JSON.stringify({ | |
| 290 | enabled: false, | |
| 291 | plugins: ['notify', 'outboxlog'] | |
| 292 | })) | |
| 293 | ||
| 294 | const node = new RelayNode({ | |
| 295 | storage, | |
| 296 | enableAPI: false, | |
| 297 | enableServices: true, | |
| 298 | plugins: ['notify', 'outboxlog', 'storage-proof'] | |
| 299 | }) | |
| 300 | t.teardown(async () => { | |
| 301 | try { await node.store.close() } catch (_) {} | |
| 302 | await rm(storage, { recursive: true, force: true }) | |
| 303 | }) | |
| 304 | ||
| 305 | await node._loadServicesOverride() | |
| 306 | t.is(node.config.enableServices, false, 'operator disable remains authoritative after restart') | |
| 307 | t.alike(node.config.plugins, ['notify', 'outboxlog'], 'selection is retained for later re-enable') | |
| 308 | }) | |
| 309 | ||
| 287 | 310 | test('RelayNode - startup DHT flush is bounded', async (t) => { |
| 288 | 311 | const node = new RelayNode({ storage: tmpStorage(), enableAPI: false, dhtFlushTimeoutMs: 5 }) |
| 289 | 312 | t.teardown(async () => { |
| … | … | @@ -22,12 +22,10 @@ const FLEET_RELAYS = Object.freeze([ |
| 22 | 22 | ['utah', 'canary', '42%'], |
| 23 | 23 | ['utah-us', 'stable', '51%'], |
| 24 | 24 | ['utah-2gb-a', 'stable', '50%'], |
| 25 | ['utah-0.5gb', 'canary', '49%'], | |
| 26 | 25 | ['utah-8gb', 'stable', '48%'], |
| 27 | 26 | ['miami', 'stable', '44%'], |
| 28 | 27 | ['sing-1', 'stable', '52%'], |
| 29 | 28 | ['sing-2', 'stable', '53%'], |
| 30 | ['sydney', 'stable', '45%'], | |
| 31 | 29 | ['bern', 'canary', '54%'], |
| 32 | 30 | ['amsterdam', 'stable', '46%'], |
| 33 | 31 | ['dubai', 'stable', '47%'] |
| … | … | @@ -26,12 +26,10 @@ const FLEET_RELAYS = Object.freeze([ |
| 26 | 26 | ['utah', 'canary'], |
| 27 | 27 | ['utah-us', 'stable'], |
| 28 | 28 | ['utah-2gb-a', 'stable'], |
| 29 | ['utah-0.5gb', 'canary'], | |
| 30 | 29 | ['utah-8gb', 'stable'], |
| 31 | 30 | ['miami', 'stable'], |
| 32 | 31 | ['sing-1', 'stable'], |
| 33 | 32 | ['sing-2', 'stable'], |
| 34 | ['sydney', 'stable'], | |
| 35 | 33 | ['bern', 'canary'], |
| 36 | 34 | ['amsterdam', 'stable'], |
| 37 | 35 | ['dubai', 'stable'] |
| … | … | @@ -25,6 +25,9 @@ function k (label) { |
| 25 | 25 | function mockNode (registry, opts = {}) { |
| 26 | 26 | return { |
| 27 | 27 | appRegistry: registry, |
| 28 | storageAdmission: { | |
| 29 | canAcknowledge: () => true | |
| 30 | }, | |
| 28 | 31 | swarm: opts.swarm || { |
| 29 | 32 | join: () => {}, |
| 30 | 33 | flush: () => Promise.resolve() |
| … | … | @@ -52,6 +55,40 @@ function mockDrive ({ |
| 52 | 55 | blobsComplete = true, |
| 53 | 56 | blobLength = 8 |
| 54 | 57 | } = {}) { |
| 58 | function mockCore ({ length, byteLength, canDownload = () => true }) { | |
| 59 | const state = { | |
| 60 | length, | |
| 61 | byteLength, | |
| 62 | fork: 0, | |
| 63 | update: async () => true | |
| 64 | } | |
| 65 | state.snapshot = () => { | |
| 66 | const snapshot = { | |
| 67 | length: state.length, | |
| 68 | byteLength: state.byteLength, | |
| 69 | fork: state.fork, | |
| 70 | ready: async () => {}, | |
| 71 | close: async () => {}, | |
| 72 | download: () => { | |
| 73 | const tracker = { | |
| 74 | destroy: () => {}, | |
| 75 | done: async () => { | |
| 76 | if (!canDownload()) { | |
| 77 | await new Promise(resolve => { | |
| 78 | const tmr = setTimeout(resolve, 100_000) | |
| 79 | if (typeof tmr.unref === 'function') tmr.unref() | |
| 80 | }) | |
| 81 | } | |
| 82 | } | |
| 83 | } | |
| 84 | return tracker | |
| 85 | } | |
| 86 | } | |
| 87 | return snapshot | |
| 88 | } | |
| 89 | return state | |
| 90 | } | |
| 91 | ||
| 55 | 92 | const drive = { |
| 56 | 93 | closed: false, |
| 57 | 94 | closing: false, |
| … | … | @@ -72,18 +109,25 @@ function mockDrive ({ |
| 72 | 109 | } |
| 73 | 110 | return dl |
| 74 | 111 | }, |
| 75 | blobs: { | |
| 76 | core: { | |
| 77 | length: blobLength, | |
| 78 | has: async (start, end) => { | |
| 79 | if (drive.blobs.core.length === 0) return true | |
| 80 | return blobsComplete | |
| 81 | } | |
| 82 | } | |
| 83 | }, | |
| 112 | _blobDownloadComplete: blobsComplete && downloadOk, | |
| 84 | 113 | // Test helper: flip the partial-pin signal mid-test so we can model |
| 85 | 114 | // "first repair pass pulls some blocks, second pass pulls the rest." |
| 86 | _setBlobsComplete: (v) => { drive.blobs.core.has = async () => v } | |
| 115 | _setBlobsComplete: (v) => { | |
| 116 | drive.blobs.core.has = async () => v | |
| 117 | drive._blobDownloadComplete = v | |
| 118 | } | |
| 119 | } | |
| 120 | drive.db = { core: mockCore({ length: 1, byteLength: 128 }) } | |
| 121 | drive.blobs = { | |
| 122 | core: mockCore({ | |
| 123 | length: blobLength, | |
| 124 | byteLength: blobLength * 128, | |
| 125 | canDownload: () => drive._blobDownloadComplete | |
| 126 | }) | |
| 127 | } | |
| 128 | drive.blobs.core.has = async () => { | |
| 129 | if (drive.blobs.core.length === 0) return true | |
| 130 | return blobsComplete | |
| 87 | 131 | } |
| 88 | 132 | return drive |
| 89 | 133 | } |
| … | … | @@ -112,7 +156,7 @@ test('repair: succeeds when drive update yields version > 0', async (t) => { |
| 112 | 156 | const { dir, cleanup } = tmpDir(); t.teardown(cleanup) |
| 113 | 157 | const reg = new AppRegistry(dir) |
| 114 | 158 | const drive = mockDrive({ version: 0, updateOk: true }) |
| 115 | reg.set(k('cc'), { type: 'app', drive, discoveryKey: drive.discoveryKey }) | |
| 159 | reg.set(k('cc'), { type: 'app', drive, discoveryKey: drive.discoveryKey, maxStorage: 4096 }) | |
| 116 | 160 | const lifecycle = new AppLifecycle(mockNode(reg)) |
| 117 | 161 | const ok = await lifecycle.repairUnanchored(k('cc'), { updateTimeout: 500, downloadTimeout: 500 }) |
| 118 | 162 | t.is(ok, true, 'returns true') |
| … | … | @@ -125,7 +169,7 @@ test('repair: returns false on update timeout', async (t) => { |
| 125 | 169 | const { dir, cleanup } = tmpDir(); t.teardown(cleanup) |
| 126 | 170 | const reg = new AppRegistry(dir) |
| 127 | 171 | const drive = mockDrive({ version: 0, updateOk: false }) |
| 128 | reg.set(k('dd'), { type: 'app', drive, discoveryKey: drive.discoveryKey }) | |
| 172 | reg.set(k('dd'), { type: 'app', drive, discoveryKey: drive.discoveryKey, maxStorage: 4096 }) | |
| 129 | 173 | const lifecycle = new AppLifecycle(mockNode(reg)) |
| 130 | 174 | const ok = await lifecycle.repairUnanchored(k('dd'), { updateTimeout: 200, downloadTimeout: 200 }) |
| 131 | 175 | t.is(ok, false) |
| … | … | @@ -137,7 +181,7 @@ test('repair: returns false on update throw', async (t) => { |
| 137 | 181 | const { dir, cleanup } = tmpDir(); t.teardown(cleanup) |
| 138 | 182 | const reg = new AppRegistry(dir) |
| 139 | 183 | const drive = mockDrive({ version: 0, throwsOnUpdate: true }) |
| 140 | reg.set(k('ee'), { type: 'app', drive, discoveryKey: drive.discoveryKey }) | |
| 184 | reg.set(k('ee'), { type: 'app', drive, discoveryKey: drive.discoveryKey, maxStorage: 4096 }) | |
| 141 | 185 | const lifecycle = new AppLifecycle(mockNode(reg)) |
| 142 | 186 | const ok = await lifecycle.repairUnanchored(k('ee'), { updateTimeout: 500, downloadTimeout: 500 }) |
| 143 | 187 | t.is(ok, false) |
| … | … | @@ -212,7 +256,7 @@ test('repair: partial pin (metadata replicated, blocks missing) stays unanchored |
| 212 | 256 | const reg = new AppRegistry(dir) |
| 213 | 257 | // Drive replies "metadata synced" but blob core has gaps. |
| 214 | 258 | const drive = mockDrive({ version: 5, updateOk: true, downloadOk: true, blobsComplete: false }) |
| 215 | reg.set(k('partial'), { type: 'app', drive, discoveryKey: drive.discoveryKey }) | |
| 259 | reg.set(k('partial'), { type: 'app', drive, discoveryKey: drive.discoveryKey, maxStorage: 4096 }) | |
| 216 | 260 | const lifecycle = new AppLifecycle(mockNode(reg)) |
| 217 | 261 | const ok = await lifecycle.repairUnanchored(k('partial'), { updateTimeout: 500, downloadTimeout: 500 }) |
| 218 | 262 | t.is(ok, false, 'repair reports failure on partial pin (would have returned true before the fix)') |
| … | … | @@ -224,7 +268,7 @@ test('repair: partial pin gets anchored once all blob blocks land', async (t) => |
| 224 | 268 | const { dir, cleanup } = tmpDir(); t.teardown(cleanup) |
| 225 | 269 | const reg = new AppRegistry(dir) |
| 226 | 270 | const drive = mockDrive({ version: 5, updateOk: true, downloadOk: true, blobsComplete: false }) |
| 227 | reg.set(k('eventually'), { type: 'app', drive, discoveryKey: drive.discoveryKey }) | |
| 271 | reg.set(k('eventually'), { type: 'app', drive, discoveryKey: drive.discoveryKey, maxStorage: 4096 }) | |
| 228 | 272 | const lifecycle = new AppLifecycle(mockNode(reg)) |
| 229 | 273 | |
| 230 | 274 | // First pass: blocks missing → not anchored |
| … | … | @@ -277,11 +277,13 @@ test('restricted discovery activation failure leaves a running public service ho |
| 277 | 277 | const tt = new TorTransport({ |
| 278 | 278 | socksPort, |
| 279 | 279 | localPort: 9100, |
| 280 | _controlFactory: factory | |
| 280 | _controlFactory: factory, | |
| 281 | _probeConnectionFactory: async () => ({ socket: { destroy () {} } }) | |
| 281 | 282 | }) |
| 282 | 283 | await tt.start() |
| 283 | 284 | control.emit('event', 'HS_DESC UPLOADED ' + SERVICE_ID + ' first') |
| 284 | 285 | control.emit('event', 'HS_DESC UPLOADED ' + SERVICE_ID + ' second') |
| 286 | await new Promise((resolve) => setTimeout(resolve, 0)) | |
| 285 | 287 | t.is(tt.health, TorHealth.READY) |
| 286 | 288 | |
| 287 | 289 | await t.exception(tt.addAuthClient(ALICE_PUB), /requires persistent keyFile mode/) |
| … | … | @@ -472,11 +474,17 @@ test('health probe cannot bypass descriptor uploads after a roster rebuild', asy |
| 472 | 474 | await tt.stop() |
| 473 | 475 | }) |
| 474 | 476 | |
| 475 | test('health — descriptor uploads drive readiness without probe vport', async (t) => { | |
| 477 | test('health — explicit probe opt-out uses descriptor uploads only', async (t) => { | |
| 476 | 478 | const socksPort = await fakeSocks(t) |
| 477 | 479 | const dir = tmpdir(t) |
| 478 | 480 | const { control, factory } = fakeFactory({}) |
| 479 | const tt = new TorTransport({ socksPort, localPort: 9100, keyFile: path.join(dir, 'hs-key.blob'), _controlFactory: factory }) | |
| 481 | const tt = new TorTransport({ | |
| 482 | socksPort, | |
| 483 | localPort: 9100, | |
| 484 | keyFile: path.join(dir, 'hs-key.blob'), | |
| 485 | health: { probeVport: false }, | |
| 486 | _controlFactory: factory | |
| 487 | }) | |
| 480 | 488 | const states = [] |
| 481 | 489 | tt.on('health', (s) => states.push(s)) |
| 482 | 490 | await tt.start() |
| … | … | @@ -485,7 +493,7 @@ test('health — descriptor uploads drive readiness without probe vport', async |
| 485 | 493 | control.emit('event', 'HS_DESC UPLOADED ' + SERVICE_ID + ' xyz') |
| 486 | 494 | t.is(tt.health, TorHealth.KEY_LOADED) // minDescriptorUploads = 2 |
| 487 | 495 | control.emit('event', 'HS_DESC UPLOADED ' + SERVICE_ID + ' abc') |
| 488 | t.is(tt.health, TorHealth.READY) // no probeVport → uploads are the signal | |
| 496 | t.is(tt.health, TorHealth.READY) // explicit opt-out → uploads are the signal | |
| 489 | 497 | t.ok(states.includes(TorHealth.DESCRIPTOR_UPLOADED)) |
| 490 | 498 | |
| 491 | 499 | const info = tt.getInfo() |
| … | … | @@ -493,10 +501,84 @@ test('health — descriptor uploads drive readiness without probe vport', async |
| 493 | 501 | t.is(info.persistent, true) |
| 494 | 502 | t.is(info.authClients, 0) |
| 495 | 503 | t.is(info.descriptorUploads, 2) |
| 504 | t.is(info.negativeProbe, null) | |
| 496 | 505 | await tt.stop() |
| 497 | 506 | t.is(tt.getInfo().health, TorHealth.DISABLED) |
| 498 | 507 | }) |
| 499 | 508 | |
| 509 | test('health — restricted discovery requires an unauthorized negative probe', async (t) => { | |
| 510 | const socksPort = await fakeSocks(t) | |
| 511 | const dir = tmpdir(t) | |
| 512 | const { control, factory } = fakeFactory({}) | |
| 513 | let probes = 0 | |
| 514 | const tt = new TorTransport({ | |
| 515 | socksPort, | |
| 516 | localPort: 9100, | |
| 517 | keyFile: path.join(dir, 'hs-key.blob'), | |
| 518 | rosterFile: path.join(dir, 'roster.json'), | |
| 519 | _controlFactory: factory, | |
| 520 | _probeConnectionFactory: async () => { | |
| 521 | probes++ | |
| 522 | throw new Error('Socks5 proxy rejected connection - HostUnreachable') | |
| 523 | } | |
| 524 | }) | |
| 525 | await tt.start() | |
| 526 | control.emit('event', 'HS_DESC UPLOADED ' + SERVICE_ID + ' first') | |
| 527 | control.emit('event', 'HS_DESC UPLOADED ' + SERVICE_ID + ' second') | |
| 528 | await new Promise((resolve) => setTimeout(resolve, 0)) | |
| 529 | t.is(probes, 1) | |
| 530 | t.is(tt.health, TorHealth.READY) | |
| 531 | t.is(tt.getInfo().negativeProbe, true) | |
| 532 | await tt.stop() | |
| 533 | }) | |
| 534 | ||
| 535 | test('health — restricted discovery degrades immediately if an unauthenticated probe connects', async (t) => { | |
| 536 | const socksPort = await fakeSocks(t) | |
| 537 | const dir = tmpdir(t) | |
| 538 | const { control, factory } = fakeFactory({}) | |
| 539 | const tt = new TorTransport({ | |
| 540 | socksPort, | |
| 541 | localPort: 9100, | |
| 542 | keyFile: path.join(dir, 'hs-key.blob'), | |
| 543 | rosterFile: path.join(dir, 'roster.json'), | |
| 544 | _controlFactory: factory, | |
| 545 | _probeConnectionFactory: async () => ({ socket: { destroy () {} } }) | |
| 546 | }) | |
| 547 | await tt.start() | |
| 548 | control.emit('event', 'HS_DESC UPLOADED ' + SERVICE_ID + ' first') | |
| 549 | control.emit('event', 'HS_DESC UPLOADED ' + SERVICE_ID + ' second') | |
| 550 | await new Promise((resolve) => setTimeout(resolve, 0)) | |
| 551 | t.is(tt.health, TorHealth.DEGRADED) | |
| 552 | t.is(tt.getInfo().negativeProbe, false) | |
| 553 | await tt.stop() | |
| 554 | }) | |
| 555 | ||
| 556 | test('health — an ambiguous restricted probe failure never passes the auth gate', async (t) => { | |
| 557 | const socksPort = await fakeSocks(t) | |
| 558 | const dir = tmpdir(t) | |
| 559 | const { control, factory } = fakeFactory({}) | |
| 560 | const tt = new TorTransport({ | |
| 561 | socksPort, | |
| 562 | localPort: 9100, | |
| 563 | keyFile: path.join(dir, 'hs-key.blob'), | |
| 564 | rosterFile: path.join(dir, 'roster.json'), | |
| 565 | health: { probeFailLimit: 3 }, | |
| 566 | _controlFactory: factory, | |
| 567 | _probeConnectionFactory: async () => { throw new Error('connect ETIMEDOUT') } | |
| 568 | }) | |
| 569 | await tt.start() | |
| 570 | control.emit('event', 'HS_DESC UPLOADED ' + SERVICE_ID + ' first') | |
| 571 | control.emit('event', 'HS_DESC UPLOADED ' + SERVICE_ID + ' second') | |
| 572 | await new Promise((resolve) => setTimeout(resolve, 0)) | |
| 573 | t.is(tt.health, TorHealth.DESCRIPTOR_UPLOADED) | |
| 574 | t.is(tt.getInfo().negativeProbe, null) | |
| 575 | await tt._probeNow() | |
| 576 | await tt._probeNow() | |
| 577 | t.is(tt.health, TorHealth.DEGRADED) | |
| 578 | t.is(tt.getInfo().negativeProbe, null) | |
| 579 | await tt.stop() | |
| 580 | }) | |
| 581 | ||
| 500 | 582 | test('PoW — SETCONF applied; failure surfaces with HiddenServiceDir guidance', async (t) => { |
| 501 | 583 | const socksPort = await fakeSocks(t) |
| 502 | 584 | const { control, factory } = fakeFactory({}) |
| … | … | @@ -1,23 +0,0 @@ |
| 1 | /** | |
| 2 | * Force-exit guard for the unit suite. | |
| 3 | * | |
| 4 | * Mirrors test/integration/zz-finalize.test.js. Some unit tests | |
| 5 | * instantiate real Hyperswarm / Hypercore / Corestore objects that | |
| 6 | * keep the Node event loop alive after assertions finish. Brittle | |
| 7 | * has no global afterAll hook, so we schedule a 5-second .unref()'d | |
| 8 | * force-exit timer as the last assertion. The .unref() ensures we | |
| 9 | * don't artificially block a clean natural exit. | |
| 10 | * | |
| 11 | * Filename `zz-finalize` makes brittle's glob expansion run this last | |
| 12 | * (every other unit-test file starts with a letter < 'z'). | |
| 13 | * | |
| 14 | * If this guard ever stops needing to fire, that means whatever is | |
| 15 | * leaking has been fixed and this file can be removed. | |
| 16 | */ | |
| 17 | ||
| 18 | import { test } from 'brittle' | |
| 19 | ||
| 20 | test('unit suite: schedule post-suite force-exit', async (t) => { | |
| 21 | setTimeout(() => process.exit(0), 5000).unref() | |
| 22 | t.pass('force-exit timer armed (5s, .unref())') | |
| 23 | }) |
| … | … | @@ -35,7 +35,7 @@ services: |
| 35 | 35 | # that reintroduces the audited supply-chain gap (HR-DIS-002) and CI |
| 36 | 36 | # will reject it. This baseline pin is v0.20.2 (the last release whose |
| 37 | 37 | # digest was captured); the next `release:prepare` bumps it. |
| 38 | image: ghcr.io/bigdestiny2/p2p-hiverelay:0.25.0-rc.4 | |
| 38 | image: ghcr.io/bigdestiny2/p2p-hiverelay:0.24.3@sha256:cb104aa65d7e8f57766ea7d60d64dbb6b081a0b9fc5b318c0fa75cb22c0d31c8 | |
| 39 | 39 | init: true |
| 40 | 40 | restart: on-failure |
| 41 | 41 | stop_grace_period: 1m |
| … | … | @@ -92,13 +92,19 @@ services: |
| 92 | 92 | HIVERELAY_TOR_CONTROL_HOST: tor |
| 93 | 93 | HIVERELAY_TOR_CONTROL_PORT: "9051" |
| 94 | 94 | HIVERELAY_TOR_KEY_FILE: /data/tor-onion-key.blob |
| 95 | HIVERELAY_TOR_MIN_DAEMON_VERSION: 0.4.9.5 | |
| 95 | 96 | |
| 96 | 97 | # ─── Utility services (enabled by default) ─────────────────── |
| 97 | 98 | # These make the relay useful to all apps on the substrate: |
| 98 | 99 | HIVERELAY_ENABLE_SERVICES: "true" |
| 99 | 100 | HIVERELAY_OUTBOXLOG: "true" |
| 101 | HIVERELAY_OUTBOXLOG_JOURNAL: hypercore-outboxes | |
| 102 | HIVERELAY_OUTBOXLOG_MAX_JOURNAL_STORAGE: 512MB | |
| 100 | 103 | HIVERELAY_NOTIFY: "true" |
| 101 | HIVERELAY_SHARD_STORE: "true" | |
| 104 | # Real wake egress can be mounted privately and selected with | |
| 105 | # HIVERELAY_NOTIFY_PUSH_CONFIG=/data/notify-push.json. | |
| 106 | # Shard custody stays opt-in because it consumes operator disk. | |
| 107 | # HIVERELAY_SHARD_STORE: "true" | |
| 102 | 108 | HIVERELAY_STORAGE_PROOF: "true" |
| 103 | 109 | HIVERELAY_VRF: "true" |
| 104 | 110 | HIVERELAY_WITNESSLOG: "true" |
| … | … | @@ -2,7 +2,7 @@ manifestVersion: 1.1 |
| 2 | 2 | id: blindspark |
| 3 | 3 | category: networking |
| 4 | 4 | name: Blindspark |
| 5 | version: "0.25.0-rc.4" | |
| 5 | version: "0.25.0-rc.7" | |
| 6 | 6 | tagline: A blind relay that keeps peer-to-peer apps online |
| 7 | 7 | description: >- |
| 8 | 8 | Blindspark turns your Umbrel into a blind relay for the Pear / Holepunch |