OverviewFilesCommitsIssuesDocsReleasesPeers

hyper-deploy commits / f75462d

0.4.13: serveOnConnection had no inverse, so external callers leaked every mux
0.4.11 fixed the overlay pairing leak for connections arriving through DeployServer's OWN swarm: _onConnection's close/error handlers call _detachOverlay. It did not fix the PUBLIC path. serveOnConnection is the documented way to mux content-serving onto a connection the caller already owns ("the keet many-protocols-one-connection pattern"). It calls _overlayProtocol.attach(mux), which registers a mux.pair() callback and records the mux in _paired. _peers is pruned by the channel's own onclose; _paired is pruned ONLY by detach(). There was no public inverse, so a caller could not release the pairing even knowing it should — every connection it ever served stayed pinned, along with the mux and that mux's stream. pgh-sidecar serves its registry-swarm connections exactly this way (pgh-sidecar/lib/index.js, `this._deployServer.serveOnConnection(conn)` on the shared registry topic — the torrent-style path every sidecar replicating the registry connects on, so it is the high-churn one). Measured on tabertron 2026-09-02: the sidecar reached 24.5 GB anon-rss in 8h06m — 78% of a 31 GB box, swap 100% full, 865 MB available. 23.5 GB of it sat in a single [heap] region with only 15 open sockets, which is what distinguishes this from the hyper-svc UDS fd leak. ~50 MB/min, against the ~315 MB/min the pre-0.4.11 process managed, i.e. what was left after 0.4.11 closed the other site. The fix arms the release inside serveOnConnection rather than adding a public detach for callers to remember: attach() creates the pairing, so its lifetime belongs there. WeakSet-guarded so a repeated call adds one listener, and _detachOverlay is idempotent, so overlapping with _onConnection's own handlers is harmless. Red-first: the new test drives serveOnConnection directly (the existing test drives _onConnection, which already passed) — _paired stayed at 5 after five connections were destroyed. 24/24 with the two pre-existing failures excluded (test/symlink-serve.test.js imports node fs/path/os instead of bare-*; test/e2e-release-sync.test.js aborts with "finalize failed: undefined") — both unrelated and confirmed on a clean tree. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
f75462d13039868dfda927048fb25ca2386b554b
by plankatron <fleeky@gmail.com> · 2026-09-03T01:00:40.000Z

Changed files (3) +81 -1

index.js +26 -0
……@@ -96,6 +96,11 @@ export class DeployServer extends ReadyResource {
9696 this._transferManager = null
9797 this._discovery = null
9898 this._sessions = new Set()
99 // Connections we've already armed an overlay-release handler on, so a
100 // repeated serveOnConnection (it is documented idempotent-ish, and
101 // _onConnection calls it too) adds exactly one 'close' listener. Weak so
102 // holding it never keeps a dead connection alive. See serveOnConnection.
103 this._armedConns = new WeakSet()
99104
100105 // 0.5b tree publish + 0.5c tree serving: when trees are populated
101106 // (which we do in _scan), also attach hyper-overlay/v2 to each
……@@ -678,6 +683,14 @@ export class DeployServer extends ReadyResource {
678683 this._overlayProtocol._contentHashPaths = contentHashPaths
679684 this._overlayProtocol._filePaths = filePaths
680685 this._overlayProtocol.attach(mux)
686 // attach() pairs the mux; only detach() unpairs it. Arm the inverse
687 // HERE rather than leaving it to the caller: this is the public way to
688 // mux content-serving onto a connection the caller owns, and it had no
689 // public inverse, so an external caller could not release the pairing
690 // even knowing it should. pgh-sidecar serves its registry-swarm
691 // connections this way and pinned every one of them for the life of the
692 // process. 0.4.11 fixed only connections arriving via our own swarm.
693 this._armOverlayRelease(conn)
681694 return true
682695 } catch (err) {
683696 this.emit('error', new Error('failed to attach overlay-v2 protocol: ' + err.message))
……@@ -704,6 +717,19 @@ export class DeployServer extends ReadyResource {
704717 * on an already-closed connection that is a no-op we do not want to throw
705718 * out of a 'close' handler.
706719 */
720 /**
721 * Drop the overlay pairing when `conn` closes. One listener per connection
722 * (WeakSet-guarded), and _detachOverlay is idempotent — _paired.delete is
723 * guarded and _peers.get returns early — so overlapping with
724 * _onConnection's own close/error handlers is harmless.
725 */
726 _armOverlayRelease (conn) {
727 if (!conn || typeof conn.once !== 'function') return
728 if (this._armedConns.has(conn)) return
729 this._armedConns.add(conn)
730 conn.once('close', () => this._detachOverlay(conn))
731 }
732
707733 _detachOverlay (conn) {
708734 if (!this._overlayProtocol) return
709735 try {
package.json +1 -1
……@@ -1,6 +1,6 @@
11{
22 "name": "hyper-deploy",
3 "version": "0.4.12",
3 "version": "0.4.13",
44 "description": "P2P release distribution — serve build artifacts to peers over Hyperswarm",
55 "type": "module",
66 "main": "index.js",
test/connection-release.test.js +54 -0
……@@ -91,3 +91,57 @@ test('overlay state is released when a connection closes', async (t) => {
9191 t.is(proto._paired.size, 0, 'paired muxes released on close — otherwise every dead connection is pinned for the life of the process')
9292 t.is(server.connections, 0, 'sessions released on close')
9393})
94
95/**
96 * The SAME leak via the PUBLIC serve path.
97 *
98 * `serveOnConnection` is the documented way to mux content-serving onto a
99 * connection the caller already owns ("the keet many-protocols-one-connection
100 * pattern"). pgh-sidecar uses exactly that on its registry swarm
101 * (pgh-sidecar/lib/index.js:362) — it does its own `_registryStore.replicate(conn)`
102 * first, then calls `serveOnConnection(conn)`.
103 *
104 * 0.4.11 fixed the leak only for connections that arrive through DeployServer's
105 * OWN swarm, because `_detachOverlay` is called from `_onConnection`'s close/error
106 * handlers. A caller that uses the public API gets `attach()` with no inverse:
107 * `_paired` (and the mux, and the mux's stream) stays pinned for the life of the
108 * process, which is the whole point of an always-on daemon.
109 *
110 * The public API has no public inverse, so no caller can fix this from outside.
111 */
112test('overlay state is released for a connection served via the public serveOnConnection', async (t) => {
113 t.timeout(30000)
114
115 const dir = mktemp('pub')
116 fs.writeFileSync(path.join(dir, 'a.txt'), 'hello')
117
118 const server = new DeployServer(dir)
119 await server.ready()
120 t.teardown(async () => {
121 try { await server.close() } catch {}
122 try { fs.rmSync(dir, { recursive: true, force: true }) } catch {}
123 })
124
125 const proto = server._overlayProtocol
126 t.ok(proto, 'overlay protocol is enabled')
127
128 const N = 5
129 const conns = []
130 for (let i = 0; i < N; i++) {
131 const p = pair()
132 conns.push(p)
133 // Exactly what pgh-sidecar does on its registry-swarm connections.
134 server.serveOnConnection(p.a)
135 }
136
137 t.is(proto._paired.size, N, 'one paired mux per live connection')
138
139 for (const p of conns) {
140 p.a.destroy()
141 p.b.destroy()
142 }
143
144 await waitFor(() => proto._paired.size === 0)
145
146 t.is(proto._paired.size, 0, 'paired muxes released when a publicly-served connection dies — otherwise an always-on caller pins every connection it ever served')
147})