OverviewFilesCommitsIssuesDocsReleasesPeers

hyper-overlay commits / ee3b15e

0.3.12: hashFile — ask what the bytes on disk actually hash to
A content-addressed store's correctness rests on "the bytes at this path hash to the oid the pointer claims", and nothing could ask that from outside. registerFile answers it only as a side effect of WRITING an index entry, and verifyOnDisk (used by the fetch path) was module-private and boolean-only. So a consumer holding a converged pointer had no read-only way to check its own local file against it. That gap is how a stale local file goes unnoticed: atproto-store's OverlayRepoStore.ensureLocal returns a local hit without ever comparing it to the pointer's oid, so a node keeps serving superseded bytes forever and never re-fetches. Found live on wj62, where two co-owners held the same 139 converged pointers but different bytes for app.bsky.actor.profile/self (two different avatar blobs, 6533 vs 5962) — so the signed repo flipped content on every signer change. It was only visible by diffing two signed CARs by hand. hashFile(path) -> { contentHash, size } | null. Pure read: no index entry, no registration, no fetch, safe against a live store. verifyOnDisk now delegates to it so the hash is defined exactly once (streaming, so a large file is not buffered). 16 of 17 test files pass; overlay-v2-corestore-mux.test.js fails at module load on a missing lib/overlay-v2.js, which predates this change (the 0.3.0 rename, already recorded in hyper-social #14). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ee3b15e25a7357cb2725f92368c69243cf41fa94
by plankatron <fleeky@gmail.com> · 2026-07-27T16:50:00.000Z

Changed files (2) +28 -5

lib/overlay.js +27 -4
……@@ -40,9 +40,9 @@ const DEFAULT_IDLE_TIMEOUT = 10000
4040// crypto.data(buffer): blake2b(LEAF_TYPE || uint64-LE(size) || bytes), which is
4141// exactly createStreamingHasher({ size }). Returns the size so callers can re-seed
4242// without a second stat.
43async function verifyOnDisk (filePath, contentHash) {
43async function hashOnDisk (filePath) {
4444 let size
45 try { size = fs.statSync(filePath).size } catch { return { ok: false, size: 0 } }
45 try { size = fs.statSync(filePath).size } catch { return null }
4646 const h = createStreamingHasher({ size })
4747 try {
4848 await new Promise((resolve, reject) => {
……@@ -51,8 +51,14 @@ async function verifyOnDisk (filePath, contentHash) {
5151 rs.on('end', resolve)
5252 rs.on('error', reject)
5353 })
54 } catch { return { ok: false, size } }
55 return { ok: h.digest() === contentHash, size }
54 } catch { return null }
55 return { contentHash: h.digest(), size }
56}
57
58async function verifyOnDisk (filePath, contentHash) {
59 const r = await hashOnDisk(filePath)
60 if (!r) return { ok: false, size: 0 }
61 return { ok: r.contentHash === contentHash, size: r.size }
5662}
5763
5864export class HyperOverlay extends ReadyResource {
……@@ -298,6 +304,23 @@ export class HyperOverlay extends ReadyResource {
298304 return prepared ? { contentHash: prepared.contentHash, size: prepared.size } : null
299305 }
300306
307 // READ-ONLY: the content hash the bytes currently on disk at `filePath` would be registered under,
308 // plus their size. Returns null when the file is absent or unreadable. Writes nothing — no index
309 // entry, no registration, no fetch — so it is safe to run against a live store.
310 //
311 // WHY THIS IS PUBLIC: a content-addressed store's correctness rests on "the bytes at this path hash
312 // to the oid the pointer says", and nothing could ask that question from outside. `registerFile`
313 // answers it only as a side effect of WRITING an index entry, and `verifyOnDisk` (used by the fetch
314 // path) is module-private and boolean-only. So a consumer holding a converged pointer had no way to
315 // check its own local file against it — which is exactly how a STALE local file goes unnoticed:
316 // OverlayRepoStore.ensureLocal returns a local hit without ever comparing it to the pointer's oid,
317 // so a node keeps serving superseded bytes forever and never re-fetches. Found on wj62, where two
318 // co-owners held the same 139 converged pointers but different bytes for app.bsky.actor.profile/self
319 // (two different avatar blobs), so the signed repo flipped content on every signer change.
320 async hashFile (filePath) {
321 return hashOnDisk(filePath)
322 }
323
301324 async evictContent (contentHash) {
302325 await this._ensure()
303326 return this._index.evictContent(contentHash)
package.json +1 -1
……@@ -1,6 +1,6 @@
11{
22 "name": "hyper-overlay",
3 "version": "0.3.11",
3 "version": "0.3.12",
44 "description": "Content-addressed P2P file distribution over Protomux — FastCDC chunking, multi-source fetch, resumable transfers, zero blob duplication. Files live on disk; the overlay is a thin index and transfer protocol.",
55 "type": "module",
66 "main": "index.js",