| … | … | @@ -233,9 +233,12 @@ DAG-CBOR **files**. Drop-in for `RepoStore`. |
| 233 | 233 | because re-hashing every file per boot is the cold-start cost that made |
| 234 | 234 | pgh-sidecar take ~190s to bind its socket. Passes no `rev` (a fresh TID |
| 235 | 235 | would look like a newer write to rev-wins conflict resolution) and |
| 236 | | `prepare:false`. ⚠ **It has no caller anywhere in hpp — no UDS RPC, no |
| 237 | | CLI — so it is currently unreachable.** ⚠ It also only walks `records()`, |
| 238 | | so it does **not** re-register blob files. |
| 236 | `prepare:false`. ✅ **CORRECTED 2026-08-05: it IS reachable.** hpp exposes it |
| 237 | as the `reindex-overlay` UDS RPC (`hyper-proto-peer/lib/serve-device-uds.js:78`), |
| 238 | UDS-only by design (re-hashing every file is a CPU-exhaustion trigger). The |
| 239 | handler ALSO calls `blobs.reindexOverlay()` when present, and `DualBlobStore` |
| 240 | forwards that to the overlay store (`lib/dual-blob-store.js:37`) — so the |
| 241 | "only walks `records()`, never blobs" caveat is handled by the CALLER, not here. |
| 239 | 242 | - `put/get/getRaw/del(collection, rkey[, record])` (:115-181) — `put` returns |
| 240 | 243 | `{ oid, size }` where `oid` is the overlay content hash (blake2b) a peer |
| 241 | 244 | fetches by; it stamps `rev: tidNow()` on the overlay registration only |
| … | … | @@ -292,7 +295,9 @@ files. Drop-in for `BlobStore`. |
| 292 | 295 | mapping has to be stored. |
| 293 | 296 | - `get(cidStr)` (:67-84) — reads the bee entry FIRST and **returns null if it |
| 294 | 297 | is absent** (:68-69); then the local file, else `overlay.fetchFile(meta.oid)`. |
| 295 | | See invariant #6 — this early return is a live footgun. |
| 298 | See invariant #6 — this early return is a footgun *for a direct consumer*, |
| 299 | but hpp no longer hits it: it wraps this in `BlobPointerStore`, which puts |
| 300 | the `cid -> oid` mapping in the shared autobase (shipped hpp 0.3.66). |
| 296 | 301 | - `has` / `close` (:86-91) — note `close()` correctly closes an owned |
| 297 | 302 | Corestore, unlike legacy `BlobStore.close()` (gotcha #3). |
| 298 | 303 | |
| … | … | @@ -426,25 +431,36 @@ Hyperbee `BlobStore`. Intended to age out, not to be permanent — see |
| 426 | 431 | overlayPath -> disk); the mistake was making it implicit and silent. |
| 427 | 432 | `OverlayRepoStore.ready()` now seeds from the persisted index — one scan, |
| 428 | 433 | **no re-hashing, no file reads**. `reindexOverlay()` is the heavier repair |
| 429 | | for a bad *index*, and is currently unreachable (no trigger). |
| 430 | | |
| 431 | | 6. **`OverlayBlobStore`'s metadata bee is NODE-LOCAL and is not replicated — |
| 432 | | so a replicated blob file can still be unservable.** Records solved this |
| 433 | | correctly: `PointerSubstrate` puts the `{$oid,size}` pointer in the |
| 434 | | **shared autobase**, so every co-owner converges on which records exist |
| 435 | | *and* the oid to fetch them by. Blobs have **no shared pointer**. In the |
| 436 | | live wiring (`device-node.js:367`) the bee is |
| 437 | | `overlay-blobs:<did>` inside `membraneStore` — a node-local Corestore. |
| 438 | | An upload on node A writes (a) the file + overlay registration, which |
| 439 | | **can** replicate, and (b) the bee entry, which **cannot**. Because |
| 440 | | `get()` reads the bee first and returns null when it is missing |
| 441 | | (overlay-blob-store.js:68-69), node B cannot serve the blob even once the |
| 442 | | bytes arrive. Same failure class as #5: bytes present, blocked by a |
| 443 | | node-local index. Note the asymmetry — favela-web's `lib/blob-serve.js` |
| 444 | | reads the blob FILE directly with magic-byte mime sniffing, so it is |
| 445 | | *more* robust here than hpp's own `get-blob`. A fix means either putting |
| 446 | | blob metadata in the shared StateBase (the records pattern) or letting |
| 447 | | `get()` fall back to a direct file read + sniff. |
| 434 | for a bad *index*, reachable as hpp's `reindex-overlay` UDS RPC. |
| 435 | |
| 436 | 6. **`OverlayBlobStore`'s metadata bee is NODE-LOCAL — ✅ FIXED IN THE CONSUMER, |
| 437 | read this whole entry before acting on it (corrected 2026-08-05).** |
| 438 | |
| 439 | The defect in THIS package is real and unchanged: the `{mimeType,size,oid}` |
| 440 | bee does not replicate, and `get()` reads it FIRST, returning null before it |
| 441 | ever looks at the file or the overlay (overlay-blob-store.js:68-69). The |
| 442 | atproto blob CID (raw codec) and the overlay oid (blake2b) are different |
| 443 | hashes, so the mapping must be replicated, not recomputed. |
| 444 | |
| 445 | **But hpp no longer hits it.** `BlobPointerStore` (this package, |
| 446 | `blob-pointer-store.js`) publishes a `__blob__/<cid>` pointer into the SHARED |
| 447 | autobase — the records pattern, applied to blobs — and hpp wires it at |
| 448 | `hyper-proto-peer/lib/device-node.js:496`: |
| 449 | `blobs = new DualBlobStore(new BlobPointerStore(sb, overlayBlobs, {did}), legacyBlobs)`. |
| 450 | Shipped **hpp 0.3.66** (`e0c06cb`); the fleet runs **0.3.105**. |
| 451 | |
| 452 | ⛔ **Do NOT describe this as a live cross-node data-loss risk.** Verified on |
| 453 | the product surface 2026-08-05: wj62's avatar serves `getBlob` 200 / 39779 B |
| 454 | from the membrane AND `cdn.bsky.app` 200 / 114 KB. hyper-social #12's |
| 455 | "p2p.7rnx.net avatar is broken" is CLOSED by observation. |
| 456 | |
| 457 | What genuinely remains is smaller: `DualBlobStore` is a write-new/read-both |
| 458 | MIGRATION wrapper that was meant to age out and hasn't, so legacy Hyperbee |
| 459 | blobs still need migrating before it can be ripped out |
| 460 | ([[project_blob-full-migration-todo]]). That is cleanup, not risk. |
| 461 | |
| 462 | Still true and useful: favela-web's `lib/blob-serve.js` reads the blob FILE |
| 463 | directly with magic-byte mime sniffing, so it never depended on the bee. |
| 448 | 464 | |
| 449 | 465 | 7. **Blob file paths get seeded only by coincidence of layout — verify it |
| 450 | 466 | before changing either path.** `OverlayRepoStore.ready()` seeds with |