OverviewFilesCommitsIssuesDocsReleasesPeers

hypercore-kb commits / 020cfa6

skills: audit atproto-identity + atproto-repo (both were untracked)
Neither had ever been committed. Audited against source, not memory: - atproto-identity (v0.0.3): covered 9/12 modules. Adds the three missing ones — plc-directory.js (the PLC SUBMIT ceremony: POST /:did is a permanent, irreversible append to the global op-log, nullifiable only by a higher-authority fork within 72h; offline-first with an injectable fetchImpl), labels.js (com.atproto.label build/sign/verify; retraction is NEGATION not deletion; low-S secp256k1 over the sig-less DAG-CBOR, byte-matching ozone), and web-auth.js (DID web sign-in verify — the DID is the root anchor, browser sub-keys are delegated devices). Also corrects the frontmatter: "No I/O" was wrong, plc-directory does network. - atproto-repo (v0.0.1): 5/5 modules covered and every documented export verified present (walkMst included — it is `export function *`, which a plain export grep misses). Two corrections: * @ipld/car is VESTIGIAL — declared in package.json, imported by no source file; every reference in the tree is a comment explaining why it was dropped for the bare-safe lib/car.js. The skill's open question is settled. * the consumer list had grown from 3 to 9 (adds atproto-did-peer, atproto-room, atproto-base, favela-web, ap-bridge). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
020cfa60e97b59c0afb1efb93e7073b71d58405d
by plankatron <fleeky@gmail.com> · 2026-08-05T02:26:48.000Z

Changed files (2) +457 -0

.claude/skills/atproto-identity/SKILL.md +327 -0
……@@ -0,0 +1,327 @@
1---
2name: atproto-identity
3description: atproto identity layer — one BIP39 mnemonic -> secp256k1/did:plc + Ed25519 (keet), device-attested PLC genesis, session auth (scrypt/JWT/ES256K), com.atproto.label sign/verify, DID web sign-in verify, DID-derived P2P discovery topic. Offline except the injectable PLC submit ceremony.
4category: library
5priority: medium
6location: hyper-fleeky/atproto-identity
7updated: 2026-08-05
8---
9
10**v0.0.3.** Audited 2026-08-05 against source: **12/12** `lib/*.js` modules now
11covered. ⚠ The previous description's "No I/O" claim was wrong — `plc-directory.js`
12performs the PLC **submit ceremony** (injectable `fetchImpl`; nothing runs at
13import). Everything else is genuinely offline.
14
15# atproto-identity
16
17Read this before touching did:plc genesis construction, secp256k1/Ed25519
18derivation, PDS session auth, or device-attestation for the atproto stack.
19This repo is **pure and offline**: no network, no persistent storage, no
20daemon. Every function is a deterministic transform over bytes it's handed
21— the caller (atproto-bridge / hyper-proto-peer) owns all I/O and all key
22custody. `package.json:4` states the mandate exactly: "Offline did:plc
23genesis + verification."
24
25Consumers (verified via `grep -rl atproto-identity package.json` across
26siblings): `atproto-bridge/package.json`, `atproto-bridge/pds/package.json`,
27`hyper-proto-peer/package.json`, `atproto-repo/package.json`. This repo has
28zero outbound deps on any of them — it is a leaf.
29
30## Module map (13 tests, all pass offline, ~650ms — verified 2026-07-04)
31
32### `lib/did-key.js` — did:key codec
33- `encodeDidKey(curve, compressedPubkey)` (`did-key.js:42`) — 33-byte SEC1
34 compressed pubkey + multicodec varint prefix -> `did:key:z...` (base58btc).
35- `decodeDidKey(didKey)` (`did-key.js:53`) — inverse; returns `{curve, pubkey}`.
36- Multicodec prefixes (`did-key.js:17-21`): secp256k1-pub=`0xe7` -> `zQ3s…`,
37 p256-pub=`0x1200` -> `zDna…`. Matches `@atproto/crypto`'s `formatDidKey`.
38
39### `lib/did-plc.js` — byte-exact did:plc derivation (THE pinned invariant)
40- `encodeOp(op)` (`did-plc.js:26`) — canonical DAG-CBOR bytes (`@ipld/dag-cbor`,
41 sorted keys, definite lengths) — this is why key order in the JS object
42 never matters.
43- `didForCreateOp(signedOp, truncate=24)` (`did-plc.js:31`) — `did:plc:` +
44 RFC4648 **lowercase, no-padding, no multibase-prefix** base32 of
45 `sha256(dagCbor(signedOp))`, truncated to 24 chars. Hashes the **signed**
46 op (sig included) — `did-plc.js:9`.
47- `unsignedOp(op)` (`did-plc.js:40`) — strips `sig`; this is what gets signed
48 to produce the op that then gets hashed (with `sig` reattached) to mint the DID.
49- **Gotcha**: `multiformats`' `base32.encode` prepends a multibase `'b'`
50 char — did:plc wants raw base32, so `rawBase32()` (`did-plc.js:20-23`)
51 strips it. Getting this wrong silently produces a plausible-looking but
52 wrong DID (no crash) — this is exactly what `golden-did-plc.test.js` guards.
53
54### `lib/atproto-identity.js` — secp256k1 keys + genesis-op building
55- `ATPROTO_SIGNING_PATH = "m/44'/0'/0'/0/0"` (`:22`) — a **project
56 convention**, not a registered BIP44 coin type. Changing it changes every
57 derived DID for every existing user.
58- `deriveAtprotoKey(mnemonic, path=ATPROTO_SIGNING_PATH)` (`:33`) —
59 `@scure/bip39` seed -> `@scure/bip32` HDKey -> secp256k1 keypair.
60- `keyFromPrivate(privateKey)` (`:41`) / `generateAtprotoKey()` (`:47`) —
61 raw-key and random-key entry points (server-minted accounts w/ no seed).
62- `signOp(unsigned, privateKey)` (`:52`) — ECDSA sign `sha256(encodeOp(unsigned))`,
63 **low-S, RFC6979 deterministic** (noble's default), base64url compact sig.
64- `verifyOp(unsigned, sigB64url, publicKey)` (`:59`).
65- `buildGenesisOp({signingDidKey, rotationDidKeys, handle, pdsUrl})` (`:66`)
66 — builds the unsigned PLC op shape; `rotationDidKeys` defaults to
67 `[signingDidKey]` (single-key setup) if omitted.
68- `identityFromKey` (`:80`) / `createAtprotoIdentity(mnemonic, opts)` (`:89`)
69 / `createAtprotoIdentityFromKey(privateKey, opts)` (`:94`) — full
70 mnemonic/key -> signed genesis + did:plc flow.
71- Re-exports `unsignedOp`, `didForCreateOp` from `did-plc.js` (`:98`).
72
73### `lib/binding.js` — the one-mnemonic dual-derivation
74- `deriveIdentity(mnemonic, {handle, pdsUrl, path})` (`:17`, async) — derives
75 **both** halves from one phrase: `keet-identity-key`'s `IdentityKey.from`
76 for Ed25519 (Hypercore/P2P world), this repo's `createAtprotoIdentity` for
77 secp256k1/did:plc (bsky world). Calls `idk.clear()` (`:20`) right after
78 copying the pubkey out — does not retain the Ed25519 secret.
79- `bindingRecord(mnemonic, opts)` (`:34`, async) — JSON-safe record (hex
80 pubkeys) for persistence by hyper-identity's `atproto` provider.
81- **Verified prerequisite** (`binding.js:6-8`, proven by
82 `test/binding.test.js`): keet's `bip39-mnemonic` and this repo's
83 `@scure/bip39` derive the identical standard BIP39 seed from one phrase —
84 this is *why* one mnemonic can safely anchor both curves.
85
86### `lib/device-attestation.js` — atproto-side device attestation
87The identity model, stated precisely (`device-attestation.js:1-19`):
88- `rotationKeys` = authority/recovery keys = the **ROOT** identity, kept off
89 the always-on serving box. Any rotation key can sign a PLC op to rotate
90 the signing key, change handle, or move PDS.
91- `verificationMethods.atproto` = the repo **signing** key = a per-**DEVICE**
92 key; the box running the PDS holds only this.
93- Genesis is signed by the ROOT, but *authorizes* the DEVICE key as signer.
94 Device loss -> root rotates to a new device; DID/handle/repos persist.
95 **The did:plc is baked from the genesis hash, so the rotation set must be
96 correct before the one irreversible PLC-directory publish** — there is no
97 "fix it later" for genesis composition.
98- `buildAttestedIdentity({rootMnemonic|rootKey, deviceMnemonic|deviceKey|deviceDidKey, devicePath, handle, pdsUrl, extraRotationDidKeys})`
99 (`:37`) — the real production path passes only `deviceDidKey` (public) —
100 **the device private key never leaves the device box**; the root just
101 authorizes the pubkey into the genesis (`:45-53`). Returns
102 `{did, signedOp, rootDidKey, rootPublicKey, deviceDidKey, deviceSigningKey, rotationKeys}`;
103 `deviceSigningKey` is `null` in the pubkey-only flow.
104- `verifyAttestedGenesis(signedOp, rootPublicKey)` (`:74`) — verifies the sig
105 is from a rotation key (root), not the device.
106
107### `lib/device-chain.js` — keet-world (Ed25519) parallel attestation
108Runs in parallel to the atproto PLC authority, on the same root/device pair,
109but is the chain **our P2P world** verifies (not federation-visible):
110- `deviceKeetKeyPair(mnemonic, seed)` (`:19`, async) — device's Ed25519
111 keypair via `keet-identity-key`'s `IdentityKey.from`.
112- `rootIdentityPublicKey(mnemonic, seed)` (`:25`, async).
113- `attestDeviceChain({rootMnemonic, rootSeed, devicePublicKey})` (`:31`,
114 async) — wraps `IdentityKey.bootstrap`; root attests the device pubkey.
115- `attestAtprotoBinding({deviceKeyPair, did, proof})` (`:37`) — wraps
116 `IdentityKey.attestData`; the device signs the **atproto DID itself** as
117 attested data, extending the proof: root -> device -> vouches-for(did).
118- `verifyAtprotoBinding({proof, did, expectedIdentity, expectedDevice})`
119 (`:43`) — wraps `IdentityKey.verify`; returns
120 `{receipt, identityPublicKey, devicePublicKey}` or falsy on any mismatch
121 (wrong DID, wrong expected device).
122
123### `lib/attested-anchor.js` — composes both worlds
124- `buildAttestedAnchor({rootMnemonic, rootSeed, deviceMnemonic, deviceSeed, devicePath, handle, pdsUrl, extraRotationDidKeys})`
125 (`:17`, async) — one call producing both the atproto genesis
126 (`buildAttestedIdentity`) and the keet P2P proof (`device-chain.js`),
127 bound to the same root+device. Returns everything needed to (a) publish
128 the PLC op and (b) let P2P peers verify the device chain -> DID binding.
129 This helper takes **both mnemonics** — it's the dev/single-operator
130 end-to-end case, not the split-custody production path (use
131 `buildAttestedIdentity` + `deviceDidKey` directly for that).
132
133### `lib/auth.js` — PDS-side HTTP session primitives (no I/O)
134Stated purpose (`auth.js:1-8`): the *only* auth the bsky-facing HTTP
135membrane needs. Pure JS crypto (`@noble/hashes`), bare-safe (no Node
136`crypto` global), randomness from `hypercore-crypto` (sodium-backed, bare-native).
137- `hashPassword(password, {salt})` / `verifyPassword(password, stored)`
138 (`:47`, `:53`) — scrypt, stored as `scrypt$N$r$p$saltHex$hashHex`; work
139 factor `N=16384, r=8, p=1, dkLen=32` (`:21`) — "interactive-login
140 appropriate" per the source comment, i.e. NOT tuned for high-value key
141 material, only PDS account passwords.
142- `signJwt(payload, secret)` / `verifyJwt(token, secret, {nowSec})`
143 (`:69`, `:79`) — HS256 session tokens (`accessJwt`/`refreshJwt` shape);
144 verify checks HMAC then `exp`. Throws typed errors via `authError(status, error, message)`
145 (`:133`) so the caller can map straight to an XRPC error response.
146- `signAssertion(payload, privKey)` / `verifyAssertion(token, pubKeyCompressed, {nowSec})`
147 (`:98`, `:107`) — **ES256K key-native login**: the client signs a
148 short-lived assertion with the atproto signing privkey; the PDS verifies
149 against the pubkey already published in the DID doc, so **no secret is
150 stored server-side**. This is the P2P trust root (key possession) brought
151 to the HTTP edge — same key that signs repo commits.
152- `timingSafeEqual(a, b)` (`:38`) — constant-time compare, used by both
153 password and signature verification paths.
154- `parseBearer(authHeader)` (`:127`), `decodeSegment(seg)` (`:123`, decodes
155 without verifying — for reading the issuer before choosing which key to
156 verify against), `b64url`/`b64urlToBytes` (`:25`, `:28`).
157- `ACCESS_SCOPE = 'com.atproto.access'`, `REFRESH_SCOPE = 'com.atproto.refresh'` (`:140-141`).
158
159### `lib/discovery.js` — DID-derived P2P rendezvous
160- `discoveryTopic(did)` (`:15`) — `hypercore-crypto.data(NS + did)`,
161 `NS = 'hyper-proto-peer/discovery/v1:'` (`:12`) -> deterministic 32-byte
162 Hyperswarm topic. **Design intent** (`:1-8`, "the sneaky P2P binding"):
163 the DID document stays 100% vanilla atproto (only `atproto_pds` service
164 entry, pointing at the bridge) — bsky/HTTP never sees a P2P side. A
165 self-custody peer announces on `discoveryTopic(did)`; a bridge holding
166 the DID derives the identical topic and connects. The topic lookup is
167 deterministic and **unauthenticated by design** — trust is established
168 at connection time (the peer must sign its served repo against the DID's
169 key), not at the rendezvous address, so a topic squatter can't forge
170 anything by finding the topic.
171
172### `lib/plc-directory.js` — ⚠ the PLC **submit** ceremony (the only network I/O)
173
174The SUBMIT half of did:plc; the offline half lives in `did-plc.js` /
175`atproto-identity.js`. **This is the module that breaks the package's
176"pure offline / no I/O" framing** — everything else here is offline.
177
178⚠ **`POST /:did` is a PERMANENT, irreversible append to the global PLC op-log**,
179nullifiable only by a higher-authority fork within a **72h** window. So the
180module is OFFLINE-FIRST by construction: op construction, the §6 validation
181MIRROR, and the ceremony orchestration are all pure/injectable and fully tested
182**without the network**. The only I/O is the injectable `fetchImpl`; nothing
183touches the live directory at import.
184
185The self-custody sequence:
186```
187GET /:did/log/last → live head → opCid(head) == our prev
188 → build full-state op → sign with a CURRENT rotation key
189 → LOCALLY re-validate (§6) → POST /:did → confirm via GET /:did/data
190```
191
192Exports: `verifySig`, `verifyOpSig`, `verifyOpSignedByAny`, `checkGenesisOp`,
193`checkUpdateOp`, `class PlcDirectory`. Supports **both** secp256k1 and p256
194(`@noble/curves/p256`, `CURVE_P256`). `verifySig` is also the primitive
195`lib/labels.js` verifies label signatures with.
196
197### `lib/labels.js` — `com.atproto.label` build / sign / verify
198
199A label is a SIGNED tag on a subject: `uri` is an AT-URI (a record) **or** an
200account DID; `val` is the value (`spam`, `!hide`, custom); `src` is the labeler's
201DID. **Retraction is NEGATION, not deletion** — `neg: true` with the same
202`src`/`uri`/`val`.
203
204`labelSigningBytes(label)` = DAG-CBOR of the label **minus `sig`**, undefined
205fields omitted. DAG-CBOR sorts map keys deterministically, so signer and verifier
206agree regardless of insertion order. The signature is a raw 64-byte secp256k1
207**low-S** signature — byte-matching atproto/ozone, which signs with an
208`#atproto_label` key (verified against that did:key).
209
210Exports: `labelSigningBytes`, `buildLabel`, `signLabel`, `verifyLabel`,
211`encodeLabel`, `decodeLabel`.
212
213Lives here because the crypto (dag-cbor + secp256k1 + verifySig + did:key)
214already does; extractable to a dedicated `atproto-label` module later. The
215moderation **policy** (which labelers to trust, how a `val` renders) is the
216CONSUMER's job — [[favela-web]]'s `labeler.js` (reader) and `labeler-export.js`
217(emitter) are the consumers.
218
219### `lib/web-auth.js` — DID web sign-in, verify half
220
221The atproto-native "pgv3 method". A browser holds a secp256k1 **SUB-KEY**; the
222DID's `#atproto` key **DELEGATES** it via a signed assertion anchored to the
223DID's PLC-published key. Login = the sub-key signs a fresh server challenge as an
224ES256K assertion. The atproto analog of pgv3's root-anchored multi-identity: the
225**DID is the root anchor**, sub-keys are delegated **devices** — revoke a device
226by expiring/removing its delegation, and the DID is untouched.
227
228All secp256k1/ES256K (`signAssertion`/`verifyAssertion`) — **no keet, no anproto**.
229
230This module owns **only the crypto verify**. The challenge store (single-use +
231TTL) and the session live in the consumer (favela wraps this + hyper-identity's
232`AtprotoProvider`). `resolveVerifyKey(did)` is **injected** — a PLC-directory
233resolver in prod, a mock in tests — so it stays network-free and unit-testable.
234**Fails closed** on any bad/absent/expired input.
235
236Exports: `verifyChallengeSig`, `makeVerifyDelegation`, `makeAtprotoWebVerify`.
237
238## Identity model summary (must-not-break invariants)
239
2401. **One seed, two curves, two roles.** One BIP39 mnemonic -> Ed25519 (keet,
241 P2P/Hypercore-verifiable) + secp256k1 (atproto/did:plc, federation-visible).
242 Both derivations are proven to agree on one phrase's seed
243 (`test/binding.test.js`, `binding.js:6-8`).
2442. **rotationKeys (root/recovery) vs verificationMethod (device/signing) are
245 never the same key in the attested flow.** Root box never runs the PDS;
246 device box never holds the root seed. Enforced/proven by
247 `test/device-attestation.test.js`.
2483. **did:plc = hash(genesis) is baked forever.** Any change to the rotation
249 set or verification method at genesis time yields a different, permanent
250 DID (`test/device-attestation.test.js:43-48`). There is no in-place fix
251 after the PLC directory write — get the genesis right first.
2524. **Deterministic signing (RFC6979) + low-S**, everywhere ECDSA is used
253 (`atproto-identity.js:54`, `auth.js:102`) — no signature randomness to
254 leak or get wrong on a bare/embedded RNG.
2555. **DAG-CBOR canonicalization is what makes did:plc key-order-independent**
256 — proven by the golden test's "reordered" case
257 (`test/golden-did-plc.test.js:27-37`). Never hand-roll JSON stringify for
258 anything that feeds `didForCreateOp`.
259
260## Gotchas
261
262- `did-plc.js`'s base32 handling is the single most fragile line in the
263 repo: forgetting to strip multiformats' leading `'b'` multibase char
264 produces a *plausible, valid-looking, but wrong* did:plc with no error.
265 Always validate against `test/golden-did-plc.test.js`'s real
266 millipds-minted vector (`did:plc:bh2hiuoeb34g4lif62zj2vx7`,
267 `test/vectors/bridgetest-genesis.json`) after touching this file.
268- `ATPROTO_SIGNING_PATH` is a made-up convention (no registered BIP44 coin
269 type for atproto) — it is load-bearing for every derived identity;
270 changing the default breaks backward compatibility with already-derived
271 DIDs silently (new mnemonic -> different key -> different DID, no crash).
272- `auth.js`'s scrypt params (`N=16384`) are for PDS *account passwords*,
273 not identity keys — do not confuse this with key-derivation security; the
274 actual identity root of trust is the BIP39 seed + HD derivation, never a
275 password.
276- `device-attestation.js`'s `buildAttestedIdentity` silently returns
277 `deviceSigningKey: null` when called with `deviceDidKey` (pubkey-only) —
278 callers must not assume `.deviceSigningKey.privateKey` exists.
279- Bare-runtime compatibility is patched, not native: `scripts/bare-deps.mjs`
280 (postinstall) rewrites `@noble/hashes`' `./crypto` export map to add a
281 `bare` condition pointing at the pure-JS crypto shim, because Bare can't
282 load `node:crypto` (which `cryptoNode.js` imports). This patches
283 `node_modules` post-install and must re-run after every reinstall
284 (wired via `package.json:9`'s `postinstall` script) — if a bare-runtime
285 build starts throwing on `node:crypto`, this is the first thing to check.
286 Comment at `bare-deps.mjs:5` notes it's safe because all signing here is
287 deterministic (RFC6979) — the shim degrading to `undefined` randomness is
288 never actually invoked.
289- This repo has NO network code, NO storage, NO daemon — if you find
290 yourself wanting to add persistence or an HTTP call here, it belongs in
291 atproto-bridge or hyper-proto-peer instead; keep this repo a pure function
292 library (the "purity firewall" referenced in `attested-anchor.js:1`'s
293 comment, pointing at an `ATPROTO-P2P-BRIDGE.md` §13 that lives in a
294 sibling repo, not here).
295
296## Tests (13 tests / 53 asserts, all pass offline in ~650ms — verified 2026-07-04)
297
298- `test/golden-did-plc.test.js` — **the byte-exact pin**: recomputes
299 `did:plc:bh2hiuoeb34g4lif62zj2vx7` from a real millipds-minted genesis op
300 (`test/vectors/bridgetest-genesis.json`) and also proves DID stability
301 under JSON key-reordering (DAG-CBOR canonicalization). If dag-cbor,
302 sha256, base32, or truncation logic drifts by a byte, this fails.
303- `test/derivation.test.js` — did:key round-trip against golden P-256
304 values, deterministic secp256k1 derivation (same mnemonic -> same key;
305 different path -> different key), genesis sign/verify + tamper detection,
306 and full mnemonic -> did:plc flow determinism.
307- `test/binding.test.js` — one mnemonic -> both keet Ed25519 (checked
308 against a captured golden pubkey) and atproto secp256k1, JSON-round-trip
309 of the binding record.
310- `test/device-attestation.test.js` — proves root=rotationKey,
311 device=verificationMethod, root-signed (never device-signed); proves a
312 changed rotation set yields a different permanent DID.
313- `test/device-chain.test.js` — keet chain resolves to root identity,
314 rejects wrong DID / wrong expected device.
315- `test/attested-anchor.test.js` — composed anchor: atproto genesis + keet
316 proof agree on one root/device; proves the root can build the identical
317 DID from **only** the device's public key (production split-custody path).
318
319Run: `npm test` (`brittle-node test/*.test.js`) — fully offline, no network,
320no ports. Safe to run anytime.
321
322## Dependencies
323
324`@ipld/dag-cbor`, `@noble/curves`, `@noble/hashes`, `@scure/bip32`,
325`@scure/bip39`, `hypercore-crypto`, `keet-identity-key`, `multiformats`
326(runtime); `brittle` (dev/test only). `type: module` (ESM), designed to run
327under both Node and Bare (see the `bare-deps.mjs` postinstall shim above).
.claude/skills/atproto-repo/SKILL.md +130 -0
……@@ -0,0 +1,130 @@
1---
2name: atproto-repo
3description: Byte-exact atproto CAR/DAG-CBOR/CID/MST/commit codec library (no storage, no network) — read this when reading/writing/verifying atproto repos, MST proofs, or firehose blocks
4category: library
5priority: medium
6location: hyper-fleeky/atproto-repo
7updated: 2026-08-05
8---
9
10**v0.0.1.** Audited 2026-08-05 against source: **5/5** `lib/*.js` modules covered,
11every documented export verified present (including `walkMst`, which is
12`export function *`). Two corrections below — the `@ipld/car` open question is
13now settled, and the consumer list had grown from 3 to 9.
14
15# atproto-repo
16
17## What it is
18
19A pure, dependency-light JS library implementing the atproto **repo wire
20format**: CAR v1 container, DAG-CBOR block encoding, CIDv1 addressing, the
21Merkle Search Tree (MST) that indexes records, and signed commits. It has
22**no storage layer and no network code** — it is the codec/serializer other
23repos build on. `package.json:4` states the mandate directly: *"Byte-exact
24atproto repo layer — CAR + DAG-CBOR + CID + MST — over Hypercore-friendly
25primitives. Verified against real relay-accepted repos."*
26
27Read this skill whenever you're touching: CAR encode/decode, MST
28build/mutate/delete, commit sign/verify, or the firehose minimal-proof
29block-set logic (`#commit` inversion proofs). It is the crux of the whole
30atproto-peer/atproto-bridge split (see memory
31`project_atproto-peer-bridge-split-2026-07-03`) — if this layer isn't
32byte-exact, nothing downstream (peer signing, bridge relaying) can work,
33because a real atproto relay validates every block's CID.
34
35Runs under both Node and **Bare** (no `fs`/streams/`crypto.subtle`
36dependencies in the runtime path — see Gotchas).
37
38## Public API
39
40### `lib/cid.js`
41- `cidForBytes(bytes) -> Promise<CID>` (cid.js:16) — async wrapper, same result as the sync variant.
42- `cidForBytesSync(bytes) -> CID` (cid.js:23) — **the core primitive**: CIDv1, dag-cbor (0x71), sha2-256 (0x12), using `@noble/hashes` pure-JS sha256 (not `crypto.subtle`, so it loads under Bare). Used everywhere a CID must be computed synchronously (MST recursion).
43- `cidForBlobSync(bytes) -> CID` (cid.js:32) — CIDv1, **raw** codec (0x55) not dag-cbor; for atproto blobs (images/avatars), content-addressed as opaque bytes.
44- `cidForValue(value) -> Promise<{ bytes, cid }>` (cid.js:38) — dag-cbor-encodes a JS value and returns its bytes + CID together.
45
46### `lib/car.js` (bare-safe replacement for `@ipld/car`)
47- `writeCar(rootCid, blocks) -> Promise<Buffer>` (car.js:44) — `blocks` iterable of `{cid, bytes}`. Whole-buffer, CARv1 only.
48- `readCar(carBytes) -> Promise<{ roots, blocks }>` (car.js:58) — `blocks` is a `Map<cidString, {cid, bytes}>`.
49- Re-exported from `lib/repo.js:15`.
50
51### `lib/commit.js`
52- `REPO_VERSION = 3` (commit.js:13).
53- `unsignedCommit({ did, rev, data, prev=null }) -> object` (commit.js:16) — `{ did, version: 3, data, rev, prev }`.
54- `signCommit(unsigned, privateKey) -> signedCommit` (commit.js:21) — ECDSA secp256k1, **low-S**, over `sha256(dagCbor.encode(unsigned))`; `sig` is raw 64-byte compact bytes (NOT base64url — that encoding is only for did:plc genesis ops).
55- `commitCid(signedCommit) -> CID` (commit.js:27).
56- `verifyCommit(signedCommit, publicKey) -> boolean` (commit.js:32) — strips `sig`, recomputes hash, verifies against a 33-byte compressed secp256k1 pubkey.
57
58### `lib/mst.js`
59- `keyHeight(key) -> int` (mst.js:20) — `leadingZeroBits(sha256(utf8(key))) >> 1` (fanout 4). Determines which tree layer a key lives at.
60- `class MSTNode` (mst.js:31) — `{keys, vals, subtrees}` (subtrees.length === keys.length+1). `.serialise()` (line 42) prefix-compresses keys against the previous key in the node and dag-cbor-encodes `{e: [...], l: leftSubtree}`. `.deserialise(bytes)` (line 68) is the inverse. `.cid()`, `.isEmpty()`, `.toOptional()`, `.height()`, `.gteIndex(key)`.
61- `class NodeStore` (mst.js:88) — in-memory CID-string -> `{cid,bytes}` map; `getNode(null)` returns the canonical stored empty node (matches atmst's `get_node(None)`).
62- `class NodeWrangler` (mst.js:106) — the mutation engine: `putRecord(rootCid, key, val) -> CID` (line 109), `delRecord(rootCid, key) -> CID` (line 115), plus internal `_putRecursive`/`_splitOnKey`/`_deleteRecursive`/`_merge`/`_squashTop`.
63- `buildMst(records) -> { root, ns, wrangler }` (mst.js:202) — `records: [{key, value:CID}]` in **any order** (build is proven order-independent by test B).
64- `reachableBlocks(ns, rootCid) -> Map<cidStr,{cid,bytes}>` (mst.js:211) — BFS of live nodes only, for CAR output.
65- `class OverlayNodeStore` (mst.js:242) — write-upper/read-through-lower store that **logs every fallthrough read** (`.gets`) — used to build inductive firehose proofs.
66- `findRpathProof(ns, rootCid, key) -> { value, proof }` (mst.js:265) — walks the search path for `key`, returns inclusion (`value !== null`) or exclusion proof (the visited node CIDs).
67- `nodeStoreFromBlocks(blocks) -> NodeStore` (mst.js:279).
68- `firehoseProofBlocks({ oldRecords, newRecords, ops }) -> { blocks, dataCid, oldRoot }` (mst.js:291) — computes the **minimal** MST block set for a `#commit` firehose event: created nodes ∪ delete-exclusion-proof nodes ∪ nodes read while replaying `ops` backward from the new root to reproduce the old root. Throws if a delete's key is still present in the new tree, or if backward replay doesn't reproduce `oldRoot` (invalid proof).
69
70### `lib/repo.js` (the read + write path, `main` entry)
71- `getBlock(car, cid)` (repo.js:17), `decodeBlock(car, cid)` (repo.js:23).
72- `loadCommit(car) -> { cid, value }` (repo.js:28) — requires exactly 1 CAR root; validates it decodes to something with `did` + `data`.
73- `walkMst(car, rootCid)` — generator (repo.js:38) — in-key-order `{key, value: recordCid}`, applying MST prefix-compression during decode.
74- `listRecords(car) -> [{key, cid, value}]` (repo.js:53) — full record list in key order.
75- `verifyRepo(car) -> Promise<{blocks, records, ok, mismatches}>` (repo.js:65) — **the fidelity proof**: re-encodes every block (`dagCbor.encode(dagCbor.decode(bytes))`) and checks both byte-identity and CID match; also confirms the commit's `data` CID and every record CID resolve to a present block.
76- `createRepo({ records, did, rev, privateKey }) -> Promise<{ carBytes, commitCid, dataCid, signedCommit }>` (repo.js:88) — originates a full CAR from scratch: encodes each record, builds the MST, signs the commit, assembles `[commit, ...mstBlocks, ...recordBlocks]`, writes the CAR. `records` entries take either `{key, value}` or `{collection, rkey, record}` shape.
77
78## Invariants that must NOT be broken
79
801. **CID ≡ byte identity.** `computeCid(dagCbor.encode(block)) === block's CID` is the anchor for the entire stack (cid.js:1-5). If any encode path produces bytes that don't dag-cbor round-trip byte-for-byte, the relay will reject the block (CID mismatch) even if the semantic content is identical. `verifyRepo` (repo.js:65) exists specifically to catch this.
812. **MST node serialization is prefix-compressed and canonical.** `MSTNode.serialise()` (mst.js:42-54) must produce byte-identical output to the reference (`atmst`/millipds) for the same logical tree — dag-cbor's canonical map-key ordering is relied on, and prefix compression against the *immediately preceding key in the same node* (not global) must match exactly. Verified against a real golden CAR (test A) and a 50-key/3-height atmst oracle for both build and delete (tests B/D).
823. **`keyHeight` formula is exact:** `leadingZeroBits(sha256(utf8(key))) // 2` (mst.js:20-29), fanout 4. Any deviation silently produces a structurally different (still "valid-looking") but non-matching tree — won't be caught by local invariants, only by CID mismatch against a real repo.
834. **Commit signature is raw 64-byte low-S secp256k1 over `sha256(dagCbor.encode(unsignedCommit))`** (commit.js:20-24), not base64url (that's did:plc genesis format — a different encoding used elsewhere in the stack, easy to confuse).
845. **Firehose proofs must be minimal AND round-trip-provable.** `firehoseProofBlocks` (mst.js:291-337) will throw rather than emit an unprovable or non-minimal set — this is deliberate: an oversized `blocks` payload risks a relay's `tooBig` rejection, and an invalid inductive proof means a cold relay can't invert the commit. Never bypass the throw with a "just include everything" fallback.
856. **Blob CIDs use the raw codec (0x55), record/commit/MST CIDs use dag-cbor (0x71).** Mixing these breaks addressing (cid.js:28-35 vs cid.js:23-26).
86
87## Gotchas / footguns
88
89- **`lib/car.js` is a hand-rolled CAR codec, not `@ipld/car`**, dropped specifically because `@ipld/car` imports `fs`/streams and won't load under Bare (car.js:1-8). It is explicitly **not hardened for untrusted input** — no bounds/overflow/truncation checks, no duplicate-CID handling (car.js:16-18). Fine for CARs this repo produces itself; **do not point `readCar` at attacker-controlled CAR bytes** (e.g. an `importRepo` ingress) without adding hardening first. Also whole-buffer only — no streaming, so very large repos are fully memory-buffered (car.js:14-15).
90- **`scripts/bare-deps.mjs` patches `@noble/hashes`' `package.json` exports map on `postinstall`** to add a `"bare"` export condition pointing at the WebCrypto-free `crypto.js`, because `node_modules` is wiped/reinstalled and Bare can't load `node:crypto` (bare-deps.mjs:1-11). This is idempotent and runs automatically via `npm install`, but if you ever see Bare failing to load `@noble/curves`/`@noble/hashes`/`@scure/*`, suspect this patch didn't run or was reverted by a fresh install without `postinstall`.
91- **`cidForBytesSync` uses noble's pure-JS sha256, not `crypto.subtle`** (cid.js:13-15) — deliberate, both for Bare compatibility and to guarantee the sync and async CID paths produce identical results. Don't "simplify" by switching to WebCrypto in one path only.
92- **`verifyRepo`'s re-encode check will flag ANY non-canonical dag-cbor in a CAR**, including CARs produced by other tools with different (but semantically valid) map key ordering. A `mismatches` entry doesn't necessarily mean data corruption — check `sameBytes` vs `recomputed` to distinguish.
93- **`NodeWrangler.delRecord` always finishes with `_squashTop`** (mst.js:118, 162-167) — collapses empty-key top layers down to the first node holding keys, so tree height can shrink after deletes. Don't assume height is monotonic non-decreasing across a session.
94- **`buildMst` records must carry pre-computed CIDs (`value`), not raw record values** — records are `{key, value: CID}}`, not `{key, value: object}`. `createRepo` (repo.js:88-98) does the value->bytes->CID step itself before calling `buildMst`.
95
96## Tests (`test/*.test.js`, 10 tests / 17 asserts, all offline, ~0.4s, verified green 2026-07-04)
97
98- `car-read.test.js` — Slice 1: loads `test/vectors/bridgetest-repo.car` (a **real relay-accepted repo** pulled from a live PDS), checks single-root decode, then `verifyRepo` byte-exactness (the "killer check"), then MST traversal + an independent `cidForValue` recompute against the MST-pointed CID.
99- `commit-car.test.js` — Slice 3: sign/verify/tamper round-trip (secp256k1); `createRepo` -> re-read via the slice-1 reader -> byte-exact; whole-stack compose using `atproto-identity`'s `deriveAtprotoKey`/`decodeDidKey` (dev dependency, cross-package, still fully offline) to prove a commit signed with a mnemonic-derived key verifies against its own did:key.
100- `mst-write.test.js` — Slice 2, two independent oracles: (A) rebuild the golden CAR's MST from its extracted records and confirm the root and raw node bytes match exactly; (B/D) a 50-key, 3-height tree matched byte-for-byte against **atmst's own reference root CIDs** (generated by running millipds's reference implementation on stortron) for both build (insertion-order-independent) and a 3-key delete sequence; (E) sanity check that the 50 test keys actually span >1 MST height (so splits are exercised, not just leaf inserts).
101
102## Dependencies
103
104**npm**: `@ipld/dag-cbor` (canonical CBOR codec), `@noble/curves` (secp256k1),
105`@noble/hashes` (sha256), `multiformats` (CID/digest/raw-codec primitives).
106Dev-only: `atproto-identity` (`file:../atproto-identity`, for cross-package test 3 —
107mnemonic -> key -> did:key), `brittle` (invoked via `brittle-node`).
108
109✅ **`@ipld/car` is VESTIGIAL — settled 2026-08-05.** It is declared at
110`package.json:12` but imported by **no source file**: every reference in the tree is
111a *comment* explaining why it was dropped (`lib/repo.js:14`, `lib/car.js:1,6,7,10`).
112The hand-rolled `lib/car.js` is the only codec in the runtime path. Safe to remove
113from `dependencies`; doing so also shrinks the `--production` closure the bare
114daemons ship.
115
116**Sibling repos that consume this as `file:../atproto-repo`** — **9**, not the 3
117originally listed:
118- `atproto-bridge` — the keyless multi-DID membrane fronting peers to bsky
119- `hyper-proto-peer` — the self-custody peer that holds keys and signs
120 (`signCommit`/`createRepo`)
121- `atproto-store` — the Holepunch-native storage substrate; holds records, this turns
122 them into wire-format CARs/commits
123- `atproto-did-peer` — the shared DID reader (`readCar`/`loadCommit`/`verifyCommit`
124 gate every front)
125- `atproto-room` — StateBaseV3 / RepoBase / the signer
126- `atproto-base`
127- `favela-web` — verified inbound ingest + the lens
128- `ap-bridge` — the AS2 projection reads the same canonical bytes
129
130This module has **no `.pear-git/ci.yaml`** and is not independently deployed — it's a pure library dependency consumed via `file:` links by the three repos above (part of the Model B peer/bridge split, memory `project_atproto-peer-bridge-split-2026-07-03`).