aa49353 · hypercore-kb

Changed files (6) +1208 -473

.claude/skills/ap-bridge/SKILL.md +236 -0

…	…	@@ -0,0 +1,236 @@
	1	---
	2	name: ap-bridge
	3	description: ActivityPub as a PROJECTION of the same canonical atproto record bytes — pure-JS RSA + HTTP Signatures (cavage-12) under Bare, WebFinger/actor, record→AS2, inbox Follow→Accept, firehose-driven delivery. Live at ap.7rnx.net.
	4	category: project
	5	priority: high
	6	location: hyper-fleeky/ap-bridge
	7	updated: 2026-08-05
	8	---
	9
	10	# ap-bridge
	11
	12	v0.0.1 (M0→M3 shipped 2026-08-01), runtime Bare, pgh app `ap-bridge`,
	13	port 3400, live at `https://ap.7rnx.net` (vhost → stortron:3400).
	14	70 tests / 254 asserts, all offline.
	15
	16	## ★ The thesis: ActivityPub is a projection, not a bridge
	17
	18	The record is the single source of truth (`DESIGN.md` §3). The atproto side
	19	derives MST + commit from the record's canonical DAG-CBOR bytes; this derives
	20	AS2 from the same bytes. Neither is stored. **One post, two projections, no
	21	copies, no second store, no borrowed credentials.**
	22
	23	This inverts hyper-social's original POSSE model (push a copy to N networks with
	24	N credential sets). The correct synthesis is two tiers, and the dividing line
	25	is a protocol property, not a preference:
	26
	27	\| tier \| networks \| mechanism \|
	28	\|---\|---\|---\|
	29	\| projection \| atproto, ActivityPub \| the network fetches an addressable object \|
	30	\| bridge \| IRC, Keet, Matrix, email \| delivery is the event — nothing to fetch \|
	31
	32	Realtime networks can never be projections. A realtime message and a post are
	33	different record types, not one thing with different targets.
	34
	35	⛔ Never route a projection through `bridge-envelope`. `content: string`
	36	cannot carry facets, langs, CWs, alt text or quote posts — routing AS2 through the
	37	lowest-common-denominator envelope would permanently cap every projection.
	38	`bridge-envelope` belongs only at the tier-2 realtime boundary
	39	(`lib/as2-project.js:12-16`).
	40
	41	Cross-network dedup key = the record CID, carried in AS2 as `atproto:cid`.
	42	Both projections derive from the same bytes, so it is a network-independent
	43	content identity; composed with `alsoKnownAs` it lets a dual-follower collapse two
	44	feeds. Only works for content we author — a property of projection, not a
	45	general fediverse fix.
	46
	47	## Architecture — pure core, thin edges
	48
	49	Every module that touches network or disk is injected, so the whole routing
	50	surface (including inbox signature verification and Follow/Accept) is testable
	51	offline with no port bound and no remote server.
	52
	53	\| module \| role \|
	54	\|---\|---\|
	55	\| `lib/router.js` (248) \| the ENTIRE request surface as one async fn → `{status,headers,body}`; never throws for client-caused conditions \|
	56	\| `lib/server.js` (54) \| bare-http1 adapter, deliberately thin — read, call router, write \|
	57	\| `lib/actor.js` (139) \| PURE — WebFinger (RFC 7033) + the AP actor doc, derived from config + public key \|
	58	\| `lib/as2-project.js` (133) \| PURE — canonical record → AS2; `resolveObjectUri` injected so the id-map stays the only holder of that binding \|
	59	\| `lib/rsa.js` (211) \| pure-JS RSA PKCS#1 v1.5 / SHA-256 (see below) \|
	60	\| `lib/http-signature.js` (196) \| draft-cavage-http-signatures-12, sign + verify \|
	61	\| `lib/records.js` (120) \| the keyless read side — fronts the DID via `atproto-did-peer` \|
	62	\| `lib/publisher.js` (210) \| delivery to follower inboxes, delivered-SET driven \|
	63	\| `lib/deliver.js` (88) \| sign + POST one activity \|
	64	\| `lib/remote-key.js` (100) \| resolve a Signature `keyId` → remote `publicKeyPem`, cached \|
	65	\| `lib/signer.js` (88) \| the key-custody seam — M2's whole point \|
	66	\| `lib/blobs.js` (93) \| avatar/banner/attachment proxy at `/ap/blobs/<cid>` \|
	67	\| `lib/followers.js` (72) \| follower collection (JSON file — M1 scope) \|
	68	\| `lib/state.js` (72) \| durable delivered-map + profile fingerprint \|
	69
	70	## ★ Why pure-JS RSA (`lib/rsa.js`)
	71
	72	Bare has NO RSA. `bare-crypto`'s entire WebCrypto surface is
	73	pbkdf2/sha/hmac/ed25519; `generateKeyPair`/`sign`/`verify` have an ED25519 branch
	74	and nothing else, and sodium-native / hypercore-crypto / @noble are all curve-only.
	75	ActivityPub HTTP Signatures are still RSA-2048 `rsa-sha256` across the fediverse,
	76	and FEP-521a's inbound support shipped RSA-only (mastodon#39497 — "Ed25519 is
	77	planned but will come later"). So RSA is forced.
	78
	79	- Modulus arithmetic is BigInt; SHA-256 from `bare-crypto`. Byte-identical to openssl.
	80	- Sign + verify ONLY. Key generation is deliberately absent — prime generation is
	81	the hard, dangerous part and is never needed at runtime: an actor's keypair is
	82	minted once with `openssl` and stored. Keeping keygen out keeps the file auditable.
	83
	84	`http-signature.js` implements cavage-12, not RFC 9421: the SWICG survey is
	85	explicit that cavage-12 "is still the most commonly supported version in the
	86	fediverse by far", and RFC 9421 dropped the `hs2019` placeholder, breaking
	87	compatibility both ways. The remedy is double-knocking; cavage-12 is what reaches
	88	real servers today.
	89
	90	## ★ The key-custody seam (`lib/signer.js`) — M2's remaining piece
	91
	92	M1 ships with the actor's private key held by the bridge process, a documented
	93	deviation from the target architecture (`DESIGN.md` §11). atproto-bridge is keyless
	94	because a peer's signature travels in its firehose — the membrane re-serves a
	95	signature it never produced. ActivityPub has no equivalent: HTTP Signatures are
	96	computed per-request over live request headers, so *something must sign at delivery
	97	time*.
	98
	99	The resolution is the shape hpp already uses for `apply-write`: the bridge does not
	100	hold the key, it asks the keyholder to sign over an authenticated RPC. Everything
	101	above `signer.js` depends only on the interface, so swapping local custody for a
	102	remote device changes one line of wiring.
	103
	104	★ `virtual-ap-device` collapsed to almost nothing. The instinct was to give
	105	ap-bridge its own hyper-overlay reader. It needs none — records live in the shared
	106	`OverlayRepoStore` and `atproto-did-peer` already reads through it and carries the
	107	blob relay. The device reduces to hyper-proto-peer + one RPC (`sign-http-request`).
	108
	109	## Delivery — the two traps
	110
	111	`lib/publisher.js`. Mastodon does not read an actor's outbox for timelines and
	112	does not backfill on follow. A correct, public, fetchable outbox puts nothing
	113	in anyone's timeline. Delivery is what makes a post arrive.
	114
	115	★ A SET, not a high-watermark. The obvious design — one watermark rkey plus
	116	"everything lexicographically greater is new", justified by rkeys being TIDs so byte
	117	order IS time order — is FALSE in practice. wj62's repo contains
	118	`e2e-overlay-1784495258`, a hand-written rkey that sorts above every TID (`e` > `3`)
	119	while being the OLDEST record. It became the watermark and **every future post would
	120	have been silently skipped forever.** A set is ordering-independent.
	121
	122	★ Mastodon CACHES actor documents and will not re-fetch on change. A new avatar,
	123	banner, display name or bio is invisible to every follower until an `Update{Person}`
	124	tells them to look again. Hence `refreshProfile()` + a stored fingerprint (fires only
	125	on real change, so a restart doesn't spam).
	126
	127	Per-post state is `rkey → projection fingerprint` (`lib/state.js`), not a set: it
	128	distinguishes "sent and unchanged" (skip) from "sent but the projection improved"
	129	(→ `Update{Note}`, which is exactly how Mastodon models an edit). So every future
	130	projection improvement reaches already-delivered posts automatically.
	131
	132	★ "Event-driven" quietly meant "never happens" — twice. Delivery fires only on
	133	`onHead`, so (a) backfilled posts sat undelivered awaiting an unrelated future post,
	134	and (b) a deploy that improved the projection booted clean and sent nothing. Both
	135	needed an explicit kick. **An event-driven system needs a defined boot-time
	136	reconciliation, or state changes that aren't events are invisible.**
	137	`AP_BACKFILL=N` is one-shot by construction and self-disarms.
	138
	139	## Blobs — why proxy instead of linking the membrane
	140
	141	`lib/blobs.js`. The atproto side already serves every blob at
	142	`pds.7rnx.net/xrpc/com.atproto.sync.getBlob`, so the cheapest thing would be to put
	143	that URL in the AS2. Two reasons not to:
	144
	145	1. A published URL is effectively permanent — remote servers cache and re-fetch
	146	it for years. Emitting someone else's endpoint means we can never change how blobs
	147	are served without breaking every object we ever published.
	148	2. It couples image rendering to the membrane being current, and the membrane is the
	149	component with the known liveness stall.
	150
	151	Blobs are immutable + content-addressed, so aggressive caching is trivially safe.
	152
	153	## Network posture
	154
	155	Outbound delivery does NOT go through the VPS (`DESIGN.md` §5). We POST directly
	156	from this box; receivers authenticate the HTTP Signature against our published actor
	157	key and do not care about source IP. The proxy is load-bearing only for inbound —
	158	WebFinger, actor fetch, inbox POST. That is what makes "the VPS is primarily a proxy"
	159	literally true.
	160
	161	⚠ We fetch remote actors UNSIGNED. Mastodon in "secure mode" requires the actor
	162	fetch itself to be signed; unsigned works against default configs (including
	163	prsm.space). Signed fetches are M2 — they need the actor's own key, which in the
	164	target architecture lives on the virtual-ap-device.
	165
	166	## Config (env, `pgh.defaults.env`)
	167
	168	\| var \| deployed \| note \|
	169	\|---\|---\|---\|
	170	\| `AP_HOST` \| `ap.7rnx.net` \| PUBLIC hostname; MUST match the proxy + what WebFinger claims \|
	171	\| `AP_USERNAME` \| `p2p` \| `acct:<name>@<host>` \|
	172	\| `AP_PORT` / `AP_BIND` \| `3400` / `0.0.0.0` \| behind the 7rnx proxy over the VPN \|
	173	\| `AP_DID` \| `did:plc:wj62…` \| the atproto DID projected; enables the keyless read \|
	174	\| `AP_ALSO_KNOWN_AS` \| `did:plc:wj62…` \| cross-network identity link \|
	175	\| `AP_PRIVATE_KEY` / `AP_PUBLIC_KEY` \| `~/.config/ap-bridge/actor.{key,pub}` \| PEM, minted once with openssl \|
	176	\| `AP_DATA_DIR` \| `~/.config/ap-bridge` \| `followers.json`, `publisher.json` \|
	177	\| `AP_BACKFILL` \| `20` \| ONE-SHOT on first prime only \|
	178	\| `AP_PDS_URL` \| `https://pds.7rnx.net` \| where blob bytes are fetched FROM \|
	179	\| `AP_MAX_SKEW` \| 300s \| signature clock skew \|
	180
	181	`assertUsable` fails at boot with a useful message rather than at first request.
	182
	183	## ⚠ Single-instance by construction
	184
	185	Three pieces of per-node mutable state, all local: the RSA keypair, `followers.json`,
	186	and the delivered-map. A second instance would sign with a key that doesn't match the
	187	single `publicKeyPem` in the actor doc, double-deliver from an independent
	188	delivered-set, and diverge the follower list. `ap.7rnx.net` is a static vhost to
	189	stortron:3400 — do not put two behind it.
	190
	191	Delivery retries are in-process and bounded; a restart drops the queue. A durable
	192	queue (per-host rate limiting, backoff, dead-letter) is `DESIGN.md` §5 and needs the
	193	HyperDB store.
	194
	195	## Deploy + CI ordering
	196
	197	pgh app, `runtime: bare`, `entrypoint: index.js`, readiness http on 3400.
	198	Siblings via `file:../`: `atproto-did-peer`, `atproto-identity`, `atproto-repo`,
	199	`atproto-store`.
	200
	201	★ Correct CI order is `install → tests → vendor → guard → publish` — the only
	202	order where every precondition holds. Four CI failures at M3, each a guard working:
	203	a YAML colon; declare-deps testing subpaths as directories; the vendor guard refusing
	204	because sibling repos had unpushed commits the outbox depended on; and
	205	`pgh publish` refusing because `npm install brittle` re-resolved the tree and deleted
	206	the vendored siblings (`removed 1194 packages`).
	207
	208	Deploy-shape gotchas: `readiness` takes `port`, not `url`; a missing `pgh.upgrade`
	209	means OTA can never work; `b4a`/`bare-fs` must be declared.
	210
	211	## Boot-order gotcha
	212
	213	`cborg` touches `TextEncoder` at module load → `bare-globals` must be imported
	214	first. Two sibling traps that presented identically as "the outbox is empty":
	215	`decodeDidKey` returns a descriptor, not bytes; and attaching swarm connection
	216	handlers without `swarm.join(discoveryTopic(did))` — nothing ever connects.
	217	A system can look correctly wired and silently do nothing.
	218
	219	## Open work
	220
	221	- `sign-http-request` on hpp → genuinely keyless bridge (M2's remaining piece)
	222	- exit-75 / `release-ready` handling → end the manual stop/install/start cycle
	223	- facets → link/mention markup; quotes (FEP-044f); `Delete`/`Tombstone`
	224	- multi-actor: ★ the id-map must become real state first — it is a pure function
	225	only while there is one DID and one collection
	226	- `followers.json`/`publisher.json` → HyperDB (§3), then the device (§7)
	227	- retire `MASTODON`/`ATPROTO` from `bridge-envelope`'s PLATFORMS — they are
	228	projections now, not bridges
	229
	230	## Related
	231
	232	- [[atproto-did-peer]] — the keyless reader this fronts the DID with
	233	- [[hyper-proto-peer]] — the keyholder; future `sign-http-request` host
	234	- [[atproto-bridge]] — the atproto projection of the same bytes
	235	- [[atproto-store]] — `OverlayRepoStore`, the shared record bytes
	236	- memory: [[project_ap-bridge-activitypub-2026-07-31]], hyper-social org issue #26

.claude/skills/atproto-bridge/SKILL.md +251 -180

…	…	@@ -1,189 +1,260 @@
1	1	---
2	2	name: atproto-bridge
3		description: Stateless keyless atproto membrane — fronts many self-custody hyper-proto-peers to bluesky over HTTP/WS, holds no keys, no accounts, never signs
	3	description: Stateless keyless atproto membrane — fronts many self-custody hyper-proto-peers to bluesky over HTTP/WS, holds no keys, no accounts, never signs. ONE tree, TWO apps — the membrane (pds-serve) and the blind availability relay (blind-relay), selected by install name.
4	4	category: project
5	5	priority: high
6	6	location: hyper-fleeky/atproto-bridge
7		updated: 2026-07-06
	7	updated: 2026-08-05
8	8	---
9	9
10	10	# atproto-bridge
11	11
12		The stateless, keyless membrane in the Model B peer/bridge split. It fronts self-custody
13		`hyper-proto-peer`s and projects them to the bluesky network over HTTP+WS. Read this when
14		working on per-DID routing, the keyless serve path, the relay firehose aggregation, or the
15		pgh deploy shape.
16
17		Deploy-root package `atproto-pds` (legacy name — it is the bridge; the custodial PDS was
18		removed 2026-07-04, see below). Real code is in `pds/`. Sibling libs: `atproto-repo`,
19		`atproto-identity`, `atproto-store`, plus `hyper-proto-peer` (the peer side, separate repo).
20
21		## Model B: membrane vs peer — the no-key invariant (STRUCTURAL as of 0.8.4)
22
23		- peer (`hyper-proto-peer`, separate repo): holds the DID's signing key + content, applies
24		and signs writes, serves its repo + its own signed firehose over Hyperswarm.
25		- membrane (this repo): holds NO key, NO account, NO custodial storage — by construction.
26		It replicates a peer's cores and re-serves them to bsky over HTTP. The signature always travels
27		in the peer's firehose commit; the membrane assembles CARs from replicated blocks
28		(`getRepoFromCommit`) but never re-signs. A `PeerFront` has no private key and `verifyCommit`
29		gates every front.
30		- The custodial `Pds` was ripped out (0.8.4, commit 88f6bee). Earlier versions embedded a
31		multi-tenant custodial PDS (opt-in `membraneOnly` flag in 0.8.3). Now there is no Pds at all —
32		re-custody is impossible, and a compromised membrane has no key to steal. Identity/accounts
33		live entirely in the peers. `git log` / `git show 88f6bee` for the removed custodial code
34		(pds.js, account.js, fleet-node/directory/rpc, provision, tools/plc-publish).
35
36		## Architecture (VERIFIED file:line)
37
38		- `pds-serve.js` — the runnable membrane daemon. No Pds/identity/jwt. Under Bare it installs
39		`bare-fetch/global` (Bare has no global `fetch`; PLC resolution needs one — see gotchas). Builds
40		a `Hyperswarm`, a `front-store` Corestore, a `FleetRelay` (aggregate firehose core lives in the
41		front-store, NOT any pds), and a `PeerRouter` from `cfg.frontedDids`; warms each fronted peer;
42		wires `createPdsServer({ peers, serverDid, availableUserDomains })`; starts the UDS control
43		plane `startControl({ peers, … })`. Clean shutdown closes control/peers/relay/swarm/server and
44		frees the front-store Corestore fd-lock (so a redeploy can reopen). Boot marker: `membrane listening`.
45		- `PeerRouter` (`pds/lib/peer-router.js`) — owns the `_fronted` DID set + a cache of verified
46		`PeerFront`s and the one-RPC-per-connection identity map (see "Multi-DID" below). Constructor
47		`_joinDid`s every fronted topic (symmetric `{server,client}`). On EVERY connection `_onconnection`
48		does `corestore.replicate(conn)` (keet pattern — muxes all DIDs' cores) AND `_identify(conn)`:
49		opens exactly ONE `protomux-rpc`, `get-coords`'d once, caches `did → {conn,rpc,coords}` in
50		`_byDid`. `front(did)` = resolve verify-key (`resolvePlcVerifyKey`, reads the `#atproto`
51		`verificationMethod` from `plc.directory`) → `_waitForDid` (a LOOKUP over `_byDid`, not a probe)
52		→ `PeerFront.fromEntry` → fold firehose into `relay.addSource`. Reconnect: `_onConnClose` drops
53		only the `_byDid` entry (NOT the front — its cores re-replicate); `_identify` refreshes the
54		front's write rpc on re-identify. `resolveHandle`/`_handleToDid` learned at front time.
55		- `PeerFront` (`pds/lib/peer-front.js`) — fronts ONE remote peer keylessly.
56		`fromEntry(corestore, swarm, {conn,rpc,coords}, did, {verifyKey})` BORROWS the router's
57		per-connection conn/rpc (`_ownsConn=false` — `close()` never tears down the shared channel), opens
58		the peer's `RepoStore` + `FirehoseSequencer`, and fail-closed rejects any repo whose latest
59		signed commit doesn't `verifyCommit` against the DID key (incl. an empty repo — a fresh peer must
60		author its first commit; the peer's `bootSwarmDevice` now does). Reads
61		(`getRepo`/`getRecord`/`listRecords`/`getLatestCommit`/`getRepoStatus`) are keyless; writes and
62		sessions are RPC-relayed to the peer over that same shared rpc. (`RPC_PROTOCOL =
63		'hyper-proto-peer/rpc/v1'`, exported for the router.) The old `discover()` — which PROBED every
64		connection opening a per-front rpc — is GONE (it caused `DUPLICATE_CHANNEL`; see the post-mortem).
65		- `server.js` (node) + `server-bare.js` (bare/prod) — the bsky-facing HTTP+WS membrane,
66		`createPdsServer({ peers = null, serverDid = null, availableUserDomains = [] })`. Endpoint-for-
67		endpoint identical; only transport primitives differ (node `http`+`ws` vs `bare-http1`+`bare-ws`).
68		Runtime chosen in `pds-serve.js` (`typeof Bare`). Prod runs server-bare.js. Every DID served
69		is a fronted peer: `frontIfPeer(idOrHandle)` returns the `PeerFront` or null → 404. `describeServer`
70		= `{ did: serverDid, availableUserDomains }` (synthesized, no Pds). `listRepos` = the fronted set.
71		`subscribeRepos` = the aggregate `peers.relay`. Authed/custodial endpoints 404
72		(getSession/getServiceAuth/getPreferences/getBlob/refreshSession/createAccount/uploadBlob/writes/
73		putPreferences) — identity lives in the peer; per-user session/write routing to peers is DEFERRED.
74		`createSession` routes to the fronted peer (`front.createSession`). `app.bsky.`/`chat.bsky.`
75		proxy PUBLIC views to the bsky AppView (`appview.js`; no service-auth minting — the membrane holds
76		no key).
77		- Relay layer (KEPT): `FleetRelay` (`fleet-relay.js`) aggregates many peer firehose Hypercores
78		into one monotonic `subscribeRepos` via `FleetSequencer` (`fleet-sequencer.js`); `fleet-replicate.js`
79		(`connectReplica`) + `fleet-serve.js` (`connectToServer`) are used by `PeerFront`. The custodial
80		fleet-of-PDS-nodes (`fleet-node/directory/rpc`) was removed.
81
82		## Multi-DID membrane = the keet model (SHIPPED 2026-07-06) — one RPC per connection
83
84		The membrane fronts MANY DIDs over ONE Hyperswarm — the same shape as **keet fronting many rooms
85		over one swarm** (`keet-core index.js:1239`): one corestore + one swarm, each room/DID a topic, and
86		on EVERY connection exactly ONE `rootCorestore.replicate(conn)` — hypercore muxes every room's cores
87		over it. The membrane needs an EXTRA application RPC (`get-coords`/`apply-write`/sessions) that keet
88		doesn't, so the load-bearing rule is: **establish that RPC the same way — once, on connect, in the
89		connection handler** (`_identify`), never ad-hoc per front.
90
91		- ❌ The bug this replaced: `PeerFront.discover` opened a `protomux-rpc` per front by PROBING every
92		`swarm.connections`. Fronting a 2nd DID probed the 1st front's cached connection → a 2nd channel with
93		the same protocol+id on one stream → `Protomux.createChannel` returns null → `ProtomuxRPC` ctor
94		throws `DUPLICATE_CHANNEL` synchronously → crash before `server.listen` → boot restart loop,
95		port never opened. (Two prod outages. `protomux info.opened` counts LOCAL opens; the null-return is
96		a duplicate-local-channel signal — which one-rpc-per-connection sidesteps entirely.)
97		- ✅ The design: `PeerRouter._identify(conn)` opens ONE rpc, `get-coords` once (10s timeout; a
98		non-membrane stream just times out + is skipped), caches `did → {conn,rpc,coords}`. `front()` is a
99		`_waitForDid` lookup + `PeerFront.fromEntry`. One rpc per connection ⇒ never a 2nd channel.
100		- ⚠️ Boot-warm is BACKGROUND (`pds-serve` fires `front(did)` after `server.listen`/`startControl`,
101		fire-and-forget). Awaiting fronts at boot blocked `listen` past the 30s UDS readiness window →
102		supervisor restart loop. A peer offline at boot must never delay the bridge coming up; on-demand
103		requests retry. Boot logs `front failed (will retry on demand)` then `fronting <did>` — that's normal.
104
105		## Federation — a NEW fronted account needs `requestCrawl` (does NOT auto-appear)
106
107		A fronted peer that serves a signed CAR still won't show on bsky until the RELAY indexes it. For a NEW
108		DID on an ALREADY-crawled PDS, the relay knows the PDS (an existing account) but not the new one
109		(`GET bsky.network/…/com.atproto.sync.getRepoStatus?did=<new>` → `RepoNotFound`). Trigger it:
110		`POST bsky.network/…/com.atproto.sync.requestCrawl {hostname: pds.7rnx.net}` (HTTP 200, no auth) →
111		relay re-enumerates via the PDS `com.atproto.sync.listRepos` (must list the new DID `active:true` —
112		the membrane does) → picks it up in ~25s at the account's rev → AppView indexes → live. Handle side
113		must already resolve (`https://<handle>/.well-known/atproto-did` → DID, served via the `*.7rnx.net`
114		wildcard). Posting (proven): `createSession` (handle + app-pw → peer verifies + mints JWT) →
115		`createRecord` (Bearer, repo=DID) → bridge relays to the peer's apply-writes → PEER signs. TODO:
116		emit `#account`/`#identity` on the aggregate firehose so new DIDs federate without a manual crawl.
117
118		Full go-live saga + every lesson: `hyper-fleeky/atproto-bridge/docs/POSTMORTEM-2026-07-06-multi-did-membrane.md`
119		(ashatron.7rnx.net + p2p.7rnx.net live on bsky; 2 prod outages from guess-fixing; ground in keet, not guesses).
120
121		## Config (`pds/lib/config.js`) — membrane-only
122
123		`loadConfig()` = file overlaid by `PDS_*` env (env > file > default). Keys:
124		`serverDid` (`PDS_SERVER_DID`, default `did:web:pds.7rnx.net`), `userDomain` (`PDS_USER_DOMAIN`,
125		`.7rnx.net`), `port` (`PDS_PORT`, 3399), `bind` (`PDS_BIND`, 127.0.0.1), `storage` (`PDS_STORAGE`
126		— holds the front-store, NOT accounts), `uds` (`PDS_SOCK`), `frontedDids` (`PDS_FRONTED_DIDS`,
127		JSON array or comma-separated). No identity/did/privHex/devicePrivHex/openRegistration/fleetId
128		(all removed with the custodial code). A pure membrane needs only `{ frontedDids, serverDid,
129		userDomain, port, bind, storage, uds }`. ⚠️ `PDS_CONFIG` defaults to `~/.config/atproto-pds/config.json`
130		— point a non-default instance at its own config.
131
132		## Deploy shape
133
134		- Runtime: Bare (`"start": "bare index.js"`; `index.js` re-exports `pds/bin/pds-serve.js`).
135		node membrane exists only for tests/dev.
136		- pgh app `atproto-pds`, published content-addressed via `pgh publish .` in `.pear-git/ci.yaml`
137		(path A — NOT the tarball/`artifacts:` path). Manifest `pgh.defaults.env` sets `PDS_BIND=0.0.0.0`,
138		`PDS_PORT=3399`, `PDS_STORAGE`/`PDS_SOCK` under `~/.config/atproto-pds/`.
139		- ci.yaml sibling-vendor pattern: `file:../` sibling deps (`hyper-svc`, `atproto-repo/identity/
140		store`) survive git-archive only as broken symlinks, so CI `cp -rL` each into `node_modules` after
141		`npm install --production`. Requires the sibling repos on the build machine (KB path or `$HOME`).
142		- ⚠️ Declare-your-deps (the #118 crash class): every direct import MUST be declared, else the
143		`--production` closure drops it and the bare daemon crash-loops `MODULE_NOT_FOUND`. Fixed 0.8.4:
144		`bare-fetch` (global fetch for Bare), `bare-https` + `protomux` (were only transitively hoisted).
145		Verify with: imports across `bin`+`lib` vs declared deps. Test-green ≠ deploy-ready.
146		- Version: root `package.json` 0.8.4 (the pure-membrane rip-out). 0.8.2 = last hybrid,
147		0.8.3 = opt-in `membraneOnly` flag (superseded).
148
149		## Gotchas / footguns
150
151		- ⚠️ Bare has NO global `fetch`. `resolvePlcVerifyKey` (gates fronting — verifies the peer's DID
152		key) uses `fetch`; without a shim it throws `fetch is not defined` on the bare runtime and NO peer
153		is fronted (silently drops every account). Fix: `pds-serve.js` installs `bare-fetch/global` under
154		Bare. Proof it works: the boot smoke's front-fail error is a real `PLC 404`, not `fetch is not defined`.
155		- ⚠️ Boot-test before deploy (`npm run smoke` → `scripts/smoke.mjs`): boots `bare index.js` as a
156		membrane (dummy fronted DID) and asserts describeServer synthesizes + custodial endpoints 404. This
157		caught a `./lib` vs `../lib` relative-import bug the unit tests can't see (they import `lib/` directly,
158		never `bin/pds-serve.js`). CI does NOT boot a server (a long-running boot orphans the child on step
159		timeout).
160		- ⚠️ Manual cp/mv doesn't compose with pgh. A hand-copied app dir is inert against pgh's install/
161		materialization (the launcher points at the installed tree). Roll back / install via pgh, never
162		`cp -a` over the app dir (cost a crash-loop half-state in the wj62 migration).
163		- Per-user writes/sessions for fronted DIDs are NOT yet routed (only `createSession` is). getSession/
164		refreshSession/writes 404 on the membrane — deferred (needs bridge↔peer write authz: the bridge's
165		swarm pubkey in the peer's `canWrite`).
166		- One membrane fronts MANY peers across MANY machines — `frontedDids` is a growing list (e.g. wj62's
167		peer on stortron + tabertron.7rnx.net's peer on tabertron). The membrane's own DID (`serverDid`) is a
168		service identity, not a user account.
169
170		## Tests (`pds/test/`, brittle; `brittle-node test/*.test.js`)
171
172		VERIFIED green 9/9, 64 asserts (2026-07-04) — the membrane set only (custodial tests removed):
173		- `membrane.test.js` — HTTP membrane routes a fronted DID to its peer (keyless reads + createSession),
174		`createPdsServer({ peers })` with NO Pds.
175		- `membrane-only.test.js` — `createPdsServer({ peers })` + `startControl({ peers })`: fronted reads
176		route to the peer, describeServer/listRepos synthesized, custodial/authed endpoints 404, UDS status
177		reports `membrane:true`.
178		- `peer-router.test.js`, `peer-front.test.js` — routing/discover/caching/relay-aggregation; keyless
179		getRepo/subscribeRepos, RPC-relayed write (peer signs), connect-time auth rejects a wrong key.
180		- `fleet-relay.test.js`? removed; `fleet-sequencer.test.js`, `fleet-replicate.test.js` — the relay/
181		replicate primitives.
182
183		## Dependencies + siblings
184
185		`file:../` siblings: `atproto-repo` (CAR/MST/commit codec), `atproto-identity` (did:plc, ES256K auth,
186		`discoveryTopic`), `atproto-store` (Corestore/RepoStore/FirehoseSequencer/read-view), `hyper-svc` (UDS
187		+ swarm RPC / pgh-sidecar). Runtime: `hyperswarm`, `corestore`, `protomux`+`protomux-rpc`,
188		`compact-encoding`, `bare-fetch` (fetch on Bare), `bare-http1`/`bare-https`/`bare-ws`, `hypercore-crypto`.
189		Peer counterpart: `hyper-proto-peer` (holds keys, signs, serves).
	12	v0.9.40, runtime Bare, pgh app `atproto-bridge`, port 3399, live at
	13	`https://pds.7rnx.net` (static vhost → stortron:3399 — not load-balanced).
	14	Real code is in `pds/`. Deploy-root package name is now `atproto-bridge`
	15	(older docs say `atproto-pds`; config moved to `~/.config/atproto-bridge/`).
	16
	17	## ★★ ONE TREE, TWO APPS — read this first
	18
	19	`index.js` is a two-way switch. The same published tree runs either app,
	20	selected by install name (or `ATPROTO_MODE`):
	21
	22	```js
	23	if (process.env.ATPROTO_MODE === 'relay' \|\| /blind-relay/.test(appName)) {
	24	await import('./pds/bin/blind-relay.js') // keyless AVAILABILITY RELAY
	25	} else {
	26	await import('./pds/bin/pds-serve.js') // keyless HTTP MEMBRANE
	27	}
	28	```
	29
	30	So `pgh app install <key> --name atproto-blind-relay` just works. **If you are
	31	looking for the redundancy / "two machines as one swarm" story, it is the relay,
	32	not a second membrane.** Do not put two membranes behind one hostname — see the
	33	seq contract below.
	34
	35	### `BlindRelay` (`pds/lib/blind-relay.js`, `bin/blind-relay.js`)
	36
	37	A keyless availability seeder. The membrane fronts a peer only over a LIVE
	38	connection, so a single peer that goes offline (ashatron on a laptop) has nobody
	39	serving its cores and vanishes from bsky. The relay:
	40
	41	- opens an explicitly allowlisted peer's repo + firehose cores BY KEY into a
	42	persistent corestore,
	43	- FULL seeder: `core.download({start:0, end:-1})` + follow `'append'` — a sparse
	44	replica could serve NOTHING once the origin is gone,
	45	- re-announces on the peer's DID discovery topic (`atproto/did-announce/v1`)
	46	from config + live core lengths,
	47	- one Hyperswarm per DID (each is a distinct virtual peer; an announce carries
	48	exactly one DID per connection),
	49	- joins `{server: true, client: true}` — firewalled↔firewalled needs both.
	50
	51	BLIND by construction: it opens NO write RPC — no `apply-write`,
	52	`create-session`, `get-service-auth`, `upload-blob`. It can make a peer's OWN signed
	53	data available (the membrane still verifies against the DID key) but can never sign
	54	or forge. It is not a StateBase indexer, and new signed commits still require the
	55	origin peer or an always-on 2nd device.
	56
	57	Config: `$RELAY_CONFIG` or `~/.config/atproto-blind-relay/config.json` —
	58	`{ storage, seedPeers: [{ did, handle, repoKey, firehoseKey }] }`. **`seedPeers` IS
	59	the gate** (never "every peer it can reach"). `serverKey` in older config is ignored —
	60	announce-push discovery needs no directionality. `status()` reports per-peer
	61	`repoLength/repoContiguous/firehoseLength/firehoseContiguous`.
	62
	63	## Model B: membrane vs peer — the no-key invariant
	64
	65	- peer (`hyper-proto-peer`): holds the DID signing key + content, applies and
	66	signs writes, serves repo + its own signed firehose over Hyperswarm.
	67	- membrane (this repo): NO key, NO account, NO custodial storage. The
	68	signature always travels in the peer's firehose commit; the membrane assembles CARs
	69	from replicated blocks but never re-signs. The custodial `Pds` was ripped out in
	70	0.8.4 (commit `88f6bee`) — re-custody is impossible.
	71
	72	One exception, deliberate and documented: `pds/lib/swarm-keypair.js` — a **persistent
	73	transport identity**. `new Hyperswarm()` mints a random keypair per boot, so the
	74	bridge's Noise public key changed every restart and anything added to hpp's
	75	`PEER_WRITERS` allowlist authorised exactly one boot then silently stopped. That
	76	blocked `listNotifications` end to end (#21). It never signs a repo commit and grants
	77	nothing on its own — it only makes the bridge nameable so a peer's operator can
	78	choose to trust it. Logged at boot as `{"msg":"swarm identity","publicKey":…}` —
	79	that is the value to paste into a DID's `writers` array. Each machine has its own,
	80	so a bridge that moves hosts is a different key.
	81
	82	## Boot sequence (`pds/bin/pds-serve.js`)
	83
	84	1. `bare-globals` FIRST (cborg touches `TextEncoder` at module load).
	85	2. `loadConfig()`; pick `server-bare.js` (Bare, prod) vs `server.js` (node, tests).
	86	3. Under Bare, install `bare-fetch/global` — Bare has no global `fetch` and PLC
	87	resolution needs one; without it `front()` throws `fetch is not defined` and NO
	88	peer is fronted, silently.
	89	4. `frontStore` Corestore + persistent swarm keypair + `FleetRelay` over
	90	`frontStore.get({name:'peer-relay-firehose'})` with the durable `relay-seq-base`.
	91	5. `DidPeerRouter` (from `atproto-did-peer`, a separate package — see that skill)
	92	with `{swarm, join, fronted: cfg.frontedDids, relay, resolveVerifyKey, rpc: true,
	93	handles, onHandles}`.
	94	6. `createPdsServer(...)` → `listen`.
	95	7. UDS control plane (`startControl`) + pgh-sidecar registration.
	96	8. Fronts warm in the BACKGROUND — awaiting them blew past the 30s UDS readiness
	97	window → supervisor restart loop. `front failed (will retry on demand)` at boot is
	98	normal.
	99	9. Seq guard (below), then the optional feed-registry reactor.
	100
	101	⚠ OTA is deliberately NOT pgh-runtime's fetch path. pgh-runtime fetches only ROOT
	102	trees, not dep closures, so app-driven self-update produces a BROKEN install — the
	103	exact `MODULE_NOT_FOUND` respawn loop that took the membrane down 2026-07-31. The
	104	sidecar does the full install and then calls `release-ready`; the app's only job is to
	105	exit 75 after responding.
	106
	107	## The firehose seq contract — and the guard that repairs it
	108
	109	`FleetRelay` (`fleet-relay.js`) merges each fronted peer's signed firehose into ONE
	110	monotonic `subscribeRepos` stream via `FleetSequencer` (`fleet-sequencer.js`).
	111
	112	- `FleetSequencer` owns a single-writer append-only emit log (its own Hypercore).
	113	It dedups by a stable event key and appends each NEW key exactly once in
	114	first-seen order, so an event's seq is STABLE no matter how anything reorders
	115	underneath. The emit log IS the durable cursor map (the seen-map rebuilds from it
	116	on open).
	117	- Keys are machine-independent: frames are `${sourceId}:${i}` where `sourceId` is
	118	the peer's firehose core key hex, and re-baselines are `sync:${did}:${rev}`.
	119	- `seqBase` offsets every EMITTED seq and only ever rises, persisted to
	120	`<storage>/relay-seq-base` — a plain file beside the front-store so it survives
	121	a corestore reset.
	122
	123	Why any of this matters: atproto relays (indigo) keep a strictly forward cursor
	124	per host and never roll it back. If our emitted seq falls below their stored cursor
	125	they treat us as a future-cursor, park the host, and never read again. Measured
	126	2026-07-31: our firehose held 79 frames while bsky sat at cursor 9834, `status:offline`
	127	— federation dead ~20h with every local surface healthy.
	128
	129	`seq-guard.js` is the half that NOTICES. `decideLift({emitted, cursor, margin})` is a
	130	pure, testable policy: lift iff `emitted <= cursor` (the relay can only ask for frames
	131	after `cursor`, and we cannot produce a seq that high — wedged by arithmetic). It is
	132	deliberately not keyed on `status === 'offline'`: a relay can report active while
	133	holding a cursor we can never reach. `runSeqGuard` never throws — a guard that can
	134	take the membrane down is worse than the parked state it repairs. OPT-IN: inert unless
	135	`crawlRelay` + `crawlHostname` are set.
	136
	137	### ⚠ Known defects in this area (open)
	138
	139	- The boot pass runs AFTER `server.listen` and un-awaited (`pds-serve.js:94` vs
	140	`:156`). Frames emitted in that window carry the un-lifted floor and are below the
	141	relay's cursor — the recurring dark window after every restart.
	142	- The guard is unobservable when healthy. `runSeqGuard` returns a full result but
	143	the caller only logs it, and `control.js`'s `status` exposes nothing — so "ran and
	144	correctly declined" is indistinguishable from "never ran". That ambiguity produced a
	145	plausible-but-wrong timer theory during an incident.
	146	- The aggregate core is machine-local. `frontStore.get({name:'peer-relay-firehose'})`
	147	derives its key from that machine's own corestore primary key, so two membranes = two
	148	permanently independent seq spaces. Since the dedup keys are already
	149	machine-independent, sharing the core is a contained change — but the shipped answer
	150	is one membrane, plus a blind relay, not two membranes.
	151
	152	Diagnose:
	153	```bash
	154	cat ~/.config/atproto-bridge/data/relay-seq-base
	155	curl -s "https://bsky.network/xrpc/com.atproto.sync.getHostStatus?hostname=pds.7rnx.net"
	156	```
	157
	158	## HTTP surface (`server-bare.js` — prod; `server.js` is the node twin)
	159
	160	`createPdsServer({ peers, serverDid, availableUserDomains, favelaWebUpstream })`.
	161	Every DID served is a fronted peer; `frontIfPeer()` returns the front or 404.
	162
	163	Keyless reads: `sync.listRepos` / `getRepoStatus` / `getLatestCommit` / `getRepo`,
	164	`repo.getRecord` / `listRecords` / `describeRepo`, `identity.resolveHandle`,
	165	`/.well-known/atproto-did`.
	166
	167	★ `com.atproto.sync.getRecord` — the PROOF-CARRYING single-record read (CAR with
	168	the signed commit + MST covering path + record block). Distinct from
	169	`repo.getRecord`'s hydrated JSON, which carries no commit, no signature, no proof, so
	170	nothing about it can be verified. We were missing it entirely and it is not niche:
	171	any client doing verified inbound ingest — including our own favela lens — got a
	172	404 and could only fall back to pulling the ENTIRE repo. Per the lexicon an absent
	173	record still returns 200 with an exclusion proof, not 404.
	174
	175	Relayed to the peer (peer-signs-over-RPC) — these are no longer 404 as older docs
	176	claim: `createSession`, `refreshSession`, `getSession`, `getServiceAuth`,
	177	`createRecord` / `putRecord` / `deleteRecord` / `applyWrites`, `uploadBlob`,
	178	`sync.getBlob`, `actor.getPreferences` / `putPreferences`. The bridge forwards the
	179	caller's bearer and the PEER verifies + signs. `applyWrites` normalizes the composer's
	180	batch (`$type` suffix → action, `value` → record, mint a TID rkey for create).
	181
	182	⚠ `uploadBlob` posts opaque BYTES — handled before `readJson`, which would choke
	183	on binary.
	184
	185	Proxies:
	186	- `proxyToFavela` (lens export → favela-web) — MUST be checked before
	187	`isAppViewMethod`: `getFeedSkeleton`/`describeFeedGenerator` start with `app.bsky.`
	188	and would otherwise go to bsky's AppView (wrong). `subscribeLabels` is piped as a
	189	binary WS both ways.
	190	- `proxyToAppView` (public + user-scoped views to `api.bsky.app`). The membrane
	191	holds no key, so a user-scoped read attaches a service-auth token **minted by the
	192	peer** (`front.getServiceAuth`), cached per `did\|aud\|lxm` while ≥30s of life remains.
	193	- `doFetch` picks http vs https by URL scheme — `https.request` on an `http://`
	194	URL hangs on a TLS handshake (the favela proxy-hang bug).
	195
	196	## UDS control plane (`lib/control.js`) — "re-derive, don't restart"
	197
	198	`status` · `list-repos` · `list-fronted` · `resolve-handle` · `diagnose` (read-only,
	199	secret-free) · `probe-front` (force an on-demand front, return the outcome/error) ·
	200	`set-seq-base` · `emit-sync` · `drop-front` (with `forget: true` to DENY a firehose
	201	key — peers re-announce on every append, so a plain drop is undone in milliseconds) ·
	202	`allow-front` · `forget-did` · `release-ready` (OTA).
	203
	204	`status` reports `membrane: !relay` — the blind-relay mode passes `relay` instead of
	205	`peers` and reports `relay.status()`.
	206
	207	## Config (`pds/lib/config.js`) — env > file > default
	208
	209	`~/.config/atproto-bridge/config.json` overlaid by `PDS_*`:
	210	`serverDid` (`did:web:pds.7rnx.net`), `userDomain` (`.7rnx.net`), `port` (3399),
	211	`bind` (127.0.0.1 default; manifest sets `0.0.0.0`), `storage`, `uds`, `frontedDids`
	212	(JSON array or comma-separated `PDS_FRONTED_DIDS`), `feedRegistries`,
	213	`favelaWebUpstream`, `crawlRelay`, `crawlHostname` (defaults to the host part of
	214	`serverDid`).
	215
	216	⚠ `frontedDids` is NOT in the manifest's `defaults.env` — it comes from the
	217	per-host config file. A host without one fronts nothing and serves
	218	`listRepos: []`. That is why only stortron is a functioning membrane today.
	219
	220	⚠ `crawlHostname` defaulting to `serverDid`'s host means every install targets
	221	`pds.7rnx.net` — a second install reads the first's cursor and lifts its own floor
	222	against it.
	223
	224	## Federation: a NEW fronted DID needs `requestCrawl`
	225
	226	A fronted peer serving a signed CAR still won't show on bsky until the RELAY indexes
	227	it. `POST bsky.network/xrpc/com.atproto.sync.requestCrawl {hostname}` (200, no auth) →
	228	the relay re-enumerates via `com.atproto.sync.listRepos` (must list the new DID
	229	`active:true`) → picks it up in ~25s. Handle side must already resolve
	230	(`https://<handle>/.well-known/atproto-did`, served via the `*.7rnx.net` wildcard).
	231
	232	## Deploy
	233
	234	pgh app, `runtime: bare`, `entrypoint: index.js`, `depends: [pgh-sidecar]`,
	235	readiness uds at `~/.config/atproto-bridge/svc.sock` (30s).
	236	`pgh.defaults.env`: `PDS_BIND=0.0.0.0`, `PDS_PORT=3399`, `PDS_STORAGE`/`PDS_SOCK`
	237	under `~/.config/atproto-bridge/`, `FAVELA_WEB_UPSTREAM=http://10.22.0.1:4041`,
	238	`PDS_CRAWL_RELAY=https://bsky.network`.
	239
	240	- Sibling deps via `file:../` — see [[reference_atproto-bridge-sibling-deps-canonical]]
	241	(SYMLINK siblings, never `npm install`; gate = `brittle-node`).
	242	- ⚠ Declare-your-deps: every direct import MUST be declared or the `--production`
	243	closure drops it and the bare daemon crash-loops `MODULE_NOT_FOUND`. Test-green ≠
	244	deploy-ready.
	245	- ⚠ Manual `cp`/`mv` does not compose with pgh. Roll back / install via pgh,
	246	never `cp -a` over the app dir.
	247	- Boot smoke (`npm run smoke`) boots `bare index.js` and asserts describeServer
	248	synthesizes — it caught a `./lib` vs `../lib` relative-import bug the unit tests
	249	cannot see (they import `lib/` directly, never `bin/`).
	250
	251	## Related
	252
	253	- [[atproto-did-peer]] — `DidPeer`/`DidPeerRouter`: announce-push, co-owned fronts,
	254	freshest-by-REV, the continuity gate. The routing layer lives there now, not here.
	255	- [[hyper-proto-peer]] — the peer side: holds keys, signs, `PEER_WRITERS` gate
	256	- [[favela-web]] — the lens-export upstream this proxies to
	257	- [[ap-bridge]] — the ActivityPub projection of the same bytes
	258	- [[atproto-store]] — corestore, `FirehoseSequencer`, frame codec, read-view
	259	- `docs/POSTMORTEM-2026-07-06-multi-did-membrane.md` (go-live saga, 2 outages from
	260	guess-fixing); hyper-social org issues #24 / #25

.claude/skills/atproto-did-peer/SKILL.md +207 -0

…	…	@@ -0,0 +1,207 @@
	1	---
	2	name: atproto-did-peer
	3	description: The direction-free P2P primitive for an atproto DID's data (repo + firehose Hypercores) — announce-push discovery, mandatory signature auth, co-owned multi-front with freshest-by-REV + continuity gate. The shared reader under atproto-bridge, favela-web and ap-bridge.
	4	category: library
	5	priority: high
	6	location: hyper-fleeky/atproto-did-peer
	7	updated: 2026-08-05
	8	---
	9
	10	# atproto-did-peer
	11
	12	v0.2.10. One file: `lib/did-peer.js` (1085 lines). Two exports that matter:
	13	`DidPeer` and `DidPeerRouter`.
	14
	15	Extracted from atproto-bridge as a shared module (audited vs keet + holepunch,
	16	2026-07-13). Every consumer in the fleet reads DIDs through this —
	17	`atproto-bridge`, `favela-web`, `ap-bridge`. If you are debugging "why is a stale
	18	head being served", "why did the index freeze", or "why isn't this peer being
	19	fronted", this is the file.
	20
	21	## The core idea
	22
	23	A DID's data lives in a swarm keyed by `discoveryTopic(did)`. **Every participant
	24	— signing device, availability relay/seedbox, read/HTTP bridge, favela client — is
	25	the SAME primitive.** It is NOT a "front the origin by serverKey" client: it
	26	replicates the DID's cores from whoever has them and serves the freshest commit
	27	that VERIFIES against the DID key.
	28
	29	### Three corrections baked in
	30
	31	1. ANNOUNCE, don't DERIVE. On connect a holder pushes
	32	`{repoKey, firehoseKey, repoLength, firehoseLength}` over a protomux channel
	33	(one-mux-per-connection, keet model). No round-trip, no `serverKey`, no
	34	directionality. (Replaced the old `get-coords` PULL.)
	35	2. EVENT-DRIVEN head reconciliation. The served head re-derives on the firehose
	36	core's `'append'` — which fires on local append and on replicated download.
	37	Never a data poll.
	38	3. Auth is mandatory. The DID topic is unauthenticated under any discovery
	39	scheme; trust comes from `verifyCommit(signed, verifyKey)`. The constructor
	40	throws without a `verifyKey`.
	41
	42	Transport-agnostic: the CONSUMER owns the swarm unless it hands one over.
	43
	44	## `DidPeer` — one DID, one co-owner's cores
	45
	46	```js
	47	new DidPeer(corestore, { did, verifyKey, handle, onHead, onError, noRepo })
	48	```
	49
	50	- `provide({repoKey, firehoseKey})` — this node HOLDS the cores (signing device,
	51	relay that already seeded, bootstrap). Opens them directly.
	52	- `attach(conn)` / `onAnnounce(a)` — learn coords from a peer's push.
	53	- `get holds()` — true once the FIREHOSE is open, not the repo. Keyed off the
	54	repo previously, so a stateless peer (which publishes no repo core) read as
	55	holding nothing.
	56	- `_reconcile()` — decode the last frame → `loadCommit` → `verifyCommit` → set
	57	`head = {rev, cid, signed}` → fire `onHead`. Rejects an unverifiable head, so
	58	it cannot be tricked into serving a forged repo.
	59
	60	### ★ `noRepo: true` — the "one copy" consumer
	61
	62	Follow the firehose ONLY (verified head + tail ops); **do not open or replicate the
	63	repo core**. Records come from the shared content-addressed `OverlayRepoStore`
	64	co-located with the producer. `getRepo`/`getRecord`/`listRecords` return
	65	null/[]/throw. This is what favela-web uses for every DID so there is no
	66	duplicate repo replica on the box.
	67
	68	## `DidPeerRouter` — many DIDs, many co-owners, one swarm
	69
	70	```js
	71	new DidPeerRouter(corestore, {
	72	swarm, join, fronted, relay, resolveVerifyKey, rpc,
	73	refreshMs = 10000, refreshTimeoutMs = 5000,
	74	staleFrontMs = 300000, continuityScan = 500,
	75	handles, onHandles
	76	})
	77	```
	78
	79	\| option \| effect \|
	80	\|---\|---\|
	81	\| `swarm` \| router owns it: auto-wires `onconnection` + the pull-only refresh/failover timer. Omit to drive `onconnection()` and topic joins yourself (favela / tests) \|
	82	\| `join` \| `(did)=>void` topic-join fn — keeps `atproto-identity` out of this module's runtime deps \|
	83	\| `fronted` \| DIDs to allow-list immediately (bridge) \|
	84	\| `relay` \| a `FleetRelay`: fold the freshest co-owner's firehose per DID \|
	85	\| `resolveVerifyKey` \| async `(did)=>pubkey` fallback (bridge resolves via PLC) \|
	86	\| `rpc` \| open ONE `protomux-rpc` per connection (write/session/blob relay); the front borrows it \|
	87	\| `handles`/`onHandles` \| persist handle→DID across restarts \|
	88
	89	favela's read-only posture is just `new DidPeerRouter(store)`.
	90
	91	### One channel per connection (the DUPLICATE_CHANNEL history)
	92
	93	`onconnection(conn)` does exactly three things: `corestore.replicate(conn)` once,
	94	open one announce channel, and (if `rpc`) open one `ProtomuxRPC`. Protomux
	95	allows one channel per `(protocol, id)` per mux; the predecessor opened an RPC per
	96	front by probing every connection, so fronting a 2nd DID hit the 1st front's
	97	connection → `createChannel` returns null → `ProtomuxRPC` throws
	98	`DUPLICATE_CHANNEL` synchronously, before `server.listen` → boot restart loop,
	99	port never opened. Two prod outages. One-per-connection sidesteps it entirely.
	100
	101	`RPC_PROTOCOL = 'hyper-proto-peer/rpc/v1'` is exported and **must match
	102	hyper-proto-peer's serve RPC protocol.**
	103
	104	### ★ Co-owned DIDs: a SET of fronts, keyed by firehoseKey
	105
	106	A DID can have MANY self-custody holders (wj62 = notary + co-owner). Each announces
	107	the SAME did with its OWN core keys — distinct keys, **identical deterministic
	108	content**, differing only in replication lag. So:
	109
	110	- `_fronts: did -> Map<fhKeyHex, DidPeer>` — one front per co-owner
	111	- `_byDid: did -> Map<fhKeyHex, {coords, conns: Map<conn, rpc\|null>}>` — **many peers
	112	announce one `(did, firehoseKey)`**: the key holder AND every keyless seeder. A
	113	single slot was won by whoever announced last, and a seeder answers no write RPC —
	114	hence a set, and `_call` tries candidates until one answers.
	115
	116	Reads resolve to the FRESHEST by commit REV — not by frame count. A lagging
	117	co-owner can out-FRAME a fresher one; that was the live wj62 symptom.
	118
	119	### ★★ The continuity gate — how loser-club served a rival head
	120
	121	Freshest-by-rev alone trusts a rev with no continuity check, so a **forked line
	122	whose rev sorts higher simply wins**. That is exactly how loser-club's public
	123	`getLatestCommit` served a rival 1-record head (`3mrnruepxss22`) over the
	124	authoritative 139-record one (`3mrfgnyiurk22`) until the bridge was bounced. A
	125	rev regression was guarded; a higher rev on a different line was not.
	126
	127	Real co-owners are not a different line: `atproto-store/lib/tid.js:nextRev` makes the
	128	chain a pure function of the previous rev, so two co-owners continuing from the same
	129	head over the same finalized record set produce byte-identical commits — a lagging
	130	co-owner's firehose is a strict PREFIX of a fresher one's. So "chains from the head we
	131	serve" is decidable: walk the candidate's own frames back looking for the exact
	132	commit we serve (`_scanForServed`, bounded by `continuityScan`, default 500).
	133
	134	Two subtleties that are load-bearing:
	135	- Reads are `{wait:false}` — a block we do not hold yet is **INCONCLUSIVE, not
	136	evidence of a fork. The verdict is left uncached** so the next read re-decides
	137	once replication catches up.
	138	- Enforced only while the incumbent is still live. A peer that legitimately
	139	rebuilds its cores (storage repair, `rebuild-did`) goes away and returns announcing
	140	new keys on a new line; refusing that would strand the DID on dead cores. Live
	141	incumbent + rival line ⇒ fork, keep the incumbent. No incumbent (gone, or evicted)
	142	⇒ the new line is the only truth on offer, take it.
	143
	144	### Relay folding — ONE source per DID
	145
	146	`_pickRelaySource(did)` folds the freshest chained co-owner's firehose into the
	147	`FleetRelay` — single source, because two co-owners are the SAME frames under
	148	different sourceIds and folding both duplicates every `#commit`. On a source change
	149	(fresher co-owner won, or the folded one stalled) it swaps
	150	`removeSource → addSource` and emits a `#sync` re-baseline
	151	(`_emitSync`, key `sync:${did}:${rev}`).
	152
	153	The relay must fold the same line the read surface serves — folding a fork's
	154	frames publishes it to every downstream subscriber, which is worse than serving it
	155	on one read.
	156
	157	### Eviction — "re-derive, don't restart"
	158
	159	A cached front outlives its peer's connection by design (its cores keep replicating),
	160	so before these there was no way to drop a front announcing a FORKED line short of a
	161	restart:
	162
	163	- `dropFront(did, {firehoseKey, forget})` — `forget: true` also DENIES that
	164	firehose key (`_denied`), required whenever the reason to evict is what the front
	165	is serving: peers re-announce on every append, so a plain drop is undone
	166	milliseconds later.
	167	- `allowFront(did, firehoseKey)` — the undo.
	168	- `forgetDid(did)` — drop every cached front + resolved key; nothing is denied and
	169	the DID stays fronted.
	170	- `_ageOutFronts` — drop a front whose peer has not announced for `staleFrontMs`
	171	(default 5 min), but only while a live verified sibling remains. Never drops
	172	the last front. `0` disables.
	173
	174	### Diagnostics
	175
	176	`diagnose()` — read-only, secret-free snapshot: per-DID fronts with `relaySource`,
	177	head, firehose length, liveness, denial state, plus `relay.sources`.
	178
	179	## Consumers, and what each turns on
	180
	181	\| consumer \| posture \|
	182	\|---\|---\|
	183	\| `atproto-bridge` (membrane) \| owns-swarm, `join`, `fronted`, `relay`, `rpc: true`, `resolveVerifyKey` = PLC \|
	184	\| `favela-web` \| read-only `new DidPeerRouter(store)`; every DID `noRepo: true`; drives its own swarm + join; re-baselines on front change (a new co-owner's firehose is a distinct history — following one fixed core froze wj62 and ashatron at their July failovers) \|
	185	\| `ap-bridge` \| keyless read + the peer's application RPC for blob relay \|
	186
	187	## Gotchas
	188
	189	- A freshly-opened replicated core reads `length = 0` until an explicit
	190	`update()`; `'append'` fires only for blocks appended AFTER connect. Consumers that
	191	need the backlog must `update({wait:true})` first (this is why `FleetRelay.addSource`
	192	does it — without it the peer's pre-existing `#commit` history never enters the
	193	aggregate and only a `#sync` re-baseline recovers it).
	194	- `_onConnClose` drops only the `_byDid` entry, never the front — its cores keep
	195	replicating over the next connection.
	196	- `add(did, opts)` without a `verifyKey` just allow-lists and returns `this`
	197	(chainable); the key resolves lazily at front time.
	198
	199	## Related
	200
	201	- [[atproto-bridge]] — the membrane; owns-swarm + relay consumer
	202	- [[favela-web]] — read-only + `noRepo` consumer, failover-aware ingest
	203	- [[ap-bridge]] — keyless AS2 projection consumer
	204	- [[hyper-proto-peer]] — the holder side; must speak `RPC_PROTOCOL`
	205	- [[atproto-store]] — `RepoStore`, `FirehoseSequencer`, frame codec, `read-view`
	206	- memory: [[feedback_p2p-no-directionality-event-driven]],
	207	[[project_membrane-peerfront-standing-download-2026-07-18]]

.claude/skills/atproto-store/SKILL.md +48 -2

…	…	@@ -4,9 +4,13 @@ description: Holepunch-native atproto storage substrate — TWO record backends
4	4	category: library
5	5	priority: medium
6	6	location: hyper-fleeky/atproto-store
7		updated: 2026-07-26
	7	updated: 2026-08-05
8	8	---
9	9
	10	v0.0.20. Audited 2026-08-05 against source: all 14 `lib/*.js` modules are
	11	covered. Consumers: [[hyper-proto-peer]], [[atproto-did-peer]],
	12	[[atproto-bridge]], [[favela-web]], [[ap-bridge]].
	13
10	14	# atproto-store
11	15
12	16	## What it is
…	…	@@ -292,10 +296,52 @@ files. Drop-in for `BlobStore`.
292	296	- `has` / `close` (:86-91) — note `close()` correctly closes an owned
293	297	Corestore, unlike legacy `BlobStore.close()` (gotcha #3).
294	298
	299	`lib/blob-pointer-store.js` — `BlobPointerStore`. ★ **The blob half of
	300	"pointers in the autobase, bytes in the overlay"** — to blobs exactly what
	301	`PointerSubstrate` is to records. It exists because the records side got this
	302	right and the blobs side did not.
	303
	304	The gap it closes. `OverlayBlobStore`'s `{mimeType, size, oid}` bee is
	305	NODE-LOCAL — in the live daemon it is `overlay-blobs:<did>` on the membrane
	306	corestore, which does not replicate — and `get()` reads that bee FIRST,
	307	returning null before it ever looks at the file or the overlay (invariant #6).
	308	So one upload produces two artifacts with very different fates:
	309
	310	\| artifact \| fate \|
	311	\|---\|---\|
	312	\| the FILE + overlay registration \| content-addressed, fetchable by `oid` from any peer \|
	313	\| the bee entry `{mimeType,size,oid}` \| stays on that node, forever \|
	314
	315	Another co-owner therefore cannot serve the blob even when the bytes are
	316	reachable, because the one thing it lacks is the `cid → oid` mapping needed to
	317	ask for them. The atproto blob CID (raw codec, `cidForBlobSync`) and the overlay
	318	oid (blake2b) are different hashes, so the mapping must be **replicated, not
	319	recomputed**. Production symptom: an avatar that serves or 404s depending on
	320	which co-owner the bridge happens to front.
	321
	322	The fix. A tiny pointer in the SHARED autobase, key `__blob__/<cid>`, in the
	323	reserved `"__"` namespace — `RepoBase.splitKey` and `PointerSubstrate` both
	324	return null for a collection starting with `__`, so these entries can never be
	325	mistaken for records or materialized into a signed commit.
	326
	327	- `new BlobPointerStore(statebase, blobs, { did })` — throws unless `statebase`
	328	has `set`/`get` and `blobs` has `put`/`adoptMeta`.
	329	- `static key(cidStr)` → `__blob__/<cid>`.
	330	- Same public surface as `BlobStore`/`OverlayBlobStore`
	331	(`ready`/`put`/`get`/`has`/`close`) so it composes under `DualBlobStore`
	332	unchanged. `ready()` is a no-op — the caller readies the underlying
	333	`OverlayBlobStore`.
	334	- ⚠ A failed pointer write does NOT fail the upload — the bytes are already
	335	durable and servable from this node, so throwing would turn a cross-node
	336	limitation into a total upload failure. It is logged LOUDLY rather than
	337	swallowed, because "blob local-only" is precisely the silent condition this
	338	module exists to end.
	339
295	340	`hyper-proto-peer/lib/dual-blob-store.js` — `DualBlobStore` (lives in the
296	341	consumer, not here). Write-new/read-both migration wrapper: `put` goes ONLY to
297	342	the overlay store; `get`/`has` try overlay then fall back to the legacy
298		Hyperbee `BlobStore`. Intended to age out, not to be permanent.
	343	Hyperbee `BlobStore`. Intended to age out, not to be permanent — see
	344	[[project_blob-full-migration-todo]] (rip it out once the migration lands).
299	345
300	346	## Storage-layout + key/sign invariants
301	347

.claude/skills/favela-web/SKILL.md +233 -0

…	…	@@ -0,0 +1,233 @@
	1	---
	2	name: favela-web
	3	description: Bare SSR gateway + local appview for favela — reads a roster of DIDs over P2P (atproto-did-peer), indexes them into a rebuildable local Hyperbee, serves no-JS HTML/JSON, and exports the LENS to bsky (getFeedSkeleton + labeler) that atproto-bridge proxies to
	4	category: project
	5	priority: high
	6	location: hyper-fleeky/favela-web
	7	updated: 2026-08-05
	8	---
	9
	10	# favela-web
	11
	12	v0.3.31, runtime Bare, pgh app `favela-web`, port 4041.
	13
	14	## What it is / when to read this
	15
	16	The P2P-less window onto a favela community (`DESIGN.md` §2.6/§4.1): a Bare
	17	backend that follows a roster of DIDs over Hyperswarm via `atproto-did-peer`,
	18	folds them into a local, rebuildable appview index, and serves
	19	server-rendered HTML at real URLs (no JS required, SEO-able, RSS/Atom) plus JSON
	20	APIs as progressive enhancement.
	21
	22	It is also — and this is the part that bites operationally — the lens export:
	23	the thing bsky calls when someone opens a favela custom feed or subscribes to
	24	favela moderation. `atproto-bridge` does not implement those; it **proxies them
	25	here**.
	26
	27	Read this before touching: the lens export (`feedgen.js` / `labeler-export.js` /
	28	`label-firehose.js`), the appview ingest path, the shared-overlay coupling, DID
	29	auth / pairing, or anything that moves favela-web to a different machine.
	30
	31	## ⚠ Two couplings that decide where this can run
	32
	33	### 1. It reads hpp's overlay directly, off local disk
	34
	35	`bin/favela-web.js:244`
	36	```js
	37	const OVERLAY_ROOT = process.env.ATPROTO_OVERLAY_ROOT \|\| STORE('atproto-repo')
	38	new OverlayRepoStore(join(OVERLAY_ROOT, roomKey16(did)), { did })
	39	```
	40	The deployed manifest points this at hpp's own storage:
	41	`ATPROTO_OVERLAY_ROOT=${HOME}/.config/hyper-proto-peer/data/atproto-repo`.
	42	Identical layout (`roomKey16 = hash('atproto-base/v1/room-key' ‖ did)`), byte-exact
	43	DAG-CBOR, so co-located hpp + favela-web **share the exact same files — one copy
	44	per machine**, and the appview index holds only POINTERS.
	45
	46	This drives a real behavioural fork at boot (`bin/favela-web.js:479-481`). EVERY
	47	DID is opened `noRepo: true` (follow the firehose, never replicate the repo core),
	48	and `coHosted` decides the baseline:
	49
	50	\| \| baseline source \|
	51	\|---\|---\|
	52	\| DID whose records hpp already holds on this box \| the authoritative shared store — bypasses membrane fork/desync \|
	53	\| any other DID \| firehose backfill from `subscribeRepos(0)` (each `#commit` frame carries the bytes) \|
	54
	55	So favela-web next to hpp is not incidental — it is how co-hosted DIDs get an
	56	authoritative baseline. Moving it to a box without hpp silently downgrades every
	57	DID to firehose backfill.
	58
	59	### 2. The membrane proxies to it over the VPN, by IP
	60
	61	`atproto-bridge`'s manifest sets `FAVELA_WEB_UPSTREAM=http://10.22.0.1:4041`
	62	(tabertron), and favela-web's own manifest sets
	63	`FAVELA_EXTERNAL_URL=https://pds.7rnx.net` + `FAVELA_BRIDGE_URL` back at the
	64	membrane. Circular, plain HTTP, hostname-pinned, not swarm. If favela-web is
	65	down, bsky's appview returns
	66	`UpstreamFailure: "favela proxy failed: connection refused"` for every feed —
	67	verified 2026-08-05. Profiles and posts keep resolving (those are keyless reads
	68	served from the membrane's own replicas); only the lens dies.
	69
	70	## The lens export — what bsky actually calls
	71
	72	A community DID's PLC doc carries the service endpoints; verified for
	73	`did:plc:wqnwulql7ndcz6xmtvaovl6u` (loser-club):
	74
	75	```
	76	#bsky_fg BskyFeedGenerator → https://pds.7rnx.net
	77	#atproto_labeler AtprotoLabeler → https://pds.7rnx.net
	78	#atproto_pds AtprotoPersonalDataServer → https://pds.7rnx.net
	79	```
	80
	81	So the path is bsky → membrane → favela-web, never bsky → favela-web directly.
	82	The membrane routes it in `server-bare.js` via `isFavelaMethod(nsid)` →
	83	`proxyToFavela`, and that check MUST precede `isAppViewMethod` (both are
	84	`app.bsky.*`, and getFeedSkeleton would otherwise be proxied to bsky's own
	85	AppView — wrong).
	86
	87	### Feed half — `lib/feedgen.js`
	88
	89	Stateless and hydration-free per the atproto contract: return an ordered list of
	90	post `at://` URIs, bsky hydrates the rest. A community's skeleton is its members'
	91	posts in reverse-chron, straight out of `getTimeline({authors})`.
	92
	93	- `GET /.well-known/did.json` — did:web doc, `BskyFeedGenerator` → our HTTPS
	94	- `GET /xrpc/app.bsky.feed.describeFeedGenerator` — every hosted community feed
	95	- `GET /xrpc/app.bsky.feed.getFeedSkeleton?feed=…` — one community's skeleton
	96
	97	⚠ Only `app.bsky.feed.post` URIs are returned. bsky cannot hydrate
	98	`net.favela.local.post` (unknown lexicon), so local-only posts correctly stay
	99	local (`DESIGN` §3.6 caveat). `slugRkey(id)` is deterministic and is used both when
	100	registering a feed and when publishing the generator record, so the two never drift.
	101
	102	### Moderation half — `lib/labeler-export.js` + `lib/label-firehose.js`
	103
	104	The EMITTER side (inverse of `lib/labeler.js`'s reader). favela produces byte-exact
	105	`com.atproto.label` records (DAG-CBOR + secp256k1 low-S over the sig-less bytes,
	106	`atproto-identity/lib/labels.js`), projected into three bsky-legible surfaces:
	107
	108	1. did:web identity — `#atproto_label` verificationMethod (so bsky verifies our
	109	sigs) + `#atproto_labeler` service entry
	110	2. `com.atproto.label.queryLabels` — the HTTP pull surface
	111	3. `app.bsky.labeler.service` record — the publish ceremony
	112
	113	`subscribeLabels` (the WS firehose bsky's AppView ingests for realtime) is served
	114	by `label-firehose.js` and proxied through the membrane as a WS pipe
	115	(`server-bare.js` opens a client WS to `FAVELA_WEB_UPSTREAM` and pipes binary
	116	frames both ways).
	117
	118	## The appview index — `lib/appview.js` (1017 lines)
	119
	120	FAVELA-LOCAL by design (§7.5): every peer derives its OWN view. The index is a
	121	rebuildable cache, never replicated. Bare-native Hyperbee, no shims.
	122
	123	Two ingest paths:
	124	- `indexPeer(peer)` — SNAPSHOT reconcile from the current repo (`listRecords`).
	125	Catches bulk-imported/adopted history that never arrived as per-record firehose
	126	commits, and REMOVES entries whose record is gone (delete-correctness).
	127	- `applyCommit(did, body)` — INCREMENTAL per-commit ingest from a `#commit` frame;
	128	the record is read from the frame's own CAR (race-free). This is the live tail.
	129
	130	Keys (ISO `createdAt` sorts lexicographically = chronologically; `reverse:true` =
	131	newest first):
	132	```
	133	timeline ${createdAt}\x00${did}\x00${rkey}
	134	author ${did}\x00${createdAt}\x00${rkey}
	135	ref ${did}\x00${rkey} -> { createdAt }
	136	```
	137	The `ref` sub exists because a delete op carries no record — it is how `delPost`
	138	reconstructs the `createdAt`-bearing keys, and how an UPDATE that moved `createdAt`
	139	clears its stale ones.
	140
	141	`INGEST_VERSION` is exported and gates re-ingest.
	142
	143	## ★ Failover-aware ingest (the loser-club stall)
	144
	145	`bin/favela-web.js:487-500`. A **co-owned DID can move to a different co-owner's
	146	firehose core** (a new `firehoseKey`) at any time. Following one fixed core FREEZES
	147	the index at the failover point — this is exactly how wj62 and ashatron both froze
	148	at their July co-owner failovers.
	149
	150	So the ingest loop resolves the freshest verified front (`router.front(did)` →
	151	highest-rev co-owner), baselines + live-tails it, and when the router opens a
	152	fresher front it aborts the tail and re-baselines — a different co-owner's
	153	firehose is a distinct history, not a continuation. `#sync` re-baselines within a
	154	front. Event-driven; no poll.
	155
	156	## Module map
	157
	158	\| module \| role \|
	159	\|---\|---\|
	160	\| `bin/favela-web.js` (2139) \| entry: swarm + router + ingest loops + all HTTP routes \|
	161	\| `lib/render.js` (1510) \| SSR — timeline/profile/post/search/lists/notifications pages, Atom \|
	162	\| `lib/appview.js` (1017) \| the local index (above) \|
	163	\| `lib/spacedust.js` (322) \| backlink source; ≠ Constellation on subjects \|
	164	\| `lib/cli.js` (304) \| `pair devices revoke fronts lens status conformance repair-strongrefs` \|
	165	\| `lib/retention.js` (269) \| inbound-lens retention budget (`FAVELA_RETENTION_BUDGET`) \|
	166	\| `lib/lens-import.js` (251) \| INBOUND lens: thread parents/replies extraction \|
	167	\| `lib/record-fetch.js` (211) \| verified single-record read (`sync.getRecord` proof CARs) \|
	168	\| `lib/inbound.js` (210) \| pointer → verify → retain \|
	169	\| `lib/conformance.js` (210) \| self-check; wired to `favela-web conformance` \|
	170	\| `lib/writer.js` (198) \| authoring path (like/follow/post/list/gate records) \|
	171	\| `lib/link-drain.js` (187) \| link/backlink drain \|
	172	\| `lib/constellation.js` (172) \| Constellation backlink client \|
	173	\| `lib/feedgen.js` (138) \| lens export — feed \|
	174	\| `lib/labeler-export.js` (98) \| lens export — labels (pull) \|
	175	\| `lib/label-firehose.js` (69) \| lens export — labels (WS push) \|
	176	\| `lib/auth.js` / `lib/pairing.js` \| browser DID sub-key auth + blind-pairing invites \|
	177	\| `lib/communities.js` / `favela-community` \| community rooms, discovery, FeedRegistries \|
	178	\| `lib/blob-serve.js` \| `roomKey16`, `blobPath`, `sniffMime`, `isSafeCid` \|
	179
	180	## Config (env, `pgh.defaults.env`)
	181
	182	\| var \| deployed value \| meaning \|
	183	\|---\|---\|---\|
	184	\| `FAVELA_PORT` \| `4041` \| HTTP port \|
	185	\| `FAVELA_HOST` \| `0.0.0.0` \| ⚠ bare-http1 `listen(port)` defaults to 127.0.0.1 unlike Node — must be set or a remote membrane cannot reach the lens XRPC \|
	186	\| `FAVELA_EXTERNAL_URL` \| `https://pds.7rnx.net` \| public URL in did:web / feed records \|
	187	\| `FAVELA_COMMUNITY_ROOMS` \| `loser-club=91ce4dbc…` \| `name=roomKey` pairs \|
	188	\| `ATPROTO_OVERLAY_ROOT` \| hpp's `data/atproto-repo` \| the shared byte store (see above) \|
	189	\| `FAVELA_STORE_ROOT` \| `~/.config/favela-web/data` \| corestore for the index \|
	190	\| `FAVELA_SPACEDUST` \| `1` \| backlink source on \|
	191	\| `FAVELA_RETENTION_BUDGET` \| `268435456` (256 MB) \| inbound-lens retention ceiling \|
	192	\| `FAVELA_BRIDGE_URL` \| `https://pds.7rnx.net` \| set per-host in `apps.json` \|
	193
	194	Readiness is http on 4041 (not UDS, unlike the bridge/hpp).
	195
	196	## Deploy
	197
	198	pgh app, `runtime: bare`, `entrypoint: bin/favela-web.js`, `autostart: true`,
	199	`depends: [pgh-sidecar]`, `pgh.stage.ignore: ["data/"]`.
	200	Siblings via `file:../`: `atproto-did-peer`, `atproto-identity`, `atproto-repo`,
	201	`atproto-store`, `favela-community`, `hyper-identity`, `hyper-svc` — same
	202	sibling-vendor rule as the rest of the fleet
	203	([[reference_atproto-bridge-sibling-deps-canonical]]: SYMLINK siblings, never
	204	`npm install`).
	205
	206	## Diagnostics
	207
	208	```bash
	209	favela-web status # includes the HOSTNAME — a status page that doesn't name its box
	210	# is how you diagnose the wrong machine
	211	favela-web fronts # per-DID front state (lib/fronts-diag.js)
	212	favela-web lens # lens export state
	213	favela-web conformance # self-check
	214	favela-web repair-strongrefs
	215	```
	216
	217	Is the lens actually up, end to end (the only decisive probe — our own endpoint
	218	being correct proves nothing about bsky):
	219	```bash
	220	curl -s "https://api.bsky.app/xrpc/app.bsky.feed.getFeed?feed=at://<communityDid>/app.bsky.feed.generator/<slug>&limit=2"
	221	```
	222	`UpstreamFailure: favela proxy failed: connection refused` ⇒ favela-web is down or
	223	unreachable from the membrane's `FAVELA_WEB_UPSTREAM`.
	224
	225	## Related
	226
	227	- [[atproto-did-peer]] — the DID reader/router this is built on (`noRepo` mode, front selection)
	228	- [[atproto-bridge]] — the membrane that proxies the lens here (`proxyToFavela`)
	229	- [[hyper-proto-peer]] — owns the overlay this shares, and signs the records
	230	- [[atproto-store]] — `OverlayRepoStore`, firehose frame codec
	231	- `DESIGN.md` §2.6/§3.6/§4.1/§7.5 in-repo; `FEATURES.md`; `spec/lens-export.md`
	232	- memory: [[project_favela-inbound-lens-2026-07-31]], [[project_favela-overlay-migration-2026-07-24]],
	233	[[project_favela-p2p-local-social-2026-07-13]], [[project_bsky-custom-feed-lande-langs-drop]]

.claude/skills/hyper-proto-peer/SKILL.md +233 -291

…	…	@@ -1,300 +1,242 @@
1	1	---
2	2	name: hyper-proto-peer
3		description: Self-custody atproto peer daemon — holds the DID signing key and Hypercore content, authors + signs its own repo/firehose over Hyperswarm, fronted keylessly by atproto-bridge
	3	description: Self-custody atproto peer daemon — holds the DID signing key, authors + signs its own repo/firehose over Hyperswarm, fronted keylessly by atproto-bridge. Deployed path is the MULTI-DID atproto-base daemon (StateBaseV3 + lease + shared signing key), not the legacy single-tenant Peer.
4	4	category: project
5	5	priority: high
6	6	location: hyper-fleeky/hyper-proto-peer
7		updated: 2026-07-06
	7	updated: 2026-08-05
8	8	---
9	9
10	10	# hyper-proto-peer
11	11
12		## What it is / when to read this
13
14		The self-custody peer half of the peer/bridge split (Model B; see
15		`atproto-bridge/docs/DECISION-2026-07-03-peer-bridge-split.md`). It is the
16		account: it holds the user's atproto signing key + all repo content as
17		Hypercores, authors and SIGNS every write locally, and serves the signed
18		repo + a signed firehose over Hyperswarm. `atproto-bridge` (a separate,
19		stateless repo) discovers this peer by DID, replicates it, and projects it
20		to bsky — the bridge never holds a key and never signs (`lib/peer.js:1-11`).
21
22		Read this skill before: restoring/redeploying the wj62 (`p2p.7rnx.net`)
23		peer, adding new RPC methods, changing vault/config shape, or touching
24		`.pear-git/ci.yaml`. Runtime is plain Node (`package.json:15`,
25		`.pear-git/ci.yaml:9-10`) — no bare shims, unlike most of this fleet.
26
27		## Public API — `Peer` (`lib/peer.js`)
28
29		Onboarding / lifecycle (all static factories, all persist a vault at
30		`<storage>/peer-identity.json`, `VAULT_FILE` at `lib/peer.js:29`):
31
32		- `Peer.fromMnemonic({storage, mnemonic, handle, pdsUrl, clock, now})` (`lib/peer.js:56-59`) — derive identity straight from a seed via `createAtprotoIdentity`, no vault durability contract enforced by itself (tests use it for throwaway dirs); calls `Peer.open`.
33		- `Peer.fromPrivateKey({storage, privateKey, handle, pdsUrl, ...})` (`lib/peer.js:62-65`) — bring-your-own raw key (e.g. already in a hyperident vault).
34		- `Peer.init({storage, rootMnemonic, deviceMnemonic?, handle, pdsUrl, appPassword?, clock, now})` (`lib/peer.js:77-107`) — the fresh onboarding path: two-role attestation, ROTATION authority = hyperident root (never persisted to the vault — `t.absent(vault.rootMnemonic...)` in `test/peer-init.test.js:39`), SIGNING key = a per-device key (`buildAttestedIdentity`, `atproto-identity/lib/device-attestation.js`). Throws if a vault already exists at that path (`lib/peer.js:82`) — never clobbers custody.
35		- `Peer.adopt({storage, did, handle, privateKey, didKey, pdsUrl, rotationKeys?, rootDidKey?, appPassword?})` (`lib/peer.js:123-145`) — migration path: bring-your-own EXISTING did:plc + signing key, no PLC op (`signedOp: null`, `lib/peer.js:139`, DID already published so serviceEndpoint is unchanged). Also refuses to clobber an existing vault.
36		- `peer.importRecords(records)` (`lib/peer.js:150-154`) — re-authors each record under THIS peer's key via `write()`; record bytes stay canonical DAG-CBOR so CIDs are preserved (bsky sees the same posts) — only the commit is fresh, at a higher rev.
37		- `Peer.load({storage, clock, now})` (`lib/peer.js:157-163`) — the normal restart path: reload from the vault. Throws if no vault (`lib/peer.js:160`).
38		- `Peer.setAppPassword({storage, appPassword})` (`lib/peer.js:110-117`) — rewrite the vault's app-password hash (and mint a jwtSecret if absent) without touching the signing key.
39		- `Peer.open(...)` / `Peer._openFromVault(...)` (`lib/peer.js:165-192`) — the shared internal constructor: opens a Corestore, a `RepoStore`, a `RepoManager`, and a `FirehoseSequencer`.
40
41		Writes / reads:
42
43		- `peer.write({collection, rkey, record})`, `peer.update(...)`, `peer.delete(...)` (`lib/peer.js:207-217`) — thin wrappers over `_authorAndSequence` (`lib/peer.js:201-205`): `RepoManager.applyWrite` signs the commit, then `FirehoseSequencer.sequenceCommit` appends the signed commit block itself into the peer's own firehose core — so a keyless replica reads the signature off the replicated firehose instead of needing to re-sign (this is the "signed-commit gap" the firehose closes).
44		- `peer.getRepo()` (`lib/peer.js:220-222`) — the full signed repo as a CAR (what a bridge fetches once, or a client calls `com.atproto.sync.getRepo` against).
45		- `peer.repoKey` / `peer.repoKeyHex` (`lib/peer.js:226-227`), `peer.firehoseKey` / `peer.firehoseKeyHex` (`lib/peer.js:232-233`) — the two Hypercore keys a replica opens read-only after replicating the Corestore.
46		- `peer.rev` / `peer.commitCid` (`lib/peer.js:194-195`).
47
48		Sessions (peer is its own auth authority, `lib/peer.js:235-266`):
49
50		- `peer.canLogin` — true iff both `jwtSecret` and an `appPasswordHash` are set.
51		- `peer.createSession({identifier, password})` — verifies the app-password (`verifyPassword`), mints access/refresh JWTs with the peer's own `_jwtSecret` (not the atproto signing key). ACCESS_TTL 2h, REFRESH_TTL 90d (`lib/peer.js:31-32`).
52		- `peer.refreshSession(refreshJwt)` — verifies + re-mints.
53
54		Serving (`lib/peer.js:268-326`):
55
56		- `peer.serve(swarm, {canWrite?})` — joins two topics: the swarm's own `keyPair.publicKey` (fleet-replicate convention, matches `atproto-bridge`'s `FleetNode`/`connectReplica`) AND `discoveryTopic(peer.did)` from `atproto-identity/lib/discovery.js` (findable from the DID alone — "Option 2" — no DID-doc entry needed). Returns `{serverKey, discoveryTopic, ready(), close()}`. On every connection it calls `this.corestore.replicate(conn)` (content) and `this._attachRpc(conn, {canWrite})` (control), muxed over the same connection via `Protomux.from(conn)` under the hood (protomux-rpc).
57		- RPC protocol string: `'hyper-proto-peer/rpc/v1'` (`lib/peer.js:30`). Methods offered by `_attachRpc` (`lib/peer.js:301-326`):
58		- `get-coords` — always available, no auth: returns `{did, handle, serverKey, repoKey, firehoseKey}` (public keys only). This is what `PeerFront.discover` (in `atproto-bridge/pds/lib/peer-front.js`) calls after finding the peer via `discoveryTopic`.
59		- `apply-write` — only registered if `canWrite` is truthy; gates on `conn.remotePublicKey` passed through the caller-supplied `canWrite(remoteHex)` predicate; throws `unauthorized: ...` otherwise. On success calls `_authorAndSequence` and returns `{did, commitCid, rev, seq}`.
60		- `create-session` / `refresh-session` — only registered if `peer.canLogin` is true.
61
62		Close (`lib/peer.js:328-334`): closes `seq`, `store`, then `corestore` — the comment at `lib/peer.js:331-332` is load-bearing: the Corestore holds the RocksDB fd-lock, and skipping this close makes a subsequent `Peer.load` on the same dir fail with "File descriptor could not be locked".
63
64		## The pgh-app entry — `index.js`
65
66		`index.js` is what `pgh`/pear-git launches as `node index.js` in production
67		(`package.json:7` bin, `package.json:11-26` `pgh` manifest block:
68		`"type": "daemon"`, `"runtime": "node"`, `"autostart": true`, `"depends":
69		["pgh-sidecar"]`, readiness = pidfile).
70
71		Mode selection is config-driven, first-boot-only (`index.js:26-63`),
72		checked in this exact order:
73
74		1. `load` — if `<storage>/peer-identity.json` already exists → `Peer.load({storage: cfg.storage})` (`index.js:26-28`). This is the durable restart path; nothing else in config matters once a vault exists.
75		2. `adopt` — else if `cfg.adoptDid && cfg.adoptDidKey && cfg.adoptDevicePrivHex` are all set → `Peer.adopt({storage, did, handle, privateKey: adoptDevicePrivHex, didKey: adoptDidKey, pdsUrl, appPassword})` (`index.js:29-39`); if `cfg.adoptCar` is also set, reads the CAR (`atproto-repo/lib/repo.js#readCar` + `#listRecords`) and calls `peer.importRecords(...)` (`index.js:40-50`).
76		3. `init` — else if `cfg.rootMnemonic` is set → `Peer.init({storage, rootMnemonic, handle, pdsUrl, appPassword})` (`index.js:51-59`) — fresh onboarding.
77		4. Else: logs an error naming which config keys are missing and `process.exit(1)` (`index.js:60-62`).
78
79		After mode resolution, if `peer.seq.length === 0` it performs one dummy
80		`write()` (`index.js:68-74`) so the peer always has at least one signed
81		commit — required for a bridge to authenticate it on first connect (the
82		bridge replays the signed commit against the DID key). An adopted peer with
83		an imported CAR already has commits, so this only fires on a truly fresh
84		`init`. Then it opens a `Hyperswarm`, calls `peer.serve(swarm)`, awaits
85		`serving.ready()`, and prints one JSON status line with `did`, `handle`,
86		`didKey`, `serverKey`, `discoveryTopic`, `repoKey`, `firehoseKey`,
87		`firehoseLen` (`index.js:80-91`) — this line is what an operator/monitor
88		greps for on boot. `SIGINT`/`SIGTERM` close serving → destroy swarm →
89		`peer.close()` (`index.js:93-98`).
90
91		### Config keys — `lib/config.js` (this is what drives a pgh deploy)
92
93		`loadConfig()` reads `${PEER_CONFIG env or ~/.config/hyper-proto-peer/config.json}`
94		(`lib/config.js:22-24`), then env wins over file wins over default
95		(`lib/config.js:32-36`) so both the pgh manifest's `defaults.env` and an
96		operator's config.json work. `${HOME}`/`~` are expanded (`lib/config.js:17-20`).
97		Keys (file key / env var):
98
99		\| purpose \| file key \| env var \| notes \|
100		\|---\|---\|---\|---\|
101		\| storage dir \| `storage` \| `PEER_STORAGE` \| default `~/.config/hyper-proto-peer/data`; the pgh manifest sets this as a package default (`package.json:19-20`) so it survives app updates (outside the install tree) \|
102		\| handle \| `handle` \| `PEER_HANDLE` \| default `peer.local` \|
103		\| PDS url (cosmetic/legacy field carried in vault) \| `pdsUrl` \| `PEER_PDS_URL` \| default `https://pds.local` \|
104		\| app-password \| `appPassword` \| `PEER_APP_PASSWORD` \| optional; enables `create-session`/`refresh-session` RPCs \|
105		\| adopt: existing DID \| `adoptDid` \| `ADOPT_DID` \| migration only \|
106		\| adopt: DID's did:key \| `adoptDidKey` \| `ADOPT_DID_KEY` \| migration only \|
107		\| adopt: device signing key (hex) \| `adoptDevicePrivHex` \| `ADOPT_DEVICE_PRIV_HEX` \| secret — keep out of git, file lives on the box only \|
108		\| adopt: repo CAR to import \| `adoptCar` \| `ADOPT_CAR` \| optional; path, `${HOME}`-expanded \|
109		\| init: hyperident recovery seed \| `rootMnemonic` \| `PEER_ROOT_MNEMONIC` \| secret; only consulted while no vault exists \|
110
111		Restoring the wj62 peer: per
112		`atproto-bridge/docs/POSTMORTEM-2026-07-03-wj62-migration.md:12-16,45-48`,
113		wj62 was already adopted + proven live on stortron as a pgh app before the
114		membrane cutover broke `:3399` (the peer itself was untouched/safe). To
115		redeploy: either restart the existing pgh-installed app (vault already
116		exists → falls straight into the `load` branch, none of the `adopt`/`ADOPT_`
117		config matters anymore), or, if standing up fresh, populate
118		`ADOPT_DID=did:plc:wj62ebbusln6gxo5azo6ogzo`, `ADOPT_DID_KEY=did:key:zQ3shtugtPDvzXzfyMCytcEjdvwEvtZ8g4s8vfuPXQa3mhrgX`,
119		`ADOPT_DEVICE_PRIV_HEX=<the wj62 device key, off-box secret>`, and
120		optionally `ADOPT_CAR=<path to backed-up repo.car>` (the postmortem
121		mentions `stortron:~/wj62-backup/wj62-repo.car`). **Do not re-adopt if a
122		vault already exists** — `Peer.adopt`/`Peer.init` both refuse to clobber
123		(`lib/peer.js:82,128`); `index.js`'s own `load`-first check means this is
124		already the default behavior, but hand-invoking `bin/peer-migrate.js`
125		against a live vault dir would still error correctly rather than silently
126		overwrite.
127
128		### Deploy shape — `.pear-git/ci.yaml`
129
130		Runtime is node, so — unlike bare-runtime pgh apps — there is no
131		`bare-deps` postinstall step (`.pear-git/ci.yaml:9-10`). Three steps
132		(`.pear-git/ci.yaml:17-33`):
133
134		1. `npm install --production` (root closure).
135		2. Vendor step: for each of `atproto-repo`, `atproto-identity`,
136		`atproto-store` (declared as `file:../` siblings in `package.json:28-30`),
137		`rm -rf node_modules/$pkg && cp -rL $SRC node_modules/$pkg` where `$SRC`
138		is resolved from the KB path or `$HOME/hyper-fleeky/$pkg`
139		(`.pear-git/ci.yaml:22-29`). This is necessary because CI `git-archive`s
140		only this repo into `/tmp`; a plain `npm install` of `file:../`
141		deps leaves symlinks that don't survive the archive, so the published
142		tree would be missing them entirely without the `cp -rL` dereference.
143		3. `pgh publish . --accept-conflict "..."` — content-addresses the closure
144		into the pgh ReleaseRegistry (installable via `pgh app install`), same
145		pattern as atproto-bridge/hypernote/hyper-wallet.
146
147		Declare-your-deps lesson (verified in git log, not inferred): commit
148		`c51651d` ("hyper-peer: hyperswarm is a runtime dep") moved `hyperswarm`
149		from devDependencies into `dependencies` in this repo's own history —
150		because `bin/peer-serve.js`/`bin/peer-migrate.js`/`index.js` all `import
151		Hyperswarm from 'hyperswarm'` at runtime, and a `--production` install
152		closure drops devDependencies. The exact same class of bug bit
153		`atproto-bridge` one commit-cycle later (`protomux-rpc` +
154		`compact-encoding` used by `peer-front.js` but undeclared — published
155		`0.8.2`) and is called out explicitly in the wj62 postmortem
156		(`POSTMORTEM-2026-07-03-wj62-migration.md:49-50,96-98`) as the root cause
157		of that outage: **test-green (hoisted node_modules) does not prove
158		deploy-ready (`--production` closure)** — boot-test the actual built
159		closure before cutting a live account over to it.
160
161		## Serving the membrane — two load-bearing fixes (SHIPPED 2026-07-06)
162
163		- ★ Symmetric swarm join. `peer.serve` + `serveMembrane` join the serverKey topic AND
164		`discoveryTopic(did)` with `{server:true, client:true}` — NOT server-only. `client:true` is
165		load-bearing for HOLEPUNCH, not just discovery: both the self-custody peer and the bridge host are
166		typically NAT'd, and a `client:false` member never punches outbound, so its NAT mapping never opens
167		and the connection never completes. Proven on two firewalled boxes: symmetric connects in ~2s;
168		asymmetric (peer server-only / bridge client-only) never opens (`cAtt>0` but `cOpen:0`). Mirrors
169		hyperident-gateway's proven mesh. Repro: `scripts/gw-mirror.mjs`; guard: `test/membrane-holepunch-symmetry`.
170		- ★ A fresh account MUST author an initial signed commit or it is unfrontable. The single-tenant
171		path (`index.js:112`) always did this; the SWARM path (`device-node.js bootSwarmDevice`) did NOT —
172		its empty-repo branch handled adopt + restore but not a brand-new empty account, so every fresh
173		multi-device DID came up empty. A bridge refuses to front an unverifiable/empty repo (`PeerFront`
174		verifies the signed commit vs the DID key), and the account can't post its way out (the composer
175		posts THROUGH the bridge). Fix: `bootSwarmDevice` authors + signs one initial commit when the repo
176		is empty and there's nothing to adopt (writable creator only; peers get it via `__commit__`
177		replication). Guard: `test/fresh-account-initial-commit`. See the bridge post-mortem:
178		`../atproto-bridge/docs/POSTMORTEM-2026-07-06-multi-did-membrane.md`.
179
180		## Invariants / gotchas
181
182		- `PEER_CAN_ADOPT` invariant: adopting an existing DID is only valid
183		custody transfer if the supplied `adoptDevicePrivHex` actually derives to
184		the DID-doc's `verificationMethod` did:key. This was proven for wj62:
185		its `devicePrivHex` derives to `did:key:zQ3shtugtPDvzXzfyMCytcEjdvwEvtZ8g4s8vfuPXQa3mhrgX`
186		(`POSTMORTEM-2026-07-03-wj62-migration.md:45-48`). `Peer.adopt` itself
187		does not verify this on its own (`lib/peer.js:123-145` trusts the
188		caller-supplied `did`/`privateKey`/`didKey` triple) — the check happens
189		downstream, e.g. `test/peer-adopt.test.js:31-32` decodes `peer.didKey`
190		and calls `verifyCommit` against the served repo, and a bridge's
191		`PeerFront.discover` independently verifies the repo signs against the
192		DID key before fronting (`atproto-bridge/pds/lib/peer-front.js:72`) —
193		refusing to front on mismatch. Never adopt with an unverified
194		key/DID pairing.
195		- Corestore directory-wipe: `Peer.open` nests the Corestore under
196		`<storage>/store/` rather than `<storage>/` directly, because Corestore
197		clears the directory it's given on init — the identity vault
198		(`peer-identity.json`) must live in the parent where the wipe can't reach
199		it (`lib/peer.js:183-186`, explicit `GOTCHA` comment).
200		- `close()` fd-lock: always close `seq` → `store` → `corestore` in that
201		order before a process exits or before calling `Peer.load` again on the
202		same storage dir, or the reload fails with "File descriptor could not be
203		locked" (`lib/peer.js:328-334`).
204		- `init`/`adopt` never clobber: both throw if a vault already exists at
205		the target path (`lib/peer.js:82`, `lib/peer.js:128`) — custody is never
206		silently overwritten. `index.js` additionally short-circuits to `load`
207		whenever a vault is present (`index.js:26-28`), so `adopt`/`init` config
208		keys are dead weight once a peer has onboarded once.
209		- Root vs device key separation (`Peer.init` only): rotation authority
210		= hyperident root recovery seed, which is never written to the vault
211		(`test/peer-init.test.js:36-39` asserts `t.absent(vault.rootMnemonic \|\|
212		vault.rootPrivateKey, ...)`); signing authority = a per-device key that
213		IS persisted. `Peer.adopt` is simpler/flatter — no root/device split,
214		just one bring-your-own signing key (matches migrating a legacy
215		single-key custodial account).
216		- Firehose carries the signature, not just the fact of a write:
217		`_authorAndSequence` (`lib/peer.js:201-205`) puts the **signed commit
218		block itself** into the firehose frame. This is what lets a
219		keyless replica (`test/peer-serves.test.js:71-82`) extract and verify a
220		signature it could never have produced — closing the "peer signs on
221		write, but a replica reading only content couldn't otherwise get a
222		signature" gap that an earlier iteration of this design left open (see
223		git log `0acd7ea "surface signed-commit gap"` → `649b0e9 "keyless
224		signed-commit serving"`).
225		- Secure-default write authorization: `serve()` only registers
226		`apply-write` at all when `canWrite` is passed; with no `canWrite`, no
227		writes are ever accepted over RPC (`lib/peer.js:313-320`,
228		`test/peer-write-rpc.test.js:65-84` proves a `canWrite: () => false`
229		peer rejects every write and applies nothing).
230		- **`bin/peer-serve.js` mentions `PeerFront.discover(swarm, store, did,
231		{verifyKey})`** (`bin/peer-serve.js:9`) as the bridge-side counterpart —
232		that class lives in `atproto-bridge/pds/lib/peer-front.js`, not in this
233		repo; don't go looking for it here.
234		- Two runnable bins besides `index.js`: `bin/peer-serve.js` (plain
235		env-var driven, `Peer.init`/`Peer.load` only — no adopt) and
236		`bin/peer-migrate.js` (env-var driven `Peer.adopt` + `ADOPT_CAR` import,
237		for one-shot manual migration runs outside the pgh config path). `index.js`
238		is the actual pgh deploy entry and supersedes both for production use.
239
240		## Tests (`test/*.test.js`, brittle, 8 files → 8 pass / 49 asserts, offline)
241
242		Run with `npm test` (`brittle-node test/*.test.js`, `package.json:9`) — all
243		8 suites pass fully offline; the discovery/serve/write-rpc suites spin up
244		an in-process `@hyperswarm/testnet` (no real network needed). Verified
245		by direct run 2026-07-04.
246
247		- `peer-init.test.js` — fresh `Peer.init`: root-signs-genesis /
248		device-signs-commits separation, vault persists device key + genesis but
249		not the root seed, reload yields identical DID/key, `init` refuses to
250		clobber an existing vault.
251		- `peer-adopt.test.js` — `Peer.adopt` + `importRecords`: bring-your-own DID
252		serves a signed repo verifying against the adopted key, imported record
253		present, reload durable.
254		- `peer-authors.test.js` — core Model B custody proof: `fromMnemonic` ->
255		peer holds its own 32-byte key, mints a `did:plc`, signs a commit that
256		verifies against its own `did:key`, `getRepo()` CAR verifies too.
257		- `peer-discovery.test.js` — Option 2: a resolver that knows **only the
258		DID** derives `discoveryTopic(did)`, connects, and gets coords back —
259		proves DID-alone discoverability with no pre-shared keys/DID-doc entry.
260		- `peer-serves.test.js` — full P2P backhaul: a consumer replicates the
261		Corestore, reads records byte-exact, and extracts+verifies the **signed
262		commit from the replicated firehose** with zero key access (the keyless
263		proof); also proves live-append propagation.
264		- `peer-write-rpc.test.js` — `apply-write` over RPC: authorized caller's
265		write gets signed and applied; unauthorized caller is refused and
266		nothing is applied; `create-session`/`refresh-session` round-trip
267		against the peer's own `jwtSecret`, wrong password rejected.
268
269		## Dependencies + ecosystem relationship
270
271		Runtime deps (`package.json:27-36`): `atproto-store`, `atproto-identity`,
272		`atproto-repo` as `file:../` siblings (pear-git-v3 monorepo-adjacent
273		layout), plus `b4a`, `protomux-rpc`, `compact-encoding`,
274		`hypercore-crypto`, `hyperswarm`. Dev-only: `brittle`,
275		`@hyperswarm/testnet`.
276
277		Ecosystem position (5-repo atproto-peer stack, see
278		`project_atproto-peer-bridge-split-2026-07-03` memory):
279
280		- `atproto-identity` — supplies `createAtprotoIdentity(FromKey)`,
281		`buildAttestedIdentity`/`verifyAttestedGenesis` (root/device attestation),
282		`discoveryTopic(did)`, JWT + password helpers (`lib/auth.js`) used
283		directly by `Peer`.
284		- `atproto-store` — supplies `openCorestore`, `RepoStore`,
285		`RepoManager` (applies + signs writes, tracks rev), `FirehoseSequencer`
286		(signed-commit frames), `getRepo`/`getRepoFromCommit` (CAR assembly) —
287		the actual storage substrate `Peer` sits on top of.
288		- `atproto-repo` — pure CAR/commit codec: `readCar`, `listRecords`,
289		`loadCommit`, `verifyCommit` — used both by this repo (`importRecords`
290		path) and by consumers verifying this peer's output.
291		- `atproto-bridge` — the stateless membrane. Its `PeerRouter`/
292		`PeerFront` (`pds/lib/peer-router.js`, `pds/lib/peer-front.js`) discover
293		this peer over `discoveryTopic`, replicate its Corestore, verify the repo
294		against the DID key, and relay `com.atproto.*` HTTP calls to this peer's
295		`get-coords`/`apply-write`/`create-session` RPCs. The bridge never
296		holds this peer's key — that is the entire point of the split. As of
297		the 2026-07-03 postmortem the bridge is still hybrid (a local `Pds`
298		opens unconditionally in `pds-serve.js`); making it a pure stateless
299		membrane (`pds=null` path) is tracked as unfinished work in
300		`atproto-bridge`, not in this repo.
	12	v0.3.105, runtime Bare, pgh app `hyper-proto-peer`, swarm-only (no
	13	HTTP; a UDS control socket + per-DID swarm RPC). Runs on tabertron and
	14	stortron.
	15
	16	## ⚠ Which boot path is real
	17
	18	`index.js` has three, and the fleet runs the third:
	19
	20	\| condition \| path \| status \|
	21	\|---\|---\|---\|
	22	\| `cfg.swarm` false \| `Peer.load` / `Peer.adopt` / `Peer.init` (`lib/peer.js`) \| legacy single-tenant — still in-tree, still tested, NOT deployed \|
	23	\| `cfg.swarm` + no `cfg.dids` \| `bootSwarmDevice` — one DID, multi-device \| single-DID swarm \|
	24	\| `cfg.swarm` + `cfg.dids[]` \| `bootMultiDevice` (`lib/device-node.js`) \| ★ what prod runs \|
	25
	26	Multi-DID = ONE process serving N self-custody DIDs over ONE shared
	27	Corestore / Hyperswarm / ProtomuxWakeup (each DID a namespace, one storage
	28	lock). A user's own DID and the communities they co-own are entries in
	29	`cfg.dids`.
	30
	31	⚠ `index.js`'s own header comment says `node index.js` and describes only the
	32	`Peer` path — both stale; the manifest is `runtime: bare` and imports `bare-fs`.
	33
	34	## Architecture of the deployed path
	35
	36	```
	37	bootMultiDevice(storage, dids[])
	38	├─ ONE Corestore + Hyperswarm + ProtomuxWakeup, shared
	39	├─ AccountRegistry (Hyperbee 'hpp-dids') — durable roster of hosted DIDs
	40	└─ per DID: bootSwarmDevice({...entry, shared, resyncStore})
	41	├─ StateBaseV3 — the shared multi-writer INTENT LOG (atproto-room)
	42	├─ RepoBase — repo ops layered on it
	43	├─ Lease — single-signer lease
	44	├─ PointerSubstrate — pointers in the autobase, BYTES in the overlay
	45	├─ signer/materialize — lease-holder turns the linearized view into
	46	│ ONE canonical signed commit chain, continuing
	47	│ from the last signed head
	48	├─ serveDevice(...) — swarm RPC, canWrite-gated (below)
	49	└─ serveMembrane(...) — the keyless read surface the bridge fronts
	50	swarm.join(node.sb.base.discoveryKey) per DID
	51	```
	52
	53	Every attested device writes repo intents into the shared autobase; **whichever
	54	device currently holds the lease materializes** them into the signed chain. The
	55	firehose lives in the SAME StateBaseV3 — each `sign()` writes the signed commit
	56	into the shared log, which is why a keyless replica can extract a signature it
	57	could never produce.
	58
	59	## ★ Roles: creator, signer, indexer — three different things
	60
	61	The single most confusing area, and where the live fleet's asymmetry comes from.
	62
	63	- Creator is elected `lowest-keyed member == self` by default — so **growing
	64	the roster can silently demote the incumbent**, which is the loser-club fork
	65	(`lib/coowner.js:91-99`, `test/coowner-join-role.test.js`). A room's creator is a
	66	property of the room, not of key ordering. Hence the explicit flags.
	67	- `forceCreator: true` — pin the incumbent (set automatically by `coowner-invite`
	68	on a non-joiner).
	69	- `joiner: true` — set by `coowner-join --bundle`. Makes the node `forceNonCreator`
	70	AND converge-only at boot: skip every local write
	71	(migratePointers/restore/adopt/fresh-account) until replication converges, then
	72	the forward-only `sign()` picks up at the tip. Without it a joiner whose key
	73	sorts below the incumbent's opens the roomSeed-derived genesis writer
	74	empty-and-writable in its fresh store and authors a rival genesis.
	75	- `resyncStore` also forces non-creator: `forceNonCreator: !!resyncStore \|\| !!joiner`
	76	(`lib/device-node.js:178`).
	77	- ⚠ A joiner is admitted as a PLAIN WRITER, never an indexer. Two indexers is
	78	the N=2 quorum trap: the majority becomes 2, one loss freezes `indexedLength`,
	79	the room forks. One indexer + plain writers is the shape a 2-peer room wants.
	80
	81	Read the live truth, never the config:
	82	```bash
	83	hyper-proto-peer diag # per-DID: writable / signs / amCreator / forcedRole /
	84	# indexers / indexed / writers / conflicts / roster /
	85	# pointerChecked / stale / missing
	86	```
	87	`forcedRole` is `forced-creator` \| `forced-non-creator` \| null. ⚠ Use `sb._isCreator`,
	88	not `sb._amCreator` — the latter is a prototype METHOD, so `!!sb._amCreator` was
	89	`!!<function>` and reported `true` for every node (fixed atproto-room 0.2.16).
	90
	91	## Co-ownership ceremony (`lib/coowner.js`)
	92
	93	Four things must line up on two machines: the joiner has its own device keypair, an
	94	existing co-owner has attested it to the DID's root, both configs list both
	95	device keys, and the joiner holds the shared `#atproto` signing key.
	96
	97	```
	98	joiner$ hyper-proto-peer coowner-join <did> → prints its device PUBLIC key
	99	existing$ hyper-proto-peer coowner-invite <did> --device <hex> → prints a bundle, adds the key locally
	100	joiner$ hyper-proto-peer coowner-join <did> --bundle <file> → writes its dids entry
	101	(restart both daemons)
	102	```
	103
	104	- NO ROOT MNEMONIC IS INVOLVED. The invite is signed by the existing co-owner's
	105	own device key via keet's chained attestation (`attestDeviceViaDevice`):
	106	root → existing → joiner. The root mnemonic is never stored on a node; only its
	107	PUBLIC key (`rootIdentityKey`) is needed.
	108	- ⚠ The bundle carries the SHARED SIGNING KEY — inherent, since a co-owner
	109	materializes signed commits. Treat the file as a secret: private path, trusted
	110	channel, delete after joining.
	111	- Config is never written without a `.bak.<ts>` beside it — a mistyped roster locks
	112	a machine out of a room.
	113	- `coowner-join` is idempotent: an existing mnemonic is REUSED, never regenerated
	114	(regenerating orphans an already-attested key → permanently unadmitted).
	115
	116	## Recovery tools (subcommands on `index.js`)
	117
	118	\| command \| what it does \|
	119	\|---\|---\|
	120	\| `diag [--remote]` \| the live per-DID truth (above) \|
	121	\| `cores [<dkey-prefix>] [--compare] [--all] [--at N]` \| map a `[hypercore] conflict detected in <dkey>` log line back to a core: which store, public key, corestore alias, and whether THIS machine can sign it. `--compare` says PREFIX (ordinary lag) vs FORK and bisects the first diverging length. Read-only, `{readOnly:true}` → safe against a running daemon, no lock \|
	122	\| `resync-did <did>` \| flag a DID to re-join in a fresh per-DID corestore on next boot (`statebase-resync/<did16>-r<gen>`) → clean local writer, forced non-creator \|
	123	\| `rebuild-did <did>` \| rebuild as genesis creator/notary from an exported CAR (`forceCreator` + `adoptCar`) — belongs in FRESH storage, never alongside the old cores \|
	124	\| `repair-pointers` \| repair converged pointers whose bytes exist nowhere \|
	125	\| `reindex-overlay` \| rebuild the overlay index (UDS-only — it hashes every record) \|
	126	\| `notifications` \| notification read \|
	127
	128	⚠ A fresh NAMESPACE does not heal a poisoned writer — autobase's local writer key
	129	is primaryKey-derived, so `#r<gen>` re-derives the identical key. Only a fresh
	130	CORESTORE sheds it.
	131
	132	⚠ `index.js` dispatches on `sub === '<name>'` and falls through to the daemon when
	133	nothing matches — so running a subcommand that is not in the installed tree tries to
	134	open the corestore the daemon holds: `File descriptor could not be locked`. That is the
	135	signature of "committed but never published".
	136
	137	## Write authorization — `canWrite` / `PEER_WRITERS`
	138
	139	`lib/serve-device.js`. `canWrite` is an allowlist of client Noise pubkeys
	140	(`config.writers`). It is fail-closed by non-registration: the whole privileged
	141	block is inside `if (canWrite) { … }`, so with `writers` absent/empty the handlers
	142	`pointer-diag`, `get-service-auth`, `apply-write`, `delegate`, `list-devices`,
	143	`revoke-device`, `is-device-revoked`, `upload-blob` are never registered at all.
	144
	145	`get-service-auth` needs two gates, neither sufficient alone: `canWrite(remote)`
	146	(may this PEER ask?) and `sessions.verifyAccess(token)` (is this a valid in-scope
	147	access JWT for THIS did?). Nothing answered it for a long time, which is exactly why
	148	`app.bsky.notification.listNotifications` silently fell back to the public proxy and
	149	never worked.
	150
	151	⚠ The allowlist is only meaningful against a stable key — hence
	152	`lib/swarm-keypair.js` on both sides. Each bridge/machine has its own swarm
	153	pubkey, so a bridge that moves hosts is a different identity and must be re-listed.
	154	A missing entry looks identical to "not allowlisted yet".
	155
	156	The UDS control surface (`serve-device-uds.js`) deliberately has no `canWrite` —
	157	local UDS is trusted; the swarm surface keeps the gate.
	158
	159	## Storage layout (`~/.config/hyper-proto-peer/data/`)
	160
	161	```
	162	store/ shared Corestore (all DIDs, one lock)
	163	statebase/ StateBaseV3
	164	statebase-swarm/
	165	statebase-resync/ index.json = { "<did>": <gen> } ← flags forced non-creator
	166	<did16>-r<gen>/ a fresh per-DID corestore (clean local writer)
	167	atproto-repo/ ★ the SHARED content-addressed record bytes (OverlayRepoStore)
	168	favela-web reads THIS via ATPROTO_OVERLAY_ROOT
	169	membrane/ membrane-repo Hyperbee (skipped when projectRepoCore:false)
	170	dids/<did16>/…
	171	control.sock UDS control (the PRIMARY DID's is what pgh-sidecar registers)
	172	peer-identity.json legacy Peer vault
	173	swarm-keypair.json persistent transport identity
	174	diag-keypair.json
	175	```
	176
	177	★ `atproto-repo/` being shared is load-bearing: co-located hpp + favela-web use the
	178	exact same files (`roomKey16 = hash('atproto-base/v1/room-key' ‖ did)`, byte-exact
	179	DAG-CBOR), so a co-hosted DID gets an authoritative baseline instead of rebuilding
	180	from firehose backfill.
	181
	182	## Config (`~/.config/hyper-proto-peer/config.json`, `PEER_*` env overlays)
	183
	184	Top level: `swarm`, `storage`, `dids[]`. Per-DID entry:
	185
	186	\| key \| meaning \|
	187	\|---\|---\|
	188	\| `did`, `handle`, `pdsUrl` \| identity \|
	189	\| `signingKey` \| ⚠ SECRET — the shared `#atproto` key \|
	190	\| `deviceMnemonic` \| ⚠ SECRET — this device's own identity \|
	191	\| `deviceProof`, `rootIdentityKey` \| the attestation chain root → … → this device \|
	192	\| `members` \| device roster — hex array or a comma string (`normalizeMembers` handles both) \|
	193	\| `writers` \| the `canWrite` Noise allowlist; absent ⇒ no privileged handlers \|
	194	\| `forceCreator` / `joiner` \| the role pins (above) \|
	195	\| `indexer` \| deliberately NOT set on a joiner \|
	196	\| `storeBackend` \| `"overlay"` in prod \|
	197	\| `primary` \| which DID's control.sock the sidecar registers \|
	198	\| `appPasswordHash`, `jwtSecret` \| the peer is its own auth authority (bsky login) \|
	199	\| `records` \| declarative records to ensure (e.g. the feed generator record) \|
	200	\| `adoptCar` \| continue-migrate an existing repo (with `forceCreator`) \|
	201	\| `projectRepoCore` \| `false` = the stateless virtual device — no membrane-repo Hyperbee; consumers derive via `get-repo-car` \|
	202
	203	Secrets stay out of git — never in the committed manifest.
	204
	205	## Legacy: the `Peer` class (`lib/peer.js`) — still valid, not deployed
	206
	207	Single-tenant, vault at `<storage>/peer-identity.json`.
	208	`Peer.fromMnemonic` / `fromPrivateKey` / `init` (fresh onboarding, root-vs-device
	209	split, root seed never persisted) / `adopt` (bring-your-own published DID, no PLC
	210	op) / `load` (normal restart) / `setAppPassword`. `init` and `adopt` **throw rather
	211	than clobber** an existing vault. `importRecords` re-authors under this key while
	212	record bytes stay canonical DAG-CBOR, so CIDs are preserved and only the commit is
	213	fresh at a higher rev.
	214
	215	⚠ `close()` order is `seq → store → corestore` or the next open fails
	216	`File descriptor could not be locked`.
	217
	218	`bin/peer-serve.js` and `bin/peer-migrate.js` drive this path env-only; both are
	219	superseded by `index.js` for production.
	220
	221	## Gotchas
	222
	223	- `bare-globals` must be imported FIRST — cborg touches `TextEncoder` at module load.
	224	- Reads of git/subprocess in a bare daemon must be async spawn, never `spawnSync`
	225	(sync blocks the always-on event loop).
	226	- `pgh app update` ≠ restart; `INSTALLED ≠ RUNNING` — check `ps -eo etime`.
	227	- OTA: the sidecar installs, then pushes `release-ready`; `setOtaHandler` closes the
	228	daemon and exits 75 so the supervisor respawns.
	229	- The sidecar registration uses the primary DID's `control.sock`; without it the
	230	sidecar has no udsPath and autoupdate installs a tree the daemon never adopts.
	231
	232	## Related
	233
	234	- [[atproto-did-peer]] — the consumer side; must speak `hyper-proto-peer/rpc/v1`
	235	- [[atproto-bridge]] — the keyless membrane that fronts this
	236	- [[favela-web]] — shares this daemon's `atproto-repo/` overlay on the same box
	237	- [[atproto-store]] — `PointerSubstrate`, `OverlayRepoStore`, `FirehoseSequencer`
	238	- [[autobase]] / [[reference_autobase-quorum-2f1]] — the N=2 quorum trap
	239	- memory: [[project_wj62-write-path-saga-2026-07-24]],
	240	[[project_atproto-storage-consensus-redesign-2026-07-19]],
	241	[[project_retention-clobber-wj62-2026-07-31]],
	242	[[reference_statebase-indexing-vs-did-signing]]

hypercore-kb commits / aa49353

Changed files (6) +1208 -473