a63842b · wirecore · pear-git-v3

Changed files (4) +1914 -117

pear-app/CODE_CHANGES.md +530 -0

…	…	@@ -0,0 +1,530 @@
	1	# Wirecore Pear App - Code Changes Overview
	2
	3	Date: 2026-01-21
	4	Purpose: High-level explanation of how the code will transform from current to refactored architecture
	5
	6	---
	7
	8	## Current Architecture (How It Works Now)
	9
	10	### Identity System
	11	- Uses IdentityKey from `keet-identity-key` package
	12	- Generates a seed (32 bytes), stores it in a Hypercore named 'identity'
	13	- Derives ed25519 keypair from seed
	14	- Public key is shown in hex format
	15	- Separate from feed key - two different keypairs per user
	16
	17	### Message Storage
	18	- Posts are full JSON objects: `{ type, text, timestamp, author, username }`
	19	- Entire message stored in Hypercore blocks (200-500 bytes per post)
	20	- Images stored in Hyperdrive, but reference is in the JSON
	21	- Feed replication downloads all message content immediately
	22
	23	### Content Flow
	24	1. User posts: "Hello world"
	25	2. Create JSON: `{ type: 'post', text: 'Hello world', timestamp: 1234, author: 'abc123...', username: 'fleeky' }`
	26	3. Convert JSON to Buffer
	27	4. Append entire JSON to Hypercore (300+ bytes)
	28	5. Peers replicate feed, get full messages
	29
	30	### Network Protocol
	31	- Hyperswarm replicates Hypercore feeds automatically
	32	- No custom content protocol
	33	- Everything flows through Hypercore's built-in replication
	34
	35	### Storage Structure
	36	```
	37	wirecore-storage/cores/
	38	├── identity/ # IdentityKey seed (32 bytes)
	39	├── my-feed/ # Full JSON messages (200-500 bytes each)
	40	├── metadata/ # Hyperbee: username, follows list
	41	└── user-files/ # Hyperdrive: uploaded images
	42	```
	43
	44	---
	45
	46	## Target Architecture (How It Will Work After Refactor)
	47
	48	### Identity System
	49	- Uses ANProto keypair only (remove IdentityKey completely)
	50	- Generate ed25519 keypair directly: `ANProto.gen()`
	51	- Returns 132-character base64 string: public key (44 chars) + secret key (88 chars)
	52	- Store keypair as string in Hyperbee metadata: `anproto-keypair`
	53	- Same keypair used for both ANProto signatures AND Hypercore feed
	54	- Public key shown in base64 format (ANProto standard)
	55
	56	### Message Storage
	57	- Posts are YAML format with metadata:
	58	```yaml
	59	name: fleeky
	60	image: pZGm1Av0IEBKARczz7exkNYsZb8LzaMrV7J32a2fFG4=
	61	previous: aBcd2Ef3ghIjKlMnOpQrStUvWxYz123456789ABCDEF=
	62	---
	63	Hello world
	64	```
	65	- Only ANProto signature stored in Hypercore blocks (132 bytes)
	66	- Content stored separately in Hyperbee by hash
	67	- Feed replication downloads only signatures, content fetched on-demand
	68
	69	### Content Flow
	70	1. User posts: "Hello world"
	71	2. Create YAML with metadata (name, previous post hash, etc.)
	72	3. Hash the YAML: `postHash = ANProto.hashData(yaml)`
	73	4. Store content in Hyperbee: `contentCache.put(postHash, yaml)`
	74	5. Sign the hash: `signature = ANProto.signHash(postHash, keypair)`
	75	6. Append only signature to Hypercore (132 bytes)
	76	7. Peers replicate feed, get only signatures
	77	8. When viewing post, peer requests content by hash from network
	78
	79	### Network Protocol
	80	- Two channels per connection:
	81	1. Hypercore feed replication (automatic, built-in)
	82	2. Content gossip protocol (custom Protomux channel)
	83
	84	- Content Gossip Messages:
	85	- Request: `{ type: 'content-request', id: 'req-123', hash: 'pZGm1...' }`
	86	- Response: `{ type: 'content-response', id: 'req-123', hash: 'pZGm1...', found: true, content: Buffer(...) }`
	87
	88	- Each node acts as content server:
	89	- Receives content request
	90	- Checks local Hyperbee cache
	91	- If found, sends content to requester
	92	- If not found, ignores request
	93
	94	### Storage Structure
	95	```
	96	wirecore-storage/cores/
	97	├── my-feed/ # ANProto signatures ONLY (132 bytes each)
	98	├── content-cache/ # Hyperbee: hash → content (YAML + images)
	99	├── metadata/ # Hyperbee: anproto-keypair, username, follows
	100	└── (no identity/) # REMOVED - no longer needed
	101	└── (no user-files/) # REMOVED - images now in content-cache
	102	```
	103
	104	---
	105
	106	## Key Transformations
	107
	108	### 1. Initialization (cmdInit)
	109
	110	BEFORE:
	111	1. Generate random seed (32 bytes)
	112	2. Store seed in 'identity' Hypercore
	113	3. Create IdentityKey from seed
	114	4. Create personal feed with auto-generated keypair
	115	5. Store username and follows in metadata
	116
	117	Result: Two separate identities (IdentityKey + Hypercore keypair)
	118
	119	AFTER:
	120	1. Generate ANProto keypair: `ANProto.gen()`
	121	2. Store keypair string in Hyperbee metadata
	122	3. Create personal feed using ANProto keypair
	123	4. Feed key = ANProto public key (unified!)
	124	5. Store username and follows in metadata
	125
	126	Result: Single unified identity (ANProto keypair = feed key)
	127
	128	---
	129
	130	### 2. Posting Messages (cmdPost)
	131
	132	BEFORE:
	133	1. Get username from metadata
	134	2. Create JSON object: `{ type: 'post', text, timestamp, author, username }`
	135	3. Convert to Buffer
	136	4. Append full JSON to Hypercore (300+ bytes)
	137	5. Done
	138
	139	AFTER:
	140	1. Get username from metadata
	141	2. Get latest post hash (for 'previous' field)
	142	3. Create YAML: `YAMLHelper.create({ name, previous }, text)`
	143	4. Hash YAML: `postHash = ANProto.hashData(yaml)`
	144	5. Store in Hyperbee: `contentCache.put(postHash, yaml)`
	145	6. Sign hash: `signature = ANProto.signHash(postHash, anprotoKeypair)`
	146	7. Append only signature to Hypercore (132 bytes)
	147	8. Broadcast content to connected peers via gossip protocol
	148
	149	Efficiency gain: 132 bytes vs 300+ bytes per post in feed
	150
	151	---
	152
	153	### 3. Reading Timeline (cmdTimeline)
	154
	155	BEFORE:
	156	1. Connect to swarm
	157	2. Wait for peer connections
	158	3. Update followed feeds (download blocks)
	159	4. For each feed, read blocks:
	160	- `block = await feed.get(i)`
	161	- `entry = JSON.parse(block.toString())`
	162	- Message is complete, display immediately
	163	5. Sort all messages by timestamp
	164	6. Display timeline
	165
	166	AFTER:
	167	1. Connect to swarm (feed replication + content gossip)
	168	2. Wait for peer connections
	169	3. Update followed feeds (download signatures only)
	170	4. For each feed, read signature blocks:
	171	- `signature = (await feed.get(i)).toString()`
	172	- `{ hash, timestamp } = ANProto.open(signature)`
	173	- Try local cache: `contentCache.get(hash)`
	174	- If not cached, request from peers via gossip
	175	- Parse YAML: `{ metadata, text } = YAMLHelper.parse(content)`
	176	5. Sort all messages by timestamp
	177	6. Display timeline
	178
	179	Benefit: Faster initial sync (only signatures), content fetched as needed
	180
	181	---
	182
	183	### 4. Following Users (cmdFollow)
	184
	185	BEFORE:
	186	1. Take user's feed public key (hex)
	187	2. Compute discovery key
	188	3. Create read-only feed from public key
	189	4. Store in follows map
	190	5. Save follows list to metadata
	191	6. Join swarm topic
	192
	193	AFTER:
	194	1. Take user's ANProto public key (base64, 44 chars)
	195	2. Convert to Buffer
	196	3. Compute discovery key
	197	4. Create read-only feed from ANProto public key
	198	5. Store in follows map
	199	6. Save follows list to metadata
	200	7. Join swarm topic (feed replication + content gossip)
	201
	202	Key difference: Public key format changes from hex to base64
	203
	204	---
	205
	206	### 5. Image Upload (New Feature)
	207
	208	BEFORE:
	209	- Images stored in Hyperdrive ('user-files')
	210	- Reference stored in message JSON
	211	- Large images bloat Hypercore blocks
	212
	213	AFTER:
	214	1. User attaches image to post
	215	2. Hash image data: `imageHash = ANProto.hashData(imageData)`
	216	3. Store image in Hyperbee: `contentCache.put(imageHash, imageData)`
	217	4. Create YAML with image reference:
	218	```yaml
	219	name: fleeky
	220	image: pZGm1Av0IEBKARczz7exkNYsZb8LzaMrV7J32a2fFG4=
	221	---
	222	Check out this photo!
	223	```
	224	5. Hash YAML, sign, append signature to feed
	225	6. When viewing post:
	226	- Fetch YAML by hash
	227	- Parse metadata.image
	228	- Separately fetch image by hash from peers
	229	- Display post with image
	230
	231	Benefit: Images fetched lazily, not replicated unless viewed
	232
	233	---
	234
	235	### 6. Content Distribution (New P2P Protocol)
	236
	237	BEFORE:
	238	- All content flows through Hypercore replication
	239	- No peer-to-peer content requests
	240	- Relay could serve content via APDS WebSocket
	241
	242	AFTER:
	243	- Hypercore replication: Feed signatures only
	244	- Protomux content gossip: Peer-to-peer content requests
	245	- Each node acts as server:
	246	```
	247	Alice posts → Bob views (fetches from Alice)
	248	→ Charlie views (fetches from Bob!)
	249	→ Network scales organically
	250	```
	251
	252	Protocol Flow:
	253	1. Node needs content with hash `pZGm1...`
	254	2. Send `content-request` to all connected peers via Protomux
	255	3. Peers check local Hyperbee cache
	256	4. If peer has it, sends `content-response` with content
	257	5. Requester receives content, caches in Hyperbee
	258	6. Requester can now serve this content to others
	259
	260	Fallback: If no peer has content, request from relay via WebSocket
	261
	262	---
	263
	264	### 7. Web Interface (No Code Changes to Protocol)
	265
	266	BEFORE:
	267	- Server reads Hypercore blocks
	268	- Parses JSON messages
	269	- Sends to browser via WebSocket
	270
	271	AFTER:
	272	- Server reads Hypercore blocks (signatures)
	273	- Verifies ANProto signatures
	274	- Fetches content from Hyperbee cache or peers
	275	- Parses YAML messages
	276	- Sends to browser via WebSocket
	277
	278	Browser sees same data format, just backend changes
	279
	280	---
	281
	282	## New Modules To Create
	283
	284	### `/pear-app/lib/anproto.js`
	285	- Copied from `lib/core/anproto.js`
	286	- No changes needed, already complete
	287	- Provides: `gen()`, `loadKeypair()`, `hash()`, `sign()`, `open()`
	288
	289	### `/pear-app/lib/yaml-helper.js`
	290	- Copied from `lib/core/yaml-helper.js`
	291	- No changes needed, already complete
	292	- Provides: `create(metadata, text)`, `parse(yaml)`
	293
	294	### `/pear-app/lib/content-cache.js` (NEW)
	295	- Purpose: Wrapper around Hyperbee for content-by-hash storage
	296	- Methods:
	297	- `put(hash, content)` - Store content by hash
	298	- `get(hash)` - Retrieve content by hash
	299	- `has(hash)` - Check if content exists
	300
	301	### `/pear-app/lib/content-gossip.js` (NEW)
	302	- Purpose: Protomux channel for peer-to-peer content requests
	303	- Methods:
	304	- `setupChannel(connection)` - Create gossip channel on connection
	305	- `requestContent(hash)` - Request content from all peers
	306	- `handleRequest(msg)` - Serve content to requesting peer
	307
	308	### `/pear-app/lib/content-sync.js` (NEW)
	309	- Purpose: Multi-transport content fetching
	310	- Methods:
	311	- `fetch(hash)` - Try cache → peers → relay
	312	- `fetchViaGossip(hash)` - Request from Hyperswarm peers
	313	- `fetchViaRelay(hash)` - Request from relay WebSocket
	314
	315	---
	316
	317	## Files To Modify
	318
	319	### `index.js` (Main App)
	320
	321	Functions to change:
	322	- `loadState()` - Load ANProto keypair instead of IdentityKey
	323	- `cmdInit()` - Generate ANProto keypair, use for feed
	324	- `cmdPost()` - Create YAML, hash, sign, store content
	325	- `cmdTimeline()` - Read signatures, fetch content by hash
	326	- `cmdFollow()` - Handle base64 public keys instead of hex
	327	- Web server - Parse signatures instead of JSON
	328
	329	New global variables:
	330	- `anprotoKeypair` - Loaded ANProto keypair
	331	- `contentCache` - Hyperbee for content storage
	332	- `contentGossip` - Content gossip protocol instance
	333
	334	Remove:
	335	- `identity` - IdentityKey no longer needed
	336	- `userDrive` - Images now in content-cache
	337
	338	### `package.json`
	339
	340	Add dependencies:
	341	- `sodium-universal` - For ANProto signatures
	342
	343	Remove dependencies:
	344	- `keet-identity-key` - No longer needed
	345
	346	Keep:
	347	- All other dependencies unchanged
	348
	349	---
	350
	351	## Migration Path
	352
	353	### For Users
	354
	355	Option 1: Fresh Start (Recommended)
	356	1. Rename old storage: `mv wirecore-storage wirecore-storage-old`
	357	2. Run new version: `pear run . init`
	358	3. Create new ANProto identity
	359	4. Re-follow users
	360	5. Start posting with new system
	361
	362	Option 2: Manual Export/Import
	363	1. Export old posts to JSON file
	364	2. Run new version, create new identity
	365	3. Import old posts (creates YAML, signs with new keypair)
	366	4. Old feed still accessible with old version
	367
	368	### For Developers
	369
	370	Phase 1: Add ANProto Layer
	371	1. Copy `anproto.js` and `yaml-helper.js` to `pear-app/lib/`
	372	2. Update `cmdInit()` to generate ANProto keypair
	373	3. Use ANProto keypair for Hypercore feed creation
	374	4. Store keypair in Hyperbee metadata
	375	5. Test: Create new identity, verify unified keypair
	376
	377	Phase 2: Add Content Cache
	378	1. Create `content-cache.js` wrapper for Hyperbee
	379	2. Initialize in `loadState()` and `cmdInit()`
	380	3. Test: Store and retrieve content by hash
	381
	382	Phase 3: Add YAML Format
	383	1. Update `cmdPost()` to create YAML instead of JSON
	384	2. Store YAML in content cache
	385	3. Test: Post message, verify YAML format
	386
	387	Phase 4: Switch to Signatures
	388	1. Update `cmdPost()` to append signature instead of JSON
	389	2. Update `cmdTimeline()` to read signatures
	390	3. Fetch content from cache for now
	391	4. Test: Post → Read timeline, verify signature → content flow
	392
	393	Phase 5: Add P2P Gossip
	394	1. Create `content-gossip.js` Protomux channel
	395	2. Create `content-sync.js` multi-transport fetcher
	396	3. Update `startSwarm()` to setup gossip channel
	397	4. Update `cmdTimeline()` to fetch missing content from peers
	398	5. Test: Two nodes, one posts, other fetches via gossip
	399
	400	Phase 6: Polish
	401	1. Remove IdentityKey code completely
	402	2. Update all display output (hex → base64)
	403	3. Add image upload support
	404	4. Test full workflow end-to-end
	405
	406	---
	407
	408	## Testing Strategy
	409
	410	### Unit Tests (Using Brittle)
	411
	412	Test: ANProto Integration
	413	- Generate keypair
	414	- Sign hash
	415	- Verify signature
	416	- Load keypair from string
	417
	418	Test: YAML Helper
	419	- Create YAML with metadata
	420	- Parse YAML back to metadata + content
	421	- Handle edge cases (no metadata, special characters)
	422
	423	Test: Content Cache
	424	- Store content by hash
	425	- Retrieve content by hash
	426	- Handle missing content
	427
	428	Test: Content Gossip
	429	- Setup Protomux channel
	430	- Send content request
	431	- Receive content response
	432	- Handle not found
	433
	434	### Integration Tests
	435
	436	Test: Two-Node P2P
	437	1. Node A: Init, post message
	438	2. Node B: Init, follow Node A
	439	3. Node B: View timeline
	440	4. Verify: Node B fetches content from Node A via gossip
	441
	442	Test: Three-Node Relay
	443	1. Node A: Post with image
	444	2. Node B: View, fetches from A
	445	3. Node C: View, fetches from B (not A!)
	446	4. Verify: Content propagates through network
	447
	448	Test: Relay Fallback
	449	1. Node A: Post message
	450	2. Node A: Goes offline
	451	3. Node B: Follow A, view timeline
	452	4. Verify: Content fetched from relay, not peers
	453
	454	### Manual Testing
	455
	456	Web Interface:
	457	- Start server
	458	- Connect via browser
	459	- Post messages
	460	- View timeline
	461	- Upload images
	462	- Follow users
	463
	464	Terminal:
	465	- Run all CLI commands
	466	- Verify output format
	467	- Check storage structure
	468	- Monitor network activity
	469
	470	---
	471
	472	## Rollout Plan
	473
	474	### Week 1: Phase 1-2 (Identity + Storage)
	475	- Implement ANProto identity
	476	- Implement content cache
	477	- Test initialization and keypair storage
	478
	479	### Week 2: Phase 3-4 (YAML + Signatures)
	480	- Implement YAML format
	481	- Switch to signature-only feeds
	482	- Test posting and reading
	483
	484	### Week 3: Phase 5 (P2P Gossip)
	485	- Implement Protomux content gossip
	486	- Implement multi-transport sync
	487	- Test peer-to-peer content distribution
	488
	489	### Week 4: Phase 6 (Polish + Testing)
	490	- Remove legacy IdentityKey
	491	- Add image upload
	492	- Full integration testing
	493	- Documentation updates
	494
	495	### Week 5: Alpha Release
	496	- Package as Pear app
	497	- Test on clean systems
	498	- User testing with fresh identities
	499	- Gather feedback
	500
	501	---
	502
	503	## Success Criteria
	504
	505	### Technical
	506	- ✅ Unified identity (ANProto key = feed key)
	507	- ✅ Feed blocks ≤ 132 bytes (signatures only)
	508	- ✅ Content served peer-to-peer (not just relay)
	509	- ✅ YAML format compatible with Wiredove
	510	- ✅ Multi-transport content fetching works
	511	- ✅ Images fetched lazily (on-demand)
	512
	513	### User Experience
	514	- ✅ Timeline loads within 10 seconds
	515	- ✅ Images appear without manual refresh
	516	- ✅ Works offline with cached content
	517	- ✅ Works when relay is down
	518	- ✅ Public key is copyable (for follows)
	519
	520	### Code Quality
	521	- ✅ All brittle tests pass
	522	- ✅ No regression in existing features
	523	- ✅ Clean separation of concerns
	524	- ✅ Documentation complete
	525
	526	---
	527
	528	Status: Planning complete, ready for implementation
	529	Next Step: Begin Phase 1 - Add ANProto layer
	530	Date: 2026-01-21

pear-app/FEATURE_PARITY.md +612 -0

…	…	@@ -0,0 +1,612 @@
	1	# Wirecore Pear App - Feature Parity Analysis
	2
	3	Date: 2026-01-21
	4	Purpose: Ensure all current features are maintained after refactor to ANProto architecture
	5
	6	---
	7
	8	## Current Features Inventory
	9
	10	### CLI Commands
	11
	12	\| Command \| Description \| Status After Refactor \|
	13	\|---------\|-------------\|----------------------\|
	14	\| `init` \| Initialize new identity \| ✅ Enhanced - ANProto keypair \|
	15	\| `post <text>` \| Post a text message \| ✅ Enhanced - YAML format, P2P gossip \|
	16	\| `setname <name>` \| Set your username \| ✅ Maintained - No changes \|
	17	\| `follow <pubkey> [name]` \| Follow a user by public key \| ✅ Updated - Base64 pubkey format \|
	18	\| `unfollow <pubkey>` \| Unfollow a user \| ✅ Updated - Base64 pubkey format \|
	19	\| `timeline [limit]` \| View merged timeline \| ✅ Enhanced - Content-by-hash fetching \|
	20	\| `whoami` \| Show your public key \| ✅ Updated - Shows ANProto key (base64) \|
	21	\| `follows` \| List followed users \| ✅ Maintained - No changes \|
	22	\| `directory [query]` \| Search user directory \| ✅ Maintained - No changes \|
	23	\| `profile <pubkey>` \| View user profile \| ✅ Updated - Base64 pubkey format \|
	24	\| `storage-path` \| Show storage location \| ✅ Maintained - No changes \|
	25	\| `migrate-storage` \| Migrate storage data \| ⚠️ Not needed - Fresh start approach \|
	26
	27	Total: 12 commands → 11 maintained/enhanced, 1 obsolete
	28
	29	### Web Interface Features
	30
	31	\| Feature \| Description \| Status After Refactor \|
	32	\|---------\|-------------\|----------------------\|
	33	\| Timeline view \| Display merged timeline of posts \| ✅ Enhanced - Content-by-hash fetching \|
	34	\| Compose post \| Write and submit new posts \| ✅ Enhanced - YAML format backend \|
	35	\| Image upload \| Attach images to posts \| ✅ Enhanced - Hash-based storage \|
	36	\| Image display \| View images in timeline \| ✅ Enhanced - Lazy loading \|
	37	\| Follow user \| Follow by public key \| ✅ Updated - Base64 format \|
	38	\| Unfollow user \| Remove from follows \| ✅ Maintained - No changes \|
	39	\| User search \| Search directory by name \| ✅ Maintained - Relay proxy unchanged \|
	40	\| Profile view \| View user profiles \| ✅ Updated - Base64 pubkey \|
	41	\| Set username \| Update your display name \| ✅ Maintained - No changes \|
	42	\| WebSocket updates \| Real-time feed updates \| ✅ Maintained - No changes \|
	43	\| Refresh timeline \| Manually refresh posts \| ✅ Enhanced - P2P sync improved \|
	44	\| Toast notifications \| Status messages \| ✅ Maintained - No changes \|
	45
	46	Total: 12 features → All maintained or enhanced
	47
	48	### HTTP API Endpoints
	49
	50	\| Endpoint \| Method \| Purpose \| Status After Refactor \|
	51	\|----------\|--------\|---------\|----------------------\|
	52	\| `/api/whoami` \| GET \| Get current user identity \| ✅ Updated - Returns ANProto pubkey \|
	53	\| `/api/profile` \| GET \| Get user profile \| ✅ Maintained - No changes \|
	54	\| `/api/profile` \| POST \| Update username \| ✅ Maintained - No changes \|
	55	\| `/api/timeline` \| GET \| Fetch timeline messages \| ✅ Enhanced - Signature verification \|
	56	\| `/api/post` \| POST \| Submit new post \| ✅ Enhanced - Creates YAML + signature \|
	57	\| `/api/follow` \| POST \| Follow a user \| ✅ Updated - Base64 pubkey \|
	58	\| `/api/unfollow` \| POST \| Unfollow a user \| ✅ Updated - Base64 pubkey \|
	59	\| `/api/follows` \| GET \| List followed users \| ✅ Maintained - No changes \|
	60	\| `/api/directory` \| GET \| Search user directory \| ✅ Maintained - Relay proxy \|
	61	\| `/api/directory/:pubkey` \| GET \| Get user profile from directory \| ✅ Updated - Base64 pubkey \|
	62	\| `/api/upload` \| POST \| Upload image \| ✅ Enhanced - Hash-based storage \|
	63	\| Static files \| GET \| Serve HTML/CSS/JS \| ✅ Maintained - No changes \|
	64	\| WebSocket \| - \| Real-time updates \| ✅ Maintained - No changes \|
	65
	66	Total: 13 endpoints → All maintained or enhanced
	67
	68	---
	69
	70	## Feature-by-Feature Analysis
	71
	72	### 1. Identity Management
	73
	74	CURRENT:
	75	- IdentityKey-based identity
	76	- Public key displayed in hex format
	77	- Separate identity keypair and feed keypair
	78
	79	AFTER REFACTOR:
	80	- ANProto-based identity
	81	- Public key displayed in base64 format (44 characters)
	82	- Unified keypair - ANProto key IS the feed key
	83	- Public key is portable across transports
	84
	85	User Impact:
	86	- ⚠️ Breaking change: New identity required (fresh start)
	87	- ✅ Simpler: One identity system instead of two
	88	- ✅ Better: ANProto is portable (works in browser too)
	89	- ⚠️ Format change: Users need to share base64 keys, not hex
	90
	91	Mitigation:
	92	- Document clearly: "This is a new version, create new identity"
	93	- Old version still works with old storage
	94	- Future: Could add import tool for old posts
	95
	96	---
	97
	98	### 2. Posting Messages
	99
	100	CURRENT:
	101	- Create JSON object: `{ type: 'post', text, timestamp, author, username }`
	102	- Append full JSON to Hypercore (300+ bytes)
	103	- Images stored inline as base64 in JSON
	104
	105	AFTER REFACTOR:
	106	- Create YAML with metadata:
	107	```yaml
	108	name: username
	109	previous: hashOfPreviousPost
	110	---
	111	Message text here
	112	```
	113	- Hash YAML → Sign hash → Append signature only (132 bytes)
	114	- Store YAML in Hyperbee by hash
	115	- Images stored separately by hash
	116
	117	User Impact:
	118	- ✅ No visible change - UI stays the same
	119	- ✅ Better performance - Faster feed sync (smaller blocks)
	120	- ✅ Better storage - Images not duplicated
	121	- ✅ P2P distribution - Content served peer-to-peer
	122
	123	Features Maintained:
	124	- ✅ Post text messages
	125	- ✅ Attach images
	126	- ✅ Display username with post
	127	- ✅ Timestamp ordering
	128	- ✅ WebSocket broadcast on new post
	129
	130	---
	131
	132	### 3. Timeline / Reading Posts
	133
	134	CURRENT:
	135	1. Connect to swarm
	136	2. Replicate all followed feeds
	137	3. Download full message blocks
	138	4. Parse JSON
	139	5. Display timeline
	140
	141	AFTER REFACTOR:
	142	1. Connect to swarm (feed replication + content gossip)
	143	2. Replicate signature blocks only
	144	3. Verify ANProto signatures
	145	4. Fetch content by hash (cache → peers → relay)
	146	5. Parse YAML
	147	6. Display timeline
	148
	149	User Impact:
	150	- ✅ Faster initial sync - Only signatures downloaded
	151	- ✅ Lazy content loading - Fetch as you scroll
	152	- ✅ P2P resilience - Works when relay down
	153	- ⚠️ Possible delay if content not cached (mitigated by P2P gossip)
	154
	155	Features Maintained:
	156	- ✅ Merged timeline from all followed feeds
	157	- ✅ Sorted by timestamp
	158	- ✅ Shows username
	159	- ✅ Shows images
	160	- ✅ Own posts vs others differentiated
	161	- ✅ Real-time updates via WebSocket
	162
	163	---
	164
	165	### 4. Following / Unfollowing Users
	166
	167	CURRENT:
	168	- Take public key (hex format, 64 characters)
	169	- Create read-only feed
	170	- Join swarm topic
	171	- Store in follows list
	172
	173	AFTER REFACTOR:
	174	- Take public key (base64 format, 44 characters)
	175	- Create read-only feed
	176	- Join swarm topic (feed + content gossip)
	177	- Store in follows list
	178
	179	User Impact:
	180	- ⚠️ Format change: Public keys are now base64, not hex
	181	- ✅ Shorter keys: 44 chars vs 64 chars
	182	- ✅ P2P gossip: Better content availability
	183
	184	Features Maintained:
	185	- ✅ Follow by public key
	186	- ✅ Optional display name
	187	- ✅ Unfollow by public key
	188	- ✅ List follows with message count
	189	- ✅ Persistent follows list
	190
	191	Migration:
	192	- Need to re-follow users with their new base64 keys
	193	- Old follows list not compatible (fresh start)
	194
	195	---
	196
	197	### 5. Image Handling
	198
	199	CURRENT:
	200	- Images uploaded via `/api/upload`
	201	- Stored in Hyperdrive (`user-files/images/`)
	202	- Also stored inline in post JSON as base64
	203	- Displayed via data URL
	204
	205	AFTER REFACTOR:
	206	- Images uploaded via `/api/upload`
	207	- Hashed with SHA-256
	208	- Stored in Hyperbee by hash
	209	- Referenced in YAML metadata: `image: hashOfImage`
	210	- Fetched separately when viewing post
	211
	212	User Impact:
	213	- ✅ No visible change - UI identical
	214	- ✅ Better deduplication - Same image stored once
	215	- ✅ Lazy loading - Images fetched on-demand
	216	- ✅ P2P distribution - Images served peer-to-peer
	217	- ⚠️ Possible delay on first view (cached afterward)
	218
	219	Features Maintained:
	220	- ✅ Upload images
	221	- ✅ Display images in timeline
	222	- ✅ Image preview in compose modal
	223	- ✅ Multiple image formats (JPEG, PNG, etc.)
	224
	225	New Capabilities:
	226	- ✅ Image deduplication (same hash = same content)
	227	- ✅ Bandwidth savings (lazy loading)
	228	- ✅ Better caching (by hash)
	229
	230	---
	231
	232	### 6. User Directory / Search
	233
	234	CURRENT:
	235	- Proxy requests to relay: `https://7rnx.net/relay/directory`
	236	- Search by username
	237	- View user profiles
	238	- Get public keys for following
	239
	240	AFTER REFACTOR:
	241	- Unchanged - Still proxy to relay
	242	- Relay serves directory based on ANProto public keys
	243	- Search still works by username
	244	- Profile lookups by base64 public key
	245
	246	User Impact:
	247	- ✅ No change - Same functionality
	248	- ⚠️ Directory needs to be updated on relay side (separate task)
	249
	250	Features Maintained:
	251	- ✅ Search users by name
	252	- ✅ View user profiles
	253	- ✅ Copy public key for following
	254	- ✅ See user's post count
	255
	256	---
	257
	258	### 7. Real-Time Updates (WebSocket)
	259
	260	CURRENT:
	261	- WebSocket connection from browser to server
	262	- Server broadcasts on feed append events
	263	- Clients receive: `{ type: 'new_post', feed: 'own' }`
	264	- Clients refresh timeline
	265
	266	AFTER REFACTOR:
	267	- Same WebSocket protocol
	268	- Server still broadcasts on feed events
	269	- Clients still receive same messages
	270	- Backend now verifies signatures + fetches content
	271
	272	User Impact:
	273	- ✅ No change - Same real-time experience
	274	- ✅ Transparent - Browser doesn't know about architecture change
	275
	276	Features Maintained:
	277	- ✅ Real-time post notifications
	278	- ✅ Automatic timeline refresh
	279	- ✅ Multi-client support
	280
	281	---
	282
	283	### 8. Profile Management
	284
	285	CURRENT:
	286	- Set username via `/api/profile` POST
	287	- Stored in Hyperbee metadata
	288	- Displayed in timeline
	289	- Included in post JSON
	290
	291	AFTER REFACTOR:
	292	- Same API endpoint
	293	- Same storage location (Hyperbee metadata)
	294	- Displayed in timeline
	295	- Included in YAML metadata field
	296
	297	User Impact:
	298	- ✅ No change - Identical functionality
	299
	300	Features Maintained:
	301	- ✅ Set username
	302	- ✅ View own username
	303	- ✅ Username in posts
	304	- ✅ Username in profile API
	305
	306	---
	307
	308	## New Features (Added Capabilities)
	309
	310	### 1. P2P Content Serving ⭐ NEW
	311
	312	What:
	313	- Each node acts as a content server
	314	- Nodes request content from peers, not just relay
	315	- Network scales organically
	316
	317	Benefit:
	318	- Better reliability (works when relay down)
	319	- Lower relay bandwidth costs
	320	- Faster content delivery (nearby peers)
	321
	322	### 2. Content Deduplication ⭐ NEW
	323
	324	What:
	325	- Same content = same hash
	326	- Stored only once, referenced many times
	327
	328	Benefit:
	329	- Storage efficiency
	330	- Bandwidth savings
	331	- Identical images shared across users
	332
	333	### 3. Lazy Content Loading ⭐ NEW
	334
	335	What:
	336	- Signatures synced immediately (fast)
	337	- Content fetched as you view it
	338
	339	Benefit:
	340	- Faster timeline load
	341	- Lower initial bandwidth
	342	- Scroll to see more (progressive loading)
	343
	344	### 4. ANProto Signatures ⭐ NEW
	345
	346	What:
	347	- Cryptographic proof of authorship
	348	- Portable across transports
	349	- Compatible with Wiredove
	350
	351	Benefit:
	352	- Stronger security (verifiable signatures)
	353	- Cross-platform compatibility
	354	- Future-proof architecture
	355
	356	### 5. YAML Format ⭐ NEW
	357
	358	What:
	359	- Human-readable message format
	360	- Metadata in front matter
	361	- Compatible with Wiredove
	362
	363	Benefit:
	364	- Better debugging (readable)
	365	- Extensible metadata
	366	- Cross-platform compatibility
	367
	368	---
	369
	370	## Features NOT Impacted
	371
	372	These features work exactly the same:
	373
	374	✅ Username management - Set/get username, stored in metadata
	375	✅ Follows list persistence - Stored in Hyperbee, reloaded on restart
	376	✅ Graceful shutdown - goodbye() hook cleanup
	377	✅ Hyperswarm networking - DHT discovery, NAT traversal
	378	✅ Corestore architecture - Multi-core storage management
	379	✅ Web server - HTTP server, CORS, static files
	380	✅ WebSocket server - Real-time connections, broadcasting
	381	✅ Storage path - Same location (can be customized)
	382	✅ CLI mode vs Web mode - Both modes still supported
	383
	384	---
	385
	386	## Breaking Changes Summary
	387
	388	### For Users
	389
	390	1. New Identity Required ⚠️
	391	- Must create new ANProto identity
	392	- Old identity not compatible
	393	- Fresh start approach
	394
	395	2. Public Key Format Change ⚠️
	396	- Old: Hex (64 characters)
	397	- New: Base64 (44 characters)
	398	- Need new keys from followed users
	399
	400	3. Follows List Reset ⚠️
	401	- Old follows not compatible
	402	- Need to re-follow users
	403	- Easy to do via directory search
	404
	405	### For Developers
	406
	407	1. Feed Block Format ⚠️
	408	- Old: Full JSON messages
	409	- New: ANProto signatures only
	410	- Not backward compatible
	411
	412	2. Content Storage ⚠️
	413	- Old: Embedded in Hypercore
	414	- New: Separate Hyperbee cache
	415	- Requires content fetching layer
	416
	417	3. Identity System ⚠️
	418	- Old: IdentityKey (keet-identity-key)
	419	- New: ANProto (sodium-universal)
	420	- Remove IdentityKey dependency
	421
	422	---
	423
	424	## Migration Strategy for Users
	425
	426	### Recommended: Fresh Start
	427
	428	1. Backup old version (optional)
	429	```bash
	430	mv wirecore-storage wirecore-storage-old
	431	```
	432
	433	2. Run new version
	434	```bash
	435	pear run . cli init
	436	```
	437
	438	3. Create new ANProto identity
	439	- Get your new base64 public key
	440	- Share with network
	441
	442	4. Re-follow users
	443	- Use directory search: `pear run . cli directory`
	444	- Follow users by new base64 keys
	445
	446	5. Start posting
	447	- All new posts use YAML + signatures
	448	- P2P content distribution active
	449
	450	### Optional: Keep Old Version Running
	451
	452	- Old storage still works with old version
	453	- Run old version from separate directory
	454	- Export posts manually if needed
	455	- Gradually migrate content
	456
	457	---
	458
	459	## Testing Plan for Feature Parity
	460
	461	### Unit Tests (All Features)
	462
	463	```bash
	464	# Test all CLI commands
	465	brittle test/cli.test.js
	466
	467	# Test all API endpoints
	468	brittle test/api.test.js
	469
	470	# Test web interface
	471	brittle test/web.test.js
	472
	473	# Test ANProto integration
	474	brittle test/anproto.test.js
	475
	476	# Test content gossip
	477	brittle test/gossip.test.js
	478	```
	479
	480	### Manual Testing Checklist
	481
	482	CLI:
	483	- [ ] `init` - Create new identity
	484	- [ ] `post` - Post text message
	485	- [ ] `post` - Post with image (future)
	486	- [ ] `setname` - Set username
	487	- [ ] `follow` - Follow user by base64 key
	488	- [ ] `unfollow` - Remove follow
	489	- [ ] `timeline` - View merged timeline
	490	- [ ] `whoami` - Display ANProto key
	491	- [ ] `follows` - List followed users
	492	- [ ] `directory` - Search users
	493	- [ ] `profile` - View user profile
	494
	495	Web Interface:
	496	- [ ] Load timeline
	497	- [ ] Post text message
	498	- [ ] Post with image
	499	- [ ] View images in timeline
	500	- [ ] Follow user via search
	501	- [ ] Unfollow user
	502	- [ ] Set username
	503	- [ ] View profile
	504	- [ ] Real-time updates (WebSocket)
	505	- [ ] Refresh timeline
	506
	507	P2P Features:
	508	- [ ] Content fetches from peers
	509	- [ ] Content serves to peers
	510	- [ ] Relay fallback works
	511	- [ ] Offline caching works
	512	- [ ] Images lazy load
	513
	514	---
	515
	516	## Deployment Checklist
	517
	518	Before releasing refactored version:
	519
	520	### Code Complete
	521	- [ ] All 6 phases implemented
	522	- [ ] All brittle tests pass
	523	- [ ] Manual testing complete
	524	- [ ] No regressions found
	525
	526	### Documentation Updated
	527	- [ ] README.md updated
	528	- [ ] CLI help text updated
	529	- [ ] API docs updated
	530	- [ ] Migration guide created
	531
	532	### User Communication
	533	- [ ] Announce new version
	534	- [ ] Explain breaking changes
	535	- [ ] Provide migration guide
	536	- [ ] Support old version temporarily
	537
	538	### Deployment
	539	- [ ] Package as Pear app
	540	- [ ] Test on clean system
	541	- [ ] Alpha release to select users
	542	- [ ] Gather feedback
	543	- [ ] Beta release
	544	- [ ] Production release
	545
	546	---
	547
	548	## Success Criteria
	549
	550	### Functional Requirements ✅
	551
	552	All current features work after refactor:
	553	- ✅ Initialize identity
	554	- ✅ Post messages
	555	- ✅ Post images
	556	- ✅ Follow/unfollow users
	557	- ✅ View timeline
	558	- ✅ Search directory
	559	- ✅ View profiles
	560	- ✅ Real-time updates
	561
	562	### Performance Requirements ✅
	563
	564	- ✅ Timeline loads < 10 seconds
	565	- ✅ Post submission < 1 second
	566	- ✅ Image display < 3 seconds (cached or P2P)
	567	- ✅ Content fetches from peers, not just relay
	568
	569	### Quality Requirements ✅
	570
	571	- ✅ All tests pass
	572	- ✅ No memory leaks
	573	- ✅ Graceful error handling
	574	- ✅ Clear error messages
	575
	576	### User Experience Requirements ✅
	577
	578	- ✅ No visible UI changes (transparent backend update)
	579	- ✅ Same or better performance
	580	- ✅ Clear migration path
	581	- ✅ Documentation complete
	582
	583	---
	584
	585	## Conclusion
	586
	587	### Feature Parity: ✅ CONFIRMED
	588
	589	Current features: 12 CLI commands + 12 web features + 13 API endpoints = 37 features
	590
	591	After refactor:
	592	- ✅ 36 features maintained or enhanced (97%)
	593	- ⚠️ 1 feature obsolete (migrate-storage - not needed)
	594	- ⭐ 5 new features added (P2P gossip, deduplication, lazy loading, ANProto, YAML)
	595
	596	### Breaking Changes: ⚠️ ACCEPTABLE
	597
	598	- Fresh start approach is clearest path
	599	- Users keep old version for reference
	600	- Gradual migration possible
	601	- Network bootstraps with new identities
	602
	603	### Net Result: 🚀 IMPROVED
	604
	605	The refactored version maintains all user-facing features while adding:
	606	- Better P2P architecture (true distributed network)
	607	- Better storage efficiency (content-by-hash)
	608	- Better protocol design (ANProto signatures)
	609	- Better cross-platform compatibility (Wiredove)
	610
	611	Status: Feature parity confirmed, ready for implementation
	612	Date: 2026-01-21

pear-app/P2P_ARCHITECTURE.md +545 -0

…	…	@@ -0,0 +1,545 @@
	1	# Wirecore P2P Architecture - Systematic Analysis
	2
	3	Date: 2026-01-21
	4	Purpose: Systematic analysis of P2P content distribution architecture
	5
	6	---
	7
	8	## Core Question
	9
	10	Should each Hypercore node act as a content server, or should content only flow through a central relay?
	11
	12	Answer: Each node MUST act as a content server for true P2P architecture.
	13
	14	---
	15
	16	## First Principles Analysis
	17
	18	### What APDS/Wiredove Teaches Us
	19
	20	Wiredove.net architecture:
	21	- Every browser node is a content server
	22	- Content gossip: "Do you have hash X?" → "Yes, here it is" / "No, try someone else"
	23	- WebRTC mesh network for content distribution
	24	- Relay exists but is NOT required for content flow
	25	- Content-by-hash is transport-agnostic (works over any connection)
	26
	27	Key insight: Distributed content = distributed load + resilience
	28
	29	### What Holepunchto Ecosystem Teaches Us
	30
	31	Hypercore primitives and their purposes:
	32
	33	\| Primitive \| Purpose \| Use Case \|
	34	\|-----------\|---------\|----------\|
	35	\| Hypercore \| Append-only log \| Sequential data (signatures, events) \|
	36	\| Hyperbee \| Key-value store \| Mappings (hash → content) \|
	37	\| Hyperdrive \| File system \| Directory structures (files/folders) \|
	38	\| Corestore \| Multi-core manager \| Organize multiple cores \|
	39
	40	Our data types:
	41
	42	\| Data \| Structure \| Best Primitive \|
	43	\|------\|-----------\|----------------\|
	44	\| Signatures \| Sequential (block 0, 1, 2...) \| Hypercore ✓ \|
	45	\| Content by hash \| Key-value (hash → content) \| Hyperbee ✓ \|
	46	\| Metadata \| Key-value (setting → value) \| Hyperbee ✓ \|
	47
	48	Critical insight: Content-by-hash is NOT sequential, so Hypercore blocks aren't right. It's a mapping, so Hyperbee IS right!
	49
	50	### What ANProto Teaches Us
	51
	52	ANProto philosophy:
	53	- Signatures are portable - Work over any transport
	54	- Content is portable - Referenced by hash, transport-agnostic
	55	- No network assumptions - ANProto doesn't care about P2P vs HTTP vs sneakernet
	56
	57	This means: Content distribution should be flexible, multi-transport, peer-to-peer!
	58
	59	---
	60
	61	## Architecture Decision
	62
	63	### YES - Each Node Must Serve Content
	64
	65	Centralized (what we DON'T want):
	66	```
	67	User A ──┐
	68	User B ──┼─→ Relay ──┐
	69	User C ──┘ ├─→ User D
	70	User E ──────────────┘
	71
	72	❌ Single point of failure
	73	❌ Relay bandwidth bottleneck
	74	❌ Not truly P2P
	75	❌ Doesn't scale
	76	```
	77
	78	Distributed (what we DO want):
	79	```
	80	User A ←→ User B
	81	↕ ↕
	82	User C ←→ User D ←→ Relay
	83	↕ ↕
	84	User E ←→ User F
	85
	86	✅ Distributed load
	87	✅ Resilient (works if relay down)
	88	✅ Scales with network size
	89	✅ True P2P
	90	✅ Network effect (more nodes = more capacity)
	91	```
	92
	93	### Content Flow Example
	94
	95	Alice posts a message with image:
	96
	97	1. Alice creates post:
	98	- Hashes content → `hash123`
	99	- Signs hash with ANProto → `signature`
	100	- Stores content in local Hyperbee: `hash123 → content`
	101	- Appends signature to Hypercore feed
	102
	103	2. Bob follows Alice:
	104	- Replicates Alice's feed via Hyperswarm
	105	- Gets signature from feed
	106	- Extracts `hash123` from signature
	107	- Requests content from peers (Protomux channel)
	108
	109	3. Charlie (connected to Bob):
	110	- Receives Bob's content request
	111	- Checks local Hyperbee cache: `contentCache.get('hash123')`
	112	- If found, sends content to Bob
	113	- If not found, ignores request
	114
	115	4. Bob receives content:
	116	- Stores in local Hyperbee: `hash123 → content`
	117	- Displays post with image
	118	- Bob now serves this content to others!
	119
	120	5. Diana (new user, follows Alice):
	121	- Replicates Alice's feed
	122	- Needs `hash123`
	123	- Requests from peers
	124	- Gets it from Bob (not from Alice or relay!)
	125
	126	This is how content propagates through the network organically!
	127
	128	---
	129
	130	## Storage Architecture
	131
	132	### The Hyperbee Solution
	133
	134	Decision: Use Hyperbee for content cache (not Hyperdrive, not content.json)
	135
	136	```javascript
	137	// Initialize
	138	const contentCache = new Hyperbee(store.get({ name: 'content-cache' }), {
	139	keyEncoding: 'utf-8', // Hash as string
	140	valueEncoding: 'binary' // Content as Buffer
	141	})
	142
	143	// Store
	144	await contentCache.put(hash, Buffer.from(yaml))
	145
	146	// Retrieve
	147	const node = await contentCache.get(hash)
	148	if (node) {
	149	const content = node.value.toString()
	150	}
	151
	152	// Check existence
	153	const node = await contentCache.get(hash)
	154	const exists = node !== null
	155	```
	156
	157	### Why Hyperbee Over Alternatives?
	158
	159	vs content.json (original lib/core):
	160	- ✅ Hypercore-native structure
	161	- ✅ Efficient B-tree lookups (O(log n) vs O(n))
	162	- ✅ Can be partially replicated (if needed)
	163	- ✅ Larger-than-memory support (Hyperbee pages data)
	164	- ✅ Transactional (Hyperbee batches writes)
	165
	166	vs Hyperdrive (previous plan):
	167	- ✅ Semantically correct (key-value, not file system)
	168	- ✅ Not abusing paths as hashes (`put(hash, content)` vs `put('/pZ/pZGm...', content)`)
	169	- ✅ Cleaner API
	170	- ✅ Better for random access by hash
	171	- ✅ Hyperdrive is designed for hierarchical files, we need flat key-value
	172
	173	vs separate database:
	174	- ✅ Integrated with Corestore
	175	- ✅ Backed up automatically
	176	- ✅ Can replicate if needed
	177	- ✅ Native to Hypercore ecosystem
	178
	179	### Complete Storage Structure
	180
	181	```
	182	{storage-path}/cores/ (Corestore)
	183	│
	184	├── my-feed/ # Hypercore: Your signatures
	185	│ └── data
	186	│ ├── Block 0: signature (132 bytes)
	187	│ ├── Block 1: signature (132 bytes)
	188	│ └── Block N: signature (132 bytes)
	189	│
	190	├── {followed-feed-A}/ # Hypercore: Alice's signatures
	191	│ └── data (signatures)
	192	│
	193	├── {followed-feed-B}/ # Hypercore: Bob's signatures
	194	│ └── data (signatures)
	195	│
	196	├── content-cache/ # Hyperbee: Content by hash
	197	│ └── data (B-tree)
	198	│ ├── "pZGm1Av0..." → "---\nname: fleeky\n---\nHello!"
	199	│ ├── "aBcd2Ef3..." → <base64-image-data>
	200	│ └── "xYz789AB..." → "---\nname: alice\n---\nGreetings"
	201	│
	202	└── metadata/ # Hyperbee: User settings
	203	└── data (B-tree)
	204	├── "anproto-keypair" → "BSY7/er4..."
	205	├── "profile" → { username: "fleeky", ... }
	206	├── "follow:BSY7..." → { name: "alice", ... }
	207	└── "config" → { theme: "dark", ... }
	208	```
	209
	210	---
	211
	212	## Network Protocol
	213
	214	### Protomux Content Channel
	215
	216	Each Hyperswarm connection has two channels:
	217
	218	1. Feed Replication (built-in)
	219	```
	220	Protocol: hypercore-replication
	221	Purpose: Replicate signature feeds
	222	Automatic: Hypercore/Corestore handles this
	223	```
	224
	225	2. Content Gossip (custom)
	226	```
	227	Protocol: wirecore-content-gossip
	228	Purpose: Request/serve content by hash
	229	Manual: We implement this
	230	```
	231
	232	### Content Gossip Protocol
	233
	234	Message Types:
	235
	236	```javascript
	237	// 1. Request content
	238	{
	239	type: 'content-request',
	240	id: 'req-12345', // Request ID for matching responses
	241	hash: 'pZGm1Av0...' // Content hash (44 chars base64)
	242	}
	243
	244	// 2. Serve content
	245	{
	246	type: 'content-response',
	247	id: 'req-12345', // Matches request ID
	248	hash: 'pZGm1Av0...', // Content hash
	249	found: true, // Whether content was found
	250	content: Buffer(...) // Content data (if found)
	251	}
	252
	253	// 3. Announce content (optional optimization)
	254	{
	255	type: 'content-announce',
	256	hashes: ['pZGm1...', 'aBcd2...', ...] // Hashes this node has
	257	}
	258	```
	259
	260	### Implementation
	261
	262	```javascript
	263	// Initialize Protomux channel
	264	swarm.on('connection', (conn) => {
	265	// 1. Feed replication (automatic)
	266	store.replicate(conn)
	267
	268	// 2. Content gossip channel
	269	const mux = Protomux.from(conn)
	270	const channel = mux.createChannel({
	271	protocol: 'wirecore-content-gossip'
	272	})
	273
	274	// Register message handlers
	275	channel.addMessage({
	276	encoding: c.json,
	277	onmessage: async (msg) => {
	278	if (msg.type === 'content-request') {
	279	await handleContentRequest(msg, channel)
	280	} else if (msg.type === 'content-response') {
	281	await handleContentResponse(msg)
	282	}
	283	}
	284	})
	285
	286	// Open channel
	287	await channel.open()
	288	})
	289
	290	// Serve content to peers
	291	async function handleContentRequest(msg, channel) {
	292	const { id, hash } = msg
	293
	294	// Check local cache
	295	const node = await contentCache.get(hash)
	296
	297	if (node) {
	298	// We have it! Send to peer
	299	channel.messages[0].send({
	300	type: 'content-response',
	301	id,
	302	hash,
	303	found: true,
	304	content: node.value
	305	})
	306	} else {
	307	// Don't have it, send "not found"
	308	channel.messages[0].send({
	309	type: 'content-response',
	310	id,
	311	hash,
	312	found: false
	313	})
	314	}
	315	}
	316
	317	// Request content from peers
	318	async function requestContent(hash) {
	319	const promises = []
	320
	321	// Send request to all connected peers
	322	for (const conn of swarm.connections) {
	323	const channel = getChannel(conn, 'wirecore-content-gossip')
	324	if (channel) {
	325	const promise = new Promise((resolve) => {
	326	const id = randomId()
	327	pendingRequests.set(id, resolve)
	328
	329	channel.messages[0].send({
	330	type: 'content-request',
	331	id,
	332	hash
	333	})
	334
	335	// Timeout after 5 seconds
	336	setTimeout(() => {
	337	pendingRequests.delete(id)
	338	resolve(null)
	339	}, 5000)
	340	})
	341	promises.push(promise)
	342	}
	343	}
	344
	345	// Wait for first successful response
	346	const results = await Promise.all(promises)
	347	return results.find(r => r !== null)
	348	}
	349	```
	350
	351	---
	352
	353	## Multi-Transport Content Fetching
	354
	355	### Content Retrieval Strategy
	356
	357	```javascript
	358	async function getContent(hash) {
	359	// 1. LOCAL CACHE (fastest)
	360	const cached = await contentCache.get(hash)
	361	if (cached) {
	362	console.log('✓ Content from local cache')
	363	return cached.value
	364	}
	365
	366	// 2. HYPERSWARM PEERS (P2P, preferred)
	367	const fromPeer = await requestViaHyperswarm(hash)
	368	if (fromPeer) {
	369	console.log('✓ Content from Hyperswarm peer')
	370	await contentCache.put(hash, fromPeer) // Cache for future
	371	return fromPeer
	372	}
	373
	374	// 3. RELAY WEBSOCKET (fallback for NAT/bootstrap)
	375	const fromRelay = await requestViaRelay(hash)
	376	if (fromRelay) {
	377	console.log('✓ Content from relay (fallback)')
	378	await contentCache.put(hash, fromRelay) // Cache for future
	379	return fromRelay
	380	}
	381
	382	// 4. NOT FOUND
	383	console.error('✗ Content not found:', hash)
	384	return null
	385	}
	386	```
	387
	388	### Why Multiple Transports?
	389
	390	Hyperswarm (primary):
	391	- ✅ True P2P (no intermediary)
	392	- ✅ Direct encrypted connections
	393	- ✅ NAT traversal (DHT hole-punching)
	394	- ✅ Distributed load
	395	- ❌ Requires UDP (doesn't work in browsers)
	396
	397	Relay WebSocket (fallback):
	398	- ✅ Works in browsers
	399	- ✅ Helps with bootstrap (new nodes)
	400	- ✅ NAT traversal when hole-punching fails
	401	- ✅ Content discovery (indexes hashes)
	402	- ❌ Centralized (single point of failure)
	403	- ❌ Bandwidth costs for relay operator
	404
	405	Together:
	406	- ✅ Best of both worlds
	407	- ✅ Resilient (works if relay down)
	408	- ✅ Efficient (P2P preferred, relay fallback)
	409	- ✅ Compatible (browsers via relay, native via Hyperswarm)
	410
	411	---
	412
	413	## The Relay's Role
	414
	415	### What The Relay Should Do
	416
	417	1. Directory Service ✅ Already implemented
	418	- Index users by name
	419	- Search for users
	420	- Profile lookups
	421
	422	2. Content Bootstrap ✅ Should implement
	423	- New nodes request content via WebSocket
	424	- Relay serves from its Hyperbee cache
	425	- Helps with cold-start problem
	426
	427	3. Web User Gateway ✅ Should implement
	428	- Browsers can't use Hyperswarm
	429	- Relay bridges WebSocket ↔ Hyperswarm
	430	- Web users access P2P network
	431
	432	4. APDS Compatibility ✅ Already implemented
	433	- Relay implements APDS protocol
	434	- Wiredove.net users can interoperate
	435	- Content-by-hash bridge
	436
	437	### What The Relay Should NOT Do
	438
	439	1. ❌ Primary Content Distribution
	440	- Content should flow peer-to-peer
	441	- Relay is fallback, not primary path
	442	- Don't make relay a bottleneck
	443
	444	2. ❌ Store All Content
	445	- Relay caches what it sees
	446	- Doesn't need to be exhaustive index
	447	- LRU cache is fine
	448
	449	3. ❌ Mandatory Intermediary
	450	- Peers should connect directly via Hyperswarm
	451	- Relay only needed when P2P fails
	452	- Network should work if relay is offline
	453
	454	---
	455
	456	## Benefits of This Architecture
	457
	458	### Scalability
	459
	460	Traditional client-server:
	461	- Bandwidth: O(n) - relay serves all n users
	462	- Latency: 2 hops (user → relay → user)
	463	- Failure: Single point (relay down = network down)
	464
	465	P2P gossip network:
	466	- Bandwidth: O(1) per node - distributed across network
	467	- Latency: 1 hop (user → user, if connected)
	468	- Failure: No single point - network degrades gracefully
	469
	470	Network effect:
	471	- 10 nodes = 10x capacity
	472	- 100 nodes = 100x capacity
	473	- 1000 nodes = 1000x capacity
	474
	475	### Resilience
	476
	477	- Relay offline? Peers still serve each other
	478	- Node offline? Content served by others who have it
	479	- Network partition? Subgraphs continue operating
	480	- Cold start? Relay helps bootstrap, then P2P takes over
	481
	482	### Efficiency
	483
	484	- Content locality - Frequently accessed content is cached by many peers
	485	- Bandwidth distribution - Popular content served by multiple nodes
	486	- Smart routing - Can request from geographically close peers
	487	- Reduced relay load - Relay only serves when P2P unavailable
	488
	489	### Privacy
	490
	491	- Peer metadata - Relay doesn't see all content requests
	492	- Direct connections - Encrypted Hyperswarm connections
	493	- Content propagation - Can't tell original source vs cached copy
	494
	495	---
	496
	497	## Implementation Summary
	498
	499	### Components
	500
	501	1. ContentCache (Hyperbee wrapper) - NEW
	502	- Local content-by-hash storage
	503	- Key = hash, Value = content (YAML or image)
	504
	505	2. ContentGossip (Protomux channel) - NEW
	506	- Request/serve content over Hyperswarm
	507	- Each node serves what it has cached
	508
	509	3. ContentSync (multi-transport) - UPDATED
	510	- Tries Hyperbee cache first
	511	- Falls back to Hyperswarm gossip
	512	- Falls back to relay WebSocket
	513
	514	4. RelayBridge (WebSocket) - EXISTS
	515	- Already implemented (APDS protocol)
	516	- Already serves content-by-hash
	517	- Just needs to cache in Hyperbee
	518
	519	### File Structure
	520
	521	```
	522	pear-app/lib/
	523	├── anproto.js # ANProto signature protocol
	524	├── yaml-helper.js # YAML parsing/creation
	525	├── content-cache.js # Hyperbee wrapper (NEW)
	526	├── content-gossip.js # Protomux channel (NEW)
	527	└── content-sync.js # Multi-transport fetching (NEW)
	528	```
	529
	530	---
	531
	532	## Next Steps
	533
	534	1. ✅ Architecture analysis complete
	535	2. ⏳ Update REFACTOR_PLAN.md with Hyperbee decision
	536	3. ⏳ Implement ContentCache (Hyperbee wrapper)
	537	4. ⏳ Implement ContentGossip (Protomux channel)
	538	5. ⏳ Implement ContentSync (multi-transport)
	539	6. ⏳ Test P2P content distribution
	540
	541	---
	542
	543	Status: Architecture finalized, ready to implement
	544	Key Decision: Hyperbee for content cache, P2P gossip protocol
	545	Date: 2026-01-21

pear-app/REFACTOR_PLAN.md +227 -117

…	…	@@ -41,34 +41,64 @@ This achieves all design criteria while following Hypercore best practices for e
41	41	### Message Flow (Refactored)
42	42
43	43	```javascript
	44	// INITIALIZATION: Create feed using ANProto keypair
	45	async function init() {
	46	// 1. Generate or load ANProto keypair
	47	const { keypair, publicKey, secretKey } = ANProto.gen()
	48	// publicKey: "BSY7/er4VJIu08o39NaRAiPY/MAvd7oQhlGCRDABjYU=" (44 chars)
	49
	50	// 2. Store in Hyperbee metadata
	51	await metaBee.put('anproto-keypair', keypair)
	52
	53	// 3. Create Hypercore feed using SAME keypair
	54	const publicKeyBuf = b4a.from(publicKey, 'base64')
	55	const secretKeyBuf = b4a.from(secretKey, 'base64')
	56
	57	myFeed = store.get({
	58	publicKey: publicKeyBuf,
	59	secretKey: secretKeyBuf
	60	})
	61
	62	// NOW: myFeed.key = publicKeyBuf = ANProto publicKey!
	63	// User identity = Feed key = ANProto public key (UNIFIED!)
	64	}
	65
44	66	// POST FLOW
45		async function post(text) {
46		// 1. Create YAML message with metadata
	67	async function post(text, imageData = null) {
	68	// 1. If there's an image, hash and store it
	69	let imageHash = null
	70	if (imageData) {
	71	imageHash = ANProto.hashData(imageData)
	72	await contentCache.put(imageHash, imageData)
	73	}
	74
	75	// 2. Create YAML message with metadata
47	76	const yaml = YAMLHelper.create({
48	77	name: await getUsername(),
49		image: await getProfileImageHash(),
	78	image: imageHash, // Reference to image (or null)
50	79	previous: await getLatestMessageHash()
51	80	}, text)
52	81
53		// 2. Hash the content (SHA256 → base64)
54		const contentHash = ANProto.hashData(yaml)
	82	// 3. Hash the YAML content (SHA256 → base64)
	83	const postHash = ANProto.hashData(yaml)
55	84	// → "pZGm1Av0IEBKARczz7exkNYsZb8LzaMrV7J32a2fFG4="
56	85
57		// 3. Store content in Hyperdrive by hash
58		const hashPath = `/${contentHash.substring(0, 2)}/${contentHash}`
59		await contentDrive.put(hashPath, Buffer.from(yaml))
60		// → Writes to cores/content-drive/{hash}
	86	// 4. Store YAML in Hyperbee by hash
	87	await contentCache.put(postHash, Buffer.from(yaml))
	88	// → Writes to cores/content-cache/ (B-tree key-value)
61	89
62		// 4. Sign the hash with ANProto
63		const signature = ANProto.signHash(contentHash, anprotoKeypair)
	90	// 5. Sign the post hash with ANProto
	91	const signature = ANProto.signHash(postHash, anprotoKeypair)
64	92	// → "BSY7/er4VJIu08o39...wnlzLSoZ1QC7rHJc..." (132 bytes)
65		// → Format: publicKey(44) + signed(timestamp + hash)(88+)
	93	// → Format: publicKey(44) + signed(timestamp + postHash)(88+)
66	94
67		// 5. Append ONLY signature to Hypercore
	95	// NOTE: signature's publicKey = myFeed's publicKey (SAME!)
	96
	97	// 6. Append ONLY signature to Hypercore
68	98	await myFeed.append(Buffer.from(signature))
69		// → Hypercore block = 132 bytes, not KB!
	99	// → Hypercore block = 132 bytes
70	100
71		return { hash: contentHash, signature, timestamp }
	101	return { postHash, imageHash, signature, timestamp }
72	102	}
73	103
74	104	// READ FLOW
…	…	@@ -77,37 +107,60 @@ async function getTimeline(limit) {
77	107
78	108	// 1. Read signatures from Hypercore feeds
79	109	for (const feed of [myFeed, ...followedFeeds]) {
	110	// feed.key = author's ANProto public key (base64 as Buffer)
	111	const authorPublicKey = b4a.toString(feed.key, 'base64')
	112
80	113	for (let i = 0; i < feed.length; i++) {
81	114	const block = await feed.get(i)
82	115	const signature = block.toString()
83	116
84		// 2. Verify signature and extract hash
85		const { timestamp, hash } = ANProto.verifySignature(signature)
86
87		// 3. Try to get content from Hyperdrive
88		const hashPath = `/${hash.substring(0, 2)}/${hash}`
89		let content = await contentDrive.get(hashPath)
90
91		// 4. If not cached, request from peers
92		if (!content) {
93		content = await contentSync.fetch(hash)
94		// Cache in Hyperdrive
95		await contentDrive.put(hashPath, content)
	117	// 2. Verify ANProto signature and extract post hash
	118	const { timestamp, hash: postHash } = ANProto.verifySignature(signature)
	119	// postHash = hash of the YAML content
	120
	121	// 3. Get YAML content by hash
	122	let postNode = await contentCache.get(postHash)
	123	if (!postNode) {
	124	// Request YAML from peers (gossip protocol)
	125	const yamlBuf = await contentSync.fetch(postHash)
	126	if (yamlBuf) {
	127	await contentCache.put(postHash, yamlBuf)
	128	postNode = { value: yamlBuf }
	129	}
96	130	}
97	131
98		content = content.toString()
99
100		// 5. Parse YAML and extract metadata
101		const { metadata, text } = YAMLHelper.parse(content)
	132	if (!postNode) continue // Skip if can't fetch content
	133
	134	const yaml = postNode.value.toString()
	135
	136	// 4. Parse YAML and extract metadata
	137	const { metadata, text } = YAMLHelper.parse(yaml)
	138	// metadata = { name, image: imageHash, previous, ... }
	139
	140	// 5. If there's an image, fetch it separately
	141	let imageData = null
	142	if (metadata.image) {
	143	let imageNode = await contentCache.get(metadata.image)
	144	if (!imageNode) {
	145	// Request image from peers (same gossip protocol)
	146	const imgBuf = await contentSync.fetch(metadata.image)
	147	if (imgBuf) {
	148	await contentCache.put(metadata.image, imgBuf)
	149	imageNode = { value: imgBuf }
	150	}
	151	}
	152	imageData = imageNode ? imageNode.value : null
	153	}
102	154
103	155	posts.push({
104	156	signature,
105		hash,
	157	postHash,
	158	imageHash: metadata.image,
106	159	timestamp,
107		author: signature.substring(0, 44),
	160	feedKey: authorPublicKey, // Feed key = ANProto public key
108	161	name: metadata.name,
109		image: metadata.image,
110		text
	162	text,
	163	imageData // Separate from YAML!
111	164	})
112	165	}
113	166	}
…	…	@@ -131,25 +184,29 @@ async function getTimeline(limit) {
131	184	│
132	185	├── {followed-feed-N}/ # More followed feeds
133	186	│
134		├── content-drive/ # Hyperdrive: Content by hash
135		│ └── {hash-prefix}/ # Organized by first 2 chars
136		│ └── {full-hash} # Each file = YAML content or image
137		│ # Example:
138		│ # pZ/pZGm1Av0... = "---\nname: fleeky\n---\nHello!"
139		│ # aB/aBcd2Ef3... = <base64-image-data>
	187	├── content-cache/ # Hyperbee: Content by hash
	188	│ └── data (B-tree) # Key-value store
	189	│ ├── "pZGm1Av0..." → YAML text content (Buffer)
	190	│ ├── "aBcd2Ef3..." → Image data (Buffer)
	191	│ └── "xYz789AB..." → More content (Buffer)
	192	│ # Efficient O(log n) lookup by hash
	193	│ # Larger-than-memory support (paged to disk)
140	194	│
141	195	└── metadata/ # Hyperbee for user metadata
142		├── anproto-keypair # ANProto ed25519 keypair (132 chars)
143		├── profile # { username, bio, avatarHash }
144		├── follow:{pubkey} # Follow list
145		└── config # App settings
	196	└── data (B-tree)
	197	├── "anproto-keypair" → "BSY7/er4VJIu08o39..." (132 chars)
	198	├── "profile" → { username, bio, avatarHash }
	199	├── "follow:BSY7..." → { name: "alice", feedKey: "..." }
	200	├── "follow:xYz1..." → { name: "bob", feedKey: "..." }
	201	└── "config" → { theme: "dark", ... }
146	202	```
147	203
148	204	Key Changes from Current:
149	205	- ❌ Removed: `identity/` core (IdentityKey)
150		- ✅ Added: `content-drive/` (Hyperdrive for ALL content)
	206	- ✅ Added: `content-cache/` (Hyperbee for content-by-hash)
151	207	- ✅ Added: `anproto-keypair` in metadata Hyperbee
152	208	- ✅ Changed: my-feed stores signatures, not full messages
	209	- ✅ Each node serves content via Protomux gossip protocol
153	210
154	211	### Network Architecture
155	212
…	…	@@ -234,68 +291,88 @@ class YAMLHelper {
234	291	}
235	292	```
236	293
237		### 3. Content Drive (Hyperdrive Wrapper)
	294	### 3. Content Cache (Hyperbee Wrapper)
238	295
239		NEW MODULE: `lib/content-drive.js`
	296	NEW MODULE: `lib/content-cache.js`
240	297
241		Purpose: Wrap Hyperdrive for content-by-hash storage
	298	Purpose: Wrap Hyperbee for content-by-hash storage
242	299
243	300	API:
244	301	```javascript
245		class ContentDrive {
	302	class ContentCache {
246	303	constructor(corestore, options)
247	304
248		async ready() // Initialize Hyperdrive
	305	async ready() // Initialize Hyperbee
249	306	async put(hash, content) → void
250		async get(hash) → Buffer \| null
	307	async get(hash) → { key, value } \| null
251	308	async has(hash) → boolean
252	309	async delete(hash) → void
253		async list() → AsyncIterator<{ hash, size }>
	310	async *list() → AsyncIterator<{ key, value }>
254	311	async close() → void
255	312	}
256	313	```
257	314
258	315	Implementation:
259	316	```javascript
260		import Hyperdrive from 'hyperdrive'
	317	import Hyperbee from 'hyperbee'
261	318
262		class ContentDrive {
	319	class ContentCache {
263	320	constructor(corestore, options = {}) {
264		this.drive = new Hyperdrive(corestore, { name: 'content-drive' })
	321	this.bee = new Hyperbee(corestore.get({ name: 'content-cache' }), {
	322	keyEncoding: 'utf-8', // Hash as string
	323	valueEncoding: 'binary' // Content as Buffer
	324	})
265	325	}
266	326
267	327	async ready() {
268		await this.drive.ready()
	328	await this.bee.ready()
269	329	}
270	330
271	331	async put(hash, content) {
272		const path = `/${hash.substring(0, 2)}/${hash}`
273		await this.drive.put(path, content)
	332	// Content should be Buffer
	333	const buf = Buffer.isBuffer(content) ? content : Buffer.from(content)
	334	await this.bee.put(hash, buf)
274	335	}
275	336
276	337	async get(hash) {
277		const path = `/${hash.substring(0, 2)}/${hash}`
278		return await this.drive.get(path)
	338	// Returns { key, value } or null
	339	return await this.bee.get(hash)
279	340	}
280	341
281	342	async has(hash) {
282		const content = await this.get(hash)
283		return content !== null
	343	const node = await this.get(hash)
	344	return node !== null
	345	}
	346
	347	async delete(hash) {
	348	await this.bee.del(hash)
284	349	}
285	350
286		// ... other methods
	351	async *list() {
	352	// Iterate over all entries
	353	for await (const node of this.bee.createReadStream()) {
	354	yield node
	355	}
	356	}
	357
	358	async close() {
	359	// Hyperbee doesn't need explicit close
	360	// Corestore handles cleanup
	361	}
287	362	}
288	363	```
289	364
290		Storage Format (Hyperdrive):
	365	Storage Format (Hyperbee B-tree):
291	366	```
292		content-drive/
293		├── pZ/
294		│ └── pZGm1Av0... (YAML content as Buffer)
295		├── aB/
296		│ └── aBcd2Ef3... (Image data as Buffer)
297		└── xY/
298		└── xYz789... (More content)
	367	content-cache/
	368	└── data (B-tree structure)
	369	├── Key: "pZGm1Av0..." → Value: Buffer("---\nname: fleeky\n---\nHello!")
	370	├── Key: "aBcd2Ef3..." → Value: Buffer(<image-data>)
	371	└── Key: "xYz789AB..." → Value: Buffer("---\nname: alice\n---\nPost")
	372
	373	Efficient O(log n) lookup by hash
	374	Larger-than-memory support (paged to disk)
	375	Transactional writes
299	376	```
300	377
301	378	### 4. Content Sync Manager (`lib/content-sync.js`)
…	…	@@ -391,29 +468,29 @@ Message Types:
391	468	Note: This is a BREAKING CHANGE - users must create new identities
392	469	Migration: Fresh start (no data migration)
393	470
394		### Phase 2: Add Content Storage (Hyperdrive)
395		Goal: Add Hyperdrive for content-by-hash, still store full messages in Hypercore
	471	### Phase 2: Add Content Storage (Hyperbee)
	472	Goal: Add Hyperbee for content-by-hash, still store full messages in Hypercore
396	473
397	474	Tasks:
398		1. Create `lib/content-drive.js` (Hyperdrive wrapper)
399		2. Initialize ContentDrive on startup
400		- Create in Corestore: `{ name: 'content-drive' }`
	475	1. Create `lib/content-cache.js` (Hyperbee wrapper)
	476	2. Initialize ContentCache on startup
	477	- Create in Corestore: `new Hyperbee(store.get({ name: 'content-cache' }))`
401	478	- Ready after store initialization
402		3. Store content in Hyperdrive when posting
	479	3. Store content in Hyperbee when posting
403	480	- Hash content
404		- Store in Hyperdrive: `/{prefix}/{hash}`
	481	- Store in Hyperbee: `put(hash, content)`
405	482	- Keep storing full message in Hypercore (for now)
406		4. Read from Hyperdrive when available
407		- Try Hyperdrive first
	483	4. Read from Hyperbee when available
	484	- Try Hyperbee first (`get(hash)`)
408	485	- Fall back to Hypercore
409	486
410	487	Testing:
411	488	- Posts work as before
412		- Content cached in Hyperdrive
413		- Timeline reads from Hyperdrive
414		- Hyperdrive replicates via Hyperswarm
	489	- Content cached in Hyperbee (B-tree)
	490	- Timeline reads from Hyperbee
	491	- Efficient O(log n) lookups by hash
415	492
416		Benefit: Content is now P2P-replicable (unlike content.json)
	493	Benefit: Content stored in Hypercore-native key-value structure
417	494
418	495	### Phase 3: Add YAML Format (Non-Breaking)
419	496	Goal: Convert messages to YAML, dual-format reader
…	…	@@ -462,28 +539,50 @@ Message Types:
462	539
463	540	Rollback: BREAKING - need to restore from backup
464	541
465		### Phase 5: Add Content Sync (Feature Complete)
466		Goal: Fetch content from peers on-demand
	542	### Phase 5: Add P2P Content Gossip (Critical Feature)
	543	Goal: Each node serves content to peers - true distributed network
467	544
468	545	Tasks:
469		1. Copy `content-sync.js` to pear-app
470		2. Copy `hyperswarm-content-transport.js` to pear-app
471		3. Initialize ContentSync with Hyperswarm transport
472		4. Attach Protomux channel to Hyperswarm connections
473		5. Request content from peers when not cached
474		- Check content.json first
475		- If missing, request via ContentSync
476		- Cache response
477		6. Respond to peer content requests
478		- Listen for content-request messages
479		- Serve from content.json if available
	546	1. Create `lib/content-gossip.js` (Protomux channel)
	547	- Protocol: `wirecore-content-gossip`
	548	- Messages: content-request, content-response
	549	2. Attach gossip channel to Hyperswarm connections
	550	```javascript
	551	swarm.on('connection', (conn) => {
	552	store.replicate(conn) // Feed replication
	553	attachContentGossip(conn) // Content gossip
	554	})
	555	```
	556	3. Serve content to peers (relay behavior)
	557	```javascript
	558	channel.on('content-request', async ({ hash }) => {
	559	const node = await contentCache.get(hash)
	560	if (node) {
	561	channel.send('content-response', { hash, content: node.value })
	562	}
	563	})
	564	```
	565	4. Request content from peers when not cached
	566	```javascript
	567	const content = await requestFromPeers(hash)
	568	if (content) {
	569	await contentCache.put(hash, content)
	570	}
	571	```
	572	5. Add multi-transport ContentSync
	573	- Try Hyperbee cache first
	574	- Try Hyperswarm peers (gossip)
	575	- Fallback to relay WebSocket
480	576
481	577	Testing:
482		- Content requests work over Hyperswarm
483		- Peers can fetch missing content
484		- Content cached after fetch
	578	- Node A posts with image
	579	- Node B requests image from Node A (Protomux)
	580	- Node B caches image
	581	- Node C requests image from Node B (not A!)
	582	- Content propagates through network
485	583
486		Feature Complete: Full ANProto + content-by-hash implementation
	584	Critical: This makes EACH node act as a content relay!
	585	Result: Truly distributed P2P content network
487	586
488	587	### Phase 6: Wiredove Bridge (Optional)
489	588	Goal: Bridge with wiredove.net users
…	…	@@ -647,30 +746,41 @@ pear-app/test/
647	746
648	747	Updated: 2026-01-21
649	748
650		### 1. Content Storage: Hyperdrive (DECIDED ✓)
	749	### 1. Content Storage: Hyperbee (DECIDED ✓ - UPDATED)
651	750
652		Decision: Use Hyperdrive for ALL content (text + images)
	751	Decision: Use Hyperbee for content-by-hash storage (NOT Hyperdrive)
653	752
654	753	```
655	754	{storage-path}/cores/
656	755	├── my-feed/ # Hypercore: ANProto signatures only (132 bytes)
657		├── content-drive/ # Hyperdrive: ALL content by hash
658		│ ├── pZ/ # First 2 chars of hash
659		│ │ └── pZGm1Av0IEBKARczz7exkNYsZb8LzaMrV7J32a2fFG4=
660		│ └── aB/
661		│ └── aBcd2Ef3...
	756	├── content-cache/ # Hyperbee: Content by hash (key-value)
	757	│ └── data (B-tree)
	758	│ ├── "pZGm1Av0..." → "---\nname: fleeky\n---\nHello!"
	759	│ ├── "aBcd2Ef3..." → <base64-image-data>
	760	│ └── "xYz789AB..." → "---\nname: alice\n---\nPost text"
662	761	└── metadata/ # Hyperbee: username, follows, anproto-keypair
663	762	```
664	763
665	764	Rationale:
666		- ✅ Pear app already uses Hyperdrive for images
667		- ✅ Native P2P replication
668		- ✅ Sparse downloads (fetch only needed content)
669		- ✅ Efficient for all sizes (text and images)
670		- ✅ Consistent: one storage mechanism for all content
671
672		Alternative considered: content.json (original lib/core design)
673		Why rejected: Not P2P-native, poor for large files, loads everything into memory
	765	- ✅ Semantically correct: Hash → content is key-value, Hyperbee is for key-value
	766	- ✅ Not a hack: `put(hash, content)` vs Hyperdrive's `put('/pZ/pZGm...', content)`
	767	- ✅ Efficient lookups: B-tree O(log n) vs Hyperdrive path traversal
	768	- ✅ Hypercore-native: Built on Hypercore, integrates with Corestore
	769	- ✅ Can be partially replicated: If needed for advanced features
	770	- ✅ Larger-than-memory support: Hyperbee pages data to disk
	771	- ✅ Transactional: Hyperbee batches writes for consistency
	772
	773	Why NOT Hyperdrive:
	774	- ❌ Hyperdrive is for file systems (hierarchical directories)
	775	- ❌ Using paths as hashes (`/pZ/pZGm...`) is abusing the abstraction
	776	- ❌ File system operations (mkdir, readdir) not needed
	777	- ❌ More complex than needed for simple key-value
	778
	779	Why NOT content.json:
	780	- ❌ Not Hypercore-native (separate JSON file)
	781	- ❌ Entire file loaded into memory
	782	- ❌ Linear scan O(n) vs Hyperbee O(log n)
	783	- ❌ Not replicable
674	784
675	785	### 2. Identity: ANProto Only (DECIDED ✓)
676	786

wirecore commits / a63842b

Changed files (4) +1914 -117