OverviewFilesCommitsIssuesDocsReleasesPeers

wirecore commits / 19dc41a

Add storage migration and fix directory search
New features: - Add migrate-storage CLI command for account migration - Validates migrated data (identity, posts, follows, Hyperdrive) - Fix directory search regex to find names anywhere in YAML frontmatter - Add auto-detection for base64/hex pubkey formats in followUser() - Fix profile API proxy missing .end() call - Add storage-path debug command Documentation: - Add comprehensive storage migration guide (docs/STORAGE_MIGRATION.md) - Document Hypercore signing architecture - Explain storage paths for dev vs production mode Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
19dc41a50f7eca43bbbfefe514cec7bf4b92e61a
by plankatron <fleeky@gmail.com> · 2026-01-21T14:35:08.000Z

Changed files (3) +1329 -17

docs/STORAGE_MIGRATION.md +169 -0
……@@ -0,0 +1,169 @@
1# Storage Migration Guide
2
3## Storage Architecture
4
5### What Gets Stored
6
7Wirecore uses **Corestore** to manage all data storage. The storage directory contains:
8
91. **`primary-key`** (32 bytes)
10 - Corestore's master keypair
11 - Used to deterministically derive all named cores
12 - Critical for identity - must be preserved
13
142. **`cores/`** directory
15 - Subdirectories for each Hypercore (named by discovery key)
16 - Contains:
17 - **Identity core** (`name: 'identity'`) - IdentityKey seed
18 - **Feed core** (`name: 'my-feed'`) - User's posts with Hypercore signatures
19 - **Metadata core** (`name: 'metadata'`) - Hyperbee with username, follows
20 - **Hyperdrive cores** (`name: 'user-files'`) - Image storage
21 - **Followed feeds** - Replicated cores from people you follow
22
23### Storage Paths
24
25**Development mode** (`pear run -d .`):
26- Uses: `./wirecore-storage/` (local directory)
27- OR: `~/.config/pear/app-storage/by-random/<random-id>/` (new storage each run!)
28
29**Production mode** (`pear run pear://...`):
30- Uses: `~/.config/pear/app-storage/by-dkey/<discovery-key>/`
31- Discovery key is deterministic based on the pear:// link
32- SAME storage on every run
33
34## Message Signing
35
36### Hypercore Protocol Signing
37
38All messages in Wirecore are signed at the **Hypercore protocol level**:
39
401. Each Hypercore feed has its own ed25519 keypair (derived from primary-key)
412. When you append data: `myFeed.append(data)`
423. Hypercore automatically signs the block with the feed's secret key
434. Signatures are embedded in the Hypercore's data structure
445. On read, Hypercore verifies signatures against the feed's public key
45
46**Important**: Message signing is NOT done at the application level. It's handled by Hypercore itself.
47
48### ANProto (Not Used in Pear App)
49
50The `lib/core/anproto.js` module implements ANProto (timestamp + hash signing), but this is **not used** in the Pear app. The Pear app relies entirely on Hypercore's built-in signing.
51
52## Migration Process
53
54### What Migration Preserves
55
56When you migrate storage, you preserve:
57
581. **Identity** - Your public key remains the same
592. **Posts** - All your messages with valid signatures
603. **Follows** - List of people you follow
614. **Username** - Your display name
625. **Images** - All uploaded images in Hyperdrive
636. **Replicated feeds** - Downloaded posts from people you follow
64
65### How Migration Works
66
67```bash
68# Check current storage location
69pear run . cli storage-path
70
71# Migrate from local dev storage to production
72pear run . cli migrate-storage ./wirecore-storage
73```
74
75**Migration steps:**
76
771. **Validate source** - Checks for valid `primary-key` and `cores/` directory
782. **Close Corestore** - Stops the current instance if running
793. **Copy data** - Recursively copies:
80 - `primary-key` file (32 bytes)
81 - `cores/` directory (all subdirectories and files)
824. **Validate migration** - Loads the migrated data and verifies:
83 - Identity can be loaded
84 - Feed exists and shows post count
85 - Metadata Hyperbee works
86 - Username is preserved (if set)
87 - Follows are preserved
88 - Hyperdrive works and shows image count
89
90### Why Simple Copy Works
91
92Corestore is a **directory-based storage system**. When both the source and destination Corestores are closed:
93
94- Copying the `primary-key` preserves the master keypair
95- Named cores derive the same keypairs from the same primary-key
96- Copying `cores/` preserves all data with signatures intact
97- Hypercore signatures are in the core files, not computed at runtime
98
99This is similar to copying a Git repository - the cryptographic integrity is preserved because the signatures are embedded in the data structures.
100
101## Common Pitfalls
102
103### ❌ Don't Run Dev Mode in Production
104
105Running `pear run -d .` in production may use random storage:
106- Creates new storage at `/by-random/<id>/`
107- Loses all your data on restart
108- Creates new identity every time
109
110### ✅ Use Production Mode
111
112Running `pear run pear://...` uses deterministic storage:
113- Always uses same path: `/by-dkey/<discovery-key>/`
114- Data persists across restarts
115- Same identity every time
116
117### ⚠️ Migration Overwrites Destination
118
119If the destination storage already exists, migration will **overwrite** it. If you want to keep existing data, back it up first.
120
121## Troubleshooting
122
123### Lost Data After Restart
124
125**Symptom**: Posts and identity disappear after restarting
126
127**Cause**: Ran in dev mode with random storage
128
129**Solution**:
1301. Check for local storage: `ls -la ./wirecore-storage/`
1312. Migrate: `pear run . cli migrate-storage ./wirecore-storage`
1323. Use production mode: `pear run pear://...`
133
134### Identity Changed
135
136**Symptom**: Public key is different after migration
137
138**Cause**: Migration failed or copied wrong storage
139
140**Solution**:
1411. Verify source has correct data: `cat ./wirecore-storage/primary-key | wc -c` (should be 32)
1422. Check identity matches: `pear run -d . cli whoami`
1433. Re-migrate with correct source path
144
145### Migration Validation Failed
146
147**Symptom**: "Migration copied files but validation failed"
148
149**Cause**: Source data was corrupted or incomplete
150
151**Solution**:
1521. Check source directory has both `primary-key` and `cores/`
1532. Verify cores directory isn't empty: `ls -la ./wirecore-storage/cores/`
1543. Try starting with source storage to verify it works first
155
156## Future Improvements
157
158Potential enhancements to migration:
159
1601. **Backup before migration** - Automatically backup destination
1612. **Merge migrations** - Combine data from multiple sources
1623. **Export/Import** - Portable format for sharing identity
1634. **Cross-device sync** - Sync storage between machines
164
165## References
166
167- [Corestore Documentation](https://github.com/holepunchto/corestore)
168- [Hypercore Protocol](https://hypercore-protocol.org/)
169- [Pear Deployment Lessons](../PEAR_STAGING_GUIDE.md)
pear-app/index.js +1037 -17
……@@ -14,14 +14,36 @@
1414import Corestore from 'corestore'
1515import Hyperswarm from 'hyperswarm'
1616import Hyperbee from 'hyperbee'
17import Hyperdrive from 'hyperdrive'
1718import crypto from 'hypercore-crypto'
1819import b4a from 'b4a'
1920import goodbye from 'graceful-goodbye'
2021import IdentityKey from 'keet-identity-key'
22import http from 'bare-http1'
23import https from 'bare-https'
24import bareWs from 'bare-ws'
25import fs from 'bare-fs'
26import path from 'bare-path'
27import process from 'bare-process'
28
29const { Server: WebSocketServer } = bareWs
2130
2231// Check if running in Pear or Node
2332const isPear = typeof Pear !== 'undefined'
2433
34// Access the app bundle using pear-appdrive (read-only)
35let appDrive = null
36if (isPear) {
37 try {
38 const { default: AppDrive } = await import('pear-appdrive')
39 appDrive = new AppDrive()
40 await appDrive.ready()
41 console.log('[DEBUG] pear-appdrive ready - can read app bundle files')
42 } catch (err) {
43 console.log('[DEBUG] pear-appdrive failed:', err.message)
44 }
45}
46
2547// Configuration
2648const STORAGE_PATH = isPear && Pear.config.storage ? Pear.config.storage : './wirecore-storage'
2749
……@@ -30,6 +52,7 @@ let store = null
3052let swarm = null
3153let myFeed = null
3254let metaBee = null
55let userDrive = null // Hyperdrive for user files (images, etc.)
3356let identity = null
3457let follows = new Map() // discoveryKey -> { feed, publicKey, name }
3558
……@@ -69,13 +92,19 @@ async function loadState() {
6992 })
7093 await metaBee.ready()
7194
95 // User Hyperdrive will be created lazily when first image is uploaded
7296 return true
7397 } catch (err) {
7498 // Storage doesn't exist or is corrupted
7599 if (store) {
76100 try { await store.close() } catch {}
77 store = null
78101 }
102 // Reset all globals
103 store = null
104 myFeed = null
105 metaBee = null
106 userDrive = null
107 identity = null
79108 return null
80109 }
81110}
……@@ -148,6 +177,17 @@ async function saveFollows() {
148177 await metaBee.put('follows', followList)
149178}
150179
180/**
181 * Ensure user Hyperdrive is initialized (lazy loading)
182 */
183async function ensureUserDrive() {
184 if (userDrive) return userDrive
185
186 userDrive = new Hyperdrive(store, { name: 'user-files' })
187 await userDrive.ready()
188 return userDrive
189}
190
151191/**
152192 * Cleanup on exit
153193 */
……@@ -156,6 +196,10 @@ async function cleanup() {
156196 await swarm.destroy()
157197 swarm = null
158198 }
199 if (userDrive) {
200 await userDrive.close()
201 userDrive = null
202 }
159203 if (store) {
160204 await store.close()
161205 store = null
……@@ -200,10 +244,13 @@ async function cmdInit() {
200244 })
201245 await metaBee.ready()
202246 await metaBee.put('follows', [])
247 await metaBee.put('profile', { username: null })
203248
204249 console.log('✅ Wirecore initialized!')
205250 console.log('Your public key:', publicKey)
206251 console.log('Feed key:', b4a.toString(myFeed.key, 'hex'))
252 console.log('')
253 console.log('💡 Set your username with: pear run . setname "YourName"')
207254}
208255
209256/**
……@@ -216,18 +263,49 @@ async function cmdPost(message) {
216263 return
217264 }
218265
266 // Get username from profile
267 const profileNode = await metaBee.get('profile')
268 const username = profileNode && profileNode.value && profileNode.value.username
269
219270 const timestamp = Date.now()
220271 const entry = {
221272 type: 'post',
222273 text: message,
223274 timestamp,
224 author: b4a.toString(myFeed.key, 'hex')
275 author: b4a.toString(myFeed.key, 'hex'),
276 username: username || null
225277 }
226278
227279 await myFeed.append(Buffer.from(JSON.stringify(entry)))
228280 console.log('✅ Posted:', message)
229281}
230282
283/**
284 * Command: Set username
285 */
286async function cmdSetName(username) {
287 await loadState()
288 if (!metaBee) {
289 console.error('❌ Not initialized. Run: init')
290 return
291 }
292
293 if (!username || username.trim().length === 0) {
294 console.error('❌ Username cannot be empty')
295 return
296 }
297
298 // Get existing profile or create new one
299 const profileNode = await metaBee.get('profile')
300 const profile = (profileNode && profileNode.value) || {}
301
302 // Update username
303 profile.username = username.trim()
304 await metaBee.put('profile', profile)
305
306 console.log('✅ Username set to:', profile.username)
307}
308
231309/**
232310 * Command: Follow a user
233311 */
……@@ -404,6 +482,871 @@ async function cmdFollows() {
404482 }
405483}
406484
485async function cmdDirectory(query = '') {
486 const endpoint = query ? `/directory/search?q=${encodeURIComponent(query)}` : '/directory'
487
488 return new Promise((resolve) => {
489 const req = https.request(`https://7rnx.net/relay${endpoint}`, (res) => {
490 if (res.statusCode !== 200) {
491 console.error(`❌ Failed to fetch directory: HTTP ${res.statusCode}`)
492 resolve()
493 return
494 }
495
496 let data = ''
497 res.on('data', chunk => { data += chunk })
498 res.on('end', () => {
499 try {
500 const result = JSON.parse(data)
501
502 if (result.total === 0) {
503 console.log(query ? `\n🔍 No users found matching "${query}"` : '\n📭 No users in directory')
504 resolve()
505 return
506 }
507
508 console.log(query ? `\n🔍 Found ${result.total} users matching "${query}":\n` : `\n👥 Directory (${result.total} users):\n`)
509
510 for (const user of result.users) {
511 console.log(` ${user.name}`)
512 console.log(` Pubkey: ${user.pubkey}`)
513 console.log(` Last seen: ${new Date(parseInt(user.lastSeen)).toLocaleString()}`)
514 console.log('')
515 }
516
517 resolve()
518 } catch (err) {
519 console.error('❌ Error parsing directory response:', err.message)
520 resolve()
521 }
522 })
523 res.on('error', (err) => {
524 console.error('❌ Error fetching directory:', err.message)
525 resolve()
526 })
527 })
528
529 req.on('error', (err) => {
530 console.error('❌ Network error:', err.message)
531 resolve()
532 })
533
534 req.end()
535 })
536}
537
538async function cmdProfile(pubkey) {
539 if (!pubkey) {
540 console.error('❌ Usage: profile <pubkey>')
541 return
542 }
543
544 return new Promise((resolve) => {
545 const req = https.request(`https://7rnx.net/relay/directory/${encodeURIComponent(pubkey)}`, (res) => {
546 if (res.statusCode === 404) {
547 console.error(`❌ User not found: ${pubkey}`)
548 resolve()
549 return
550 }
551
552 if (res.statusCode !== 200) {
553 console.error(`❌ Failed to fetch profile: HTTP ${res.statusCode}`)
554 resolve()
555 return
556 }
557
558 let data = ''
559 res.on('data', chunk => { data += chunk })
560 res.on('end', () => {
561 try {
562 const profile = JSON.parse(data)
563
564 console.log(`\n👤 ${profile.name}\n`)
565 console.log(`Pubkey: ${profile.pubkey}`)
566 console.log(`Messages: ${profile.messageCount}`)
567 console.log(`Last seen: ${new Date(parseInt(profile.lastSeen)).toLocaleString()}`)
568 console.log(`\n📝 Recent posts:\n`)
569
570 for (const post of profile.recentPosts) {
571 // Remove YAML frontmatter for cleaner display
572 const text = post.text.replace(/^---\s*\n[\s\S]*?\n---\s*\n/, '')
573 const date = new Date(parseInt(post.timestamp)).toLocaleString()
574 console.log(` [${date}]`)
575 console.log(` ${text.substring(0, 200)}${text.length > 200 ? '...' : ''}`)
576 console.log('')
577 }
578
579 resolve()
580 } catch (err) {
581 console.error('❌ Error parsing profile response:', err.message)
582 resolve()
583 }
584 })
585 res.on('error', (err) => {
586 console.error('❌ Error fetching profile:', err.message)
587 resolve()
588 })
589 })
590
591 req.on('error', (err) => {
592 console.error('❌ Network error:', err.message)
593 resolve()
594 })
595
596 req.end()
597 })
598}
599
600/**
601 * Migrate storage from local directory to production path
602 */
603async function cmdMigrateStorage(sourcePath) {
604 if (!sourcePath) {
605 console.error('❌ Usage: cli migrate-storage <source-path>')
606 console.log('\nExample:')
607 console.log(' cli migrate-storage ./wirecore-storage')
608 console.log('\nThis will copy data from source to current storage path:', STORAGE_PATH)
609 return
610 }
611
612 // Resolve paths
613 const source = path.resolve(sourcePath)
614 const dest = path.resolve(STORAGE_PATH)
615
616 console.log('📦 Storage Migration')
617 console.log(' Source:', source)
618 console.log(' Destination:', dest)
619 console.log('')
620
621 // Check if source exists
622 try {
623 const stats = fs.statSync(source)
624 if (!stats.isDirectory()) {
625 console.error('❌ Source is not a directory:', source)
626 return
627 }
628 } catch (err) {
629 console.error('❌ Source path does not exist:', source)
630 return
631 }
632
633 // Check if source has valid Corestore structure
634 const primaryKeyPath = path.join(source, 'primary-key')
635 const coresPath = path.join(source, 'cores')
636
637 try {
638 fs.statSync(primaryKeyPath)
639 const coresStats = fs.statSync(coresPath)
640 if (!coresStats.isDirectory()) {
641 console.error('❌ Source does not have valid Corestore structure (cores/ not a directory)')
642 return
643 }
644 } catch (err) {
645 console.error('❌ Source does not have valid Corestore structure (missing primary-key or cores/)')
646 return
647 }
648
649 // Check if destination already exists
650 let destExists = false
651 try {
652 fs.statSync(dest)
653 destExists = true
654 } catch (err) {
655 // Destination doesn't exist - that's fine
656 }
657
658 if (destExists) {
659 console.log('⚠️ Destination already exists. Migration will OVERWRITE existing data.')
660 console.log(' If you want to keep existing data, cancel now and backup first.')
661 console.log('')
662 // In CLI mode, we'll proceed - user can Ctrl+C to cancel
663 // In a future version, we could add confirmation prompts
664 }
665
666 // Close current Corestore if open
667 if (store) {
668 console.log('🔄 Closing current Corestore...')
669 try {
670 await swarm.destroy()
671 swarm = null
672 } catch (err) {
673 console.log(' (swarm already closed)')
674 }
675 try {
676 await store.close()
677 store = null
678 myFeed = null
679 metaBee = null
680 userDrive = null
681 } catch (err) {
682 console.error('⚠️ Error closing store:', err.message)
683 }
684 }
685
686 // Create destination directory if needed
687 try {
688 const destParent = path.dirname(dest)
689 fs.mkdirSync(destParent, { recursive: true })
690 } catch (err) {
691 // Already exists
692 }
693
694 // Copy the data
695 console.log('📋 Copying storage data...')
696 try {
697 // Copy primary-key
698 const primaryKeySrc = path.join(source, 'primary-key')
699 const primaryKeyDest = path.join(dest, 'primary-key')
700 const primaryKeyData = fs.readFileSync(primaryKeySrc)
701 fs.writeFileSync(primaryKeyDest, primaryKeyData)
702 console.log(' ✅ Copied primary-key')
703
704 // Copy cores directory recursively
705 copyDirectoryRecursive(path.join(source, 'cores'), path.join(dest, 'cores'))
706 console.log(' ✅ Copied cores/')
707
708 console.log('')
709 console.log('✅ Storage copied successfully!')
710 console.log('')
711
712 // Validate the migration by trying to load the data
713 console.log('🔍 Validating migrated data...')
714 try {
715 const testStore = new Corestore(dest)
716 await testStore.ready()
717
718 // Check if identity can be loaded
719 const testIdentityCore = testStore.get({ name: 'identity' })
720 await testIdentityCore.ready()
721
722 if (testIdentityCore.length === 0) {
723 throw new Error('Identity core is empty')
724 }
725
726 const testIdentitySeed = await testIdentityCore.get(0)
727 const testIdentity = await IdentityKey.from({ seed: testIdentitySeed })
728 const publicKey = b4a.toString(testIdentity.identityPublicKey, 'hex')
729
730 console.log(' ✅ Identity verified:', publicKey.substring(0, 16) + '...')
731
732 // Check if feed exists
733 const testFeed = testStore.get({ name: 'my-feed' })
734 await testFeed.ready()
735 console.log(' ✅ Feed verified:', testFeed.length, 'posts')
736
737 // Check if metadata exists
738 const testMetaBee = new Hyperbee(testStore.get({ name: 'metadata' }), {
739 keyEncoding: 'utf-8',
740 valueEncoding: 'json'
741 })
742 await testMetaBee.ready()
743
744 const profileNode = await testMetaBee.get('profile')
745 const username = profileNode && profileNode.value && profileNode.value.username
746 if (username) {
747 console.log(' ✅ Username verified:', username)
748 }
749
750 // Count follows
751 let followCount = 0
752 for await (const node of testMetaBee.createReadStream({ gte: 'follow:', lt: 'follow;\xff' })) {
753 followCount++
754 }
755 console.log(' ✅ Follows verified:', followCount)
756
757 // Check if Hyperdrive exists and has files
758 try {
759 const testDrive = new Hyperdrive(testStore, { name: 'user-files' })
760 await testDrive.ready()
761
762 let imageCount = 0
763 for await (const entry of testDrive.list('/images')) {
764 imageCount++
765 }
766
767 if (imageCount > 0) {
768 console.log(' ✅ Hyperdrive verified:', imageCount, 'images')
769 } else {
770 console.log(' ✅ Hyperdrive verified (no images yet)')
771 }
772
773 await testDrive.close()
774 } catch (driveErr) {
775 console.log(' ⚠️ Hyperdrive check skipped:', driveErr.message)
776 }
777
778 await testStore.close()
779
780 console.log('')
781 console.log('✅ Migration complete and validated!')
782 console.log('')
783 console.log('Your identity, posts, and follows have been successfully migrated.')
784 console.log('Storage location:', dest)
785 console.log('')
786 console.log('You can now restart the app and it will use the migrated data.')
787
788 } catch (validationErr) {
789 console.error('⚠️ Migration copied files but validation failed:', validationErr.message)
790 console.log('')
791 console.log('The data was copied, but there may be an issue.')
792 console.log('You can try restarting the app to see if it works.')
793 }
794
795 } catch (err) {
796 console.error('❌ Migration failed:', err.message)
797 console.error(err.stack)
798 }
799}
800
801/**
802 * Recursively copy a directory
803 */
804function copyDirectoryRecursive(src, dest) {
805 // Create destination directory
806 try {
807 fs.mkdirSync(dest, { recursive: true })
808 } catch (err) {
809 if (err.code !== 'EEXIST') throw err
810 }
811
812 // Read source directory
813 const entries = fs.readdirSync(src)
814
815 for (const entry of entries) {
816 const srcPath = path.join(src, entry)
817 const destPath = path.join(dest, entry)
818
819 const stats = fs.statSync(srcPath)
820
821 if (stats.isDirectory()) {
822 copyDirectoryRecursive(srcPath, destPath)
823 } else {
824 const data = fs.readFileSync(srcPath)
825 fs.writeFileSync(destPath, data)
826 }
827 }
828}
829
830/**
831 * Web Server - HTTP + WebSocket
832 */
833let wss = null
834const wsClients = new Set()
835
836async function startWebServer(port = 3003) {
837 await loadState()
838
839 if (!identity) {
840 console.error('❌ Not initialized. Run: pear run . init')
841 return
842 }
843
844 // Start P2P networking
845 await startSwarm()
846 await loadFollows()
847
848 // Create HTTP server
849 const server = http.createServer(async (req, res) => {
850 const url = new URL(req.url, 'http://localhost')
851
852 // CORS headers
853 res.setHeader('Access-Control-Allow-Origin', '*')
854 res.setHeader('Access-Control-Allow-Methods', 'GET, POST, OPTIONS')
855 res.setHeader('Access-Control-Allow-Headers', 'Content-Type')
856
857 if (req.method === 'OPTIONS') {
858 res.writeHead(200)
859 res.end()
860 return
861 }
862
863 // API routes
864 if (url.pathname.startsWith('/api/')) {
865 await handleAPI(req, res, url)
866 return
867 }
868
869 // Serve static files from public/
870 await serveStatic(req, res, url)
871 })
872
873 // Create WebSocket server
874 wss = new WebSocketServer({ server })
875
876 wss.on('connection', (ws) => {
877 wsClients.add(ws)
878 console.log('🔌 Browser connected, total:', wsClients.size)
879
880 ws.on('close', () => {
881 wsClients.delete(ws)
882 console.log('👋 Browser disconnected, total:', wsClients.size)
883 })
884
885 ws.on('error', (err) => {
886 console.error('WebSocket error:', err.message)
887 wsClients.delete(ws)
888 })
889 })
890
891 // Listen for feed events and broadcast to browsers
892 myFeed.on('append', () => {
893 broadcast({ type: 'new_post', feed: 'own' })
894 })
895
896 // Start server
897 server.listen(port, '0.0.0.0', () => {
898 console.log('🚀 Wirecore Web Server running!')
899 console.log('')
900 console.log(`📍 Local: http://localhost:${port}`)
901 console.log(`🔌 WebSocket: ws://localhost:${port}`)
902 console.log(`👤 Public key: ${b4a.toString(identity.identityPublicKey, 'hex')}`)
903 console.log(`📡 Following: ${follows.size} users`)
904 console.log(`📂 Public files: ./public (via import.meta.asset)`)
905 console.log('')
906 console.log('')
907 console.log('✨ Open in your browser and start posting!')
908 })
909}
910
911// Broadcast message to all connected browsers
912function broadcast(message) {
913 const data = JSON.stringify(message)
914 for (const client of wsClients) {
915 if (client.readyState === 1) { // OPEN
916 try {
917 client.send(data)
918 } catch (err) {
919 console.error('Failed to send to client:', err.message)
920 }
921 }
922 }
923}
924
925// Handle API requests
926async function handleAPI(req, res, url) {
927 const route = url.pathname
928
929 try {
930 switch (route) {
931 case '/api/whoami':
932 res.writeHead(200, { 'Content-Type': 'application/json' })
933 res.end(JSON.stringify({
934 pubkey: b4a.toString(identity.identityPublicKey, 'hex'),
935 feedKey: b4a.toString(myFeed.key, 'hex'),
936 discoveryKey: b4a.toString(myFeed.discoveryKey, 'hex')
937 }))
938 break
939
940 case '/api/profile':
941 if (req.method === 'GET') {
942 const profileNode = await metaBee.get('profile')
943 const profile = (profileNode && profileNode.value) || {}
944
945 res.writeHead(200, { 'Content-Type': 'application/json' })
946 res.end(JSON.stringify({
947 pubkey: b4a.toString(identity.identityPublicKey, 'hex'),
948 username: profile.username || null
949 }))
950 } else if (req.method === 'POST') {
951 const body = await readBody(req)
952 const { username } = JSON.parse(body)
953
954 if (!username || username.trim().length === 0) {
955 res.writeHead(400, { 'Content-Type': 'application/json' })
956 res.end(JSON.stringify({ error: 'Username cannot be empty' }))
957 return
958 }
959
960 const profileNode = await metaBee.get('profile')
961 const profile = (profileNode && profileNode.value) || {}
962 profile.username = username.trim()
963 await metaBee.put('profile', profile)
964
965 res.writeHead(200, { 'Content-Type': 'application/json' })
966 res.end(JSON.stringify({ success: true, username: profile.username }))
967 }
968 break
969
970 case '/api/timeline':
971 const posts = await getTimelineMessages(50)
972 res.writeHead(200, { 'Content-Type': 'application/json' })
973 res.end(JSON.stringify(posts))
974 break
975
976 case '/api/post':
977 if (req.method === 'POST') {
978 const body = await readBody(req)
979 const { message, imageData, imageMimeType } = JSON.parse(body)
980
981 // Get username from profile
982 const profileNode = await metaBee.get('profile')
983 const username = profileNode && profileNode.value && profileNode.value.username
984
985 const entry = {
986 text: message,
987 timestamp: Date.now(),
988 username: username || null
989 }
990
991 // Add image data if provided (stored inline as base64)
992 if (imageData) {
993 entry.imageData = imageData
994 entry.imageMimeType = imageMimeType || 'image/jpeg'
995 }
996
997 await myFeed.append(Buffer.from(JSON.stringify(entry)))
998
999 res.writeHead(200, { 'Content-Type': 'application/json' })
1000 res.end(JSON.stringify({ success: true }))
1001 }
1002 break
1003
1004 case '/api/follow':
1005 if (req.method === 'POST') {
1006 const body = await readBody(req)
1007 const { publicKey, name } = JSON.parse(body)
1008
1009 await followUser(publicKey, name)
1010
1011 res.writeHead(200, { 'Content-Type': 'application/json' })
1012 res.end(JSON.stringify({ success: true }))
1013 }
1014 break
1015
1016 case '/api/unfollow':
1017 if (req.method === 'POST') {
1018 const body = await readBody(req)
1019 const { publicKey } = JSON.parse(body)
1020
1021 const pubKeyBuf = b4a.from(publicKey, 'hex')
1022 const feedKey = crypto.discoveryKey(pubKeyBuf)
1023 const discoveryKeyHex = b4a.toString(feedKey, 'hex')
1024
1025 follows.delete(discoveryKeyHex)
1026 await saveFollows()
1027
1028 res.writeHead(200, { 'Content-Type': 'application/json' })
1029 res.end(JSON.stringify({ success: true }))
1030 }
1031 break
1032
1033 case '/api/follows':
1034 const followsList = Array.from(follows.entries()).map(([dk, { publicKey, name, feed }]) => ({
1035 publicKey,
1036 name,
1037 messageCount: feed.length
1038 }))
1039
1040 res.writeHead(200, { 'Content-Type': 'application/json' })
1041 res.end(JSON.stringify(followsList))
1042 break
1043
1044 case '/api/directory':
1045 // Proxy directory search requests to relay
1046 {
1047 const query = url.searchParams.get('q') || ''
1048 const relayUrl = query
1049 ? `https://7rnx.net/relay/directory/search?q=${encodeURIComponent(query)}`
1050 : 'https://7rnx.net/relay/directory'
1051
1052 https.request(relayUrl, (relayRes) => {
1053 res.writeHead(relayRes.statusCode, {
1054 'Content-Type': 'application/json'
1055 })
1056 relayRes.pipe(res)
1057 }).on('error', (err) => {
1058 console.error('Relay request failed:', err)
1059 res.writeHead(500, { 'Content-Type': 'application/json' })
1060 res.end(JSON.stringify({ error: 'Failed to fetch directory' }))
1061 }).end()
1062 }
1063 break
1064
1065 case '/api/upload':
1066 if (req.method === 'POST') {
1067 const body = await readBody(req)
1068 const { image, filename, mimeType } = JSON.parse(body)
1069
1070 // Store images inline in posts as base64 (for P2P replication)
1071 // Also store in Hyperdrive for backward compatibility with old posts
1072 const imageData = b4a.from(image, 'base64')
1073 const hash = b4a.toString(crypto.hash(imageData), 'hex').substring(0, 16)
1074 const ext = filename ? filename.split('.').pop() : 'jpg'
1075 const imagePath = `/images/${hash}.${ext}`
1076
1077 // Lazy-load userDrive and store (for backward compatibility)
1078 try {
1079 await ensureUserDrive()
1080 await userDrive.put(imagePath, imageData)
1081 } catch (err) {
1082 console.error('Failed to store in Hyperdrive:', err)
1083 }
1084
1085 res.writeHead(200, { 'Content-Type': 'application/json' })
1086 res.end(JSON.stringify({ success: true, imageData: image, mimeType }))
1087 }
1088 break
1089
1090 default:
1091 // Check if it's a directory profile request
1092 if (url.pathname.startsWith('/api/directory/')) {
1093 const pubkey = url.pathname.substring('/api/directory/'.length)
1094 const relayUrl = `https://7rnx.net/relay/directory/${encodeURIComponent(pubkey)}`
1095
1096 const proxyReq = https.request(relayUrl, (relayRes) => {
1097 res.writeHead(relayRes.statusCode, {
1098 'Content-Type': 'application/json'
1099 })
1100 relayRes.pipe(res)
1101 })
1102
1103 proxyReq.on('error', (err) => {
1104 console.error('Relay request failed:', err)
1105 res.writeHead(500, { 'Content-Type': 'application/json' })
1106 res.end(JSON.stringify({ error: 'Failed to fetch profile' }))
1107 })
1108
1109 proxyReq.end()
1110 return
1111 }
1112
1113 // Check if it's an image request (for backward compatibility with old posts)
1114 if (url.pathname.startsWith('/api/image/')) {
1115 const imagePath = url.pathname.replace('/api/image', '')
1116 try {
1117 // Lazy-load userDrive if needed
1118 await ensureUserDrive()
1119
1120 const imageData = await userDrive.get(imagePath)
1121 if (!imageData) {
1122 res.writeHead(404, { 'Content-Type': 'text/plain' })
1123 res.end('Image not found')
1124 return
1125 }
1126
1127 // Determine content type from extension
1128 const ext = imagePath.split('.').pop().toLowerCase()
1129 const contentType = {
1130 jpg: 'image/jpeg',
1131 jpeg: 'image/jpeg',
1132 png: 'image/png',
1133 gif: 'image/gif',
1134 webp: 'image/webp',
1135 svg: 'image/svg+xml'
1136 }[ext] || 'application/octet-stream'
1137
1138 res.writeHead(200, { 'Content-Type': contentType })
1139 res.end(imageData)
1140 } catch (err) {
1141 console.error('Error loading image:', err)
1142 res.writeHead(500, { 'Content-Type': 'text/plain' })
1143 res.end('Error loading image')
1144 }
1145 return
1146 }
1147
1148 res.writeHead(404, { 'Content-Type': 'application/json' })
1149 res.end(JSON.stringify({ error: 'Not found' }))
1150 }
1151 } catch (err) {
1152 console.error('API error:', err)
1153 res.writeHead(500, { 'Content-Type': 'application/json' })
1154 res.end(JSON.stringify({ error: err.message }))
1155 }
1156}
1157
1158// Read request body
1159function readBody(req) {
1160 return new Promise((resolve, reject) => {
1161 let body = ''
1162 req.on('data', chunk => { body += chunk.toString() })
1163 req.on('end', () => resolve(body))
1164 req.on('error', reject)
1165 })
1166}
1167
1168// Serve static files
1169async function serveStatic(req, res, url) {
1170 // Remove leading slash and default to index.html
1171 let fileName = url.pathname === '/' ? 'index.html' : url.pathname.substring(1)
1172
1173 console.log(`[STATIC] Request: ${url.pathname} → ${fileName}`)
1174
1175 try {
1176 let data
1177
1178 if (appDrive) {
1179 // Running from pear:// link - read from app bundle using pear-appdrive
1180 const filePath = '/public/' + fileName
1181 console.log(`[STATIC] Reading from app bundle: ${filePath}`)
1182 data = await appDrive.get(filePath)
1183 if (!data) throw new Error('File not found in app bundle')
1184 } else {
1185 // Local development - read from filesystem
1186 const localPath = path.join('public', fileName)
1187 console.log(`[STATIC] Reading from filesystem: ${localPath}`)
1188 data = fs.readFileSync(localPath)
1189 }
1190
1191 console.log(`[STATIC] ✅ Success: ${fileName} (${data.length} bytes)`)
1192
1193 const ext = path.extname(fileName)
1194 const contentType = {
1195 '.html': 'text/html',
1196 '.css': 'text/css',
1197 '.js': 'application/javascript',
1198 '.json': 'application/json',
1199 '.svg': 'image/svg+xml'
1200 }[ext] || 'text/plain'
1201
1202 res.writeHead(200, { 'Content-Type': contentType })
1203 res.end(data)
1204 } catch (err) {
1205 console.error(`[STATIC] ❌ Failed: ${fileName}`)
1206 console.error(`[STATIC] Error: ${err.message}`)
1207 res.writeHead(404, { 'Content-Type': 'text/plain' })
1208 res.end('Not found')
1209 }
1210}
1211
1212// Get timeline messages (helper for API)
1213async function getTimelineMessages(limit) {
1214 const messages = []
1215
1216 // Read own feed
1217 for (let i = Math.max(0, myFeed.length - limit); i < myFeed.length; i++) {
1218 try {
1219 const block = await myFeed.get(i, { wait: false })
1220 if (!block) continue
1221 const entry = JSON.parse(block.toString())
1222
1223 // Use username from post if available, otherwise "You"
1224 const displayName = entry.username || 'You'
1225
1226 messages.push({
1227 ...entry,
1228 author: displayName,
1229 publicKey: b4a.toString(identity.identityPublicKey, 'hex'),
1230 isOwn: true
1231 })
1232 } catch (err) {
1233 // Skip
1234 }
1235 }
1236
1237 // Read followed feeds
1238 for (const [discoveryKey, { feed, name, publicKey }] of follows.entries()) {
1239 try {
1240 await feed.update()
1241 const feedLimit = Math.min(limit, feed.length)
1242 for (let i = Math.max(0, feed.length - feedLimit); i < feed.length; i++) {
1243 try {
1244 const block = await feed.get(i, { wait: true, timeout: 2000 })
1245 if (!block) continue
1246 const entry = JSON.parse(block.toString())
1247
1248 // Use username from post if available, otherwise follow name or pubkey
1249 const displayName = entry.username || name || publicKey.substring(0, 16)
1250
1251 messages.push({
1252 ...entry,
1253 author: displayName,
1254 publicKey,
1255 isOwn: false
1256 })
1257 } catch (err) {
1258 continue
1259 }
1260 }
1261 } catch (err) {
1262 continue
1263 }
1264 }
1265
1266 // Sort by timestamp descending
1267 messages.sort((a, b) => b.timestamp - a.timestamp)
1268
1269 return messages.slice(0, limit)
1270}
1271
1272// Helper function to follow a user (extracted for API use)
1273async function followUser(publicKeyInput, name) {
1274 // Auto-detect format: base64 (from directory/wiredove) or hex (wirecore native)
1275 let pubKeyBuf
1276 let publicKeyHex
1277
1278 if (publicKeyInput.includes('=') || publicKeyInput.includes('+') || publicKeyInput.includes('/')) {
1279 // Base64 format (from directory)
1280 pubKeyBuf = b4a.from(publicKeyInput, 'base64')
1281 publicKeyHex = b4a.toString(pubKeyBuf, 'hex')
1282 } else {
1283 // Hex format (wirecore native)
1284 publicKeyHex = publicKeyInput
1285 pubKeyBuf = b4a.from(publicKeyHex, 'hex')
1286 }
1287
1288 const feedKey = crypto.discoveryKey(pubKeyBuf)
1289 const discoveryKeyHex = b4a.toString(feedKey, 'hex')
1290
1291 if (follows.has(discoveryKeyHex)) {
1292 throw new Error('Already following this user')
1293 }
1294
1295 swarm.join(feedKey, { server: false, client: true })
1296
1297 const feed = store.get(pubKeyBuf)
1298 await feed.ready()
1299
1300 follows.set(discoveryKeyHex, {
1301 feed,
1302 publicKey: publicKeyHex,
1303 name: name || publicKeyHex.substring(0, 16)
1304 })
1305
1306 await saveFollows()
1307 broadcast({ type: 'new_follow', publicKey: publicKeyHex, name })
1308}
1309
1310/**
1311 * Show usage information
1312 */
1313function showUsage() {
1314 console.log('🕊️ Wirecore - P2P Social Network')
1315 console.log('')
1316 console.log('Usage:')
1317 console.log(' pear run . cli Start in CLI mode (interactive commands)')
1318 console.log(' pear run . web [port] Start web server (default port: 3003)')
1319 console.log('')
1320 console.log('CLI Commands:')
1321 console.log(' pear run . cli init Initialize new identity')
1322 console.log(' pear run . cli post <text> Post a message')
1323 console.log(' pear run . cli setname <name> Set your username')
1324 console.log(' pear run . cli follow <pubkey> [name] Follow a user')
1325 console.log(' pear run . cli unfollow <pubkey> Unfollow a user')
1326 console.log(' pear run . cli timeline [limit] View timeline')
1327 console.log(' pear run . cli whoami Show your public key')
1328 console.log(' pear run . cli follows List followed users')
1329 console.log(' pear run . cli directory [query] Search user directory')
1330 console.log(' pear run . cli profile <pubkey> View user profile')
1331 console.log(' pear run . cli storage-path Show current storage location')
1332 console.log(' pear run . cli migrate-storage <path> Migrate data from another storage')
1333 console.log('')
1334 console.log('Examples:')
1335 console.log(' pear run . web 3003')
1336 console.log(' pear run . cli init')
1337 console.log(' pear run . cli post "Hello, P2P world!"')
1338 console.log(' pear run . cli timeline 50')
1339 console.log(' pear run . cli directory ev')
1340 console.log(' pear run . cli profile 5Skd4SFHZkY5Syf6RiiSEvBlEHiJQvrffBnw0VKacB4=')
1341 console.log('')
1342 console.log('Migration:')
1343 console.log(' # Check current storage location')
1344 console.log(' pear run . cli storage-path')
1345 console.log('')
1346 console.log(' # Migrate from local dev storage to production')
1347 console.log(' pear run . cli migrate-storage ./wirecore-storage')
1348}
1349
4071350/**
4081351 * Main CLI handler
4091352 */
……@@ -411,23 +1354,93 @@ async function main() {
4111354 const args = isPear ? (Pear.config.args || []) : process.argv.slice(2)
4121355
4131356 if (args.length === 0) {
414 console.log('Usage: ' + (isPear ? 'pear run .' : 'node index.js') + ' <command> [args...]')
415 console.log('')
416 console.log('Commands:')
417 console.log(' init - Initialize Wirecore')
418 console.log(' post <message> - Post a message')
419 console.log(' follow <pubkey> [name] - Follow a user')
420 console.log(' unfollow <pubkey> - Unfollow a user')
421 console.log(' timeline [limit] - View timeline')
422 console.log(' whoami - Show your public key')
423 console.log(' follows - List followed users')
1357 showUsage()
4241358 return
4251359 }
4261360
427 const [command, ...params] = args
1361 const [mode, ...params] = args
1362
1363 // Long-running commands that should NOT cleanup
1364 const longRunningCommands = ['web']
1365 const shouldCleanup = !longRunningCommands.includes(mode)
4281366
4291367 try {
430 switch (command) {
1368 switch (mode) {
1369 case 'web':
1370 // Start web server with optional custom port: pear run . web 8080
1371 const customPort = params[0] ? parseInt(params[0]) : 3003
1372 await startWebServer(customPort)
1373 return // Keep server running - cleanup handled by graceful-goodbye
1374
1375 case 'cli':
1376 // CLI mode - execute command
1377 if (params.length === 0) {
1378 showUsage()
1379 return
1380 }
1381
1382 const [command, ...cmdParams] = params
1383
1384 switch (command) {
1385 case 'init':
1386 await cmdInit()
1387 break
1388
1389 case 'post':
1390 await cmdPost(cmdParams.join(' '))
1391 break
1392
1393 case 'setname':
1394 await cmdSetName(cmdParams.join(' '))
1395 break
1396
1397 case 'follow':
1398 await cmdFollow(cmdParams[0], cmdParams[1])
1399 break
1400
1401 case 'unfollow':
1402 await cmdUnfollow(cmdParams[0])
1403 break
1404
1405 case 'timeline':
1406 await cmdTimeline(parseInt(cmdParams[0]) || 20)
1407 break
1408
1409 case 'whoami':
1410 await cmdWhoami()
1411 break
1412
1413 case 'follows':
1414 await cmdFollows()
1415 break
1416
1417 case 'directory':
1418 await cmdDirectory(cmdParams.join(' '))
1419 break
1420
1421 case 'profile':
1422 await cmdProfile(cmdParams[0])
1423 break
1424
1425 case 'storage-path':
1426 console.log('Storage path:', STORAGE_PATH)
1427 console.log('Is Pear:', isPear)
1428 if (isPear) {
1429 console.log('Pear.config.storage:', Pear.config.storage)
1430 }
1431 break
1432
1433 case 'migrate-storage':
1434 await cmdMigrateStorage(cmdParams[0])
1435 break
1436
1437 default:
1438 console.log('Unknown command:', command)
1439 showUsage()
1440 }
1441 break
1442
1443 // Legacy support for old commands without 'cli' prefix
4311444 case 'init':
4321445 await cmdInit()
4331446 break
……@@ -436,6 +1449,10 @@ async function main() {
4361449 await cmdPost(params.join(' '))
4371450 break
4381451
1452 case 'setname':
1453 await cmdSetName(params.join(' '))
1454 break
1455
4391456 case 'follow':
4401457 await cmdFollow(params[0], params[1])
4411458 break
……@@ -457,14 +1474,17 @@ async function main() {
4571474 break
4581475
4591476 default:
460 console.log('Unknown command:', command)
461 console.log('Run without arguments to see usage')
1477 console.log('Unknown mode:', mode)
1478 showUsage()
4621479 }
4631480 } catch (err) {
4641481 console.error('❌ Error:', err.message)
4651482 console.error(err.stack)
4661483 } finally {
467 await cleanup()
1484 // Only cleanup for short-lived CLI commands
1485 if (shouldCleanup) {
1486 await cleanup()
1487 }
4681488 }
4691489}
4701490
relay/wirecore-relay.js +123 -0
……@@ -330,6 +330,129 @@ const handleHTTP = async (req, res) => {
330330 return
331331 }
332332
333 // Helper function to extract name from YAML frontmatter
334 const extractName = (text) => {
335 if (!text) return null
336 // Match name: field anywhere in YAML frontmatter (between --- markers)
337 const nameMatch = text.match(/^---[\s\S]*?\nname:\s*['"]?([^'"\n]+)['"]?/m)
338 return nameMatch ? nameMatch[1].trim() : null
339 }
340
341 // GET /directory/{pubkey} - view user profile with recent posts
342 if (key.startsWith('directory/') && key !== 'directory/search') {
343 try {
344 const pubkey = key.substring('directory/'.length)
345 const messages = await apds.query(pubkey) // Get all messages from this user
346
347 if (messages.length === 0) {
348 res.writeHead(404, { 'Content-Type': 'application/json' })
349 res.end(JSON.stringify({ error: 'User not found' }))
350 return
351 }
352
353 // Extract name from latest message
354 let name = null
355 for (let i = messages.length - 1; i >= 0 && !name; i--) {
356 name = extractName(messages[i].text)
357 }
358
359 // Get recent posts (last 10)
360 const recentPosts = messages.slice(-10).reverse().map(msg => ({
361 text: msg.text,
362 timestamp: msg.ts,
363 hash: msg.hash
364 }))
365
366 res.writeHead(200, { 'Content-Type': 'application/json' })
367 res.end(JSON.stringify({
368 pubkey: pubkey,
369 name: name || 'Unknown',
370 messageCount: messages.length,
371 lastSeen: messages[messages.length - 1]?.ts,
372 recentPosts: recentPosts
373 }, null, 2))
374 } catch (err) {
375 res.writeHead(500, { 'Content-Type': 'application/json' })
376 res.end(JSON.stringify({ error: err.message }))
377 }
378 return
379 }
380
381 // GET /directory/search?q=name - search users
382 if (key === 'directory/search') {
383 try {
384 const query = url.searchParams.get('q') || ''
385 const pubkeys = await apds.getPubkeys()
386 const users = []
387
388 // Get latest message from each user for their name
389 for (const pubkey of pubkeys) {
390 const latest = await apds.getLatest(pubkey)
391 if (latest && latest.text) {
392 const name = extractName(latest.text)
393 if (name) {
394 // Filter by query if provided
395 if (!query || name.toLowerCase().includes(query.toLowerCase())) {
396 users.push({
397 pubkey: pubkey,
398 name: name,
399 lastSeen: latest.ts
400 })
401 }
402 }
403 }
404 }
405
406 users.sort((a, b) => b.lastSeen - a.lastSeen)
407
408 res.writeHead(200, { 'Content-Type': 'application/json' })
409 res.end(JSON.stringify({
410 query: query,
411 total: users.length,
412 users: users
413 }, null, 2))
414 } catch (err) {
415 res.writeHead(500, { 'Content-Type': 'application/json' })
416 res.end(JSON.stringify({ error: err.message }))
417 }
418 return
419 }
420
421 // GET /directory - list all users
422 if (key === 'directory') {
423 try {
424 const pubkeys = await apds.getPubkeys()
425 const users = []
426
427 // Get latest message from each user for their name
428 for (const pubkey of pubkeys) {
429 const latest = await apds.getLatest(pubkey)
430 if (latest && latest.text) {
431 const name = extractName(latest.text)
432 if (name) {
433 users.push({
434 pubkey: pubkey,
435 name: name,
436 lastSeen: latest.ts
437 })
438 }
439 }
440 }
441
442 users.sort((a, b) => b.lastSeen - a.lastSeen)
443
444 res.writeHead(200, { 'Content-Type': 'application/json' })
445 res.end(JSON.stringify({
446 total: users.length,
447 users: users
448 }, null, 2))
449 } catch (err) {
450 res.writeHead(500, { 'Content-Type': 'application/json' })
451 res.end(JSON.stringify({ error: err.message }))
452 }
453 return
454 }
455
333456 // GET /config - view current configuration
334457 // NOTE: This endpoint should be protected in production (firewall, auth, etc.)
335458 // Currently only accessible via localhost or internal network