# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Repository Purpose
This is a **triple knowledge base** combining:
1. **holepunchto/** - Complete analysis of 529 holepunchto P2P ecosystem repositories
2. **tetherto-kb/** - Tether WDK (Wallet Development Kit) ecosystem analysis (18 repos)
3. **hyper-fleeky/** - Personal projects built using the holepunchto and WDK stack
The repository serves as both a comprehensive reference system and active development workspace for P2P applications using Hypercore, Hyperswarm, Pear, Tether WDK, and related technologies.
## System Context
**CRITICAL: You are running on tabertron.local**
- Hostname: tabertron.local
- You have direct local access - DO NOT use SSH to connect to tabertron
- Run all commands directly without SSH
- **This is the Pear development machine** - All `pear stage` and `pear seed` commands run here
- Other machines (ashatron, 7rnx.net) are remote and consume the pear:// releases
## Key Commands
### Working with holepunchto Knowledge Base
```bash
cd holepunchto
# Discover repos for a use case
node designer-agent.js "distributed database"
node designer-agent.js --use-case p2p-file-sharing
node designer-agent.js --list-use-cases
# Design system architecture
node architect-agent.js hypercore hyperbee hyperswarm
# Update knowledge base
node process_repos.js # Phase 1A: GitHub API analysis
node clone_and_analyze_tier.js 1 # Phase 1B: Deep code analysis
node build_semantic_maps.js # Phase 2: Semantic mappings
```
### Working with tetherto-kb (Tether WDK) Knowledge Base
The tetherto-kb follows the same structure as holepunchto:
```bash
cd tetherto-kb
# Explore use cases (9 use cases)
cat use-case-map.json | jq '.useCases | keys'
cat use-case-map.json | jq '.useCases["multi-chain-wallet"]'
cat use-case-map.json | jq '.useCases["pear-wallet-app"]' # hyper-wallet use case
# Find implementation patterns (12 patterns)
cat pattern-map.json | jq '.patterns | keys'
cat pattern-map.json | jq '.patterns["pear-integration"]'
cat pattern-map.json | jq '.patterns["bare-worklet"]'
# Read skills documentation (3 created so far)
cat skills/wdk-core/skills.md # WDK Core API reference (431 lines)
cat skills/wdk-wallet-btc/skills.md # Bitcoin wallet API (621 lines)
cat skills/wdk-wallet-evm/skills.md # EVM wallet API (618 lines)
ls skills/ # All available skills
# Clone repositories (10 tier 1 repos)
./clone_tier1.sh # Clone: wdk-core, wallets, pear-wrk-wdk
# Check status and progress
cat STATUS.md # Build status, discoveries, next steps
cat README.md # Overview and navigation
```
**Key Tetherto Discoveries:**
- Lightning Network: Uses **Spark blockchain** (not traditional BTC Lightning)
- Pear Integration: `pear-wrk-wdk` provides Bare worklets for crypto isolation
- Multi-chain: Bitcoin, Spark, Ethereum/EVM, TON, TRON, Solana + USDT
### Testing holepunchto Repos
Most holepunchto repos use **brittle** as the test framework:
```bash
# Node.js testing
npm test
brittle test/all.js
# Bare runtime testing
npm run test:bare
bare test/all.js
```
### Working with hyper-fleeky Projects
The personal projects are git submodules in `hyper-fleeky/repos/`:
```bash
# Initialize submodules
git submodule update --init hyper-fleeky/repos/hyperident
git submodule update --init hyper-fleeky/repos/hyperident-gateway
git submodule update --init hyper-fleeky/repos/fedicore
# Each project has its own workflow
cd hyper-fleeky/repos/hyperident
npm test # Run brittle tests
pear run -d . # Run in Pear development mode
cd hyper-fleeky/repos/hyperident-gateway
npm test # Run tests
sudo node gateway.js # Run locally (needs root for TUN)
cd hyper-fleeky/repos/fedicore
npm start # CLI mode
npm run web # Web interface on port 3000
```
### Working with hyper-wallet
hyper-wallet combines Holepunchto P2P infrastructure with Tether WDK cryptocurrency wallets:
```bash
cd hyper-fleeky/hyper-wallet
# Run tests (requires Bare runtime)
bare test/protocols/wdk.test.js # Test WDK integration
bare test/lib/*.test.js # Test library functions
npm run test:protocols # All protocol tests
npm run test:lib # All lib tests
# Build bundles (using bare-pack)
npm run bundle # Current platform
npm run bundle:linux # Linux x64
npm run bundle:macos # macOS ARM64
npm run bundle:all # All platforms
# Pear deployment
npm run pear:stage # Stage to Pear DHT
npm run pear:seed # Seed the release
# Or use pear commands directly:
pear stage hyper-wallet .
pear seed hyper-wallet .
# Format code
npm run format # Prettier (holepunchto style)
```
**Architecture:**
- **Protocols**: WDK integration adapters (test/protocols/)
- **Library**: Shared utilities (lib/)
- **Shims**: Bare/Node.js compatibility (shims/)
- **Bundles**: Standalone executables (bundle/)
**Import Maps:**
- Uses `#node:*` imports for Bare/Node.js compatibility
- WDK shims: Maps @tetherto/* to ./shims/ for Bare runtime
- Example: `#node:crypto` → `bare-crypto` (Bare) or `crypto` (Node.js)
**Testing Notes:**
- Tests MUST run with `bare` command (not `node`)
- Install bare-runtime globally: `npm install -g bare-runtime`
- Tests verify WDK works in Bare/Pear environment
### Multi-Device Testing with Hyper-Lab
hyper-lab is a complete multi-device P2P testing framework for testing Hypercore applications across real machines:
```bash
cd hyper-fleeky/hyper-lab
# Check agent connectivity
./hyper-lab status
# Sync code across machines via git
./hyper-lab sync # Full sync (push + pull + verify)
./hyper-lab sync --verify-only # Only verify all agents on same commit
# Run integration tests
./hyper-lab run <test-file> # Run test with auto-sync
./hyper-lab run <test-file> --no-sync # Skip git sync
./hyper-lab run <test-file> --attach # Auto-attach to tmux session
# Attach to running tmux session
./hyper-lab attach <session-name>
# View logs from test run
./hyper-lab logs <session-name>
# List available tests
./hyper-lab list
```
**Architecture:**
- **SSH Manager** (`lib/ssh-manager.js`) - SSH connections with ssh-agent support
- **Git Sync Manager** (`lib/git-sync-manager.js`) - Automated git push/pull between machines
- **Tmux Orchestrator** (`lib/tmux-orchestrator.js`) - Programmatic tmux control + visual debugging
- **Main CLI** (`hyper-lab`) - Command-line interface
**Key Features:**
- SSH orchestration for real machines (tabertron + ashatron)
- Tmux visualization with multi-pane layout (human-friendly)
- Programmatic control via tmux commands (AI-friendly)
- Automatic log capture with `pipe-pane`
- Pattern matching and assertions
- Output capture from tmux panes
- Git sync workflow (bare repo on tabertron)
**Example Test Workflow:**
```javascript
// tests/integration/test-address-sync.js
import { TmuxOrchestrator } from '../../hyper-lab/lib/tmux-orchestrator.js'
export default async function testAddressSync(agentConfigs, options = {}) {
const test = new TmuxOrchestrator('address-sync', agentConfigs)
await test.connect() // SSH to all agents
await test.createSession() // Create tmux session with panes
// Run commands on each device
await test.run('tabertron', 'init')
await test.run('ashatron', 'init')
await test.run('tabertron', 'address', ['new', 'bitcoin'])
// Wait for P2P sync
await test.sleep(8000)
// Capture output and assert
const output = await test.capturePane(test.paneMap.get('ashatron'), 50)
test.assert(output.includes('Total: 1 address'), 'Address should sync')
return test.sessionName
}
```
**Configuration:**
- `agents.json` - Machine configurations (tabertron, ashatron, etc.)
- Use `localhost` for tabertron when running hyper-lab ON tabertron
- Use `tabertron.local` when running from remote machines
- SSH keys managed via ssh-agent (no plaintext passwords)
**Documentation:**
- `hyper-lab/README.md` - Complete user guide
- `hyper-lab/SETUP.md` - Initial setup instructions
- `hyper-lab/TMUX_AUTOMATION.md` - Tmux automation architecture
- `hyper-lab/GIT_SYNC_WORKFLOW.md` - Git sync design
- `hyper-lab/IMPLEMENTATION_COMPLETE.md` - Implementation summary
**Use Cases:**
- Multi-device Hypercore replication testing
- P2P sync verification across NAT boundaries
- Hyperswarm discovery testing
- Autobase conflict resolution testing
- Performance benchmarking across machines
### Git Submodules
```bash
# Initialize specific submodules
git submodule update --init holepunchto/repos/tier1/hypercore
git submodule update --init hyper-fleeky/repos/hyperident
# Update all initialized submodules
git submodule update --remote
# Check status
git submodule status
```
## Architecture Overview
### Three-Part System
#### 1. holepunchto/ - Ecosystem Knowledge Base (529 repos)
**Three-Layer Architecture:**
- **Data Layer**: JSON files with dependency graphs, semantic categories, use cases, patterns
- **Documentation Layer**: `skills/` with 529 API documentation files
- **Agent Layer**: Designer and architect CLI tools
**Key Files:**
- `repo_map.json` - All 529 repos with metadata
- `dependency-graph.json` - Complete dependency relationships
- `semantic-categories.json` - 7 categories (Building Blocks, Bare Modules, etc.)
- `use-case-map.json` - 11 P2P use cases with repo recommendations
- `pattern-map.json` - 10 P2P patterns mapped to repos
- `skills/{repo}/skills.md` - API docs for each repo
- `skills/{repo}/api-extract.json` - Structured API data (133 repos)
**Repo Tiers:**
- Tier 1 (6): Core building blocks (hypercore, hyperbee, hyperdrive, autobase, hyperdht, hyperswarm)
- Tier 2 (6): Essential helpers (corestore, localdrive, compact-encoding, protomux, etc.)
- Tier 3 (50): Most popular by stars
- Tier 4 (150): Selective additional repos
- Remaining (317): Infrastructure/tools
#### 2. hyper-fleeky/ - Personal P2P Projects
**Five Active Projects:**
1. **hyperident** - Decentralized identity management using Hypercore
- Pear terminal app
- Uses: hypercore, hypercore-crypto, hyperbee
- Status: Alpha
- Location: `hyper-fleeky/repos/hyperident/`
2. **hyperident-gateway** - P2P VPN gateway using Hyperswarm and TUN devices
- Bare runtime app (requires CAP_NET_ADMIN)
- Uses: hyperswarm, hypercore-crypto, utun
- Network: 10.22.0.0/16 P2P VPN
- Deployment: systemd service on 7rnx.net
- Status: Production
- Location: `hyper-fleeky/repos/hyperident-gateway/`
3. **wirecore** - P2P social network with ANProto signatures
- **Two Active Components:**
- **pear-app/** - Pear terminal app + web interface (CURRENT FOCUS)
- **relay/** - APDS WebSocket relay server (DEPLOYED)
- Uses: hypercore, hyperbee, hyperswarm, ANProto
- Status: Active refactor (Phase 3 complete, Phase 4 next)
- Location: `hyper-fleeky/repos/wirecore/`
- **NOTE: lib/ is deprecated, use pear-app/ and relay/**
4. **fedicore** - Decentralized social network (legacy)
- HTTP server on port 3000
- Uses: hypercore, hyperbee, hyperswarm, autobase
- Status: Proof-of-concept
- Location: `hyper-fleeky/repos/fedicore/`
5. **hyper-wallet** - Self-custodial P2P cryptocurrency wallet ⭐ **NEW**
- Pear terminal app
- Uses: Holepunchto (hypercore, hyperbee, corestore, autobase) + Tether WDK
- Supports: Bitcoin (BIP-84 SegWit), Lightning (Spark), Ethereum/EVM, Solana
- Status: Active development
- Location: `hyper-fleeky/hyper-wallet/`
**Metadata Files:**
- `projects.json` - Project descriptions and dependencies
- `dependencies.json` - Cross-project and holepunchto dependencies
- `integration-map.json` - How projects integrate
- `pear-deployment-lessons.md` - Production deployment lessons learned
#### 3. tetherto-kb/ - Tether WDK Knowledge Base (18 repos)
**Same Structure as holepunchto:**
- **Data Layer**: JSON files with semantic categories, use cases, patterns
- **Documentation Layer**: `skills/` with WDK API documentation
- **Repository Layer**: Git clones of tier 1 repos
**Key Files:**
- `repo_map.json` - All 18 WDK repos with metadata
- `semantic-categories.json` - 7 categories (Core, Wallets, Protocols, etc.)
- `use-case-map.json` - 9 cryptocurrency use cases
- `pattern-map.json` - 12 WDK patterns (HD wallets, multi-chain, etc.)
- `skills/wdk-core/skills.md` - WDK Core API docs
- `skills/wdk-wallet-btc/skills.md` - Bitcoin wallet API
- `skills/wdk-wallet-evm/skills.md` - EVM wallet API
**Repo Tiers:**
- Tier 1 (10): Core, wallets, Pear integration (cloned)
- Tier 2 (4): DeFi protocols (bridge, swap, lending)
- Tier 3 (4): Applications and templates
**Key for hyper-wallet:**
- `pear-wrk-wdk` - Bare runtime integration with BareKit worklets
- Lightning via Spark blockchain (not traditional BTC Lightning)
- Multi-chain support: BTC, ETH, Solana, TON, TRON + USDT
## Technology Stack
### Testing
- **brittle** - Primary test framework for holepunchto and hyper-fleeky projects
- Run with: `brittle test/all.js` or `npm test`
- Bare support: `bare test/all.js` or `npm run test:bare`
### Runtimes
- **Node.js** - Standard Node.js (for most development)
- **Bare** - Minimal JavaScript runtime (for production apps)
- **Pear** - P2P application runtime (wraps Bare)
### Module System
- **ESM** - Modern projects use `"type": "module"` in package.json
- **CommonJS** - Some repos still use require()
- **Import Maps** - Used for Bare/Node.js compatibility via `"imports"` in package.json
Example import map pattern (hypercore):
```json
{
"imports": {
"events": {
"bare": "bare-events",
"default": "events"
}
}
}
```
### Formatting & Linting
- **prettier** - Code formatting (holepunchto standard)
- **lunte** - Linting tool
- Run: `npm run format` or `npm run lint`
## Critical Pear Deployment Knowledge
The `hyper-fleeky/pear-deployment-lessons.md` file contains essential production deployment knowledge for Pear apps. Key lessons:
### Module Resolution
- **Always include node_modules** in Pear apps (remove from `pear.stage.ignore`)
- **Dynamic requires fail** - Add transitive dependencies explicitly to package.json
- **pear:// protocol has module resolution bugs** - Use local directories for production
### Systemd Services
- Run as regular user with `CAP_NET_ADMIN` capability, not root
- Set full PATH in service file: `/home/user/.config/pear/bin:/usr/bin:/bin`
- Use `pear run pear://...` for link-based deployment
- Use `pear dump` + local directory for more reliable deployment
### Cache Issues
- Pear caches aggressively in `~/.config/pear/app-storage/`
- Clear cache when testing updates: `rm -rf ~/.config/pear/app-storage/*`
- Use `--checkout=latest` to force fresh download
- Complete reinstall if corrupted: `rm -rf ~/.config/pear && npx pear`
### Known Bugs
- **Critical**: Module resolution under `pear://` protocol is broken for node_modules
- ESM/CommonJS interop with hoisted dependencies fails in Bare runtime
- Workaround: Convert to CommonJS or use local directory deployment
## Important Conventions
### Git Commits
All commits must include co-authorship:
```bash
git commit -m "$(cat <<'EOF'
Your commit message here.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
EOF
)"
```
### Node.js Compatibility
Avoid optional chaining in holepunchto-style code:
```javascript
// DON'T
const dep = packageJson.dependencies?.hypercore
// DO
const dep = packageJson.dependencies && packageJson.dependencies.hypercore
```
### Code Style
- Follow prettier configuration from `prettier-config-holepunch`
- Use `npm run format` before committing
- Never commit uncommitted changes to submodules
## File Organization
```
/
├── holepunchto/ # Ecosystem knowledge base
│ ├── skills/ # 529 API documentation directories
│ ├── repos/ # Git submodules (optional, 529 repos)
│ ├── *.json # Semantic maps and metadata
│ ├── *-agent.js # CLI tools
│ ├── CLAUDE.md # holepunchto-specific guidance
│ └── README.md # Full documentation
│
├── tetherto-kb/ # Tether WDK knowledge base
│ ├── skills/ # WDK API documentation
│ ├── repos/ # Git clones (10 tier 1 repos)
│ ├── *.json # Semantic maps and metadata
│ ├── STATUS.md # Build progress and discoveries
│ └── README.md # Overview
│
├── hyper-fleeky/ # Personal projects
│ ├── repos/ # Git submodules (4 projects)
│ │ ├── hyperident/
│ │ ├── hyperident-gateway/
│ │ ├── wirecore/
│ │ └── fedicore/
│ ├── hyper-wallet/ # P2P cryptocurrency wallet (new)
│ ├── hyper-lab/ # Multi-device testing framework (new)
│ │ ├── lib/ # Core libraries (ssh, git-sync, tmux)
│ │ ├── hyper-lab # Main CLI executable
│ │ ├── agents.json # Machine configurations
│ │ └── *.md # Documentation
│ ├── projects.json # Project metadata
│ ├── dependencies.json # Dependency map
│ ├── integration-map.json
│ └── pear-deployment-lessons.md # CRITICAL production knowledge
│
├── CLAUDE.md # This file
├── README.md # Top-level overview
└── .gitmodules # Submodule definitions
```
## Development Workflows
### Exploring the Ecosystem
```bash
# Find repos by pattern
jq '.[] | select(.dependencies[]? == "hypercore")' holepunchto/dependency-graph.json
# Find bare runtime modules
jq '.[] | select(.runtime == "bare")' holepunchto/tech-stack.json
# Check which repos implement a pattern
jq '.patterns["append-only-log"].implementedBy' holepunchto/pattern-map.json
# Read API documentation
cat holepunchto/skills/hypercore/skills.md
jq . holepunchto/skills/hypercore/api-extract.json
```
### Building a New P2P Project
1. **Discover appropriate repos**
```bash
cd holepunchto
node designer-agent.js "your use case"
```
2. **Design architecture**
```bash
node architect-agent.js repo1 repo2 repo3
```
3. **Read API docs**
```bash
cat skills/repo1/skills.md
```
4. **Create project structure**
```bash
# Create in hyper-fleeky/repos/ as git submodule
# Add to hyper-fleeky/projects.json
# Update hyper-fleeky/dependencies.json
```
5. **Implement using learned patterns**
- Check `holepunchto/pattern-map.json` for implementation patterns
- Reference existing projects in `hyper-fleeky/repos/` for examples
- Consult `pear-deployment-lessons.md` for deployment gotchas
### Testing Changes
```bash
# Test holepunchto agent changes
cd holepunchto
node designer-agent.js --help
node architect-agent.js --use-case distributed-database
# Test personal project changes
cd hyper-fleeky/repos/hyperident
npm test
pear run -d .
# Test after submodule updates
git submodule update --remote
cd holepunchto/repos/tier1/hypercore
npm test
```
### Updating Knowledge Base
```bash
cd holepunchto
# Full update workflow
node process_repos.js # Fetch latest GitHub metadata
node clone_and_analyze_tier.js 1 # Analyze tier 1 repos
node build_semantic_maps.js # Rebuild semantic maps
# Commit with proper authorship
git add -A
git commit -m "$(cat <<'EOF'
Update knowledge base with latest repo metadata
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
EOF
)"
```
## Wirecore Deployment Architecture
### Current Production Infrastructure
```
Internet Users
↓ HTTPS
7rnx.net (142.93.23.112) - Public Gateway
├─ Apache2 SSL (Let's Encrypt)
├─ /relay → wss://10.22.0.1:9000 (wirecore-relay)
└─ hyperident-gateway (tunnel0, 10.22.0.3/16)
↓ VPN (10.22.0.0/16)
tabertron (10.22.0.1) - Backend Server
└─ wirecore-relay (port 9000)
• APDS WebSocket server
• Content-by-hash storage
• ANProto signature verification
• Gossip protocol
• Bridge to wiredove.net
Pear App Users (distributed)
└─ WebSocket → wss://7rnx.net/relay
```
**Active Services:**
- `wirecore-relay` on tabertron:9000 → Public at wss://7rnx.net/relay
- `hyperident-gateway` systemd service → VPN tunnel (10.22.0.0/16)
- Apache2 reverse proxy → SSL termination + WebSocket upgrade
**Deprecated Services:**
- ~~coturn TURN server~~ (disabled, legacy WebRTC)
- ~~wirecore lib/server~~ (replaced by relay + pear-app)
### Data Flow: Pear App ↔ Relay
**Posting Flow:**
```
1. User posts in Pear app
2. Create YAML with metadata (name, image, previous)
3. Hash YAML → postHash (SHA256)
4. Store YAML in local Hyperbee by hash
5. Sign postHash → ANProto signature (132 bytes)
6. Append signature to Hypercore feed
7. Send YAML to relay via WebSocket
8. Relay: store by hash, verify signature, gossip to peers
```
**Reading Flow:**
```
1. Read ANProto signatures from Hypercore feeds
2. Extract content hashes from signatures
3. Try local Hyperbee cache (contentCache.get(hash))
4. If miss: request from relay via WebSocket (send hash)
5. Relay responds with content (YAML or image data)
6. Cache locally in Hyperbee
7. Parse YAML and display
```
**Key Insight**: Content and signatures are separate
- Hypercore: Only 132-byte ANProto signatures (fast sync)
- Hyperbee: Full content by hash (lazy fetch)
- Relay: Bridge between Pear apps and wiredove.net
## Wirecore Pear App Development Protocol
### CRITICAL: This is a PEAR APP
**When working on `hyper-fleeky/repos/wirecore/pear-app/`:**
⚠️ **ALWAYS use `pear` commands, NEVER `npm run`**
```bash
# ✅ CORRECT
pear stage wirecore .
pear run pear://hash cli init
pear seed wirecore .
# ❌ WRONG - DO NOT USE
npm run stage
npm run seed
npm test # (npm test is OK for unit tests, but deployment uses pear)
```
### Staging and Release Workflow
**CRITICAL: Staging vs Release**
```bash
# 1. Make code changes
vim index.js
# 2. Stage (creates new version, but doesn't make it default)
pear stage wirecore .
# Output: Current version is 4080 with release set to 4034
# The pear:// link still points to 4034!
# 3. Release (makes latest version the default)
pear release wirecore
# Output: Setting release to 4081
# NOW pear:// link points to 4081 with your changes!
# 4. Test
pear run pear://hash cli init
```
**Key Understanding:**
- `pear stage` creates a new version but keeps old release
- `pear release` updates the release pointer to latest
- The pear:// link always fetches the **release version**
- If you stage without releasing, your changes won't be in pear://
### Testing Workflow
**CRITICAL: After Stage + Release, ALWAYS Restart Local Pear Apps:**
```bash
# After running pear stage + pear release:
# 1. Find running wirecore processes
ps aux | grep "pear run.*zupx3qzpwj61hdt1bxanabctcyxqqu3arogctwim4qt3wfbzn4bo" | grep -v grep | grep -v seed
# 2. Kill all running instances (use the PID from step 1)
kill <pid> # Or use Ctrl+C in each terminal
# 3. Restart with latest release
pear run pear://zupx3qzpwj61hdt1bxanabctcyxqqu3arogctwim4qt3wfbzn4bo web 3003
# ⚠️ If you don't restart, the old version keeps running and you won't see your changes!
```
**First-Time Init (New Identity):**
```bash
# ⚠️ NEVER DO: rm -rf ~/.config/pear/app-storage/*
# This destroys ALL Pear apps, not just wirecore!
# If you MUST test fresh identity, find the specific app directory:
ls ~/.config/pear/app-storage/by-dkey/
# Then remove ONLY that specific directory:
# rm -rf ~/.config/pear/app-storage/by-dkey/<wirecore-specific-hash>/
# Or better: just test without clearing
pear run pear://hash cli init
# If already initialized, it will say "Already initialized"
```
**Subsequent Testing (Existing Identity):**
```bash
# DON'T clear cache - storage persists!
# Test existing identity
pear run pear://hash cli whoami
# Test posting
pear run pear://hash cli post "Hello P2P"
# Test web interface
pear run pear://hash web 3003
# Access: http://localhost:3003
```
**Multi-Peer Testing:**
```bash
# Terminal 1 (Peer A):
pear run pear://hash web 3003
# Terminal 2 (Peer B):
pear run pear://hash cli follow <pubkey-from-A>
pear run pear://hash cli timeline
# Different machine (Peer C):
rm -rf ~/.config/pear/app-storage/* # Fresh identity
pear run pear://hash cli init
pear run pear://hash cli follow <pubkey-from-A>
```
### Before ANY Development Action
**Self-Check Protocol:**
1. ✅ Confirm: "I am working on wirecore pear-app, a PEAR APPLICATION"
2. ✅ Verify: Am I using `pear` commands? (not npm run)
3. ✅ Check phase: Currently Phase 1 (ANProto identity layer only)
4. ✅ Storage awareness: Does identity already exist? (don't clear unless testing fresh start)
### Required Reading Before Development
**Architecture & Planning:**
- `pear-app/REFACTOR_PLAN.md` - Phase 1-6 implementation plan
- `pear-app/CODE_CHANGES.md` - How code transforms
- `pear-app/FEATURE_PARITY.md` - What features to maintain
- `pear-app/REFACTOR_DEPLOYMENT.md` - Testing & deployment guide
**Wirecore Knowledge:**
- `wirecore/docs/claude/wirecore-skills.md` - ANProto protocol, content-by-hash patterns
- `wirecore/docs/CLAUDE-SKILLS.md` - Semantic keyword lookup
- `relay/wirecore-relay.js` - Relay WebSocket protocol
- **`pear-app/IMAGE_UPLOAD_LESSONS.md`** - **CRITICAL:** Image upload protocol and common mistakes (ALWAYS review "Common Mistakes" section before working on images!)
**Holepunchto Skills:**
- `holepunchto/skills/hypercore/` - Feed management patterns
- `holepunchto/skills/hyperbee/` - Key-value storage (content cache)
- `holepunchto/skills/hyperswarm/` - P2P networking
- `holepunchto/skills/protomux/` - Protocol multiplexing (content gossip)
- `holepunchto/skills/corestore/` - Multi-core management
### Current Status (Phase 1 of 6)
**Phase 1: COMPLETE ✅ (2026-01-21)**
**Completed:**
- ✅ ANProto keypair generation (lib/anproto.js)
- ✅ YAML helper module (lib/yaml-helper.js)
- ✅ sodium-universal dependency added
- ✅ Unified identity (ANProto key = Hypercore feed key)
- ✅ Keypair stored in Hyperbee metadata as string
- ✅ Updated cmdInit() to use ANProto
- ✅ Updated cmdWhoami() to show ANProto public key
- ✅ Updated web server identity display
- ✅ Updated /api/whoami endpoint
- ✅ Fixed Bug 1: publicKey display (Buffer → base64 string)
- ✅ Fixed Bug 2: loadState() checks ANProto system first
- ✅ Fixed Bug 3: web server checks for ANProto keypair
- ✅ Staged: `pear://zupx3qzpwj61hdt1bxanabctcyxqqu3arogctwim4qt3wfbzn4bo`
- ✅ Released: version 4087
- ✅ Tested: init + whoami + web all working correctly
- ✅ Committed: git SHA 63fdd73
**Phase 1.5: Real-Time Updates (2026-01-21)**
**Problem Discovery:**
- Real-time post updates weren't working in web interface
- Feeds replicate successfully (manual refresh shows new posts)
- Hypercore `download` events only fire during initial sync, NOT for new blocks
- bare-ws WebSocket server incompatible with browser WebSocket clients (error 1006)
**Solution Implemented:**
- ✅ Added `feed.update()` polling (every 5 seconds) for all followed feeds
- ✅ Polls check for new blocks and trigger downloads actively
- ✅ Added 10-second polling fallback in browser (auto-refresh timeline)
- ✅ Created test-websocket.js to verify server broadcasts independently
- ✅ Released: version 4158
**Architectural Insight:**
- Wiredove uses active polling (every 15 seconds) to relay server
- Wirecore needs active `feed.update()` polling for Hypercore replication
- Passive event-driven approach doesn't work - feeds need explicit update checks
**Known Issues:**
- 🐛 **bare-ws WebSocket incompatibility**: Browser WebSocket clients connect then immediately disconnect (error 1006). Server broadcasts work (verified with Node.js test client) but browser clients fail. Workaround: 10-second polling fallback in browser.
- 🐛 **CLI storage lock conflict**: Cannot post from CLI when web server is running (Corestore lock conflict). Both try to access same storage simultaneously.
**Testing Tools:**
- `test-websocket.js` - Node.js WebSocket test client for independent verification
- Manual multi-peer testing: ashatron ↔ tabertron via Hyperswarm
**Immediate Fixes Needed:**
- 🔧 Fix bare-ws browser compatibility or switch WebSocket library
- 🔧 Fix CLI storage access (allow CLI and web to coexist)
- 🔧 Multi-peer stress testing (2-3 nodes, extended sessions)
**Development Roadmap:**
See "Wirecore Development Roadmap" section below for detailed phase planning.
## Wirecore Development Roadmap
### Phase 1: ANProto Identity Layer ✅ COMPLETE
**Goal:** Establish unified identity system using ANProto cryptographic signatures.
**Completed:**
- ANProto keypair generation and storage
- YAML helper for structured data
- Unified Hypercore feed key with ANProto public key
- CLI commands: init, whoami
- Web interface identity display
- Real-time updates with active feed polling
- Polling fallback for UI updates
**Known Issues:**
- bare-ws WebSocket browser compatibility (workaround: polling)
- CLI storage lock when web server running
**Release:** version 4158, pear://zupx3qzpwj61hdt1bxanabctcyxqqu3arogctwim4qt3wfbzn4bo
---
### Phase 2: WebSocket & Storage Fixes 🔧 DEFERRED
**Status:** DEFERRED - Moving to Phase 3 first. See `hyper-fleeky/repos/wirecore/pear-app/PHASE2_TESTING_TODO.md` for detailed testing plan.
**⚠️ IMPORTANT:** Keep `PHASE2_TESTING_TODO.md` updated during Phase 3 development if you discover new issues or workarounds.
**Goal:** Fix critical bugs blocking development and improve real-time updates.
**Tasks:**
1. **Fix bare-ws Browser Compatibility**
- Option A: Find bare-ws configuration for browser clients
- Option B: Switch to different WebSocket library (ws, uWebSockets.js)
- Option C: Use bare-http with manual WebSocket upgrade
- Test with browser clients, not just Node.js test client
2. **Fix CLI Storage Access**
- Allow multiple Corestore instances or use IPC
- Option A: Web server exposes IPC endpoint for CLI
- Option B: Use file-based locking with queue
- Option C: Separate storage paths for CLI vs web
3. **Improve Feed Replication**
- Current: 5-second polling works but inefficient
- Investigate Hypercore event timing (why download doesn't fire for new blocks?)
- Consider hybrid: events + fallback polling
4. **Testing & Validation**
- Multi-peer stress test (3+ nodes, 24 hours)
- Test feed replication across NAT boundaries
- Verify Hyperswarm discovery and reconnection
**Success Criteria:**
- Browser WebSocket shows live updates without polling fallback
- CLI commands work while web server running
- 3 peers can follow each other and see real-time updates
- No disconnections or missed messages over 24 hours
**Estimated Scope:** 1-2 weeks
---
### Phase 3: Content-by-Hash Storage ✅ COMPLETE
**Goal:** Implement Hyperbee content cache and hash-based storage.
**Completed (2026-01-22):**
1. ✅ **Hyperbee Content Cache**
- Created contentCache Hyperbee instance (shared between pear-app and bridge)
- Store post content by SHA256 hash using ANProto.hash()
- Get/put operations for content retrieval
2. ✅ **Update Post Format**
- Posts stored as YAML with metadata (name, image, previous)
- Hash YAML content → contentHash (base64)
- Store YAML in contentCache by hash on creation
3. ✅ **Signature-Only Feeds (Transition)**
- Feeds store full content AND contentHash (dual storage)
- Timeline reads from contentCache first, falls back to feed content
- Backward compatible with old format posts
4. ✅ **Relay Integration**
- Bridge pull stores all 600+ relay posts in contentCache by hash
- Timeline can access relay posts via contentCache
- Posts pushed to relay successfully with correct YAML format
5. ✅ **Image Upload Protocol (CRITICAL FIX)**
- **Problem:** Images not appearing on pub.wiredove.net (404 errors)
- **Root Cause:** Hash mismatch - hashing raw bytes but sending data URLs
- **Solution:** Hash data URLs (not raw bytes) to match wiredove-cli protocol
- **Protocol:** Images sent as SEPARATE WebSocket messages (not bundled with posts)
- **Format:** Store and send `data:image/jpeg;base64,<base64>` format
- **Result:** Images now display correctly on wiredove.net (HTTP 200)
- **Documentation:** See `pear-app/IMAGE_UPLOAD_LESSONS.md` for complete analysis
**Bug Fixes:**
- Fixed bare-ws Socket API: `.write()` instead of `.send()`, `.destroy()` instead of `.close()`
- Fixed ANProto signature: pass contentHash only (sign() adds timestamp internally)
- Fixed YAML format: Added opening `---` delimiter for proper parsing
- Fixed image hashing: Hash data URLs instead of raw bytes
- Fixed image protocol: Send images as separate blob messages
**Released:** v4257, git commit [current]
**Success Criteria Met:**
- ✅ All new posts stored in Hyperbee by hash
- ✅ Timeline reads from contentCache successfully
- ✅ YAML format matches wirecore relay expectations (with opening delimiter)
- ✅ No data loss - backward compatible with old format
- ✅ 658 relay posts stored in contentCache
- ✅ Images upload successfully to pub.wiredove.net
- ✅ Images display on wiredove.net web interface
**Key Files:**
- `pear-app/IMAGE_UPLOAD_LESSONS.md` - Complete protocol documentation
- `pear-app/lib/relay-client.js` - Separate message implementation
- `pear-app/index.js` - Data URL hashing for posts
- `pear-app/bridge/wirecore-bridge.js` - Bridge image extraction
- `pear-app/lib/yaml-helper.js` - YAML delimiter fix
---
### Phase 4: Signature-Only Feeds 🔐 PLANNED
**Goal:** Switch Hypercore feeds from full content to 132-byte ANProto signatures only.
**Tasks:**
1. **Update Feed Schema**
- Append only ANProto signatures (132 bytes) to feeds
- Signature covers: timestamp, contentHash, previous signature hash
- Remove full content storage from feeds
2. **Content Separation**
- Hypercore: signatures only (fast sync, ~50 signatures/block)
- Hyperbee: full content by hash (lazy fetch)
- Relay: content bridge for new users
3. **Migration Strategy**
- Detect old vs new feed format
- Support reading both formats during transition
- Provide migration tool to convert old feeds
4. **Update API Endpoints**
- `/api/timeline` reads signatures, fetches content from cache
- `/api/post` creates signature, stores content in cache
- Content miss → request from relay (Phase 6)
**Success Criteria:**
- Feeds contain only 132-byte signatures
- Feed sync 10x faster than before (less data)
- Timeline displays correctly from contentCache
- Signature verification working (ANProto)
**Estimated Scope:** 2-3 weeks
---
### Phase 5: P2P Content Gossip Protocol 🌐 PLANNED
**Goal:** Implement Protomux protocol for P2P content sharing without relay.
**Tasks:**
1. **Protomux Integration**
- Add protomux to Hyperswarm connections
- Define wirecore gossip protocol (channels, message types)
- Handle multiplexing with replication streams
2. **Content Request/Response**
- Request content by hash from peers
- Respond to content requests from local cache
- LRU cache eviction for space management
3. **Discovery & Routing**
- Discover which peers have content (DHT hints?)
- Route requests to likely peers
- Fallback to relay if no peer has content
4. **Bandwidth Management**
- Limit concurrent requests per peer
- Prioritize recent/followed content
- Implement backpressure and flow control
**Success Criteria:**
- Peers can fetch missing content from each other
- No relay dependency for established networks
- Latency < 500ms for peer content fetch
- Bandwidth usage reasonable (< 1MB/min idle)
**Estimated Scope:** 3-4 weeks
---
### Phase 6: Full Relay Integration 🔗 PLANNED
**Goal:** Integrate wirecore pear-app with wirecore-relay for bootstrapping and bridge to wiredove.net.
**Tasks:**
1. **Relay WebSocket Client**
- Connect to wss://7rnx.net/relay on startup
- Send new posts as YAML to relay
- Subscribe to followed users' updates
2. **Content Fetching**
- Request missing content from relay by hash
- Cache relay responses in local Hyperbee
- Prefer P2P gossip over relay when available
3. **Bootstrapping New Users**
- New users fetch initial content from relay
- Populate contentCache from relay responses
- Transition to P2P as cache builds
4. **Bridge to wiredove.net**
- Relay forwards wirecore posts to wiredove
- Relay forwards wiredove posts to wirecore
- Cross-protocol following (wirecore ↔ wiredove)
**Success Criteria:**
- New users can bootstrap from relay
- Existing users get updates via P2P + relay fallback
- wiredove.net users visible in wirecore timeline
- Relay connection resilient to disconnections
**Estimated Scope:** 2-3 weeks
---
### Phase 7: UI/UX Polish & Production 🎨 PLANNED
**Goal:** Production-ready UI, error handling, and deployment.
**Tasks:**
1. **UI Improvements**
- Better timeline rendering (images, threads)
- Loading states and error messages
- Offline mode indication
- Profile pages and user discovery
2. **Error Handling**
- Graceful degradation when relay unavailable
- Content fetch timeout and retry logic
- Clear user feedback for all error states
3. **Performance Optimization**
- Virtual scrolling for long timelines
- Lazy image loading
- Debounce/throttle UI updates
- Minimize feed.update() polling overhead
4. **Production Deployment**
- Systemd service for web interface (optional)
- Documentation for users
- Migration guide from wiredove
- Public announcement and testing
**Success Criteria:**
- App feels responsive and polished
- No confusing error states
- Timeline with 1000+ posts performs well
- Users can self-host or use pear:// link
**Estimated Scope:** 2-3 weeks
---
### Phase 8: Advanced Features 🚀 FUTURE
**Possible Future Work:**
- Image/file attachments (content-by-hash)
- Threads and reply chains
- Mentions and notifications
- Search and filtering
- Moderation and blocking
- Multiple identities
- Encrypted DMs
- Mobile app (Pear mobile runtime)
---
### Current Priority
**RIGHT NOW:** Phase 4 (Signature-Only Feeds) - READY TO START
**Phase 3 Status:** ✅ COMPLETE (2026-01-22)
- All posts stored in contentCache by hash
- Relay integration working with bidirectional sync
- **Image uploads working** - Successfully syncing to pub.wiredove.net
- Released: v4257
- **Documentation:** See `pear-app/IMAGE_UPLOAD_LESSONS.md` for image protocol details
**Next Steps (Phase 4):**
1. Switch feeds from full content to 132-byte ANProto signatures only
2. Feeds will contain: timestamp + contentHash signatures
3. Timeline will fetch content from contentCache by hash
4. Implement migration for old format feeds
5. Test P2P sync with signature-only feeds
**Phase 2 Status:** DEFERRED - See `hyper-fleeky/repos/wirecore/pear-app/PHASE2_TESTING_TODO.md` for issues to fix later.
**Immediate Priorities:**
- Fix Phase 2 issues (bare-ws WebSocket browser compatibility, CLI storage lock)
- OR proceed to Phase 4 (signature-only feeds for 10x faster sync)
- OR proceed to Phase 5 (P2P content gossip via Protomux)
### Testing Real-Time Updates
**Automated Testing (Recommended):**
```bash
# Terminal 1: Start web server
pear run pear://zupx3qzpwj61hdt1bxanabctcyxqqu3arogctwim4qt3wfbzn4bo web 3003
# Terminal 2: Run WebSocket test client
cd hyper-fleeky/repos/wirecore/pear-app
node test-websocket.js
# Terminal 3: Post from another machine to trigger broadcast
# (On ashatron or another peer)
pear run pear://zupx3qzpwj61hdt1bxanabctcyxqqu3arogctwim4qt3wfbzn4bo cli post "test message"
# Terminal 2 should show:
# 📨 Received message: {"type":"new_post","feed":"own"}
```
**Manual Browser Testing:**
```bash
# Terminal 1: tabertron web server
pear run pear://zupx3qzpwj61hdt1bxanabctcyxqqu3arogctwim4qt3wfbzn4bo web 3003
# Terminal 2: ashatron posting (different machine)
pear run pear://zupx3qzpwj61hdt1bxanabctcyxqqu3arogctwim4qt3wfbzn4bo cli post "hello from ashatron"
# Browser on tabertron at http://localhost:3003
# Should see post appear within 10 seconds (polling fallback)
# Check browser console for "[POLL] Auto-refreshing timeline..." every 10 seconds
```
**Debugging WebSocket Issues:**
- Server logs: Look for `[BROADCAST] Sending to N clients`
- test-websocket.js: Proves server broadcasts work
- Browser console: Check for WebSocket disconnect errors (1006)
- Feed updates: Look for `📥 Downloaded block X from ...` in server logs
## Common Pitfalls
### Pear Deployment
- **Never ignore node_modules** in pear.stage.ignore
- **Test on clean machine** - local cache hides bugs
- **Use capabilities, not root** - Pear's permission model breaks with root
- **Report blocking bugs** - Module resolution under pear:// is broken
- **DON'T use npm run for deployment** - Use pear commands
### Wirecore Pear App Specific
- **🚨 NEVER run `rm -rf ~/.config/pear/app-storage/*`** - This destroys ALL Pear apps, not just wirecore!
- **Init only once per identity** - Storage persists, don't clear cache unless testing fresh identity
- **Check specific app directory only** - If must clear: `ls ~/.config/pear/app-storage/by-dkey/` then remove only that specific hash
- **Use pear commands, not npm** - This is a Pear app, not npm package
- **Phase awareness** - Currently Phase 1 only, don't jump ahead
- **Real-time updates require active polling** - Hypercore `download` events only fire during initial sync. Use `setInterval` with `feed.update()` to actively check for new blocks (every 5 seconds recommended).
- **bare-ws browser incompatibility** - bare-ws WebSocket server disconnects browser clients immediately (error 1006). Works with Node.js `ws` package. Workaround: implement polling fallback in browser.
- **CLI + web server conflict** - Cannot run CLI commands when web server is running (Corestore lock). Both access same storage. Needs architectural fix.
- **ANProto is append-only** - No editing or deleting messages possible. Design UIs accordingly.
- **Image upload mistakes (CRITICAL)** - See `pear-app/IMAGE_UPLOAD_LESSONS.md` "Common Mistakes" section:
- Backend MUST add `\n` to message text (not just metadata)
- Frontend MUST parse markdown `` to `<img>` tags (not show raw markdown)
- Canvas MUST be drawn before `toDataURL()` (wiredove-specific)
### Git Submodules
- **Don't commit submodule changes** unless intentionally updating references
- **Initialize before working** - `git submodule update --init path/to/module`
- **Update carefully** - `git submodule update --remote` updates all at once
### Testing
- **Use brittle** not mocha/jest - it's the holepunchto standard
- **Test both Node and Bare** if targeting Bare runtime
- **Run formatters first** - `npm run format` before `npm test`
### Bare Runtime Testing
- **Use `bare` command, not `node`** - WDK tests require Bare runtime
- **Install bare-runtime globally** - `npm install -g bare-runtime`
- **Import maps required** - Use `#node:*` pattern for Bare/Node.js compatibility
- **Shims for WDK** - @tetherto/* packages need shims in Bare environment
### Multi-Device Testing (Hyper-Lab)
- **Use localhost for local agent** - When running hyper-lab ON tabertron, use `"host": "localhost"` in agents.json, NOT `"host": "tabertron.local"` (avoids SSH to self)
- **SSH agent required** - Encrypted SSH keys need ssh-agent: `eval $(ssh-agent) && ssh-add ~/.ssh/id_rsa`
- **Git parent directory** - Git operations work on parent directory (hypercore-kb) not subdirectory (hyper-fleeky)
- **Bare repo location** - Use `/home/fleeky/git-repos/hypercore-kb.git` (not `/storage/projects/code/hypercore-kb/hyper-fleeky.git`)
- **No submodules needed** - When cloning on ashatron, don't use `--recurse-submodules` (529 holepunchto repos not needed for testing)
- **Tmux sessions persist** - Use `tmux ls` to see active sessions, `tmux kill-session -t <name>` to cleanup
- **Log files accumulate** - Check `hyper-lab/logs/` directory, clean up old test logs periodically
## Data Consistency Rules
When updating metadata:
- Repos in `use-case-map.json` must exist in `repo_map.json`
- Pattern references must point to existing repos
- Keep repo counts consistent (529 in holepunchto, 18 in tetherto-kb, 5 in hyper-fleeky)
- Update both `projects.json` and `dependencies.json` when adding hyper-fleeky projects
- tetherto-kb follows same structure as holepunchto (semantic-categories, use-case-map, pattern-map)
## Current Status
### holepunchto Knowledge Base
- ✅ Phase 1A: Complete (529/529 repos analyzed)
- ✅ Phase 1B: Selective complete (133/529 deeply analyzed)
- ✅ Phase 2: Complete (all semantic mappings)
- ✅ Phase 3: Complete (designer + architect agents)
- ⏳ Phase 4-6: Pending (additional tooling)
### hyper-fleeky Projects
- hyperident: Alpha (Pear terminal app, basic functionality working)
- hyperident-gateway: Production (VPN gateway, systemd deployment on 7rnx.net)
- wirecore: Active refactor (Phase 3 complete, Phase 4 next)
- fedicore: Proof-of-concept (basic social network, HTTP interface)
- hyper-wallet: Active development (P2P cryptocurrency wallet with WDK)
### tetherto-kb Knowledge Base
- ✅ Semantic lookup system: 18 repos, 9 use cases, 12 patterns
- ✅ Repository clones: 10 tier 1 repos cloned
- ✅ Skills documentation: 3 of 10 core skills created (wdk-core, wdk-wallet-btc, wdk-wallet-evm)
- 🚧 Next: pear-wrk-wdk skills (critical for hyper-wallet)
All projects are under active development. See `hyper-fleeky/pear-deployment-lessons.md` for latest deployment knowledge.
## Quick Reference
**Read first**: holepunchto/QUICK_START.md (5-minute introduction)
**AI workflow**: holepunchto/AI_WORKFLOW_GUIDE.md (complete guide)
**Production deployment**: hyper-fleeky/pear-deployment-lessons.md (critical)
**Ecosystem docs**:
- holepunchto/README.md (P2P ecosystem - comprehensive)
- tetherto-kb/README.md (Tether WDK ecosystem)
- tetherto-kb/STATUS.md (Build progress and discoveries)
**Project details**: hyper-fleeky/projects.json (metadata)