| … | … | @@ -0,0 +1,500 @@ |
| 1 | # Image Upload Protocol Lessons Learned |
| 2 | |
| 3 | **Date:** 2026-01-22 |
| 4 | **Context:** Debugging image uploads from wirecore pear-app to pub.wiredove.net via relay |
| 5 | |
| 6 | ## Problem Overview |
| 7 | |
| 8 | Images were not appearing on pub.wiredove.net despite posts syncing correctly. The issue involved multiple protocol mismatches between wirecore and wiredove. |
| 9 | |
| 10 | ## Root Causes Discovered |
| 11 | |
| 12 | ### 1. Hash Mismatch (CRITICAL) |
| 13 | |
| 14 | **Problem:** Wirecore hashed raw image bytes, but wiredove expected data URL hash. |
| 15 | |
| 16 | **Wrong Approach:** |
| 17 | ```javascript |
| 18 | // Hashing raw bytes |
| 19 | const imgBuffer = b4a.from(imageData, 'base64') |
| 20 | const imageHash = b4a.toString(crypto.hash(imgBuffer), 'base64') |
| 21 | ``` |
| 22 | |
| 23 | **Correct Approach (from wiredove-cli):** |
| 24 | ```javascript |
| 25 | // Create data URL first |
| 26 | const dataUrl = `data:${mimeType};base64,${imageData}` |
| 27 | |
| 28 | // Hash the data URL string (not raw bytes!) |
| 29 | const imageHash = ANProto.hash(dataUrl) |
| 30 | ``` |
| 31 | |
| 32 | **Why:** APDS stores blobs by their content hash. If you hash raw bytes but send a data URL, the hashes don't match and the relay can't retrieve the image. |
| 33 | |
| 34 | ### 2. Separate Message Protocol |
| 35 | |
| 36 | **Problem:** Initially tried to send images bundled with post in same WebSocket message. |
| 37 | |
| 38 | **Wrong Approach:** |
| 39 | ```javascript |
| 40 | ws.write(pubkey) |
| 41 | ws.write(yaml) |
| 42 | ws.write(imageData) // ❌ Wrong! Bundled with post |
| 43 | ws.write(signature) |
| 44 | ``` |
| 45 | |
| 46 | **Correct Approach:** |
| 47 | ```javascript |
| 48 | // First connection: Send post |
| 49 | ws.write(pubkey) |
| 50 | ws.write(yaml) |
| 51 | ws.write(signature) |
| 52 | ws.destroy() |
| 53 | |
| 54 | // Second connection: Send image blob separately |
| 55 | const ws2 = new WebSocket(url) |
| 56 | ws2.write(imageData) // ✅ Separate message |
| 57 | ws2.destroy() |
| 58 | ``` |
| 59 | |
| 60 | **Why:** Wiredove/APDS treats images as independent blobs that can be requested by hash. Posts reference images by hash, but images are gossiped separately. |
| 61 | |
| 62 | **Evidence from wiredove-cli (composer.js):** |
| 63 | ```javascript |
| 64 | // First: send signature and content |
| 65 | await send(signed) |
| 66 | await send(blob) |
| 67 | |
| 68 | // Then: extract image refs and send blobs separately |
| 69 | const images = blob.match(/!\[.*?\]\((.*?)\)/g) |
| 70 | if (images) { |
| 71 | for (const image of images) { |
| 72 | const src = image.match(/!\[.*?\]\((.*?)\)/)[1] // Extract hash |
| 73 | const imgBlob = await apds.get(src) // Get raw data |
| 74 | if (imgBlob) { await send(imgBlob) } // Send separately! |
| 75 | } |
| 76 | } |
| 77 | ``` |
| 78 | |
| 79 | ### 3. Data URL Format |
| 80 | |
| 81 | **Format:** `data:image/jpeg;base64,<base64-encoded-data>` |
| 82 | |
| 83 | **Storage:** |
| 84 | - Store in feed: Full data URL string |
| 85 | - Send to relay: Full data URL string |
| 86 | - Hash for YAML reference: Hash of full data URL string |
| 87 | |
| 88 | **Why:** Consistency across the pipeline. APDS hashes whatever you send it, so if you send data URLs, you must hash data URLs. |
| 89 | |
| 90 | ### 4. YAML Format Requirements |
| 91 | |
| 92 | **Problem:** Missing opening `---` delimiter caused posts to not render on wiredove.net. |
| 93 | |
| 94 | **Wrong:** |
| 95 | ```yaml |
| 96 | name: tabertron |
| 97 | image: abc123... |
| 98 | previous: xyz789... |
| 99 | --- |
| 100 | Message content |
| 101 | ``` |
| 102 | |
| 103 | **Correct:** |
| 104 | ```yaml |
| 105 | --- |
| 106 | name: tabertron |
| 107 | image: abc123... |
| 108 | previous: xyz789... |
| 109 | --- |
| 110 | Message content |
| 111 | ``` |
| 112 | |
| 113 | **Fix:** |
| 114 | ```javascript |
| 115 | static create(metadata, content) { |
| 116 | const lines = [] |
| 117 | lines.push('---') // ← Opening delimiter! |
| 118 | |
| 119 | if (metadata.name) lines.push(`name: ${metadata.name}`) |
| 120 | if (metadata.image) lines.push(`image: ${metadata.image}`) |
| 121 | if (metadata.previous) lines.push(`previous: ${metadata.previous}`) |
| 122 | |
| 123 | lines.push('---') |
| 124 | lines.push(content) |
| 125 | |
| 126 | return lines.join('\n') |
| 127 | } |
| 128 | ``` |
| 129 | |
| 130 | ### 5. bare-ws WebSocket API Differences |
| 131 | |
| 132 | **Problem:** bare-ws Socket uses Duplex stream API, not standard WebSocket API. |
| 133 | |
| 134 | **Wrong (standard WebSocket):** |
| 135 | ```javascript |
| 136 | ws.send(data) // ❌ Method doesn't exist |
| 137 | ws.close() // ❌ Method doesn't exist |
| 138 | ``` |
| 139 | |
| 140 | **Correct (bare-ws):** |
| 141 | ```javascript |
| 142 | ws.write(data) // ✅ Duplex stream method |
| 143 | ws.destroy() // ✅ Duplex stream method |
| 144 | ``` |
| 145 | |
| 146 | **Import:** |
| 147 | ```javascript |
| 148 | import { Socket as WebSocket } from 'bare-ws' |
| 149 | ``` |
| 150 | |
| 151 | ## Complete Working Flow |
| 152 | |
| 153 | ### 1. Create Post with Image (pear-app) |
| 154 | |
| 155 | ```javascript |
| 156 | // User uploads image via UI |
| 157 | const file = input.files[0] |
| 158 | const reader = new FileReader() |
| 159 | reader.readAsDataURL(file) // Read as data URL |
| 160 | |
| 161 | reader.onload = async (e) => { |
| 162 | const dataUrl = e.target.result // "data:image/jpeg;base64,..." |
| 163 | |
| 164 | // Hash the data URL |
| 165 | const imageHash = await apds.hash(dataUrl) |
| 166 | |
| 167 | // Store in IndexedDB for local display |
| 168 | await apds.put(`post-image:${imageHash}`, dataUrl) |
| 169 | |
| 170 | // Post to backend |
| 171 | await fetch('/api/post', { |
| 172 | method: 'POST', |
| 173 | body: JSON.stringify({ |
| 174 | message: text, |
| 175 | imageData: dataUrl.split(',')[1], // Extract base64 part |
| 176 | imageMimeType: 'image/jpeg' |
| 177 | }) |
| 178 | }) |
| 179 | } |
| 180 | ``` |
| 181 | |
| 182 | ### 2. Backend Stores Post (index.js) |
| 183 | |
| 184 | ```javascript |
| 185 | app.post('/api/post', async (req, res) => { |
| 186 | const { message, imageData, imageMimeType } = req.body |
| 187 | |
| 188 | let imageHash = null |
| 189 | let dataUrl = null |
| 190 | |
| 191 | if (imageData) { |
| 192 | // Reconstruct data URL |
| 193 | const mimeType = imageMimeType || 'image/jpeg' |
| 194 | dataUrl = `data:${mimeType};base64,${imageData}` |
| 195 | |
| 196 | // Hash the data URL (not raw bytes!) |
| 197 | imageHash = ANProto.hash(dataUrl) |
| 198 | } |
| 199 | |
| 200 | // Create YAML with image hash |
| 201 | const metadata = { |
| 202 | name: username, |
| 203 | image: imageHash, // Reference by hash |
| 204 | previous: prevHash |
| 205 | } |
| 206 | const yaml = YAMLHelper.create(metadata, message) |
| 207 | |
| 208 | // Store in feed with full data URL |
| 209 | const entry = { |
| 210 | yaml, |
| 211 | contentHash: ANProto.hash(yaml), |
| 212 | timestamp: Date.now(), |
| 213 | imageData: dataUrl // Store full data URL! |
| 214 | } |
| 215 | |
| 216 | await myFeed.append(Buffer.from(JSON.stringify(entry))) |
| 217 | }) |
| 218 | ``` |
| 219 | |
| 220 | ### 3. Bridge Pushes to Relay (wirecore-bridge.js) |
| 221 | |
| 222 | ```javascript |
| 223 | async push() { |
| 224 | for (let i = startIndex; i < feedLength; i++) { |
| 225 | const entry = await this.myFeed.get(i) |
| 226 | |
| 227 | // Extract YAML and image |
| 228 | const yaml = entry.yaml |
| 229 | const imageData = entry.imageData // Full data URL |
| 230 | |
| 231 | // Sign content |
| 232 | const contentHash = ANProto.hash(yaml) |
| 233 | const signature = ANProto.sign(contentHash, this.keypair) |
| 234 | |
| 235 | // Push to relay |
| 236 | await this.relayClient.push(pubkey, yaml, signature, imageData) |
| 237 | } |
| 238 | } |
| 239 | ``` |
| 240 | |
| 241 | ### 4. Relay Client Sends (relay-client.js) |
| 242 | |
| 243 | ```javascript |
| 244 | async push(pubkey, yaml, signature, imageData = null) { |
| 245 | // First: Send post |
| 246 | await this._pushPost(pubkey, yaml, signature) |
| 247 | |
| 248 | // Then: Send image blob separately |
| 249 | if (imageData) { |
| 250 | await this._pushImageBlob(imageData) |
| 251 | } |
| 252 | } |
| 253 | |
| 254 | async _pushPost(pubkey, yaml, signature) { |
| 255 | const ws = new WebSocket(this.wsUrl) |
| 256 | ws.on('open', () => { |
| 257 | ws.write(pubkey) // Author |
| 258 | ws.write(yaml) // Content (with image hash reference) |
| 259 | ws.write(signature) // Signature |
| 260 | setTimeout(() => ws.destroy(), 2000) |
| 261 | }) |
| 262 | } |
| 263 | |
| 264 | async _pushImageBlob(imageData) { |
| 265 | const ws = new WebSocket(this.wsUrl) |
| 266 | ws.on('open', () => { |
| 267 | ws.write(imageData) // Send data URL as blob |
| 268 | setTimeout(() => ws.destroy(), 3000) |
| 269 | }) |
| 270 | } |
| 271 | ``` |
| 272 | |
| 273 | ### 5. Relay Processes (wirecore-relay.js) |
| 274 | |
| 275 | ```javascript |
| 276 | ws.on('message', async (data) => { |
| 277 | const msg = data.toString('utf8') |
| 278 | |
| 279 | if (msg.length === 44) { |
| 280 | // Hash request - retrieve blob |
| 281 | const blob = await apds.get(msg) |
| 282 | if (blob) ws.send(blob) |
| 283 | } else { |
| 284 | // New content or signature - store and gossip |
| 285 | const hash = await apds.make(msg) // Creates hash of msg |
| 286 | await apds.add(msg) // Stores msg by hash |
| 287 | |
| 288 | // Gossip to all peers |
| 289 | for (const peer of sockets) { |
| 290 | peer.send(msg) |
| 291 | } |
| 292 | |
| 293 | // Forward to wiredove.net |
| 294 | if (bridgeWs) bridgeWs.send(msg) |
| 295 | } |
| 296 | }) |
| 297 | ``` |
| 298 | |
| 299 | **Key Insight:** The relay stores BOTH the post YAML and the image data URL by their respective hashes. When the relay stores the data URL, it hashes it exactly as wiredove would, so the hashes match. |
| 300 | |
| 301 | ### 6. Wiredove Stores & Displays |
| 302 | |
| 303 | - Post stored: `hash(YAML)` → YAML content |
| 304 | - Image stored: `hash(dataURL)` → data URL content |
| 305 | - Post references image by `hash(dataURL)` in YAML |
| 306 | - When displaying, wiredove fetches image by hash from APDS |
| 307 | |
| 308 | ## Testing & Verification |
| 309 | |
| 310 | ### Verify Image Upload |
| 311 | |
| 312 | ```bash |
| 313 | # 1. Create post with image via pear-app |
| 314 | curl -X POST http://localhost:3003/api/post \ |
| 315 | -H "Content-Type: application/json" \ |
| 316 | -d '{ |
| 317 | "message": "Test image", |
| 318 | "imageData": "<base64>", |
| 319 | "imageMimeType": "image/jpeg" |
| 320 | }' |
| 321 | |
| 322 | # 2. Trigger bridge sync |
| 323 | curl -X POST http://localhost:3003/api/bridge/sync |
| 324 | |
| 325 | # 3. Check post on wiredove (wait 10s for propagation) |
| 326 | curl -s https://pub.wiredove.net/all | \ |
| 327 | jq --arg pubkey "YOUR_PUBKEY" \ |
| 328 | '[.[] | select(.author == $pubkey)][-1]' |
| 329 | |
| 330 | # 4. Extract image hash from YAML |
| 331 | # Example: "image: abc123..." |
| 332 | |
| 333 | # 5. Verify image blob exists |
| 334 | curl -I https://pub.wiredove.net/<image-hash> |
| 335 | # Should return HTTP 200 (not 404!) |
| 336 | |
| 337 | # 6. View on web |
| 338 | # Visit https://wiredove.net/ - image should display |
| 339 | ``` |
| 340 | |
| 341 | ### Common Errors |
| 342 | |
| 343 | **404 on image blob:** |
| 344 | - Hash mismatch - you hashed raw bytes but sent data URL |
| 345 | - Image not sent separately - bundled with post instead |
| 346 | |
| 347 | **Post not appearing on wiredove.net:** |
| 348 | - Missing opening `---` in YAML |
| 349 | - Signature verification failed |
| 350 | - Relay not forwarding to wiredove bridge |
| 351 | |
| 352 | **Old posts have broken images:** |
| 353 | - Expected - they were created with wrong hash algorithm |
| 354 | - Only new posts (after fix) will work |
| 355 | |
| 356 | ## Critical Research Sources |
| 357 | |
| 358 | These wiredove-cli files were essential to understanding the protocol: |
| 359 | |
| 360 | 1. **`wiredove-cli/wiredove.js:181-206`** - `uploadImage()` function |
| 361 | - Shows: Hash the data URL, not raw bytes |
| 362 | - Shows: Send data URL as string |
| 363 | |
| 364 | 2. **`wiredove/composer.js:95-107`** - Post sending logic |
| 365 | - Shows: Send post first, then extract image refs and send separately |
| 366 | - Shows: Images are independent blobs requested by hash |
| 367 | |
| 368 | 3. **`apds/index.js`** - APDS storage implementation |
| 369 | - Shows: Everything is content-addressed by hash |
| 370 | - Shows: No special handling for images vs posts |
| 371 | |
| 372 | ## Backward Compatibility |
| 373 | |
| 374 | ### Feed Storage Evolution |
| 375 | |
| 376 | **Old Format (broken images):** |
| 377 | ```javascript |
| 378 | { |
| 379 | yaml: "...", |
| 380 | imageData: "base64string", // Just base64 |
| 381 | imageMimeType: "image/jpeg", |
| 382 | imageHash: hash(rawBytes) // Wrong hash! |
| 383 | } |
| 384 | ``` |
| 385 | |
| 386 | **New Format (working images):** |
| 387 | ```javascript |
| 388 | { |
| 389 | yaml: "...", |
| 390 | imageData: "data:image/jpeg;base64,base64string", // Full data URL |
| 391 | imageHash: hash(dataURL) // Correct hash! |
| 392 | } |
| 393 | ``` |
| 394 | |
| 395 | ### UI Compatibility |
| 396 | |
| 397 | Handle both formats in display code: |
| 398 | |
| 399 | ```javascript |
| 400 | if (msg.imageData) { |
| 401 | let dataUrl |
| 402 | if (msg.imageData.startsWith('data:')) { |
| 403 | // New format - already data URL |
| 404 | dataUrl = msg.imageData |
| 405 | } else { |
| 406 | // Old format - construct data URL |
| 407 | dataUrl = `data:${msg.imageMimeType || 'image/jpeg'};base64,${msg.imageData}` |
| 408 | } |
| 409 | imageHtml = `<img src="${dataUrl}" />` |
| 410 | } |
| 411 | ``` |
| 412 | |
| 413 | ## Key Takeaways |
| 414 | |
| 415 | 1. **Always hash what you send** - If sending data URLs, hash data URLs |
| 416 | 2. **Images are separate messages** - Not bundled with posts |
| 417 | 3. **Research existing implementations** - wiredove-cli was the reference |
| 418 | 4. **YAML format matters** - Opening delimiter is required |
| 419 | 5. **API differences matter** - bare-ws uses Duplex stream API |
| 420 | 6. **Content-addressed systems require consistency** - Hash must match content exactly |
| 421 | |
| 422 | ## Related Files |
| 423 | |
| 424 | - `/pear-app/index.js` - Post creation endpoint |
| 425 | - `/pear-app/bridge/wirecore-bridge.js` - Relay sync logic |
| 426 | - `/pear-app/lib/relay-client.js` - WebSocket push/pull |
| 427 | - `/pear-app/lib/yaml-helper.js` - YAML formatting |
| 428 | - `/pear-app/lib/anproto.js` - Hashing and signing |
| 429 | - `/pear-app/public/app-simple.js` - UI display logic |
| 430 | - `/relay/wirecore-relay.js` - Relay message handling |
| 431 | |
| 432 | ## Version History |
| 433 | |
| 434 | - **v4241** (2026-01-21): Initial broken implementation |
| 435 | - Hashed raw bytes, sent data URLs |
| 436 | - Images bundled with posts |
| 437 | - Result: 404 on all images |
| 438 | |
| 439 | - **v4253** (2026-01-22): Fixed hashing |
| 440 | - Hash data URLs instead of raw bytes |
| 441 | - Store data URLs in feeds |
| 442 | - Result: Still 404 (still bundling images) |
| 443 | |
| 444 | - **v4254** (2026-01-22): Fixed message protocol |
| 445 | - Send images as separate messages |
| 446 | - Two WebSocket connections per post with image |
| 447 | - Result: Images work! HTTP 200 on wiredove.net |
| 448 | |
| 449 | - **v4257** (2026-01-22): UI backward compatibility |
| 450 | - Handle both old (base64) and new (data URL) formats |
| 451 | - Current stable version |
| 452 | |
| 453 | ## Future Improvements |
| 454 | |
| 455 | 1. **Batch image uploads** - Queue multiple images, send in batch |
| 456 | 2. **Progress indicators** - Show upload progress for large images |
| 457 | 3. **Image compression** - Resize/compress before upload |
| 458 | 4. **P2P image gossip** - Direct peer-to-peer image sharing via Protomux |
| 459 | 5. **Cache management** - LRU eviction for stored images |
| 460 | 6. **Thumbnail generation** - Create smaller previews for timeline |
| 461 | |
| 462 | ## Debugging Tips |
| 463 | |
| 464 | 1. **Check relay logs** - See what messages relay receives |
| 465 | ```bash |
| 466 | tail -f /tmp/wirecore-relay.log | grep -A 5 "image\|blob" |
| 467 | ``` |
| 468 | |
| 469 | 2. **Verify hashes match** - Compare YAML reference to stored blob |
| 470 | ```bash |
| 471 | # Extract image hash from YAML |
| 472 | curl -s https://pub.wiredove.net/all | jq '.[-1].text' |
| 473 | |
| 474 | # Check if blob exists at that hash |
| 475 | curl -I https://pub.wiredove.net/<hash-from-yaml> |
| 476 | ``` |
| 477 | |
| 478 | 3. **Test WebSocket separately** - Use test client to verify protocol |
| 479 | ```bash |
| 480 | node test-websocket.js |
| 481 | ``` |
| 482 | |
| 483 | 4. **Compare with wiredove-cli** - Run side-by-side, compare behavior |
| 484 | ```bash |
| 485 | # Wiredove-cli post |
| 486 | wiredove post "test" /path/to/image.jpg |
| 487 | |
| 488 | # Check wiredove.net |
| 489 | curl -s https://pub.wiredove.net/all | jq '.[-1]' |
| 490 | |
| 491 | # Wirecore post |
| 492 | curl -X POST http://localhost:3003/api/post ... |
| 493 | |
| 494 | # Compare YAML format |
| 495 | ``` |
| 496 | |
| 497 | 5. **Monitor network traffic** - Use wireshark/tcpdump on relay |
| 498 | ```bash |
| 499 | tcpdump -i any -A 'tcp port 9000' |
| 500 | ``` |