OverviewFilesCommitsIssuesDocsReleasesPeers

godot-keet commits / 8d97132

Add UID files + tests; daemon: selective GC for broken rooms
CI copies updated keet-daemon with selective GC — tracks broken rooms from KEET_CONTRACT_ABORT errors and purges only those instead of nuking the entire database. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
8d971327e69c28780c38497e1e20deedeec467e3
by plankatron <fleeky@gmail.com> · 2026-02-23T13:55:31.000Z

Changed files (17) +562 -0

SYSTEMATIC_PLAN.md +218 -0
……@@ -0,0 +1,218 @@
1# Systematic Plan: Making "Rooms Are The Computer" Work
2
3## Problem Statement
4
5Steps 1-7 replaced working Autobase discovery with room-based alternatives.
6The result works LESS well than before because:
7
81. **Deploys are broken** — pear-runtime OTA fails (missing transitive deps)
92. **No metadata input** — users cannot add tags/topics to rooms
103. **Clustering is blind** — MI computes but can't label or explain groups
114. **No auto-emergence** — computed clusters don't become visible groups
12
13## Approach: Bottom-Up, Each Step Independently Verifiable
14
15No step depends on future steps. Each step is testable in isolation.
16Each step has a concrete success criterion that can be verified on-device.
17
18---
19
20## Phase A: Fix Deploy (must work before anything else)
21
22### Problem
23`npm install --production` resolves godot-keet's 10 public deps + their
24transitive trees. But keet-daemon imports `@holepunchto/*` packages (private,
25not on npm). The CI must overlay these without breaking npm's resolved tree.
26
27### Root Cause
28The current CI either:
29- rsync's ALL of keet-cli's node_modules (overkill, may conflict with npm's tree)
30- cp's only @holepunchto (insufficient, misses keet-daemon's own transitive deps)
31
32### Fix
33`npm install --production` already resolves pear-runtime → corestore →
34random-access-file correctly. The ONLY private deps needed are:
35- `node_modules/@holepunchto/*` (25 packages)
36- `node_modules/keet-daemon/` (our wrapper)
37
38The fix: cp private deps AFTER npm install, don't rsync the entire tree.
39If npm install works locally (it does — 132 packages including random-access-file),
40it will work in CI.
41
42### Success Criterion
43```
44On ashatron: run godot-keet, check logs for:
45- NO "MODULE_NOT_FOUND" errors
46- pear-runtime initializes successfully
47- OTA check completes (success or "up to date")
48```
49
50### Verification Command
51```bash
52pgh deploy log godot-keet | tail -20
53# Should show: all CI steps pass, stage + release succeed
54```
55
56---
57
58## Phase B: Enable Metadata Input (tags + discoverable)
59
60### Problem
61The daemon supports `meta.set(roomId, {topics: ["rust", "p2p"], discoverable: true})`
62but the Godot UI never calls it. `register_room()` hardcodes `topics: []`.
63
64### What Exists (daemon side — WORKING)
65- `meta.set(roomId, metadata)` — merges arbitrary metadata into pinned _hs:room-meta message
66- `meta.get(roomId)` — reads pinned _hs:room-meta message
67- `topic.announce(roomId)` — reads metadata, announces on DHT per topic
68- `registry.add(registryRoomId, {topics: [...]})` — stores topics in listing
69
70### What's Missing (Godot side)
711. Room Settings → no topics field
722. Room Settings → no discoverable toggle
733. register_room() → topics always `[]`
744. No metadata display anywhere
75
76### Changes (all in room_settings.gd)
77
78**Add to General tab:**
79- `LineEdit` for topics (comma-separated, e.g. "rust, p2p, gaming")
80- `CheckButton` for "Discoverable on DHT"
81- On save: call `meta.set(roomId, {topics: [...], discoverable: bool})`
82
83**Fix Discovery tab:**
84- `register_room()` reads topics from metadata instead of hardcoding `[]`
85- Show current topics next to each registration checkbox
86
87### Success Criterion
88```
891. Open Room Settings for any room
902. Type "rust, p2p" in topics field
913. Toggle "Discoverable" on
924. Click Save
935. Close and reopen Room Settings
946. Topics field shows "rust, p2p"
957. Discoverable toggle is on
968. Register room in a group → group listing shows topics
97```
98
99### Verification
100```bash
101# From daemon debug:
102echo '{"method":"meta.get","params":{"roomId":"ROOM_ID"}}' | ...
103# Should return: {topics: ["rust", "p2p"], discoverable: true}
104```
105
106---
107
108## Phase C: Label Clusters Automatically
109
110### Problem
111MI clustering finds groups of rooms with overlapping members but can't
112explain WHY they're grouped. Clusters show as "cluster:0" with no name.
113
114### Solution: Topic Overlap Labels
115When MI identifies a cluster, scan the topics of all rooms in that cluster.
116The most common topics across the cluster's rooms become the cluster label.
117
118### Changes (daemon: local-graph.js)
119
120After computing clusters via MI, for each cluster:
1211. Collect all topics from rooms in the cluster (via meta.get)
1222. Count topic frequency
1233. Cluster label = top 2 topics joined by " + " (e.g. "rust + p2p")
1244. If no topics: label = "N rooms" (fallback)
125
126### Changes (Godot: group_bar.gd)
127
128Display `cluster.label` in the icon tooltip and group header.
129Currently the label field exists but is ignored.
130
131### Success Criterion
132```
1331. Add topics "rust" to 3 rooms that share members
1342. Add topics "gaming" to 2 other rooms that share members
1353. Refresh graph
1364. GroupBar shows cluster icon with tooltip "rust" (not "cluster:0")
1375. Selecting cluster shows rooms with the label at the top
138```
139
140### Verification
141```bash
142# From daemon:
143echo '{"method":"graph.compute","params":{"force":true}}' | ...
144# Response clusters should include: {id: 0, rooms: [...], label: "rust + p2p"}
145```
146
147---
148
149## Phase D: Graph Rendering Validation
150
151### Problem
152Graph data computes (15 nodes, 97 edges, 1 cluster) and is sent to canvas,
153but user reports "graph is broken." Need to verify rendering on-device.
154
155### Possible Issues
1561. GraphPanel not visible (member panel in the way)
1572. Force-directed layout produces degenerate positions (all nodes on top of each other)
1583. Canvas draw coordinates off-screen
1594. Graph toggle button not functional
160
161### Diagnostic Steps (ON DEVICE, not code changes)
1621. Click the Graph button in chat panel header
1632. Check: does GraphPanel appear? (should replace member panel)
1643. Check: are nodes visible? (colored circles)
1654. Check: are edges visible? (white/gray lines)
1665. Check: can you zoom/pan?
167
168### If Nodes Overlap
169The force-directed layout has REPULSION=500, but with 15 nodes and 97 edges
170(very dense), attraction may dominate. Fix: increase REPULSION or reduce
171ATTRACTION proportional to edge density.
172
173### Success Criterion
174```
1751. Click Graph button → graph panel visible
1762. Nodes are spread across the canvas (not overlapping)
1773. Edges visible between connected nodes
1784. Cluster nodes share the same color
1795. Hover shows room title
1806. Click navigates to room
181```
182
183---
184
185## Implementation Order
186
187```
188A (fix deploy) → B (metadata input) → C (auto-labels) → D (graph validation)
189```
190
191Each phase is independently verifiable.
192No phase writes code that depends on a future phase.
193Each phase can be deployed and tested before starting the next.
194
195## Estimated Sizes
196
197| Phase | Files Changed | Lines Added | Lines Removed |
198|-------|--------------|-------------|---------------|
199| A | ci.yaml | 0 | 0 (fix params) |
200| B | room_settings.gd | ~50 | ~5 |
201| C | local-graph.js, group_bar.gd | ~30 | ~5 |
202| D | graph_canvas.gd (if needed) | ~10 | ~5 |
203| **Total** | **4-5 files** | **~90 lines** | **~15 lines** |
204
205## What This Does NOT Do
206
207- Does not auto-create registry rooms from clusters (manual > magical)
208- Does not add complex tag taxonomy (simple comma-separated topics is enough)
209- Does not change the MI algorithm (it works correctly)
210- Does not add new daemon routes (all routes already exist)
211- Does not refactor architecture (uses what's already built)
212
213## Key Principle
214
215The daemon already has everything needed. The problem is that the Godot UI
216doesn't expose the daemon's capabilities. Phase B (metadata input) is the
217critical missing piece — without it, clustering has nothing meaningful to
218work with.
addons/keet_social/keet_discovery.gd.uid +1 -0
……@@ -0,0 +1 @@
1uid://047208br4nig
addons/keet_social/keet_dm.gd.uid +1 -0
……@@ -0,0 +1 @@
1uid://cuwrxh4um3avj
addons/keet_social/keet_graph.gd.uid +1 -0
……@@ -0,0 +1 @@
1uid://do7fidf1i0n3f
scripts/avatar_picker.gd.uid +1 -0
……@@ -0,0 +1 @@
1uid://88jc0kvn1mgh
scripts/chat_input.gd.uid +1 -0
……@@ -0,0 +1 @@
1uid://mkmrpnmpvex4
scripts/graph_canvas.gd.uid +1 -0
……@@ -0,0 +1 @@
1uid://bko8vvkdwsesl
scripts/graph_panel.gd.uid +1 -0
……@@ -0,0 +1 @@
1uid://b5utceogmyc8w
scripts/group_bar.gd.uid +1 -0
……@@ -0,0 +1 @@
1uid://bb8wknk3tmmwi
scripts/group_settings.gd.uid +1 -0
……@@ -0,0 +1 @@
1uid://nvg05ysjwbs7
scripts/image_lightbox.gd.uid +1 -0
……@@ -0,0 +1 @@
1uid://cvflmb6wwpbem
scripts/invite_dialog.gd.uid +1 -0
……@@ -0,0 +1 @@
1uid://bqfoh4cyw8kge
scripts/room_settings.gd.uid +1 -0
……@@ -0,0 +1 @@
1uid://dme84p0yfspwx
tests/check-ait.mjs +49 -0
……@@ -0,0 +1,49 @@
1import Pipe from 'bare-pipe'
2import FramedStream from 'framed-stream'
3import b4a from 'b4a'
4
5const pipe = new Pipe('/tmp/keet-daemon.sock')
6const framed = new FramedStream(pipe)
7let id = 0
8
9function call (m, p) {
10 return new Promise((res, rej) => {
11 const i = ++id
12 const t = setTimeout(() => rej(new Error('timeout')), 10000)
13 framed.on('data', function h (buf) {
14 const msg = JSON.parse(b4a.toString(buf))
15 if (msg.id === i) {
16 framed.removeListener('data', h)
17 clearTimeout(t)
18 if (msg.error) rej(new Error(msg.error.message || JSON.stringify(msg.error)))
19 else res(msg.result || msg.data)
20 }
21 })
22 framed.write(b4a.from(JSON.stringify({ jsonrpc: '2.0', id: i, method: m, params: p || {} })))
23 })
24}
25
26async function run () {
27 const rooms = await call('rooms')
28 const ait = rooms.find(r => r.title === 'AI Testing Town')
29 if (!ait) { console.log('Not found'); framed.destroy(); pipe.destroy(); return }
30 const AIT = ait.roomId || ait.id
31 console.log('roomId:', AIT)
32 const msgs = await call('read', { roomId: AIT, limit: 20, reverse: true })
33 const messages = Array.isArray(msgs) ? msgs : (msgs?.messages || msgs?.data || [])
34
35 console.log('Last 15 messages in AI Testing Town:\n')
36 for (const m of messages) {
37 const text = m.chat?.text || m.text || m.message?.text || ''
38 const ts = m.chat?.timestamp || m.timestamp || 0
39 const date = ts ? new Date(ts).toISOString().slice(0, 19) : 'unknown'
40 const prefix = text.startsWith('{"_hs":') ? '[_hs]' : '[chat]'
41 console.log(`${date} ${prefix} ${text.slice(0, 120)}`)
42 }
43
44 framed.destroy()
45 pipe.destroy()
46 setTimeout(() => Bare.exit(0), 500)
47}
48
49run().catch(err => { console.error('FAIL:', err.message); Bare.exit(1) })
tests/check-meta-key.mjs +34 -0
……@@ -0,0 +1,34 @@
1import Pipe from 'bare-pipe'
2import FramedStream from 'framed-stream'
3import b4a from 'b4a'
4
5let rpcId = 0
6const pipe = new Pipe('/tmp/keet-daemon.sock')
7const framed = new FramedStream(pipe)
8const pending = new Map()
9
10framed.on('data', buf => {
11 const msg = JSON.parse(b4a.toString(buf))
12 const cb = pending.get(msg.id)
13 if (cb) { pending.delete(msg.id); cb(msg) }
14})
15
16function call (method, params = {}) {
17 return new Promise((resolve, reject) => {
18 const id = ++rpcId
19 const timeout = setTimeout(() => { pending.delete(id); reject(new Error('timeout')) }, 10000)
20 pending.set(id, msg => {
21 clearTimeout(timeout)
22 if (msg.ok !== undefined && msg.data !== undefined) msg.result = msg.data
23 if (msg.error) reject(new Error(msg.error.message || String(msg.error)))
24 else resolve(msg.result)
25 })
26 framed.write(b4a.from(JSON.stringify({ jsonrpc: '2.0', id, method, params })))
27 })
28}
29
30console.log('MetadataBase key:', await call('meta.key'))
31console.log('Meta invite:', await call('meta.invite'))
32
33pipe.destroy()
34setTimeout(() => Bare.exit(0), 500)
tests/meta-sync.mjs +105 -0
……@@ -0,0 +1,105 @@
1/**
2 * meta-sync.mjs — Check MetadataBase state and optionally join a remote key
3 *
4 * Usage:
5 * bare tests/meta-sync.mjs # show current state
6 * bare tests/meta-sync.mjs <64-char-hex-key> # join remote MetadataBase
7 */
8import Pipe from 'bare-pipe'
9import FramedStream from 'framed-stream'
10import b4a from 'b4a'
11
12let rpcId = 0
13const pipe = new Pipe('/tmp/keet-daemon.sock')
14const framed = new FramedStream(pipe)
15const pending = new Map()
16
17framed.on('data', buf => {
18 const msg = JSON.parse(b4a.toString(buf))
19 const cb = pending.get(msg.id)
20 if (cb) { pending.delete(msg.id); cb(msg) }
21})
22
23function call (method, params = {}) {
24 return new Promise((resolve, reject) => {
25 const id = ++rpcId
26 const timeout = setTimeout(() => { pending.delete(id); reject(new Error('timeout: ' + method)) }, 15000)
27 pending.set(id, msg => {
28 clearTimeout(timeout)
29 if (msg.ok !== undefined && msg.data !== undefined) msg.result = msg.data
30 if (msg.error) reject(new Error(msg.error.message || String(msg.error)))
31 else resolve(msg.result)
32 })
33 framed.write(b4a.from(JSON.stringify({ jsonrpc: '2.0', id, method, params })))
34 })
35}
36
37// Show current state
38console.log('=== MetadataBase State ===\n')
39try {
40 const keyInfo = await call('meta.key')
41 console.log('Key:', keyInfo.key)
42 console.log('Local key:', keyInfo.localKey)
43 console.log('Writable:', keyInfo.writable)
44} catch (err) {
45 console.log('meta.key error:', err.message)
46}
47
48// Check how many rooms have metadata
49try {
50 const rooms = await call('rooms')
51 let withMeta = 0
52 let withoutMeta = 0
53 for (const r of rooms) {
54 const rid = r.roomId || r.id
55 try {
56 const meta = await call('meta.get', { roomId: rid })
57 if (meta && meta.category) withMeta++
58 else withoutMeta++
59 } catch {
60 withoutMeta++
61 }
62 }
63 console.log(`\nRooms: ${rooms.length} total, ${withMeta} with metadata, ${withoutMeta} without`)
64} catch (err) {
65 console.log('rooms error:', err.message)
66}
67
68// Join if key provided
69const joinKey = Bare.argv[Bare.argv.length - 1]
70if (joinKey && joinKey.length === 64 && /^[0-9a-f]+$/.test(joinKey)) {
71 console.log(`\n=== Joining MetadataBase: ${joinKey.slice(0, 16)}... ===\n`)
72 try {
73 const result = await call('meta.join', { key: joinKey })
74 console.log('Join result:', JSON.stringify(result, null, 2))
75
76 // Verify state after join
77 console.log('\n=== Post-Join State ===\n')
78 const keyInfo = await call('meta.key')
79 console.log('Key:', keyInfo.key)
80 console.log('Writable:', keyInfo.writable)
81
82 // Wait a bit for replication, then check metadata
83 console.log('\nWaiting 5s for replication...')
84 await new Promise(r => setTimeout(r, 5000))
85
86 const rooms = await call('rooms')
87 let withMeta = 0
88 for (const r of rooms) {
89 const rid = r.roomId || r.id
90 try {
91 const meta = await call('meta.get', { roomId: rid })
92 if (meta && meta.category) {
93 withMeta++
94 console.log(` ${(r.title || '?').padEnd(20)} cat=${meta.category}`)
95 }
96 } catch {}
97 }
98 console.log(`\n${withMeta} rooms now have metadata`)
99 } catch (err) {
100 console.log('Join error:', err.message)
101 }
102}
103
104pipe.destroy()
105setTimeout(() => Bare.exit(0), 500)
tests/query-rooms.mjs +144 -0
……@@ -0,0 +1,144 @@
1/**
2 * query-rooms.mjs — List all rooms and their current metadata
3 * Usage: bare tests/query-rooms.mjs
4 */
5
6import Pipe from 'bare-pipe'
7import FramedStream from 'framed-stream'
8import b4a from 'b4a'
9
10const SOCKET = '/tmp/keet-daemon.sock'
11let rpcId = 0
12
13function connect () {
14 const pipe = new Pipe(SOCKET)
15 const framed = new FramedStream(pipe)
16 const pending = new Map()
17
18 framed.on('data', (buf) => {
19 try {
20 const msg = JSON.parse(b4a.toString(buf))
21 const cb = pending.get(msg.id)
22 if (cb) {
23 pending.delete(msg.id)
24 if (msg.ok !== undefined && msg.data !== undefined) msg.result = msg.data
25 cb(msg)
26 }
27 } catch {}
28 })
29
30 function call (method, params = {}) {
31 return new Promise((resolve, reject) => {
32 const id = ++rpcId
33 const timeout = setTimeout(() => {
34 pending.delete(id)
35 reject(new Error(`RPC timeout: ${method}`))
36 }, 15000)
37 pending.set(id, (msg) => {
38 clearTimeout(timeout)
39 if (msg.error) reject(new Error(msg.error.message || msg.error))
40 else resolve(msg.result)
41 })
42 const payload = JSON.stringify({ jsonrpc: '2.0', id, method, params })
43 framed.write(b4a.from(payload))
44 })
45 }
46
47 function close () { framed.destroy(); pipe.destroy() }
48 return { call, close }
49}
50
51async function run () {
52 const client = connect()
53
54 // Check available meta routes
55 let hasList = true
56 try {
57 await client.call('introspect')
58 } catch {}
59
60 const rooms = await client.call('rooms')
61
62 // Try meta.list, fall back to per-room meta.get
63 let allMeta = {}
64 try {
65 allMeta = await client.call('meta.list')
66 } catch {
67 // meta.list not available, fetch per room
68 for (const room of rooms) {
69 const id = room.roomId || room.id
70 try {
71 const meta = await client.call('meta.get', { roomId: id })
72 if (meta && Object.keys(meta).length > 0) {
73 allMeta[id] = meta
74 }
75 } catch {}
76 }
77 }
78
79 console.log(`\n=== ${rooms.length} rooms total, ${Object.keys(allMeta).length} have metadata ===\n`)
80
81 const withMeta = []
82 const withoutMeta = []
83
84 for (const room of rooms) {
85 const id = room.roomId || room.id
86 const title = room.title || '(untitled)'
87 const meta = allMeta[id]
88
89 if (meta && Object.keys(meta).filter(k => !k.startsWith('_')).length > 0) {
90 withMeta.push({ id, title, meta })
91 } else {
92 withoutMeta.push({ id, title })
93 }
94 }
95
96 if (withMeta.length > 0) {
97 console.log('--- ROOMS WITH METADATA ---\n')
98 for (const { id, title, meta } of withMeta) {
99 const clean = {}
100 for (const [k, v] of Object.entries(meta)) {
101 if (!k.startsWith('_')) clean[k] = v
102 }
103 console.log(` "${title}" (${id.slice(0, 16)}...)`)
104 console.log(` ${JSON.stringify(clean)}`)
105 }
106 }
107
108 if (withoutMeta.length > 0) {
109 console.log('\n--- ROOMS WITHOUT METADATA ---\n')
110 for (const { id, title } of withoutMeta) {
111 console.log(` "${title}" (${id.slice(0, 16)}...)`)
112 }
113 }
114
115 // Get room details for ALL rooms
116 console.log('\n--- ROOM DETAILS ---\n')
117 for (const room of rooms) {
118 const id = room.roomId || room.id
119 const title = room.title || '(untitled)'
120 try {
121 const info = await client.call('room', { roomId: id })
122 const roomType = info.digest?.settings?.roomType
123 const memberCount = info.status?.memberCount || info.digest?.memberCount || '?'
124 const connected = info.status?.connectedPeers || '?'
125 const isWritable = info.status?.isWritable
126 const hasMeta = !!allMeta[id]
127 const typeLabel = roomType === 1 ? 'Default' : roomType === 2 ? 'Broadcast' : roomType === 3 ? 'DM' : `type=${roomType}`
128 console.log(` "${title}" [${typeLabel}] members=${memberCount} connected=${connected} writable=${isWritable} meta=${hasMeta}`)
129 console.log(` id: ${id}`)
130 } catch (err) {
131 console.log(` "${title}" — error: ${err.message}`)
132 }
133 }
134
135 console.log('\n=== SUMMARY ===')
136 console.log(` With metadata: ${withMeta.length}`)
137 console.log(` Without metadata: ${withoutMeta.length}`)
138 console.log(` Total rooms: ${rooms.length}`)
139
140 client.close()
141 setTimeout(() => Bare.exit(0), 500)
142}
143
144run().catch(err => { console.error('FATAL:', err.message); Bare.exit(1) })