OverviewFilesCommitsIssuesDocsReleasesPeers

favela-surface-android commits / 48aa826

the bundle store can stage a release's addons, and hasTree means the whole tree
Two changes to bundleStore, one of them a latent bug fix that stands on its own. materialize() took the single entry named worklet.bundle and dropped everything else on the floor. It now also reads an `addons/` directory entry via the fetchTree pgh-runtime already passes it, writing each file .part-then-rename and honouring the entry's exec bit — which the tree format carries inside the identity hash, so it cannot drift from what was published. Android needs execute on the file object to dlopen out of app storage, and a mode that quietly defaulted to 0644 would fail at load with an error naming the path, not the permission. A release with no addons/ is the single-file store it has always been; every release published so far is { package.json, worklet.bundle } and those keep working untouched. hasTree() checked only that the bundle was non-empty. installTree SHORT-CIRCUITS on hasTree, so a run that wrote the bundle and died before the addons would have reported complete for ever, and the bundle that then booted would demand a soname nothing on the device provides. That is the "marker before the bytes" bug this file already guards against one level down, at the level that cannot be fixed remotely. The marker now records the addon names and hasTree verifies each. addonsFor() returns the staged addon directory, or null when the release carries none — never an empty dir, because a loader has to tell "no addons in this release" apart from "addons that did not arrive". Why this shape: a `linked:` bundle names the sonames the APK must supply, so a bundle whose addon set the APK cannot satisfy could only die at dlopen. Bare's own Addon class is the way out — `new Addon(fileURL)` loads from an absolute path, and Addon.cache is keyed by url.href and consulted by the `linked:` branch of Addon.resolve, so a staged addon can be registered under the name the bundle asks for and wins before any dlopen-by-soname, while an unstaged one still falls through to the APK. Measured on linux-x64 against a real --linked bundle: the same bundle that failed ADDON_NOT_FOUND loads and runs once its addons are preloaded, and a mixed run (2 staged, 4 from a simulated APK lib dir) also runs. pgh-runtime needs nothing for this: installTree hands the whole entries array to store.materialize along with a fetchTree to recurse with, and never inspects a name. The single-file assumption was ours alone. No version bump: the loader that consumes these files is not written yet, and a version is what CI publishes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
48aa826b69e6bb3a7c2bed300818b17f43b2f63e
by plankatron <fleeky@gmail.com> · 2026-09-11T00:01:24.000Z

Changed files (2) +158 -6

lib/host/ota.js +66 -5
……@@ -27,7 +27,7 @@
2727import fs from 'bare-fs'
2828import path from 'bare-path'
2929import crypto from 'hypercore-crypto'
30import { computeTreeHash, KIND_FILE } from 'hyper-overlay/tree'
30import { computeTreeHash, KIND_FILE, KIND_DIR } from 'hyper-overlay/tree'
3131
3232const DEFAULT_BASE = 'https://7rnx.net/git'
3333// ⛔ THIS REPO'S OWN NAME, and the reason is not tidiness. This file was copied verbatim from
……@@ -49,6 +49,8 @@ const DEFAULT_INTERVAL = 6 * 60 * 60 * 1000 // 6h — a phone is not a server; d
4949// How long to let a COLD registry replica settle before reading it. See swarmFetcher.
5050const COLD_SETTLE_MS = 30000
5151const BUNDLE_NAME = 'worklet.bundle'
52// The directory a release puts its native addons in. CI publishes it beside the bundle.
53const ADDONS_NAME = 'addons'
5254
5355// ⛔ THE STACK, NOT JUST THE MESSAGE. Three separate homedir defaults have now produced the SAME
5456// ENOENT on a handset, each masked by the one before it, and each cost a build-and-sideload cycle to
……@@ -224,6 +226,7 @@ export function bundleFetcher ({ base = DEFAULT_BASE, repo = DEFAULT_REPO, doFet
224226 */
225227export function bundleStore ({ dir }) {
226228 const markerPath = (treeHash) => path.join(dir, treeHash + '.json')
229 const addonsDir = (treeHash) => path.join(dir, treeHash + '.addons')
227230 // The release currently being installed. pgh-runtime's installTree does not pass the release down to
228231 // the store, but it emits `updating` with it immediately beforehand — so the caller notes it here and
229232 // materialize can stamp the staged marker with a comparable build time. Without that the shell has no
……@@ -247,7 +250,16 @@ export function bundleStore ({ dir }) {
247250 hasTree (treeHash) {
248251 const m = readMarker(treeHash)
249252 if (!m || !m.file) return false
250 try { return fs.statSync(path.join(dir, m.file)).size > 0 } catch { return false }
253 try { if (fs.statSync(path.join(dir, m.file)).size === 0) return false } catch { return false }
254 // ⛔ AND EVERY ADDON. installTree SHORT-CIRCUITS on hasTree, so a run that wrote the bundle and
255 // died before the addons would report complete for ever — and the bundle that then boots demands
256 // a soname nothing on the device provides. Same "marker before the bytes" bug as above, one level
257 // up, and the level where it cannot be fixed remotely.
258 const addons = Array.isArray(m.addons) ? m.addons : []
259 for (const name of addons) {
260 try { if (fs.statSync(path.join(addonsDir(treeHash), name)).size === 0) return false } catch { return false }
261 }
262 return true
251263 },
252264
253265 /** Where the staged bundle for this tree lives, or null. */
……@@ -256,7 +268,22 @@ export function bundleStore ({ dir }) {
256268 return m && m.file ? path.join(dir, m.file) : null
257269 },
258270
259 async materialize (rootHash, entries, { fetchContent } = {}) {
271 /**
272 * Where this tree's staged addons live, or null when the release carries none.
273 *
274 * The loader registers each of these under the `linked:` URL the bundle asks for — Bare's
275 * `Addon.cache` is keyed by url.href and the `linked:` branch of `Addon.resolve` goes through
276 * `Addon.load`, so a staged addon wins before any dlopen-by-soname and an unstaged one still falls
277 * through to the APK. Null, never an empty dir: a loader must be able to tell "no addons in this
278 * release" from "addons that did not arrive".
279 */
280 addonsFor (treeHash) {
281 const m = readMarker(treeHash)
282 if (!m || !Array.isArray(m.addons) || m.addons.length === 0) return null
283 return addonsDir(treeHash)
284 },
285
286 async materialize (rootHash, entries, { fetchContent, fetchTree } = {}) {
260287 // ⛔ RECORD A BUILD TIME, WHATEVER THE RELEASE CALLS IT. chooseBundle compares this against the
261288 // APK's stamp, and a staged bundle with no time is one an APK install cannot supersede — the
262289 // failure that left a handset running 0.1.6's JS through four installs on 2026-08-25.
……@@ -291,14 +318,48 @@ export function bundleStore ({ dir }) {
291318 fs.writeFileSync(part, Buffer.from(bytes))
292319 fs.renameSync(part, dest)
293320 const file = want.childHash + '.bundle'
321
322 // ⛔ THE ADDONS, BEFORE THE MARKER. A `linked:` bundle names the sonames the APK must supply, and
323 // until now a bundle demanding one the APK lacks could only die at dlopen — on a device with no
324 // adb and no UI left to say why. Bare's Addon class is the way out: `new Addon(fileURL)` loads
325 // from an absolute path, so an addon delivered here can be registered under the `linked:` name
326 // the bundle asks for and win before any dlopen-by-soname.
327 //
328 // BY NAME AND BY KIND, for the same reason the bundle is: a release tree's shape is not ours to
329 // assume. Absent `addons/`, this is exactly the single-file store it has always been — every
330 // release published so far is { package.json, worklet.bundle } and must keep working untouched.
331 const addonDirEntry = (entries || []).find((e) => e && e.kind === KIND_DIR && e.name === ADDONS_NAME)
332 const addons = []
333 if (addonDirEntry) {
334 if (typeof fetchTree !== 'function') {
335 throw new Error('release carries ' + ADDONS_NAME + '/ but no fetchTree to read it with')
336 }
337 const adir = addonsDir(rootHash)
338 fs.mkdirSync(adir, { recursive: true })
339 const sub = await fetchTree(addonDirEntry.childHash)
340 for (const e of (sub || [])) {
341 if (!e || e.kind !== KIND_FILE) continue
342 const b = await fetchContent(e.childHash)
343 const d = path.join(adir, e.name)
344 const pt = d + '.part'
345 fs.writeFileSync(pt, Buffer.from(b))
346 // ⛔ THE EXEC BIT IS CARRIED IN THE TREE and it is part of the entry's identity hash, so it
347 // cannot drift from what was published. Android needs execute on the FILE OBJECT to dlopen
348 // it out of app storage; a mode that silently defaulted to 0644 would fail at load with an
349 // error naming the path and not the permission.
350 if (e.exec === 1) fs.chmodSync(pt, 0o755)
351 fs.renameSync(pt, d)
352 addons.push(e.name)
353 }
354 }
294355 // `builtAt` is what lets the SHELL compare a staged bundle with the one inside the APK. Without it
295356 // an APK install loses to whatever was staged before it — chooseBundle prefers staged, and tree
296357 // hashes carry no ordering, so a phone kept running the older JS until the next OTA pass caught up.
297 fs.writeFileSync(markerPath(rootHash), JSON.stringify({ treeHash: rootHash, file, builtAt }, null, 2) + '\n')
358 fs.writeFileSync(markerPath(rootHash), JSON.stringify({ treeHash: rootHash, file, builtAt, addons }, null, 2) + '\n')
298359 // current.json is how the REACT NATIVE side learns what is staged: it runs before the worklet and
299360 // cannot read this store's code (bare-fs), only its files. Written last, after the bytes and the
300361 // marker, so it never advertises something that is not fully on disk.
301 fs.writeFileSync(path.join(dir, 'current.json'), JSON.stringify({ treeHash: rootHash, file, builtAt }, null, 2) + '\n')
362 fs.writeFileSync(path.join(dir, 'current.json'), JSON.stringify({ treeHash: rootHash, file, builtAt, addons }, null, 2) + '\n')
302363 }
303364 }
304365}
test/ota.test.js +92 -1
……@@ -11,7 +11,7 @@ import fs from 'bare-fs'
1111import path from 'bare-path'
1212import os from 'bare-os'
1313import crypto from 'hypercore-crypto'
14import { computeTreeHash } from 'hyper-overlay/tree'
14import { computeTreeHash, KIND_FILE, KIND_DIR } from 'hyper-overlay/tree'
1515import { installTree } from 'pgh-runtime'
1616import { treeFor, bundleFetcher, bundleStore, pickNewest, chooseBundle } from '../lib/host/ota.js'
1717
……@@ -321,3 +321,94 @@ test('a condemned bundle is still refused', (t) => {
321321 t.is(chooseBundle({ current: staged, assetBuiltAt: 1, failed: ['a'.repeat(64)] }).use, 'asset',
322322 'condemned wins over any build-time comparison')
323323})
324
325// ── addons in the release tree ────────────────────────────────────────────────────────────────────
326//
327// ⛔ WHY THIS EXISTS. A `linked:` bundle names the sonames the APK must supply, so a bundle whose addon
328// set the APK cannot satisfy dies at dlopen with no way in from outside. Bare's own Addon class is the
329// way out: `new Addon(fileURL)` loads from an ABSOLUTE PATH, and `Addon.cache` is keyed by url.href and
330// consulted by the `linked:` branch of Addon.resolve — so an addon staged by OTA can be registered under
331// the `linked:` name the bundle asks for and wins before any dlopen-by-soname. That makes shipping
332// native code over OTA a question of getting the FILES onto the device, which is this store's job.
333//
334// pgh-runtime needs nothing for it: installTree hands `store.materialize` the whole entries array and a
335// `fetchTree` to recurse with, and never inspects a name. The single-file assumption is ours alone.
336
337// ota.js keeps its own; the definition is one line and duplicating it here keeps the test honest about
338// what a content hash IS rather than trusting the module under test to tell it.
339const hashOf = (bytes) => crypto.data(Buffer.from(bytes)).toString('hex')
340
341// A release tree shaped like the one CI would publish: the bundle beside an `addons/` directory.
342function treeWithAddons (bundleBytes, addons) {
343 const sub = Object.entries(addons)
344 .map(([name, bytes]) => ({ kind: KIND_FILE, exec: 1, name, childHash: hashOf(bytes) }))
345 .sort((a, b) => (a.name < b.name ? -1 : 1))
346 const subHash = computeTreeHash(sub)
347 const entries = [
348 { kind: KIND_DIR, exec: 0, name: 'addons', childHash: subHash },
349 { kind: KIND_FILE, exec: 0, name: 'worklet.bundle', childHash: hashOf(bundleBytes) }
350 ].sort((a, b) => (a.name < b.name ? -1 : 1))
351 return { entries, treeHash: computeTreeHash(entries), sub, subHash }
352}
353
354// installTree's fetcher contract, backed by memory — the forge routes are irrelevant here.
355function treeFetcher ({ trees, contents }) {
356 return {
357 async getLatestRelease () { return null },
358 async getTree (h) { return trees[h] ? { entries: trees[h] } : null },
359 async getContent (h) { return contents[h] }
360 }
361}
362
363const ADDONS = {
364 'libbare-fs.4.8.1.so': Buffer.from('\x7fELF pretend bare-fs'),
365 'libsodium-native.5.1.0.so': Buffer.from('\x7fELF pretend sodium')
366}
367
368test('a release tree carrying addons stages the addons, not just the bundle', async (t) => {
369 const dir = tmpdir('addons')
370 const { entries, treeHash, sub, subHash } = treeWithAddons(BUNDLE, ADDONS)
371 const contents = { [hashOf(BUNDLE)]: BUNDLE }
372 for (const [name, bytes] of Object.entries(ADDONS)) contents[hashOf(bytes)] = bytes
373
374 const store = bundleStore({ dir })
375 await installTree(store, treeHash, treeFetcher({ trees: { [treeHash]: entries, [subHash]: sub }, contents }))
376
377 t.ok(store.pathFor(treeHash), 'the bundle is staged')
378 const staged = store.addonsFor(treeHash)
379 t.ok(staged, 'the store reports where the addons went')
380 for (const [name, bytes] of Object.entries(ADDONS)) {
381 const p = path.join(staged, name)
382 t.ok(fs.existsSync(p), name + ' is on disk')
383 t.alike(fs.readFileSync(p), bytes, name + ' has the right bytes')
384 }
385})
386
387test('hasTree is false when the BUNDLE arrived but an addon did not', async (t) => {
388 // The same "marker before the bytes" bug one level up. installTree short-circuits on hasTree, so a
389 // run that wrote the bundle and died before the addons would report complete FOREVER — and the bundle
390 // that then boots demands a soname nothing on the device provides.
391 const dir = tmpdir('addons-partial')
392 const { entries, treeHash, sub, subHash } = treeWithAddons(BUNDLE, ADDONS)
393 const contents = { [hashOf(BUNDLE)]: BUNDLE }
394 for (const [name, bytes] of Object.entries(ADDONS)) contents[hashOf(bytes)] = bytes
395
396 const store = bundleStore({ dir })
397 await installTree(store, treeHash, treeFetcher({ trees: { [treeHash]: entries, [subHash]: sub }, contents }))
398 t.ok(store.hasTree(treeHash), 'complete after a clean install')
399
400 fs.unlinkSync(path.join(store.addonsFor(treeHash), 'libbare-fs.4.8.1.so'))
401 t.absent(store.hasTree(treeHash), 'one missing addon means the tree is NOT installed')
402})
403
404test('a bundle-only tree still stages, and reports no addons', async (t) => {
405 // Every release published so far is { package.json, worklet.bundle }. Those must keep working
406 // unchanged, and must not grow an empty addons dir that a loader would then trust.
407 const dir = tmpdir('addons-none')
408 const { entries, treeHash } = treeFor(BUNDLE)
409 const store = bundleStore({ dir })
410 await installTree(store, treeHash, treeFetcher({ trees: { [treeHash]: entries }, contents: { [hashOf(BUNDLE)]: BUNDLE } }))
411
412 t.ok(store.hasTree(treeHash), 'staged')
413 t.is(store.addonsFor(treeHash), null, 'no addons dir is reported for a bundle-only release')
414})