OverviewFilesCommitsIssuesDocsReleasesPeers

favela-surface-android commits / 8ad2c44

preload-addons: staged native addons can satisfy a bundle's linked: sonames
The loader half of OTA-delivered native code. bare-pack --preset android sets linked:true, so the worklet bundle carries no native bytes — it NAMES the libraries the APK must supply. That is why a JS-only OTA works and any native change has needed a 154 MB sideload: a staged bundle whose addon set the APK cannot satisfy dies at dlopen, on a device with no adb and no UI left to say why. Bare's own Addon class is the way out, from bare/src/addon.js: - new Addon(url) accepts file: and loads from an ABSOLUTE PATH - Addon.cache is a live object keyed by url.href - Addon.resolve()'s linked: branch returns Addon.load(resolution).url, and Addon.load checks that cache BEFORE constructing anything So an addon whose bytes arrived over the swarm can be loaded out of app storage and registered under the linked: URL the bundle will ask for. It wins; anything NOT staged still falls through to dlopen-by-soname and the APK. Staging is additive, which is what lets a release ship the one addon that changed rather than all 58. The filename is the whole contract — each staged file is named after the soname it satisfies, so the directory listing IS the mapping. No manifest to fall out of step with the bytes and no bundle-header parsing on a phone. CI names them from that header, the same list stage-addons.mjs already reads to build jniLibs. ⛔ This deliberately does NOT live inside the staged bundle. worklet.js hands bare-kit a path with a null source, so Bare loads the staged bundle as its own entry — preload code inside it runs too late, and its own import of bare-fs would resolve to linked:libbare-fs.<staged version>.so, the very soname that might be missing. The loader must be its own bundle packed against the APK's addon set. Wiring that entry is the next commit; nothing calls this yet. Tested, including the real thing rather than only a double: a genuine prebuild copied to libstaged-probe.9.9.9.so — a path the linker has never seen — loads through Bare's actual Addon, registers under the soname a bundle would ask for, and reports live exports. Also covered: additive staging, a missing directory (every release published so far has no addons), .part and junk files ignored, and one bad addon not taking the boot down with it. Suite 58/58, 224/224. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
8ad2c44c4d986a458f535b727b2a781547507cac
by plankatron <fleeky@gmail.com> · 2026-09-11T00:08:29.000Z

Changed files (2) +208 -0

lib/host/preload-addons.js +82 -0
……@@ -0,0 +1,82 @@
1// Register OTA-staged native addons so a bundle can demand a soname the APK does not carry.
2//
3// THE PROBLEM THIS SOLVES. `bare-pack --preset android` sets `linked: true`, so the worklet bundle does
4// not contain native code — it NAMES the libraries the APK must supply (`linked:librocksdb-native.3.17.2.so`).
5// That is why a JS-only OTA works and a native change has needed a 154 MB sideload: a staged bundle whose
6// addon set the APK cannot satisfy dies at dlopen, on a device with no adb and no UI left to say why.
7//
8// THE WAY OUT IS BARE'S OWN ADDON CLASS. From bare/src/addon.js:
9//
10// - `new Addon(url)` accepts `file:` and loads from an ABSOLUTE PATH — no linker search involved.
11// - `Addon.cache` is a live object keyed by `url.href`.
12// - `Addon.resolve()`'s `linked:` branch returns `Addon.load(resolution).url`, and `Addon.load` checks
13// that cache BEFORE constructing anything.
14//
15// So an addon whose bytes arrived over the swarm can be loaded from app storage and registered under the
16// `linked:` URL the bundle will ask for. It then wins, and anything NOT staged still falls through to
17// dlopen-by-soname and the APK. Staging is additive, which is what lets a release ship the one addon that
18// changed instead of all 58.
19//
20// ⛔ THIS MUST NOT BE PART OF THE STAGED BUNDLE. worklet.js hands bare-kit a path with a null source, so
21// Bare loads the staged bundle as its own entry — anything inside it runs too late, and its own import of
22// bare-fs would resolve to `linked:libbare-fs.<staged version>.so`, the very soname that might be missing.
23// The loader is its own bundle, packed against the APK's addon set, and loads the real one afterwards.
24//
25// ⚠ Addon.cache / Addon.load / Addon.resolve are all marked @deprecated upstream while bare-module still
26// calls them on every `require.addon`. If they are ever removed, the replacement is the same idea one
27// layer down — construct the Addon and hand it to whatever resolves addons then.
28import fs from 'bare-fs'
29import path from 'bare-path'
30import { pathToFileURL } from 'bare-url'
31
32// ⛔ THE FILENAME IS THE WHOLE CONTRACT. Each staged file is named after the soname it satisfies, so the
33// directory listing IS the mapping — no manifest to fall out of step with the bytes, and no parsing of
34// the bundle header on a phone. CI names them from that header, which is the same list stage-addons.mjs
35// already reads to build the APK's jniLibs.
36export function addonHref (filename) {
37 return 'linked:' + filename
38}
39
40/**
41 * Load every staged addon and register it under its linked: URL. Returns how many were registered.
42 *
43 * Never throws: this runs on every boot, and the overwhelmingly common case is a release with no addons
44 * at all. A loader that died on an empty or missing directory would break the path it exists to protect.
45 *
46 * @param {object} opts
47 * @param {?string} opts.dir the staged addon directory (bundleStore.addonsFor), or null for none
48 * @param {object} [opts.cache] addon cache to populate — defaults to Bare's
49 * @param {function}[opts.Addon] addon constructor — defaults to Bare's
50 * @param {function}[opts.say] diagnostic sink
51 */
52export function preloadAddons ({ dir, cache = Bare.Addon.cache, Addon = Bare.Addon, say = () => {} } = {}) {
53 if (!dir) return 0
54
55 let names
56 try {
57 names = fs.readdirSync(dir)
58 } catch {
59 // Not an error: a release with no addons has no directory, and bundleStore.addonsFor returns null
60 // for exactly that case. Reaching here means the dir was removed under us, which the next boot's
61 // hasTree check will notice properly.
62 return 0
63 }
64
65 let n = 0
66 for (const name of names) {
67 // `.so` and nothing else. A `.part` is an interrupted write — the store renames into place, so a
68 // leftover one means a run that died, and dlopening a truncated ELF is a crash rather than an error.
69 if (!name.endsWith('.so')) continue
70 const file = path.join(dir, name)
71 try {
72 // ⛔ ONE addon failing must not take the boot with it. The bundle still cannot run without it, but
73 // this is not the place that discovers that — the failure surfaces at the require that needs it,
74 // where the message names the module rather than the loader.
75 cache[addonHref(name)] = new Addon(pathToFileURL(file))
76 n++
77 } catch (err) {
78 say('preload: ' + name + ' failed to load — ' + ((err && err.message) || err))
79 }
80 }
81 return n
82}
test/preload-addons.test.js +126 -0
……@@ -0,0 +1,126 @@
1// The loader half of OTA-delivered native addons.
2//
3// ⛔ WHY A SEPARATE MODULE FROM THE STAGED BUNDLE. worklet.js hands bare-kit a PATH and a null source,
4// so Bare loads the staged bundle as its own entry (NativeBareKit.startFile). Preload code placed
5// inside that bundle is too late and, worse, circular: its own `import` of bare-fs resolves to
6// `linked:libbare-fs.<staged version>.so` — the very soname the APK might not have. The loader must
7// therefore be its own bundle, packed against the APK's addon set, which then registers the staged
8// addons and loads the real bundle.
9//
10// ⛔ AND WHY THE FILENAME IS THE CONTRACT. Addon.cache is keyed by url.href and the `linked:` branch of
11// Addon.resolve goes through Addon.load, so registering under `linked:<soname>` wins before any
12// dlopen-by-soname. Naming each staged file after the soname it satisfies means the loader needs no
13// manifest and never parses the bundle — the directory listing IS the mapping.
14import test from 'brittle'
15import fs from 'bare-fs'
16import path from 'bare-path'
17import os from 'bare-os'
18import { fileURLToPath } from 'bare-url'
19import { addonHref, preloadAddons } from '../lib/host/preload-addons.js'
20
21let seq = 0
22function tmpdir (name) {
23 const dir = path.join(os.tmpdir(), 'favela-preload-' + name + '-' + (++seq))
24 fs.rmSync(dir, { recursive: true, force: true })
25 fs.mkdirSync(dir, { recursive: true })
26 return dir
27}
28
29test('a staged filename maps to the linked: URL the bundle asks for', (t) => {
30 t.is(addonHref('libbare-fs.4.8.1.so'), 'linked:libbare-fs.4.8.1.so')
31 t.is(addonHref('libsodium-native.5.1.0.so'), 'linked:libsodium-native.5.1.0.so')
32})
33
34test('every staged addon is registered under its linked: URL', (t) => {
35 const dir = tmpdir('register')
36 fs.writeFileSync(path.join(dir, 'libbare-fs.4.8.1.so'), 'x')
37 fs.writeFileSync(path.join(dir, 'libsodium-native.5.1.0.so'), 'x')
38
39 const cache = Object.create(null)
40 const loaded = []
41 const Addon = function (url) { loaded.push(url.href); this.url = url }
42 const n = preloadAddons({ dir, cache, Addon })
43
44 t.is(n, 2, 'both registered')
45 t.ok(cache['linked:libbare-fs.4.8.1.so'], 'bare-fs is in the cache under its soname')
46 t.ok(cache['linked:libsodium-native.5.1.0.so'], 'sodium-native too')
47 t.ok(loaded.every((h) => h.startsWith('file://')), 'each was loaded from an absolute file: path')
48 t.ok(loaded.some((h) => h.endsWith('/libbare-fs.4.8.1.so')), 'from the staged directory')
49})
50
51test('an addon the APK already satisfies is NOT overridden if it is not staged', (t) => {
52 // Staging is additive. Anything absent from the dir must fall through to dlopen-by-soname, which is
53 // what makes a release able to ship ONE changed addon rather than all 58.
54 const dir = tmpdir('additive')
55 fs.writeFileSync(path.join(dir, 'libbare-fs.4.8.1.so'), 'x')
56
57 const cache = Object.create(null)
58 preloadAddons({ dir, cache, Addon: function (url) { this.url = url } })
59
60 t.ok(cache['linked:libbare-fs.4.8.1.so'], 'the staged one is registered')
61 t.absent(cache['linked:libsodium-native.5.1.0.so'], 'the unstaged one is left to the APK')
62})
63
64test('no addons directory is zero, not a throw', (t) => {
65 // Every release published so far carries no addons at all, and the loader runs on every boot.
66 const cache = Object.create(null)
67 t.is(preloadAddons({ dir: null, cache, Addon: function () {} }), 0, 'null dir')
68 t.is(preloadAddons({ dir: path.join(os.tmpdir(), 'does-not-exist-' + (++seq)), cache, Addon: function () {} }), 0, 'missing dir')
69 t.alike(cache, Object.create(null), 'nothing registered')
70})
71
72test('a non-library file in the directory is ignored', (t) => {
73 const dir = tmpdir('junk')
74 fs.writeFileSync(path.join(dir, 'libbare-fs.4.8.1.so'), 'x')
75 fs.writeFileSync(path.join(dir, 'README'), 'x')
76 fs.writeFileSync(path.join(dir, 'libbroken.so.part'), 'x') // an interrupted write
77
78 const cache = Object.create(null)
79 const n = preloadAddons({ dir, cache, Addon: function (url) { this.url = url } })
80 t.is(n, 1, 'only the .so is taken')
81 t.absent(cache['linked:README'], 'no junk in the cache')
82 t.absent(cache['linked:libbroken.so.part'], 'a .part is not a library')
83})
84
85test('one addon that fails to load does not stop the others', (t) => {
86 // A bundle missing ONE addon still cannot run — but the loader is not the place to decide that, and
87 // dying here would take out the boot with no diagnostic. Report and continue; the failure surfaces
88 // where it means something, at the require that needs it.
89 const dir = tmpdir('partial')
90 fs.writeFileSync(path.join(dir, 'liba.1.0.0.so'), 'x')
91 fs.writeFileSync(path.join(dir, 'libb.1.0.0.so'), 'x')
92
93 const cache = Object.create(null)
94 const said = []
95 const n = preloadAddons({
96 dir,
97 cache,
98 Addon: function (url) { if (url.href.endsWith('liba.1.0.0.so')) throw new Error('bad ELF'); this.url = url },
99 say: (m) => said.push(m)
100 })
101 t.is(n, 1, 'the good one is still registered')
102 t.ok(cache['linked:libb.1.0.0.so'], 'b loaded')
103 t.absent(cache['linked:liba.1.0.0.so'], 'a did not')
104 t.ok(said.some((m) => m.includes('liba.1.0.0.so') && m.includes('bad ELF')), 'and it said which and why: ' + said.join(' | '))
105})
106
107test('REAL: a genuine addon loads from an arbitrary staged path', (t) => {
108 // The whole route in one assertion, against Bare's real Addon rather than a double. A prebuild copied
109 // under a new name is a path the linker has never seen, which is exactly the OTA case: bytes that
110 // arrived over the swarm, sitting in app storage, loaded by absolute path.
111 const here = path.dirname(fileURLToPath(import.meta.url))
112 const src = path.join(here, '..', 'node_modules', 'bare-url', 'prebuilds', Bare.Addon.host, 'bare-url.bare')
113 if (!fs.existsSync(src)) return t.pass('no prebuild for ' + Bare.Addon.host + ' — nothing to prove here')
114
115 const dir = tmpdir('real')
116 const staged = path.join(dir, 'libstaged-probe.9.9.9.so')
117 fs.writeFileSync(staged, fs.readFileSync(src))
118 fs.chmodSync(staged, 0o755)
119
120 const cache = Object.create(null)
121 const n = preloadAddons({ dir, cache })
122 t.is(n, 1, 'loaded')
123 const addon = cache['linked:libstaged-probe.9.9.9.so']
124 t.ok(addon, 'registered under the soname a bundle would ask for')
125 t.ok(addon.exports && typeof addon.exports === 'object', 'and it really initialised — it has exports')
126})