OverviewFilesCommitsIssuesDocsReleasesPeers

favela-surface-android commits / 5a30d07

the addon guard, and CI publishes the addons it guards against
Three pieces that close the loop on OTA-delivered native code. CI publishes the addons. The pgh tree is now { package.json, worklet.bundle, addons/ }, the addons named by soname because the filename IS the contract — the loader maps a staged file straight to `linked:<filename>` and never parses a bundle. bare-link already writes them under that name, so this copies. ⛔ ALL of them, not the changed ones: a publisher cannot know which APK a device holds, so "changed" has no definable baseline. It costs nothing after the first release because the tree is content-addressed — an addon whose bytes are unchanged is already in the store and never fetched again. Exercised against the real bundle: 58 addons, 39.7 MB. The store records what the bundle DEMANDS, read while the bytes are in hand at materialize — the alternative is React Native re-opening a 32 MB file every boot to re-read a header that never changes. ⛔ Unreadable means null, never []: an empty list would read as "demands nothing", a bundle that always passes the guard. Unknown has to disable the guard, not sail through it. An unreadable header is also not grounds to reject a bundle whose bytes were content-verified upstream. chooseBundle refuses a staged bundle demanding a soname that is in neither the APK nor the addons staged beside it. Until now nothing refused: such a bundle booted, died at dlopen, never reached ready, and had its tree hash condemned permanently — an update silently burned, with no diagnosis available on a device with no adb. ⛔ REFUSED IS NOT CONDEMNED. Nothing was tried, so nothing failed; the bundle is left intact and a later APK carrying the library can still run it. Condemning would destroy a good update because the app was briefly too old. It also does not arm markPending, which exists to catch a boot that was actually attempted. ⛔ The guard fires ONLY on knowledge. No `demands` (a marker predating it) or no apkAddons (an older shell) means the question is unanswerable, and refusing on missing information would strand every device already holding a staged bundle. An empty apkAddons counts as unknown too — no APK demands nothing, so an empty list is a failed read, not a fact. stage-addons.mjs writes that list to assets/apk-addons.json, from the KEEP-LIST rather than the demanded set, because the keep-list is what survives jniLibs.excludes and therefore what the APK actually ships. Run for real: arm64-v8a, 59 entries — the 58 demanded plus libbare-buffer.3.6.2.so pulled in by the native closure. Omitted entirely when readelf is unavailable, and any stale copy deleted, because a list written from an empty keep-list would UNDER-report and refuse updates that would have run. worklet.js reads it and logs a refusal explicitly — a declined update is otherwise invisible, since the app simply keeps running the old bundle, which is exactly what a working install looks like. Suite 75/75, 259/259. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
5a30d071f5a748018261c479c6fa92bc79d77547
by plankatron <fleeky@gmail.com> · 2026-09-11T00:35:00.000Z

Changed files (8) +244 -4

.gitignore +2 -0
……@@ -6,6 +6,8 @@ android/app/.cxx/
66android/app/src/main/assets/worklet.bundle
77# The APK's entry bundle, built by `npm run pack:loader` — an artifact like the worklet bundle above.
88android/app/src/main/assets/loader.bundle
9# Written by scripts/stage-addons.mjs from the keep-list — describes the build, so never committed.
10android/app/src/main/assets/apk-addons.json
911*.apk
1012*.keystore
1113android/app/src/main/assets/worklet.bundle.size
.pear-git/ci.yaml +35 -0
……@@ -155,6 +155,41 @@ steps:
155155 set -e
156156 rm -rf .pgh-publish && mkdir -p .pgh-publish
157157 cp worklet.bundle .pgh-publish/worklet.bundle
158 # ── the native addons, so a release can change them without a 154 MB sideload ────────────────
159 #
160 # A `linked:` bundle NAMES the sonames the APK must supply, which is why every native change has
161 # needed a new APK. lib/loader.js registers these under exactly those names in Bare's Addon.cache,
162 # which the `linked:` branch of Addon.resolve consults BEFORE dlopen-by-soname — so a staged addon
163 # wins and an unstaged one still falls through to the APK.
164 #
165 # ⛔ ALL OF THEM, NOT THE CHANGED ONES. A publisher cannot know which APK any given device holds,
166 # so "only what changed" has no definable baseline. Publishing the full demanded set costs nothing
167 # after the first release: the tree is content-addressed, so an addon whose bytes are unchanged is
168 # already in the store and is never fetched again. It also guarantees bundle and addons match.
169 #
170 # Named by soname because THE FILENAME IS THE CONTRACT — the loader maps a staged file straight to
171 # `linked:<filename>` and never parses the bundle. bare-link already writes them under exactly
172 # that name, so this copies rather than renames.
173 #
174 # arm64-v8a alone, matching abiFilters. A second ABI belongs in by-arch/ with pgh-runtime's
175 # skipPath, which PghRuntime does not expose yet — see hyper-social #69.
176 mkdir -p .pgh-publish/addons
177 node -e '
178 const fs = require("fs"), path = require("path")
179 const buf = fs.readFileSync("worklet.bundle")
180 const nl = buf.indexOf(10)
181 const len = parseInt(buf.subarray(0, nl).toString(), 10)
182 const want = JSON.parse(buf.subarray(nl, nl + len).toString().trim()).addons.map(a => a.replace(/^linked:/, ""))
183 const src = "node_modules/react-native-bare-kit/android/src/main/addons/arm64-v8a"
184 let n = 0
185 for (const name of want) {
186 const from = path.join(src, name)
187 if (!fs.existsSync(from)) throw new Error("demanded addon not staged: " + name)
188 fs.copyFileSync(from, path.join(".pgh-publish/addons", name))
189 n++
190 }
191 console.log("staged " + n + " addon(s) for publication")
192 '
158193 # Version comes from the repo's package.json — the same field pgh publish reads for every other
159194 # app, so a release is identified the way the rest of the fleet identifies one.
160195 VERSION=$(node -p "require('./package.json').version")
lib/host/choose-bundle.js +28 -1
……@@ -22,7 +22,7 @@
2222 * The failure this exists to prevent is the retry loop — a bad bundle chosen again on every boot, dying
2323 * identically each time, with no way in from outside.
2424 */
25export function chooseBundle ({ current = null, pending = null, failed = [], assetBuiltAt = null } = {}) {
25export function chooseBundle ({ current = null, pending = null, failed = [], assetBuiltAt = null, apkAddons = null } = {}) {
2626 // ⛔ `addons: null` ON EVERY ASSET PATH. Booting the APK's own bundle must not preload addons staged
2727 // for a DIFFERENT one — their sonames carry versions, so a staged libbare-fs.4.9.0.so registered
2828 // while the APK's 4.8.1 bundle runs would shadow a library this binary actually ships.
……@@ -72,6 +72,33 @@ export function chooseBundle ({ current = null, pending = null, failed = [], ass
7272 return asset('the staged bundle carries no build time; an installed APK is the more recent evidence', condemn)
7373 }
7474
75 // ⛔ THE ADDON GUARD. A `linked:` bundle NAMES the sonames it needs, and two sources can supply one:
76 // the APK's jniLibs, or the addons staged beside the bundle. A demand in NEITHER means a dlopen
77 // failure, a node that never reaches ready, and this tree hash condemned for good — an update
78 // silently burned on a device with no adb and no way to say why. Refusing turns that into "this
79 // update needs a newer app", which is a sentence someone can act on.
80 //
81 // ⛔ REFUSED IS NOT CONDEMNED. Nothing was tried, so nothing failed: the bundle is left alone and a
82 // later APK carrying the missing library can still run it. Condemning here would destroy a perfectly
83 // good update because the app was briefly too old, which is the opposite of the intent.
84 //
85 // ⛔ AND THE GUARD ONLY FIRES ON KNOWLEDGE. `demands` is absent from any marker written before the
86 // store recorded it, and `apkAddons` is absent in any shell that does not ship its own list. Either
87 // way the question cannot be answered — and refusing on missing information would strand every device
88 // already holding a staged bundle. Unknown means proceed, exactly as before this existed. An empty
89 // apkAddons is treated as unknown for the same reason: no APK demands nothing, so an empty list is a
90 // failed read, not a fact.
91 const demands = Array.isArray(current.demands) ? current.demands : null
92 const apk = Array.isArray(apkAddons) && apkAddons.length > 0 ? apkAddons : null
93 if (demands && apk) {
94 const have = new Set(apk)
95 for (const a of (Array.isArray(current.addons) ? current.addons : [])) have.add(a)
96 const missing = demands.filter((d) => !have.has(d))
97 if (missing.length > 0) {
98 return asset('the staged bundle needs ' + missing.join(', ') + ' — in neither the APK nor the release', condemn)
99 }
100 }
101
75102 return {
76103 use: 'staged',
77104 hash: current.treeHash,
lib/host/ota.js +32 -2
……@@ -89,6 +89,33 @@ export function treeFor (bytes, name = BUNDLE_NAME) {
8989 return { entries, treeHash: computeTreeHash(entries), contentHash, size: bytes.length }
9090}
9191
92/**
93 * The sonames a packed bundle demands, or null when they cannot be read.
94 *
95 * bare-bundle's container is an ASCII decimal header length, a newline, then that many bytes of JSON —
96 * the length measured FROM the newline. Parsed by hand, exactly as scripts/stage-addons.mjs does at
97 * build time, so the phone needs no bundle library to answer the one question chooseBundle asks.
98 *
99 * ⛔ NULL, NEVER []. chooseBundle refuses a staged bundle whose demands nothing can satisfy, and an
100 * empty list would read as "demands nothing" — a bundle that always passes the guard. Unknown has to
101 * DISABLE the guard, not sail through it. And an unreadable header is not a reason to reject a bundle
102 * here: it is one fact missing, not a bad download, and the bytes were content-verified upstream.
103 */
104export function demandedAddons (bytes) {
105 try {
106 const buf = Buffer.from(bytes)
107 const nl = buf.indexOf(10)
108 if (nl === -1) return null
109 const len = parseInt(buf.subarray(0, nl).toString(), 10)
110 if (!Number.isInteger(len) || len <= 0) return null
111 const header = JSON.parse(buf.subarray(nl, nl + len).toString().trim())
112 if (!Array.isArray(header.addons)) return null
113 return header.addons.map((a) => String(a).replace(/^linked:/, ''))
114 } catch {
115 return null
116 }
117}
118
92119/**
93120 * Newest release by BUILD TIME, not by position in the list.
94121 *
……@@ -318,6 +345,9 @@ export function bundleStore ({ dir }) {
318345 fs.writeFileSync(part, Buffer.from(bytes))
319346 fs.renameSync(part, dest)
320347 const file = want.childHash + '.bundle'
348 // Read while the bytes are in hand. The alternative is the React Native side re-opening a 32 MB
349 // file at every boot to re-read a header that never changes.
350 const demands = demandedAddons(bytes)
321351
322352 // ⛔ THE ADDONS, BEFORE THE MARKER. A `linked:` bundle names the sonames the APK must supply, and
323353 // until now a bundle demanding one the APK lacks could only die at dlopen — on a device with no
……@@ -355,11 +385,11 @@ export function bundleStore ({ dir }) {
355385 // `builtAt` is what lets the SHELL compare a staged bundle with the one inside the APK. Without it
356386 // an APK install loses to whatever was staged before it — chooseBundle prefers staged, and tree
357387 // hashes carry no ordering, so a phone kept running the older JS until the next OTA pass caught up.
358 fs.writeFileSync(markerPath(rootHash), JSON.stringify({ treeHash: rootHash, file, builtAt, addons }, null, 2) + '\n')
388 fs.writeFileSync(markerPath(rootHash), JSON.stringify({ treeHash: rootHash, file, builtAt, addons, demands }, null, 2) + '\n')
359389 // current.json is how the REACT NATIVE side learns what is staged: it runs before the worklet and
360390 // cannot read this store's code (bare-fs), only its files. Written last, after the bytes and the
361391 // marker, so it never advertises something that is not fully on disk.
362 fs.writeFileSync(path.join(dir, 'current.json'), JSON.stringify({ treeHash: rootHash, file, builtAt, addons }, null, 2) + '\n')
392 fs.writeFileSync(path.join(dir, 'current.json'), JSON.stringify({ treeHash: rootHash, file, builtAt, addons, demands }, null, 2) + '\n')
363393 }
364394 }
365395}
lib/host/worklet.js +15 -1
……@@ -133,11 +133,19 @@ async function boot () {
133133 const bdir = RNFS.DocumentDirectoryPath + '/bundles'
134134 const readJson = async (p) => { try { return JSON.parse(await RNFS.readFile(p, 'utf8')) } catch { return null } }
135135 const failed = (await readJson(bdir + '/failed.json')) || []
136 // What THIS APK's jniLibs can satisfy, written at build time by scripts/stage-addons.mjs. Nothing
137 // at runtime can enumerate them — they are mapped out of the APK, never unpacked — so the list has
138 // to ship as an asset. Absent (an older APK, or a build where readelf was unavailable) reads as
139 // "not known", and chooseBundle then proceeds exactly as it did before the guard existed.
140 let apkAddons = null
141 try { apkAddons = JSON.parse(await RNFS.readFileAssets('apk-addons.json', 'utf8')) } catch {}
142
136143 const choice = chooseBundle({
137144 current: await readJson(bdir + '/current.json'),
138145 pending: await readJson(bdir + '/pending.json'),
139146 failed,
140 assetBuiltAt
147 assetBuiltAt,
148 apkAddons
141149 })
142150
143151 if (choice.condemn) {
……@@ -149,6 +157,12 @@ async function boot () {
149157 try { await RNFS.unlink(bdir + '/pending.json') } catch {}
150158 }
151159
160 // ⛔ SAY IT OUT LOUD. A refused update is invisible otherwise — the app just keeps running the old
161 // bundle, which is exactly what a working install looks like. The reason names the missing soname.
162 if (choice.use === 'asset' && choice.reason.includes('needs ')) {
163 console.log('[worklet] REFUSED the staged bundle: ' + choice.reason)
164 }
165
152166 if (choice.use === 'staged' && await RNFS.exists(bdir + '/' + choice.file)) {
153167 await RNFS.writeFile(bdir + '/pending.json', JSON.stringify({ treeHash: choice.hash }), 'utf8')
154168 bundlePath = bdir + '/' + choice.file
scripts/stage-addons.mjs +27 -0
……@@ -216,3 +216,30 @@ if (!readelfOk) {
216216const keepPath = path.join(root, 'android/app/addon-keep.json')
217217fs.writeFileSync(keepPath, JSON.stringify(readelfOk ? keepByAbi : {}, null, 2) + '\n')
218218console.log(' keep-list ->', path.relative(root, keepPath))
219
220// ── what THIS APK can satisfy, shipped as an asset so the running app can answer it ───────────────
221//
222// chooseBundle refuses a staged bundle demanding a soname that is in neither the APK nor the addons
223// staged beside it — otherwise that bundle boots, dies at dlopen, and its tree hash is condemned for
224// good, an update silently burned on a device with no adb. To answer that it has to know what the APK
225// carries, and nothing at runtime can enumerate jniLibs: they are mapped out of the APK, not unpacked.
226//
227// ⛔ THE KEEP-LIST, NOT `want`. The keep-list is what survives jniLibs.excludes — the demanded set PLUS
228// the transitive native closure — so it is what the APK actually ships. Writing `want` would claim a
229// library packaging had removed.
230//
231// ⚠ Written for the SHIPPED ABI only, and omitted entirely when readelf was unavailable: without the
232// closure the keep-list is empty and gradle keeps everything, so a list written from it would UNDER-
233// report and refuse updates that would have run. chooseBundle reads an absent or empty list as "not
234// known" and proceeds, which is the right failure direction.
235const shippedAbi = ABI_FOR_HOST[hosts[0]] || hosts[0]
236const apkAddonsPath = path.join(root, 'android/app/src/main/assets/apk-addons.json')
237if (readelfOk && keepByAbi[shippedAbi]) {
238 fs.mkdirSync(path.dirname(apkAddonsPath), { recursive: true })
239 fs.writeFileSync(apkAddonsPath, JSON.stringify(keepByAbi[shippedAbi], null, 2) + '\n')
240 console.log(` apk addon list (${shippedAbi}, ${keepByAbi[shippedAbi].length}) ->`, path.relative(root, apkAddonsPath))
241} else {
242 // Stale content here would be worse than none: it would answer for a build that no longer exists.
243 try { fs.unlinkSync(apkAddonsPath) } catch {}
244 console.log(' no apk addon list written — the guard will not fire (unknown means proceed)')
245}
test/lifecycle.test.js +62 -0
……@@ -207,3 +207,65 @@ test('the APK asset never reports staged addons', (t) => {
207207 t.is(r.use, 'asset', 'newer APK wins')
208208 t.is(r.addons, null, 'and brings no staged addons with it')
209209})
210
211// ── the addon guard ───────────────────────────────────────────────────────────────────────────────
212//
213// A `linked:` bundle names the sonames it needs. Two sources can satisfy one: the APK's jniLibs, or the
214// addons staged beside the bundle. If a demand is in NEITHER, running it means a dlopen failure, a node
215// that never reaches ready, and a tree hash condemned for good — the update silently burned.
216//
217// ⛔ AND THE GUARD ONLY FIRES ON KNOWLEDGE. A marker written before `demands` existed, or a shell with
218// no list of its own addons, means the question cannot be answered — and refusing on missing information
219// would strand every device holding an older staged bundle. Unknown means proceed, exactly as today.
220const APK = ['libbare-fs.4.8.1.so', 'libbare-path.3.1.2.so']
221
222test('a staged bundle whose demands the APK satisfies runs', (t) => {
223 const current = { treeHash: 'a'.repeat(64), file: 'x.bundle', builtAt: 2000, demands: ['libbare-fs.4.8.1.so'] }
224 t.is(chooseBundle({ current, assetBuiltAt: 1000, apkAddons: APK }).use, 'staged')
225})
226
227test('a staged bundle whose demands only the STAGED addons satisfy runs', (t) => {
228 // The whole point of the route: a release can bring an addon the APK has never carried.
229 const current = {
230 treeHash: 'b'.repeat(64),
231 file: 'x.bundle',
232 builtAt: 2000,
233 addons: ['librocksdb-native.9.9.9.so'],
234 demands: ['libbare-fs.4.8.1.so', 'librocksdb-native.9.9.9.so']
235 }
236 const r = chooseBundle({ current, assetBuiltAt: 1000, apkAddons: APK })
237 t.is(r.use, 'staged', 'the new addon came with the release')
238 t.is(r.addons, 'b'.repeat(64) + '.addons', 'and its directory is reported')
239})
240
241test('a demand in NEITHER source is refused, and says which', (t) => {
242 // ⛔ This is the whole ⛔ item: today nothing refuses, so such a bundle is staged, boots, dies at
243 // dlopen and is condemned permanently — an update burned with no diagnosis on a device with no adb.
244 const current = {
245 treeHash: 'c'.repeat(64),
246 file: 'x.bundle',
247 builtAt: 2000,
248 addons: ['libbare-fs.4.8.1.so'],
249 demands: ['libbare-fs.4.8.1.so', 'libsodium-native.5.1.0.so']
250 }
251 const r = chooseBundle({ current, assetBuiltAt: 1000, apkAddons: APK })
252 t.is(r.use, 'asset', 'the APK bundle runs instead')
253 t.ok(r.reason.includes('libsodium-native.5.1.0.so'), 'names the addon: ' + r.reason)
254 t.absent(r.condemn, 'REFUSED, not condemned — nothing was tried, so nothing failed')
255})
256
257test('a refusal does not mark the bundle pending', (t) => {
258 // markPending is the watchdog arming itself before a risky boot. A bundle we declined to run must not
259 // arm it, or the next boot condemns a tree that never executed and a later APK could have fixed.
260 const current = { treeHash: 'd'.repeat(64), file: 'x.bundle', builtAt: 2000, demands: ['libmissing.1.0.0.so'] }
261 t.absent(chooseBundle({ current, assetBuiltAt: 1000, apkAddons: APK }).markPending)
262})
263
264test('unknown demands or unknown APK addons mean PROCEED, never refuse', (t) => {
265 const noDemands = { treeHash: 'e'.repeat(64), file: 'x.bundle', builtAt: 2000 }
266 t.is(chooseBundle({ current: noDemands, assetBuiltAt: 1000, apkAddons: APK }).use, 'staged', 'marker predates demands')
267
268 const withDemands = { treeHash: 'f'.repeat(64), file: 'x.bundle', builtAt: 2000, demands: ['libanything.1.0.0.so'] }
269 t.is(chooseBundle({ current: withDemands, assetBuiltAt: 1000 }).use, 'staged', 'shell does not know its own addons')
270 t.is(chooseBundle({ current: withDemands, assetBuiltAt: 1000, apkAddons: [] }).use, 'staged', 'an EMPTY list is not knowledge')
271})
test/ota.test.js +43 -0
……@@ -412,3 +412,46 @@ test('a bundle-only tree still stages, and reports no addons', async (t) => {
412412 t.ok(store.hasTree(treeHash), 'staged')
413413 t.is(store.addonsFor(treeHash), null, 'no addons dir is reported for a bundle-only release')
414414})
415
416// ── what the staged bundle DEMANDS ────────────────────────────────────────────────────────────────
417//
418// ⛔ THE STORE IS THE HONEST PLACE TO READ THIS. It is holding the bundle's bytes at exactly the moment
419// they are verified; the alternative is the React Native side re-opening a 32 MB file at every boot to
420// re-read a header that never changes. chooseBundle then compares the list against the APK's own
421// addons and the ones staged beside the bundle.
422
423// bare-bundle's container: ASCII decimal header length, a newline, then that many bytes of JSON —
424// the length measured from the newline, which is what stage-addons.mjs parses.
425function bundleWithHeader (addons) {
426 const json = JSON.stringify({ version: 0, addons, resolutions: {}, files: {} })
427 const len = json.length + 1
428 return Buffer.from(String(len) + '\n' + json)
429}
430
431test('the marker records the sonames the staged bundle demands', async (t) => {
432 const dir = tmpdir('demands')
433 const bundle = bundleWithHeader(['linked:libbare-fs.4.8.1.so', 'linked:libsodium-native.5.1.0.so'])
434 const { entries, treeHash } = treeFor(bundle)
435 const store = bundleStore({ dir })
436 await installTree(store, treeHash, treeFetcher({ trees: { [treeHash]: entries }, contents: { [hashOf(bundle)]: bundle } }))
437
438 const marker = JSON.parse(fs.readFileSync(path.join(dir, treeHash + '.json'), 'utf8'))
439 t.alike(marker.demands, ['libbare-fs.4.8.1.so', 'libsodium-native.5.1.0.so'], 'linked: stripped, as chooseBundle compares bare sonames')
440 const cur = JSON.parse(fs.readFileSync(path.join(dir, 'current.json'), 'utf8'))
441 t.alike(cur.demands, marker.demands, 'and the RN side sees the same list')
442})
443
444test('a bundle whose header cannot be read is UNKNOWN, not empty', async (t) => {
445 // ⛔ null, never []. chooseBundle refuses a bundle whose demands are unsatisfied, and an empty list
446 // would read as "demands nothing" — which is a bundle that always passes the guard. Unknown must
447 // disable the guard, not sail through it.
448 const dir = tmpdir('demands-bad')
449 const junk = Buffer.from('not a bundle at all')
450 const { entries, treeHash } = treeFor(junk)
451 const store = bundleStore({ dir })
452 await installTree(store, treeHash, treeFetcher({ trees: { [treeHash]: entries }, contents: { [hashOf(junk)]: junk } }))
453
454 const marker = JSON.parse(fs.readFileSync(path.join(dir, treeHash + '.json'), 'utf8'))
455 t.is(marker.demands, null, 'unknown')
456 t.ok(store.hasTree(treeHash), 'and the bundle is still staged — an unreadable header is not a bad bundle')
457})