| … | … | @@ -0,0 +1,79 @@ |
| 1 | # Post-mortem — cthuloid lights, 2026-10-02 |
| 2 | |
| 3 | From "immense lag between Blender and studiolight" to shared control from Blender, a web UI on |
| 4 | stortron and an Android app with self-update, in one day. Records what broke, why, how it was |
| 5 | found, and what is still open. |
| 6 | |
| 7 | ## Summary |
| 8 | |
| 9 | | # | Problem | Root cause | Fix | Status | |
| 10 | |---|---|---|---|---| |
| 11 | | 1 | Blender → studiolight changes arrived seconds late or never | Addon's ONE network thread resolved every enabled fixture's `.local` name; 3 of 4 fixtures were powered off, each lookup blocked **5.0 s** | One worker thread per fixture; unticking a fixture forgets it | fixed, verified | |
| 12 | | 2 | A Blender disconnect held its slots for ~2 s | arduino-esp32 3.3.8 `NetworkClient::available()` swallows the peer's FIN, so the firmware only noticed via its 2 s idle timer | Read client sockets with `recv()` on the fd | fixed, flashed | |
| 13 | | 3 | Android 0.1.0 stuck on "loading" | Assumed `Bare.argv` had > 1 entry; on a phone bare-kit passes `[storage]` only, so the controller never started | `storageFromArgv` + a 20 s "did not answer" screen | fixed in 0.1.1 | |
| 14 | | 4 | Diagnosed "router isolates WiFi clients" | One failed ARP snapshot from one phone | Retracted: 20 min later ARP reachable, 0 % ping loss, and a 2nd phone had been controlling the lamp throughout | wrong diagnosis, corrected | |
| 15 | | 5 | In-app update aborted ("Software caused connection abort") and restarted from 0 % | Android freezes the app when the screen locks / battery saver; the download ran in-process and the retry deleted the partial file | `UpdaterModule.download` resumes with `Range` from the bytes on disk; SHA-256 of the whole file still gates the install | fixed in 0.1.5 | |
| 16 | | 6 | Lamp connections drop when the phone is backgrounded | react-native-bare-kit suspends the worklet on every `background` (incl. screen lock) — not Android itself (favela `lib/host/lifecycle.js`) | none yet — see Open items | open | |
| 17 | | 7 | No way to see the app version or check for updates | 0.1.3 showed the update strip only when an update existed | Always-visible "Cthuloid Lights x.y.z · up to date · checked hh:mm", tap = check | fixed in 0.1.4 | |
| 18 | |
| 19 | ## Details |
| 20 | |
| 21 | ### 1. Addon starvation (the original "lag") |
| 22 | - Measured: `getaddrinfo` on a powered-off fixture's `.local` name blocks 5.0 s; tcpdump showed 0 |
| 23 | pwm frames reaching studiolight in 30 s while the ESP32 answered other traffic in 4 ms. |
| 24 | Firmware and network were fine. |
| 25 | - Unticking "Enabled" did not help: the sender never dropped targets. |
| 26 | - Lesson: for "lag / not connecting", count fixtures enabled vs powered **before** suspecting firmware. |
| 27 | |
| 28 | ### 2. Swallowed FIN |
| 29 | - `available()` returns 0 for both "no data" and "peer closed", so a closed socket looked idle. |
| 30 | - Found by the e2e test (`esp324xlight/test/e2e_v3.py`) — which itself first needed `shutdown()` |
| 31 | before `close()`, because Python sends no FIN while a reader thread sits in `recv`. |
| 32 | |
| 33 | ### 3. Android argv |
| 34 | - The desktop Bare path passes extra argv; bare-kit passes the storage path alone. Code written |
| 35 | against the desktop shape failed silently on the phone — no exit event from a worklet, so the |
| 36 | UI spun forever. Two fixes: correct parsing, and never wait without a deadline. |
| 37 | |
| 38 | ### 4. Misdiagnosis (process failure) |
| 39 | - Concluded a router setting from one snapshot on one device. The user's second phone disproved it. |
| 40 | - Lesson: re-test, and compare a second device, before naming a cause. The outage itself was |
| 41 | transient and its cause is unknown. |
| 42 | |
| 43 | ### 5. Update download vs app freeze |
| 44 | - The forge serves `Range` (206) but no `ETag` / `Last-Modified`. Resume is still safe: each |
| 45 | release URL contains its version and never changes, and the SHA-256 of the complete file is |
| 46 | checked before install. Verified on the host: first 40 MB + ranged rest → published SHA-256. |
| 47 | - The release-index `hash` field is **not** the file's SHA-256 (8e99… vs 9d54… for 0.1.2); the |
| 48 | updater uses the published `.sha256`. |
| 49 | |
| 50 | ### 7. Version visibility |
| 51 | - A local `gradle assembleRelease` build stamps `package.json`'s 0.1.0 (only CI sets the version |
| 52 | from the tag), so an adb-installed dev build reports 0.1.0 whatever its code. Releases are |
| 53 | correct (0.1.0–0.1.5 → versionCode 100–105). |
| 54 | |
| 55 | ### Tooling |
| 56 | - A background gradle task reported exit 0 while gradle had failed; exit codes are now recorded |
| 57 | explicitly. |
| 58 | - Android XML comments may not contain `--`; a committed `app/.cxx/` build dir had to be untracked. |
| 59 | |
| 60 | ## What went well |
| 61 | - Firmware proto 3 (per-slot grants, lamp as referee, ed25519-signed `free`): host-tested grant |
| 62 | logic, hardware e2e 23/23; Blender + web UI + studiolight shared-control e2e 14/14. |
| 63 | - Android release pipeline: tag `android-vX.Y.Z` → forge CI (~95–130 s) → public APK + `.sha256`, |
| 64 | each release checked for size, hash, versionName and signing certificate before announcing. |
| 65 | - In-app self-update worked on a real phone: 0.1.3 → 0.1.4 (0.1.5's resume path is not yet |
| 66 | exercised on a phone). |
| 67 | |
| 68 | ## Open items |
| 69 | 1. **Background behaviour** (#6): decide with phase 3. favela's answer is a foreground service plus |
| 70 | taking over bare-kit's suspend policy; phase 3's autobee needs the phone online anyway. |
| 71 | 2. **Phase 3 — autobee** (approved to proceed): shared looks/swatches, a durable record that |
| 72 | outlives a lamp reboot, groundwork for off-LAN control (phase 4). |
| 73 | 3. **P2P updates**: the APK is 92 MB but the part that changes is a 764 KB worklet bundle; favela's |
| 74 | pgh-registry bundle OTA fits once phase 3 brings Hyperswarm to the phone. |
| 75 | 4. On-phone shared-control e2e (grab on the phone → "held by" on stortron → background → slot |
| 76 | frees) is not yet run. |
| 77 | 5. Boot fade leaves all channels at 65535, red above its `RED_MAX_PWM` cap — question unanswered. |
| 78 | 6. Keystore `~/android-toolchain/cthuloid.keystore` is only backed up when `backup.sh` is run |
| 79 | by hand; losing it means no further updates to installed phones. |