| … | … | @@ -0,0 +1,49 @@ |
| 1 | # Jigsaw transformer v2 — per-occurrence evidence from actual neighbours. Pre-registration |
| 2 | |
| 3 | 2026-10-18. Written and committed **before any v2 code or run**. Follows `RESULTS-v1.md`: v1's sense gain |
| 4 | appeared equally with word order shuffled (J6 FAIL). Hypothesis: v1 judges an occurrence's class against |
| 5 | **class-averaged** neighbour profiles Φ(y | z), which blurs the evidence that separates senses. v2 tests two |
| 6 | replacements, as two arms of one run. Everything else (corpus, seeds, tiers, K = 30, layer 0, exchange start, |
| 7 | 10 EM passes, 2 layers, doubling checkpoints, audit) is v1 unchanged. |
| 8 | |
| 9 | ## Arms |
| 10 | |
| 11 | **V2-WORD (actual neighbour words).** The context term of the class update uses the attended neighbour |
| 12 | **words**: q_t(z) ∝ π(z | w_t) · exp(Σ_j α_tj log Ψ_side(w_j | z)), side = left if j < t else right. Ψ are |
| 13 | class→word counts (Ψ_side[z][w_j] += q_t(z)·α_tj), Dirichlet-smoothed toward the class-level estimate: |
| 14 | Ψ(v | z) = (n(z, v) + β·Σ_y Φ(y | z) P(v | y)) / (n(z) + β), with P(v | y) from π's counts, **β = 10**. |
| 15 | |
| 16 | **V2-DJ (Linas disjuncts).** Each layer, the sentence is parsed by Eisner over the current attention |
| 17 | (α_tj + α_jt, as the v1 attention parse). Occurrence t's **disjunct** d_t is the position-ordered sequence of its |
| 18 | links as connectors (side, argmax class of the linked word): a Section with class connectors, as after |
| 19 | Linas's clustering. Update q_t(z) ∝ π(z | w_t) · P(d_t | z), with disjunct counts D[z][d] += q_t(z) and |
| 20 | P(d | z) = (n(z, d) + β·Π_connectors Φ_side(y | z)) / (n(z) + β), **β = 10**. Initial counts: disjuncts from the |
| 21 | Eisner parse of the layer-0 word attention. |
| 22 | |
| 23 | **V1** is re-run in the same job (deterministic, identical code path) as the reference. |
| 24 | |
| 25 | ## New measurement: collapse |
| 26 | |
| 27 | For each learner X, **X-COLL** labels every held-out occurrence of a word with X's most frequent label for |
| 28 | that word over the held-out set (same classes, one label per word). X − X-COLL on sense accuracy is the effect |
| 29 | of **per-occurrence context alone**; better word classes cannot produce it. This replaces TYPE-A as the sense |
| 30 | comparator (TYPE-A is still reported). |
| 31 | |
| 32 | ## Predictions (paired over 5 seeds at 30K; "beats" = 95% CI of the paired difference excludes 0) |
| 33 | |
| 34 | For each v2 arm A ∈ {V2-WORD, V2-DJ}: |
| 35 | |
| 36 | - **K1 (context senses):** A beats A-COLL on sense accuracy. |
| 37 | - **K2 (control):** with words shuffled within sentences, A does **not** beat A-COLL on sense accuracy. |
| 38 | - **K3 (beats v1):** A beats V1 on sense accuracy. |
| 39 | - **K4 (strong):** A beats MAJ on sense accuracy. |
| 40 | - **K5 (no class damage):** token V-measure of A − V1 has 95% CI lower bound > −0.02. |
| 41 | - **K6 (audit):** replay reproduces every checkpoint; one altered checkpoint is detected there (all seeds/tiers). |
| 42 | |
| 43 | Also scored, no prediction: V1 − V1-COLL (does v1 have any per-occurrence effect?), V2-WORD vs V2-DJ, and the |
| 44 | attention parse F1 of each arm. |
| 45 | |
| 46 | ## Procedure |
| 47 | |
| 48 | One smoke run (seed 1, tier 3K) only to check it runs and the audit passes; **no parameter changes after |
| 49 | it**. Then `run-v2.sh`: 5 seeds × {ordered, shuffled}, scored by `summarize-v2.py` written before the full run. |