| … | … | @@ -0,0 +1,33 @@ |
| 1 | # Re-parse with classes — does each discovered level improve the next parse? Pre-registration |
| 2 | |
| 3 | 2026-10-18. Written and committed **before any code or run**. Follows #15 (audit) and #16 (Test A-U). Linas planned |
| 4 | "perform MST with and without clusters, try to see which is better" (`learn-lang-diary-part-nine.lyx:275`) and a |
| 5 | re-parse with the discovered classes (`gram-class/README.md:219-221`), but never ran it. Test A-U: the layer-1 parse |
| 6 | limits level 2 (link F1 0.735 even with gold classes). This is the core hypothesis in Linas's own terms: the |
| 7 | discovered level (classes) constrains the next search (the parse), and the improved parse yields better classes. |
| 8 | |
| 9 | ## The loop (training text; same window counts, W = 6, and the MST parser of PREREG-U: ungated ordered-pair MI, |
| 10 | undirected, projective, spanning) |
| 11 | |
| 12 | - **Round 0:** symbols = words. Parse = MST over word-pair MI. |
| 13 | - **Round r = 1…5:** classes = exchange clustering (Martin/Liermann/Ney objective, as `jigsaw-transformer/jt.c` |
| 14 | `exchange_weighted`; frequency-rank start; K = 30 declared) over the word pairs **linked in the round r−1 parse** |
| 15 | (each link counts once, as an ordered left/right pair). Symbols = classes. Parse = MST over class-pair MI from the |
| 16 | window counts. |
| 17 | |
| 18 | The clusterer is a stand-in (one class per word); Linas's own clusterer is step 3 (#15 recommendation 3). |
| 19 | |
| 20 | ## Measures per round (held-out, 10 seeds, tiers 3K/10K/30K) |
| 21 | |
| 22 | Undirected link precision / recall / F1 against the gold tree; many-to-one accuracy of the classes on held-out |
| 23 | tokens; Test A-U's ucomp type recall and candidate precision with that round's symbols and parse. |
| 24 | References from the same run: **G-MST** (gold classes) and **L-MST** (exchange over adjacent bigrams, Test A-U). |
| 25 | |
| 26 | ## Predictions (paired over 10 seeds; "beats" = 95% CI of the paired difference excludes 0) |
| 27 | |
| 28 | - **P1 (classes constrain the parse):** at every tier, round 1 beats round 0 on link F1. |
| 29 | - **P2 (the loop helps):** at 30K, round 5 beats round 1 on link F1. |
| 30 | - **P3 (parse-derived classes beat window classes):** at 30K, round 5 beats L-MST on link F1. |
| 31 | - **P4 (and carry to level 2):** at 30K, round 5 beats L-MST on ucomp type recall. |
| 32 | |
| 33 | **Control R0:** with `--reparse` off, output is unchanged (Test A lines reproduce `results/1792340082`). |