| … | … | @@ -0,0 +1,43 @@ |
| 1 | # Level 2, Test A-U — undirected, Linas-style layer-1 parses. Pre-registration |
| 2 | |
| 3 | 2026-10-18. Written and committed **before any code or run**. Audit #15 (`docs/reports/audit-2026-10-18.md`): the |
| 4 | Test A parser read arc directions that were a tie-break artifact, and the `comp` rule was directional. Linas's links |
| 5 | are undirected (Yuret's model; LG `+`/`-` is word order), his MST spans the sentence with MI as cost (no gate), and |
| 6 | his MPG adds high-MI links to the tree, giving a planar graph with loops. This test re-scores Test A with undirected |
| 7 | rules on three layer-1 parsers. |
| 8 | |
| 9 | ## Parsers (per layer-1 source x ∈ {G, L, R}; same window pair counts, W = 6, as Test A) |
| 10 | |
| 11 | - **FOREST:** Test A's parse (gated PMI, root edges 0), read as **undirected links**. |
| 12 | - **MST:** maximum-weight projective spanning tree. Edge score = ordered-pair MI log2(n·N / (R·C)) of the pair, any |
| 13 | sign, **no gate**; window pairs never counted get −10; pairs farther apart than W are not allowed. Eisner with |
| 14 | every root edge −10⁶, so the tree has one root child (spans the sentence). Read as undirected links. |
| 15 | - **MPG:** MST plus extra links: window pairs with MI > 0, in descending MI, added if not already linked and not |
| 16 | crossing any existing link (planarity). Linas's `graph-add-linear` and `graph-add-bridges` steps are not done |
| 17 | (declared simplification). |
| 18 | |
| 19 | Declared deviations kept from Test A: window pair counts (not LG "any" random planar parses); one parse per sentence. |
| 20 | |
| 21 | ## Undirected unit rules (span [i, j], length 2–6) |
| 22 | |
| 23 | - **uconn:** the parse links with both ends inside the span connect all its tokens. |
| 24 | - **ucomp:** uconn, and **at most one** parse link has exactly one end inside the span (one free connector; zero |
| 25 | allowed for a span that is a whole connected component). |
| 26 | |
| 27 | Arms per source x and parser p: **M-x-uconn-p** and **M-x-ucomp-p**, with Test A's measures (candidate types, instance |
| 28 | recall, type recall, candidate precision). Also per x and p: undirected **link precision / recall / F1** on held-out |
| 29 | text against the gold tree. |
| 30 | |
| 31 | ## Predictions (at 30K unless stated; paired over 10 seeds; "beats" = 95% CI of the paired difference excludes 0) |
| 32 | |
| 33 | - **U1 (the artifact was real):** M-G-ucomp-FOREST beats M-G-comp (Test A's directional rule, same parse) on type recall. |
| 34 | - **U2 (Linas's tree is the better parse):** for x ∈ {G, L}, MST beats FOREST on link F1. |
| 35 | - **U3 (A1 re-tested):** for the better of MST and FOREST by link F1 with G: type recall of M-G-ucomp ≥ 0.7 × that |
| 36 | of C-G, and its candidate types ≤ 10% of C-G's. |
| 37 | - **U4 (a real layer):** at every tier, M-G-ucomp-MST and M-L-ucomp-MST beat M-R-ucomp-MST on type recall. |
| 38 | - **U5 (focus):** at every tier, M-G-ucomp-MST beats C-G on candidate precision. |
| 39 | |
| 40 | Reported, no prediction: MPG arms (loops make ucomp stricter), uconn arms, link F1 of MPG. |
| 41 | |
| 42 | **Controls:** R0 — every Test A line reproduces `results/1792340082` exactly. **C5-U** — with the gold tree as the |
| 43 | parse, every gold span is uconn and ucomp (must be 100%). |