OverviewFilesCommitsIssuesDocsReleasesPeers

atomspace-layer-discovery commits / a1cd10d

experiments/level2: correction notes — learned-parse comp numbers carry the arc-direction artifact
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
a1cd10dbdb3eda9d829b9f5448c308a7104f8d14
by plankatron <fleeky@gmail.com> · 2026-10-18T22:33:06.000Z

Changed files (3) +21 -0

experiments/level2/RESULTS-JT.md +7 -0
……@@ -19,3 +19,10 @@ full table in its `summary.txt`. Control R0 PASS: all Test A arms reproduce `res
19192. **Per-occurrence labels fragment level 2.** C-JT has 49K span types vs 27K for exchange classes and 24K for
2020 gold: the same phrase appears under more label sequences, so counts thin out and type recall drops.
21213. Better layer-1 class metrics (v2) did not carry into level 2. Gold classes with the old parse remain best.
22
23---
24
25**Correction (2026-10-18, after Test B):** the learned layer-1 parse scores word pairs symmetrically, so its arc
26directions are a tie-break artifact (76–94% of heads on the left vs 34% gold). The directional "complete assembly"
27(`comp`, HARD/SOFT) numbers with a learned parse inherit that bias. See `RESULTS-B.md`. Gold-tree controls and
28`conn` numbers are unaffected.
experiments/level2/RESULTS-soft.md +7 -0
……@@ -23,3 +23,10 @@ At 30K, recall / precision with the top 3K types, G: CNT 0.950 / 0.128, HARD 0.8
2323 ranking at 3K it covers 0.883). But it does not help find more of them than counting does.
24244. Caveat on the measure: R@B asks whether a phrase's *type* is in the searched set, not whether a phrase was
2525 chosen. Choosing units (Test B) remains the open step, and that is where precision should matter.
26
27---
28
29**Correction (2026-10-18, after Test B):** the learned layer-1 parse scores word pairs symmetrically, so its arc
30directions are a tie-break artifact (76–94% of heads on the left vs 34% gold). The directional "complete assembly"
31(`comp`, HARD/SOFT) numbers with a learned parse inherit that bias. See `RESULTS-B.md`. Gold-tree controls and
32`conn` numbers are unaffected.
experiments/level2/RESULTS.md +7 -0
……@@ -55,3 +55,10 @@ Run `results/1792340082` (harness `854d470`, 10 seeds, tiers 3K/10K/30K). Scored
5555- **Several parses per sentence** (Linas's convention): a span is a candidate if it is an assembly in any of
5656 k parses. Tests whether the loss is parse noise that multiple linkages average out.
5757- Test B (choosing units) still needs a unit criterion.
58
59---
60
61**Correction (2026-10-18, after Test B):** the learned layer-1 parse scores word pairs symmetrically, so its arc
62directions are a tie-break artifact (76–94% of heads on the left vs 34% gold). The directional "complete assembly"
63(`comp`, HARD/SOFT) numbers with a learned parse inherit that bias. See `RESULTS-B.md`. Gold-tree controls and
64`conn` numbers are unaffected.