OverviewFilesCommitsIssuesDocsReleasesPeers

atomspace-layer-discovery commits / a0411ce

experiments/jigsaw-transformer: v2 results — order-dependent per-occurrence effect, small next to class error
K1 both arms PASS; K2 PASS for disjuncts (+0.013 ordered, -0.039 shuffled), FAIL for word arm (+0.006 shuffled); K3 word PASS (barely), dj FAIL; K4 FAIL; K5, K6 PASS. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
a0411ce38a7c232f3fe2d78786b9e9b1ba02ff78
by plankatron <fleeky@gmail.com> · 2026-10-18T21:38:52.000Z

Changed files (34) +1253 -0

experiments/jigsaw-transformer/RESULTS-v2.md +29 -0
……@@ -0,0 +1,29 @@
1# Jigsaw transformer v2 — results
2
3Run `results-v2/1792358782` (code `a2f3695`, 5 seeds, tiers 3K/10K/30K, ordered + shuffled). Scored against PREREG-v2
4(`6c2c668`, before any v2 code) by `summarize-v2.py`; full output in `results-v2/1792358782/summary.txt`.
5
6| | V2-WORD (actual neighbour words) | V2-DJ (Linas disjuncts) |
7|---|---|---|
8| K1 beats own collapse on sense | **PASS** +0.020 [+0.013, +0.027] | **PASS** +0.013 [+0.006, +0.021] |
9| K2 shuffled: does not | **FAIL** +0.006 [+0.002, +0.010] | **PASS** −0.039 [−0.073, −0.005] |
10| K3 beats V1 on sense | **PASS** (barely) +0.012 [+0.000, +0.024] | **FAIL** +0.006 [−0.006, +0.017] |
11| K4 beats MAJ on sense | **FAIL** −0.236 | **FAIL** −0.243 |
12| K5 V-measure not worse than V1 | **PASS** +0.008 | **PASS** +0.021 |
13| K6 audit | **PASS** 60/60 | **PASS** 60/60 |
14
15Unpredicted: V1 − V1-COLL = +0.010 [+0.001, +0.018] ordered, +0.003 [−0.002, +0.008] shuffled.
16
17## What it says
18
191. **Per-occurrence context now has a measured, order-dependent effect.** The collapse comparison isolates it.
20 V2-DJ: +0.013 with order, −0.039 shuffled. V2-WORD: +0.020 with order, +0.006 shuffled (the strict K2 fails,
21 but the effect is 3× larger with order). Even V1 had a small one (+0.010) — v1's J6 failure was partly
22 the TYPE-A comparator mixing in better word classes.
232. **The effect is small next to the real gap.** Every learner's *collapsed* labels are ~0.63 sense accuracy;
24 MAJ is 0.886. Most polysemous-word error is the word's class being wrong at the type level, not a failure to
25 pick the sense in context. Senses cannot be fixed before the classes are.
263. **Disjuncts give the best classes and parse:** V-measure 0.864 (V1 0.842, TYPE 0.901), attention parse F1
27 0.642 (V1 0.625), and the steepest learning curve (0.693 at 4K vs 0.653). But V2-DJ is *below* its own
28 collapse at 3K and 10K (−0.029, −0.030): disjunct evidence is sparse and only helps senses once there is
29 enough data.
experiments/jigsaw-transformer/results-v2/1792358782/commit.txt +1 -0
……@@ -0,0 +1 @@
1a2f36959ec58aae0a30c2848821550d37a381532
experiments/jigsaw-transformer/results-v2/1792358782/dj-seed-1.tsv +44 -0
……@@ -0,0 +1,44 @@
1C 1 3000 30 JT-V2D 1000 0.45557 0.51938 0.42579
2C 1 3000 30 JT-V2D 2000 0.58058 0.61082 0.54717
3C 1 3000 30 JT-V2D 3000 0.59996 0.62719 0.53585
4V 1 3000 30 JT-V2D 0.59996 0.62719 0.53585
5V 1 3000 30 JT-V2D-COLL 0.63935 0.64438 0.58491
6L 1 3000 30 ATTN-V2D 0.51421
7V 1 3000 30 TYPE-A 0.49386 0.55137 0.45220
8A 1 3000 30 replay PASS events=3 first_mismatch=-1
9A 1 3000 30 mutation PASS altered=1 detected_at=1
10V 1 3000 30 TYPE 0.77499 0.61384 0.63396
11V 1 3000 30 MAJ 0.97629 0.98795 0.88931
12L 1 3000 30 WORD-MI 0.14961
13L 1 3000 30 GOLD-MI 0.70421
14C 1 10000 30 JT-V2D 1000 0.45557 0.51938 0.42579
15C 1 10000 30 JT-V2D 2000 0.58058 0.61082 0.54717
16C 1 10000 30 JT-V2D 4000 0.67755 0.69027 0.62642
17C 1 10000 30 JT-V2D 8000 0.80670 0.78610 0.73270
18C 1 10000 30 JT-V2D 10000 0.79773 0.78308 0.68491
19V 1 10000 30 JT-V2D 0.79773 0.78308 0.68491
20V 1 10000 30 JT-V2D-COLL 0.81294 0.79521 0.70503
21L 1 10000 30 ATTN-V2D 0.61452
22V 1 10000 30 TYPE-A 0.64821 0.69973 0.59371
23A 1 10000 30 replay PASS events=5 first_mismatch=-1
24A 1 10000 30 mutation PASS altered=2 detected_at=2
25V 1 10000 30 TYPE 0.90545 0.78986 0.76164
26V 1 10000 30 MAJ 0.97820 0.98904 0.89937
27L 1 10000 30 WORD-MI 0.26366
28L 1 10000 30 GOLD-MI 0.72903
29C 1 30000 30 JT-V2D 1000 0.45557 0.51938 0.42579
30C 1 30000 30 JT-V2D 2000 0.58058 0.61082 0.54717
31C 1 30000 30 JT-V2D 4000 0.67755 0.69027 0.62642
32C 1 30000 30 JT-V2D 8000 0.80670 0.78610 0.73270
33C 1 30000 30 JT-V2D 16000 0.84560 0.81596 0.75220
34C 1 30000 30 JT-V2D 30000 0.86165 0.82274 0.73585
35V 1 30000 30 JT-V2D 0.86165 0.82274 0.73585
36V 1 30000 30 JT-V2D-COLL 0.86595 0.82555 0.72390
37L 1 30000 30 ATTN-V2D 0.64111
38V 1 30000 30 TYPE-A 0.79801 0.79651 0.69811
39A 1 30000 30 replay PASS events=6 first_mismatch=-1
40A 1 30000 30 mutation PASS altered=3 detected_at=3
41V 1 30000 30 TYPE 0.92877 0.84308 0.76289
42V 1 30000 30 MAJ 0.97776 0.98870 0.89623
43L 1 30000 30 WORD-MI 0.37422
44L 1 30000 30 GOLD-MI 0.73022
experiments/jigsaw-transformer/results-v2/1792358782/dj-seed-2.tsv +44 -0
……@@ -0,0 +1,44 @@
1C 2 3000 30 JT-V2D 1000 0.47334 0.53896 0.37814
2C 2 3000 30 JT-V2D 2000 0.53048 0.57892 0.40054
3C 2 3000 30 JT-V2D 3000 0.64090 0.66147 0.63612
4V 2 3000 30 JT-V2D 0.64090 0.66147 0.63612
5V 2 3000 30 JT-V2D-COLL 0.68292 0.68107 0.64223
6L 2 3000 30 ATTN-V2D 0.51357
7V 2 3000 30 TYPE-A 0.52169 0.58030 0.53360
8A 2 3000 30 replay PASS events=3 first_mismatch=-1
9A 2 3000 30 mutation PASS altered=1 detected_at=1
10V 2 3000 30 TYPE 0.81337 0.67858 0.55126
11V 2 3000 30 MAJ 0.97729 0.98834 0.88527
12L 2 3000 30 WORD-MI 0.16166
13L 2 3000 30 GOLD-MI 0.69330
14C 2 10000 30 JT-V2D 1000 0.47334 0.53896 0.37814
15C 2 10000 30 JT-V2D 2000 0.53048 0.57892 0.40054
16C 2 10000 30 JT-V2D 4000 0.69388 0.68990 0.43585
17C 2 10000 30 JT-V2D 8000 0.79827 0.76693 0.63001
18C 2 10000 30 JT-V2D 10000 0.80874 0.79957 0.77461
19V 2 10000 30 JT-V2D 0.80874 0.79957 0.77461
20V 2 10000 30 JT-V2D-COLL 0.82825 0.81020 0.78004
21L 2 10000 30 ATTN-V2D 0.60581
22V 2 10000 30 TYPE-A 0.67791 0.72137 0.69111
23A 2 10000 30 replay PASS events=5 first_mismatch=-1
24A 2 10000 30 mutation PASS altered=2 detected_at=2
25V 2 10000 30 TYPE 0.90581 0.80019 0.60353
26V 2 10000 30 MAJ 0.97879 0.98923 0.89409
27L 2 10000 30 WORD-MI 0.26885
28L 2 10000 30 GOLD-MI 0.72498
29C 2 30000 30 JT-V2D 1000 0.47334 0.53896 0.37814
30C 2 30000 30 JT-V2D 2000 0.53048 0.57892 0.40054
31C 2 30000 30 JT-V2D 4000 0.69388 0.68990 0.43585
32C 2 30000 30 JT-V2D 8000 0.79827 0.76693 0.63001
33C 2 30000 30 JT-V2D 16000 0.83613 0.80364 0.55329
34C 2 30000 30 JT-V2D 30000 0.86232 0.84202 0.56687
35V 2 30000 30 JT-V2D 0.86232 0.84202 0.56687
36V 2 30000 30 JT-V2D-COLL 0.86938 0.84719 0.55193
37L 2 30000 30 ATTN-V2D 0.64337
38V 2 30000 30 TYPE-A 0.79663 0.81006 0.50509
39A 2 30000 30 replay PASS events=6 first_mismatch=-1
40A 2 30000 30 mutation PASS altered=3 detected_at=3
41V 2 30000 30 TYPE 0.92451 0.82138 0.60421
42V 2 30000 30 MAJ 0.97775 0.98875 0.88934
43L 2 30000 30 WORD-MI 0.38086
44L 2 30000 30 GOLD-MI 0.73453
experiments/jigsaw-transformer/results-v2/1792358782/dj-seed-3.tsv +44 -0
……@@ -0,0 +1,44 @@
1C 3 3000 30 JT-V2D 1000 0.46120 0.51020 0.26049
2C 3 3000 30 JT-V2D 2000 0.52876 0.56303 0.34574
3C 3 3000 30 JT-V2D 3000 0.62378 0.64018 0.54533
4V 3 3000 30 JT-V2D 0.62378 0.64018 0.54533
5V 3 3000 30 JT-V2D-COLL 0.66379 0.65730 0.56089
6L 3 3000 30 ATTN-V2D 0.51202
7V 3 3000 30 TYPE-A 0.51535 0.56967 0.45061
8A 3 3000 30 replay PASS events=3 first_mismatch=-1
9A 3 3000 30 mutation PASS altered=1 detected_at=1
10V 3 3000 30 TYPE 0.79529 0.61468 0.75710
11V 3 3000 30 MAJ 0.97468 0.98651 0.86942
12L 3 3000 30 WORD-MI 0.15552
13L 3 3000 30 GOLD-MI 0.70371
14C 3 10000 30 JT-V2D 1000 0.46120 0.51020 0.26049
15C 3 10000 30 JT-V2D 2000 0.52876 0.56303 0.34574
16C 3 10000 30 JT-V2D 4000 0.67645 0.67582 0.49391
17C 3 10000 30 JT-V2D 8000 0.80803 0.78393 0.52977
18C 3 10000 30 JT-V2D 10000 0.81363 0.78211 0.63464
19V 3 10000 30 JT-V2D 0.81363 0.78211 0.63464
20V 3 10000 30 JT-V2D-COLL 0.83485 0.79448 0.68065
21L 3 10000 30 ATTN-V2D 0.61227
22V 3 10000 30 TYPE-A 0.67024 0.68435 0.55210
23A 3 10000 30 replay PASS events=5 first_mismatch=-1
24A 3 10000 30 mutation PASS altered=2 detected_at=2
25V 3 10000 30 TYPE 0.92204 0.84151 0.77470
26V 3 10000 30 MAJ 0.97641 0.98763 0.88024
27L 3 10000 30 WORD-MI 0.27664
28L 3 10000 30 GOLD-MI 0.74009
29C 3 30000 30 JT-V2D 1000 0.46120 0.51020 0.26049
30C 3 30000 30 JT-V2D 2000 0.52876 0.56303 0.34574
31C 3 30000 30 JT-V2D 4000 0.67645 0.67582 0.49391
32C 3 30000 30 JT-V2D 8000 0.80803 0.78393 0.52977
33C 3 30000 30 JT-V2D 16000 0.84891 0.82264 0.68065
34C 3 30000 30 JT-V2D 30000 0.86702 0.82425 0.76252
35V 3 30000 30 JT-V2D 0.86702 0.82425 0.76252
36V 3 30000 30 JT-V2D-COLL 0.87397 0.82788 0.75372
37L 3 30000 30 ATTN-V2D 0.62998
38V 3 30000 30 TYPE-A 0.79603 0.79385 0.71854
39A 3 30000 30 replay PASS events=6 first_mismatch=-1
40A 3 30000 30 mutation PASS altered=3 detected_at=3
41V 3 30000 30 TYPE 0.92078 0.81209 0.56631
42V 3 30000 30 MAJ 0.97727 0.98819 0.88566
43L 3 30000 30 WORD-MI 0.37845
44L 3 30000 30 GOLD-MI 0.72961
experiments/jigsaw-transformer/results-v2/1792358782/dj-seed-4.tsv +44 -0
……@@ -0,0 +1,44 @@
1C 4 3000 30 JT-V2D 1000 0.43152 0.50664 0.32407
2C 4 3000 30 JT-V2D 2000 0.58560 0.61635 0.44475
3C 4 3000 30 JT-V2D 3000 0.61589 0.63543 0.49831
4V 4 3000 30 JT-V2D 0.61589 0.63543 0.49831
5V 4 3000 30 JT-V2D-COLL 0.66255 0.64941 0.56068
6L 4 3000 30 ATTN-V2D 0.51519
7V 4 3000 30 TYPE-A 0.51036 0.56611 0.47525
8A 4 3000 30 replay PASS events=3 first_mismatch=-1
9A 4 3000 30 mutation PASS altered=1 detected_at=1
10V 4 3000 30 TYPE 0.78722 0.61768 0.57763
11V 4 3000 30 MAJ 0.97339 0.98581 0.86237
12L 4 3000 30 WORD-MI 0.16105
13L 4 3000 30 GOLD-MI 0.71224
14C 4 10000 30 JT-V2D 1000 0.43152 0.50664 0.32407
15C 4 10000 30 JT-V2D 2000 0.58560 0.61635 0.44475
16C 4 10000 30 JT-V2D 4000 0.70651 0.70224 0.57017
17C 4 10000 30 JT-V2D 8000 0.81443 0.78421 0.68881
18C 4 10000 30 JT-V2D 10000 0.80674 0.79322 0.56000
19V 4 10000 30 JT-V2D 0.80674 0.79322 0.56000
20V 4 10000 30 JT-V2D-COLL 0.82644 0.80531 0.55186
21L 4 10000 30 ATTN-V2D 0.59374
22V 4 10000 30 TYPE-A 0.67055 0.71817 0.45288
23A 4 10000 30 replay PASS events=5 first_mismatch=-1
24A 4 10000 30 mutation PASS altered=2 detected_at=2
25V 4 10000 30 TYPE 0.77178 0.53529 0.55458
26V 4 10000 30 MAJ 0.97560 0.98707 0.87458
27L 4 10000 30 WORD-MI 0.27889
28L 4 10000 30 GOLD-MI 0.73639
29C 4 30000 30 JT-V2D 1000 0.43152 0.50664 0.32407
30C 4 30000 30 JT-V2D 2000 0.58560 0.61635 0.44475
31C 4 30000 30 JT-V2D 4000 0.70651 0.70224 0.57017
32C 4 30000 30 JT-V2D 8000 0.81443 0.78421 0.68881
33C 4 30000 30 JT-V2D 16000 0.84054 0.82089 0.77695
34C 4 30000 30 JT-V2D 30000 0.86424 0.84626 0.58780
35V 4 30000 30 JT-V2D 0.86424 0.84626 0.58780
36V 4 30000 30 JT-V2D-COLL 0.87032 0.84899 0.58034
37L 4 30000 30 ATTN-V2D 0.64647
38V 4 30000 30 TYPE-A 0.80053 0.81824 0.55729
39A 4 30000 30 replay PASS events=6 first_mismatch=-1
40A 4 30000 30 mutation PASS altered=3 detected_at=3
41V 4 30000 30 TYPE 0.81074 0.54787 0.52203
42V 4 30000 30 MAJ 0.97478 0.98665 0.87051
43L 4 30000 30 WORD-MI 0.37526
44L 4 30000 30 GOLD-MI 0.73533
experiments/jigsaw-transformer/results-v2/1792358782/dj-seed-5.tsv +44 -0
……@@ -0,0 +1,44 @@
1C 5 3000 30 JT-V2D 1000 0.44416 0.52313 0.45171
2C 5 3000 30 JT-V2D 2000 0.55967 0.60058 0.38069
3C 5 3000 30 JT-V2D 3000 0.63849 0.64261 0.59751
4V 5 3000 30 JT-V2D 0.63849 0.64261 0.59751
5V 5 3000 30 JT-V2D-COLL 0.67658 0.66338 0.61308
6L 5 3000 30 ATTN-V2D 0.52291
7V 5 3000 30 TYPE-A 0.51737 0.56564 0.49657
8A 5 3000 30 replay PASS events=3 first_mismatch=-1
9A 5 3000 30 mutation PASS altered=1 detected_at=1
10V 5 3000 30 TYPE 0.87463 0.79210 0.75576
11V 5 3000 30 MAJ 0.97317 0.98562 0.87103
12L 5 3000 30 WORD-MI 0.16322
13L 5 3000 30 GOLD-MI 0.67959
14C 5 10000 30 JT-V2D 1000 0.44416 0.52313 0.45171
15C 5 10000 30 JT-V2D 2000 0.55967 0.60058 0.38069
16C 5 10000 30 JT-V2D 4000 0.70990 0.70916 0.61433
17C 5 10000 30 JT-V2D 8000 0.81669 0.80765 0.61308
18C 5 10000 30 JT-V2D 10000 0.80259 0.77271 0.50093
19V 5 10000 30 JT-V2D 0.80259 0.77271 0.50093
20V 5 10000 30 JT-V2D-COLL 0.82190 0.78174 0.58816
21L 5 10000 30 ATTN-V2D 0.61625
22V 5 10000 30 TYPE-A 0.65556 0.68318 0.59502
23A 5 10000 30 replay PASS events=5 first_mismatch=-1
24A 5 10000 30 mutation PASS altered=2 detected_at=2
25V 5 10000 30 TYPE 0.91059 0.81057 0.78069
26V 5 10000 30 MAJ 0.97412 0.98618 0.87601
27L 5 10000 30 WORD-MI 0.27297
28L 5 10000 30 GOLD-MI 0.73756
29C 5 30000 30 JT-V2D 1000 0.44416 0.52313 0.45171
30C 5 30000 30 JT-V2D 2000 0.55967 0.60058 0.38069
31C 5 30000 30 JT-V2D 4000 0.70990 0.70916 0.61433
32C 5 30000 30 JT-V2D 8000 0.81669 0.80765 0.61308
33C 5 30000 30 JT-V2D 16000 0.84918 0.81016 0.71776
34C 5 30000 30 JT-V2D 30000 0.86272 0.83752 0.56137
35V 5 30000 30 JT-V2D 0.86272 0.83752 0.56137
36V 5 30000 30 JT-V2D-COLL 0.86348 0.83718 0.53894
37L 5 30000 30 ATTN-V2D 0.64747
38V 5 30000 30 TYPE-A 0.78598 0.80272 0.48847
39A 5 30000 30 replay PASS events=6 first_mismatch=-1
40A 5 30000 30 mutation PASS altered=3 detected_at=3
41V 5 30000 30 TYPE 0.92034 0.82919 0.75576
42V 5 30000 30 MAJ 0.97578 0.98729 0.88598
43L 5 30000 30 WORD-MI 0.38075
44L 5 30000 30 GOLD-MI 0.74128
experiments/jigsaw-transformer/results-v2/1792358782/dj-shuf-1.tsv +32 -0
……@@ -0,0 +1,32 @@
1C 1 3000 30 JT-V2D-SHUF 1000 0.32388 0.42671 0.28742
2C 1 3000 30 JT-V2D-SHUF 2000 0.35075 0.46664 0.34969
3C 1 3000 30 JT-V2D-SHUF 3000 0.37890 0.47664 0.39874
4V 1 3000 30 JT-V2D-SHUF 0.37890 0.47664 0.39874
5V 1 3000 30 JT-V2D-COLL-SHUF 0.47416 0.52048 0.46101
6L 1 3000 30 ATTN-V2D-SHUF 0.37817
7V 1 3000 30 TYPE-A-SHUF 0.45467 0.51144 0.46604
8A 1 3000 30 replay PASS events=3 first_mismatch=-1
9A 1 3000 30 mutation PASS altered=1 detected_at=1
10C 1 10000 30 JT-V2D-SHUF 1000 0.32388 0.42671 0.28742
11C 1 10000 30 JT-V2D-SHUF 2000 0.35075 0.46664 0.34969
12C 1 10000 30 JT-V2D-SHUF 4000 0.37992 0.47966 0.42264
13C 1 10000 30 JT-V2D-SHUF 8000 0.44914 0.53733 0.48176
14C 1 10000 30 JT-V2D-SHUF 10000 0.48553 0.56644 0.38616
15V 1 10000 30 JT-V2D-SHUF 0.48553 0.56644 0.38616
16V 1 10000 30 JT-V2D-COLL-SHUF 0.57936 0.60952 0.36604
17L 1 10000 30 ATTN-V2D-SHUF 0.37675
18V 1 10000 30 TYPE-A-SHUF 0.53248 0.58014 0.31698
19A 1 10000 30 replay PASS events=5 first_mismatch=-1
20A 1 10000 30 mutation PASS altered=2 detected_at=2
21C 1 30000 30 JT-V2D-SHUF 1000 0.32388 0.42671 0.28742
22C 1 30000 30 JT-V2D-SHUF 2000 0.35075 0.46664 0.34969
23C 1 30000 30 JT-V2D-SHUF 4000 0.37992 0.47966 0.42264
24C 1 30000 30 JT-V2D-SHUF 8000 0.44914 0.53733 0.48176
25C 1 30000 30 JT-V2D-SHUF 16000 0.54555 0.60644 0.35723
26C 1 30000 30 JT-V2D-SHUF 30000 0.61503 0.67589 0.55912
27V 1 30000 30 JT-V2D-SHUF 0.61503 0.67589 0.55912
28V 1 30000 30 JT-V2D-COLL-SHUF 0.68806 0.71158 0.61132
29L 1 30000 30 ATTN-V2D-SHUF 0.36873
30V 1 30000 30 TYPE-A-SHUF 0.64847 0.68637 0.60189
31A 1 30000 30 replay PASS events=6 first_mismatch=-1
32A 1 30000 30 mutation PASS altered=3 detected_at=3
experiments/jigsaw-transformer/results-v2/1792358782/dj-shuf-2.tsv +32 -0
……@@ -0,0 +1,32 @@
1C 2 3000 30 JT-V2D-SHUF 1000 0.34113 0.43840 0.33876
2C 2 3000 30 JT-V2D-SHUF 2000 0.36242 0.45745 0.36999
3C 2 3000 30 JT-V2D-SHUF 3000 0.38779 0.49396 0.32315
4V 2 3000 30 JT-V2D-SHUF 0.38779 0.49396 0.32315
5V 2 3000 30 JT-V2D-COLL-SHUF 0.48022 0.53447 0.49151
6L 2 3000 30 ATTN-V2D-SHUF 0.39275
7V 2 3000 30 TYPE-A-SHUF 0.45275 0.52626 0.47794
8A 2 3000 30 replay PASS events=3 first_mismatch=-1
9A 2 3000 30 mutation PASS altered=1 detected_at=1
10C 2 10000 30 JT-V2D-SHUF 1000 0.34113 0.43840 0.33876
11C 2 10000 30 JT-V2D-SHUF 2000 0.36242 0.45745 0.36999
12C 2 10000 30 JT-V2D-SHUF 4000 0.36792 0.47043 0.38900
13C 2 10000 30 JT-V2D-SHUF 8000 0.43103 0.52509 0.33469
14C 2 10000 30 JT-V2D-SHUF 10000 0.45226 0.53834 0.36660
15V 2 10000 30 JT-V2D-SHUF 0.45226 0.53834 0.36660
16V 2 10000 30 JT-V2D-COLL-SHUF 0.54093 0.57582 0.52410
17L 2 10000 30 ATTN-V2D-SHUF 0.38402
18V 2 10000 30 TYPE-A-SHUF 0.49933 0.55270 0.46436
19A 2 10000 30 replay PASS events=5 first_mismatch=-1
20A 2 10000 30 mutation PASS altered=2 detected_at=2
21C 2 30000 30 JT-V2D-SHUF 1000 0.34113 0.43840 0.33876
22C 2 30000 30 JT-V2D-SHUF 2000 0.36242 0.45745 0.36999
23C 2 30000 30 JT-V2D-SHUF 4000 0.36792 0.47043 0.38900
24C 2 30000 30 JT-V2D-SHUF 8000 0.43103 0.52509 0.33469
25C 2 30000 30 JT-V2D-SHUF 16000 0.53124 0.60584 0.43652
26C 2 30000 30 JT-V2D-SHUF 30000 0.63622 0.68314 0.39579
27V 2 30000 30 JT-V2D-SHUF 0.63622 0.68314 0.39579
28V 2 30000 30 JT-V2D-COLL-SHUF 0.71398 0.72766 0.40869
29L 2 30000 30 ATTN-V2D-SHUF 0.37977
30V 2 30000 30 TYPE-A-SHUF 0.65724 0.68983 0.39986
31A 2 30000 30 replay PASS events=6 first_mismatch=-1
32A 2 30000 30 mutation PASS altered=3 detected_at=3
experiments/jigsaw-transformer/results-v2/1792358782/dj-shuf-3.tsv +32 -0
……@@ -0,0 +1,32 @@
1C 3 3000 30 JT-V2D-SHUF 1000 0.31027 0.36261 0.14208
2C 3 3000 30 JT-V2D-SHUF 2000 0.34860 0.44633 0.31055
3C 3 3000 30 JT-V2D-SHUF 3000 0.36525 0.46219 0.35250
4V 3 3000 30 JT-V2D-SHUF 0.36525 0.46219 0.35250
5V 3 3000 30 JT-V2D-COLL-SHUF 0.45346 0.49050 0.33965
6L 3 3000 30 ATTN-V2D-SHUF 0.38716
7V 3 3000 30 TYPE-A-SHUF 0.42339 0.48106 0.35995
8A 3 3000 30 replay PASS events=3 first_mismatch=-1
9A 3 3000 30 mutation PASS altered=1 detected_at=1
10C 3 10000 30 JT-V2D-SHUF 1000 0.31027 0.36261 0.14208
11C 3 10000 30 JT-V2D-SHUF 2000 0.34860 0.44633 0.31055
12C 3 10000 30 JT-V2D-SHUF 4000 0.39143 0.49008 0.17253
13C 3 10000 30 JT-V2D-SHUF 8000 0.43799 0.52704 0.32679
14C 3 10000 30 JT-V2D-SHUF 10000 0.45793 0.54361 0.37551
15V 3 10000 30 JT-V2D-SHUF 0.45793 0.54361 0.37551
16V 3 10000 30 JT-V2D-COLL-SHUF 0.56088 0.59238 0.46008
17L 3 10000 30 ATTN-V2D-SHUF 0.38010
18V 3 10000 30 TYPE-A-SHUF 0.51740 0.56471 0.45061
19A 3 10000 30 replay PASS events=5 first_mismatch=-1
20A 3 10000 30 mutation PASS altered=2 detected_at=2
21C 3 30000 30 JT-V2D-SHUF 1000 0.31027 0.36261 0.14208
22C 3 30000 30 JT-V2D-SHUF 2000 0.34860 0.44633 0.31055
23C 3 30000 30 JT-V2D-SHUF 4000 0.39143 0.49008 0.17253
24C 3 30000 30 JT-V2D-SHUF 8000 0.43799 0.52704 0.32679
25C 3 30000 30 JT-V2D-SHUF 16000 0.53456 0.60713 0.37212
26C 3 30000 30 JT-V2D-SHUF 30000 0.59731 0.65255 0.39986
27V 3 30000 30 JT-V2D-SHUF 0.59731 0.65255 0.39986
28V 3 30000 30 JT-V2D-COLL-SHUF 0.67468 0.69399 0.48038
29L 3 30000 30 ATTN-V2D-SHUF 0.38229
30V 3 30000 30 TYPE-A-SHUF 0.62813 0.66632 0.46211
31A 3 30000 30 replay PASS events=6 first_mismatch=-1
32A 3 30000 30 mutation PASS altered=3 detected_at=3
experiments/jigsaw-transformer/results-v2/1792358782/dj-shuf-4.tsv +32 -0
……@@ -0,0 +1,32 @@
1C 4 3000 30 JT-V2D-SHUF 1000 0.35631 0.44661 0.31932
2C 4 3000 30 JT-V2D-SHUF 2000 0.35774 0.46143 0.31254
3C 4 3000 30 JT-V2D-SHUF 3000 0.35951 0.45695 0.28203
4V 4 3000 30 JT-V2D-SHUF 0.35951 0.45695 0.28203
5V 4 3000 30 JT-V2D-COLL-SHUF 0.45766 0.49413 0.29966
6L 4 3000 30 ATTN-V2D-SHUF 0.39456
7V 4 3000 30 TYPE-A-SHUF 0.42244 0.47939 0.29356
8A 4 3000 30 replay PASS events=3 first_mismatch=-1
9A 4 3000 30 mutation PASS altered=1 detected_at=1
10C 4 10000 30 JT-V2D-SHUF 1000 0.35631 0.44661 0.31932
11C 4 10000 30 JT-V2D-SHUF 2000 0.35774 0.46143 0.31254
12C 4 10000 30 JT-V2D-SHUF 4000 0.38576 0.49560 0.37627
13C 4 10000 30 JT-V2D-SHUF 8000 0.45013 0.53955 0.33492
14C 4 10000 30 JT-V2D-SHUF 10000 0.46441 0.55199 0.46780
15V 4 10000 30 JT-V2D-SHUF 0.46441 0.55199 0.46780
16V 4 10000 30 JT-V2D-COLL-SHUF 0.55303 0.59392 0.51661
17L 4 10000 30 ATTN-V2D-SHUF 0.38806
18V 4 10000 30 TYPE-A-SHUF 0.51118 0.56660 0.49424
19A 4 10000 30 replay PASS events=5 first_mismatch=-1
20A 4 10000 30 mutation PASS altered=2 detected_at=2
21C 4 30000 30 JT-V2D-SHUF 1000 0.35631 0.44661 0.31932
22C 4 30000 30 JT-V2D-SHUF 2000 0.35774 0.46143 0.31254
23C 4 30000 30 JT-V2D-SHUF 4000 0.38576 0.49560 0.37627
24C 4 30000 30 JT-V2D-SHUF 8000 0.45013 0.53955 0.33492
25C 4 30000 30 JT-V2D-SHUF 16000 0.52728 0.59343 0.40407
26C 4 30000 30 JT-V2D-SHUF 30000 0.60097 0.66911 0.55525
27V 4 30000 30 JT-V2D-SHUF 0.60097 0.66911 0.55525
28V 4 30000 30 JT-V2D-COLL-SHUF 0.67683 0.70531 0.58034
29L 4 30000 30 ATTN-V2D-SHUF 0.37969
30V 4 30000 30 TYPE-A-SHUF 0.62732 0.67862 0.55864
31A 4 30000 30 replay PASS events=6 first_mismatch=-1
32A 4 30000 30 mutation PASS altered=3 detected_at=3
experiments/jigsaw-transformer/results-v2/1792358782/dj-shuf-5.tsv +32 -0
……@@ -0,0 +1,32 @@
1C 5 3000 30 JT-V2D-SHUF 1000 0.33970 0.44068 0.35701
2C 5 3000 30 JT-V2D-SHUF 2000 0.35307 0.45554 0.31900
3C 5 3000 30 JT-V2D-SHUF 3000 0.35887 0.46353 0.28847
4V 5 3000 30 JT-V2D-SHUF 0.35887 0.46353 0.28847
5V 5 3000 30 JT-V2D-COLL-SHUF 0.45524 0.49965 0.28224
6L 5 3000 30 ATTN-V2D-SHUF 0.38811
7V 5 3000 30 TYPE-A-SHUF 0.42190 0.48104 0.22741
8A 5 3000 30 replay PASS events=3 first_mismatch=-1
9A 5 3000 30 mutation PASS altered=1 detected_at=1
10C 5 10000 30 JT-V2D-SHUF 1000 0.33970 0.44068 0.35701
11C 5 10000 30 JT-V2D-SHUF 2000 0.35307 0.45554 0.31900
12C 5 10000 30 JT-V2D-SHUF 4000 0.37192 0.45832 0.27040
13C 5 10000 30 JT-V2D-SHUF 8000 0.43690 0.53341 0.42866
14C 5 10000 30 JT-V2D-SHUF 10000 0.46930 0.55543 0.38006
15V 5 10000 30 JT-V2D-SHUF 0.46930 0.55543 0.38006
16V 5 10000 30 JT-V2D-COLL-SHUF 0.55441 0.59454 0.40249
17L 5 10000 30 ATTN-V2D-SHUF 0.37932
18V 5 10000 30 TYPE-A-SHUF 0.51517 0.57210 0.38567
19A 5 10000 30 replay PASS events=5 first_mismatch=-1
20A 5 10000 30 mutation PASS altered=2 detected_at=2
21C 5 30000 30 JT-V2D-SHUF 1000 0.33970 0.44068 0.35701
22C 5 30000 30 JT-V2D-SHUF 2000 0.35307 0.45554 0.31900
23C 5 30000 30 JT-V2D-SHUF 4000 0.37192 0.45832 0.27040
24C 5 30000 30 JT-V2D-SHUF 8000 0.43690 0.53341 0.42866
25C 5 30000 30 JT-V2D-SHUF 16000 0.52923 0.59148 0.41994
26C 5 30000 30 JT-V2D-SHUF 30000 0.61869 0.67359 0.54019
27V 5 30000 30 JT-V2D-SHUF 0.61869 0.67359 0.54019
28V 5 30000 30 JT-V2D-COLL-SHUF 0.68264 0.70513 0.56511
29L 5 30000 30 ATTN-V2D-SHUF 0.37714
30V 5 30000 30 TYPE-A-SHUF 0.63816 0.67936 0.50156
31A 5 30000 30 replay PASS events=6 first_mismatch=-1
32A 5 30000 30 mutation PASS altered=3 detected_at=3
experiments/jigsaw-transformer/results-v2/1792358782/src.sha256 +2 -0
……@@ -0,0 +1,2 @@
178f81268f749a220d53a8fb0c734628514c4775c01d1d9f95b0ca70c626f3156 jt.c
22f4343faaa8bd25592fdadbc5bd38807f63d6823bcf85aff9eaeda50e96b8396 ../across-cluster/ac.c
experiments/jigsaw-transformer/results-v2/1792358782/summary.txt +81 -0
……@@ -0,0 +1,81 @@
1## Token classes (mean): V-measure / many-to-one / sense accuracy
2tier 3000
3 JT-V1 0.614 / 0.619 / 0.523
4 JT-V1-COLL 0.616 / 0.619 / 0.525
5 JT-V2W 0.607 / 0.621 / 0.511
6 JT-V2W-COLL 0.611 / 0.619 / 0.503
7 JT-V2D 0.624 / 0.641 / 0.563
8 JT-V2D-COLL 0.665 / 0.659 / 0.592
9 JT-V1-SHUF 0.478 / 0.512 / 0.364
10 JT-V1-COLL-SHUF 0.485 / 0.513 / 0.378
11 JT-V2W-SHUF 0.482 / 0.516 / 0.381
12 JT-V2W-COLL-SHUF 0.486 / 0.516 / 0.380
13 JT-V2D-SHUF 0.370 / 0.471 / 0.329
14 JT-V2D-COLL-SHUF 0.464 / 0.508 / 0.375
15 TYPE-A 0.512 / 0.567 / 0.482
16 TYPE-A-SHUF 0.435 / 0.496 / 0.365
17 TYPE 0.809 / 0.663 / 0.655
18 MAJ 0.975 / 0.987 / 0.875
19tier 10000
20 JT-V1 0.761 / 0.757 / 0.630
21 JT-V1-COLL 0.762 / 0.756 / 0.622
22 JT-V2W 0.771 / 0.763 / 0.643
23 JT-V2W-COLL 0.773 / 0.763 / 0.636
24 JT-V2D 0.806 / 0.786 / 0.631
25 JT-V2D-COLL 0.825 / 0.797 / 0.661
26 JT-V1-SHUF 0.582 / 0.607 / 0.447
27 JT-V1-COLL-SHUF 0.585 / 0.607 / 0.449
28 JT-V2W-SHUF 0.613 / 0.629 / 0.478
29 JT-V2W-COLL-SHUF 0.618 / 0.629 / 0.477
30 JT-V2D-SHUF 0.466 / 0.551 / 0.395
31 JT-V2D-COLL-SHUF 0.558 / 0.593 / 0.454
32 TYPE-A 0.664 / 0.701 / 0.577
33 TYPE-A-SHUF 0.515 / 0.567 / 0.422
34 TYPE 0.883 / 0.755 / 0.695
35 MAJ 0.977 / 0.988 / 0.885
36tier 30000
37 JT-V1 0.842 / 0.829 / 0.637
38 JT-V1-COLL 0.841 / 0.828 / 0.628
39 JT-V2W 0.851 / 0.835 / 0.649
40 JT-V2W-COLL 0.849 / 0.833 / 0.629
41 JT-V2D 0.864 / 0.835 / 0.643
42 JT-V2D-COLL 0.869 / 0.837 / 0.630
43 JT-V1-SHUF 0.709 / 0.722 / 0.549
44 JT-V1-COLL-SHUF 0.711 / 0.722 / 0.547
45 JT-V2W-SHUF 0.738 / 0.739 / 0.558
46 JT-V2W-COLL-SHUF 0.741 / 0.738 / 0.552
47 JT-V2D-SHUF 0.614 / 0.671 / 0.490
48 JT-V2D-COLL-SHUF 0.687 / 0.709 / 0.529
49 TYPE-A 0.795 / 0.804 / 0.593
50 TYPE-A-SHUF 0.640 / 0.680 / 0.505
51 TYPE 0.901 / 0.771 / 0.642
52 MAJ 0.977 / 0.988 / 0.886
53
54## Parse link F1 (mean)
553000: ATTN-V1 0.485 ATTN-V2W 0.489 ATTN-V2D 0.516 WORD-MI 0.158 GOLD-MI 0.699
5610000: ATTN-V1 0.572 ATTN-V2W 0.577 ATTN-V2D 0.609 WORD-MI 0.272 GOLD-MI 0.734
5730000: ATTN-V1 0.625 ATTN-V2W 0.631 ATTN-V2D 0.642 WORD-MI 0.378 GOLD-MI 0.734
58
59## Continuous checkpoints, tier 30000 (mean V-measure)
60 V1 1000:0.517 2000:0.580 4000:0.653 8000:0.746 16000:0.803 30000:0.842
61 V2W 1000:0.501 2000:0.570 4000:0.655 8000:0.756 16000:0.815 30000:0.851
62 V2D 1000:0.453 2000:0.557 4000:0.693 8000:0.809 16000:0.844 30000:0.864
63
64## Predictions (sense = sense accuracy on polysemous occurrences, at 30000)
65K1 V2W: PASS — sense JT-V2W − JT-V2W-COLL +0.020 [+0.013, +0.027]
66K2 V2W: FAIL — shuffled sense JT-V2W − COLL +0.006 [+0.002, +0.010]
67K3 V2W: PASS — sense JT-V2W − JT-V1 +0.012 [+0.000, +0.024]
68K4 V2W: FAIL — sense JT-V2W − MAJ -0.236 [-0.357, -0.116]
69K5 V2W: PASS — V-measure JT-V2W − JT-V1 +0.008 [+0.006, +0.011]
70K6 V2W: PASS — 60/60 audit checks PASS
71K1 V2D: PASS — sense JT-V2D − JT-V2D-COLL +0.013 [+0.006, +0.021]
72K2 V2D: PASS — shuffled sense JT-V2D − COLL -0.039 [-0.073, -0.005]
73K3 V2D: FAIL — sense JT-V2D − JT-V1 +0.006 [-0.006, +0.017]
74K4 V2D: FAIL — sense JT-V2D − MAJ -0.243 [-0.360, -0.125]
75K5 V2D: PASS — V-measure JT-V2D − JT-V1 +0.021 [+0.011, +0.031]
76K6 V2D: PASS — 60/60 audit checks PASS
77
78## Scored, no prediction
79V1 sense − V1-COLL: +0.010 [+0.001, +0.018]
80V1 shuffled sense − COLL: +0.003 [-0.002, +0.008]
81V2W sense − V2D: +0.006 [-0.012, +0.025]
experiments/jigsaw-transformer/results-v2/1792358782/v1-seed-1.tsv +44 -0
……@@ -0,0 +1,44 @@
1C 1 3000 30 JT-V1 1000 0.51007 0.52897 0.57673
2C 1 3000 30 JT-V1 2000 0.61088 0.61267 0.49182
3C 1 3000 30 JT-V1 3000 0.59293 0.59555 0.46415
4V 1 3000 30 JT-V1 0.59293 0.59555 0.46415
5V 1 3000 30 JT-V1-COLL 0.59624 0.59596 0.46604
6L 1 3000 30 ATTN-V1 0.48016
7V 1 3000 30 TYPE-A 0.49386 0.55137 0.45220
8A 1 3000 30 replay PASS events=3 first_mismatch=-1
9A 1 3000 30 mutation PASS altered=1 detected_at=1
10V 1 3000 30 TYPE 0.77499 0.61384 0.63396
11V 1 3000 30 MAJ 0.97629 0.98795 0.88931
12L 1 3000 30 WORD-MI 0.14961
13L 1 3000 30 GOLD-MI 0.70421
14C 1 10000 30 JT-V1 1000 0.51007 0.52897 0.57673
15C 1 10000 30 JT-V1 2000 0.61088 0.61267 0.49182
16C 1 10000 30 JT-V1 4000 0.64708 0.64247 0.55409
17C 1 10000 30 JT-V1 8000 0.72887 0.73178 0.64654
18C 1 10000 30 JT-V1 10000 0.75161 0.75637 0.65346
19V 1 10000 30 JT-V1 0.75161 0.75637 0.65346
20V 1 10000 30 JT-V1-COLL 0.75354 0.75507 0.64969
21L 1 10000 30 ATTN-V1 0.57421
22V 1 10000 30 TYPE-A 0.64821 0.69973 0.59371
23A 1 10000 30 replay PASS events=5 first_mismatch=-1
24A 1 10000 30 mutation PASS altered=2 detected_at=2
25V 1 10000 30 TYPE 0.90545 0.78986 0.76164
26V 1 10000 30 MAJ 0.97820 0.98904 0.89937
27L 1 10000 30 WORD-MI 0.26366
28L 1 10000 30 GOLD-MI 0.72903
29C 1 30000 30 JT-V1 1000 0.51007 0.52897 0.57673
30C 1 30000 30 JT-V1 2000 0.61088 0.61267 0.49182
31C 1 30000 30 JT-V1 4000 0.64708 0.64247 0.55409
32C 1 30000 30 JT-V1 8000 0.72887 0.73178 0.64654
33C 1 30000 30 JT-V1 16000 0.80335 0.79726 0.75283
34C 1 30000 30 JT-V1 30000 0.84553 0.82226 0.73836
35V 1 30000 30 JT-V1 0.84553 0.82226 0.73836
36V 1 30000 30 JT-V1-COLL 0.84290 0.81979 0.72956
37L 1 30000 30 ATTN-V1 0.62905
38V 1 30000 30 TYPE-A 0.79801 0.79651 0.69811
39A 1 30000 30 replay PASS events=6 first_mismatch=-1
40A 1 30000 30 mutation PASS altered=3 detected_at=3
41V 1 30000 30 TYPE 0.92877 0.84308 0.76289
42V 1 30000 30 MAJ 0.97776 0.98870 0.89623
43L 1 30000 30 WORD-MI 0.37422
44L 1 30000 30 GOLD-MI 0.73022
experiments/jigsaw-transformer/results-v2/1792358782/v1-seed-2.tsv +44 -0
……@@ -0,0 +1,44 @@
1C 2 3000 30 JT-V1 1000 0.53185 0.55463 0.42227
2C 2 3000 30 JT-V1 2000 0.55622 0.58562 0.43245
3C 2 3000 30 JT-V1 3000 0.62410 0.64242 0.59674
4V 2 3000 30 JT-V1 0.62410 0.64242 0.59674
5V 2 3000 30 JT-V1-COLL 0.62457 0.64042 0.59538
6L 2 3000 30 ATTN-V1 0.49492
7V 2 3000 30 TYPE-A 0.52169 0.58030 0.53360
8A 2 3000 30 replay PASS events=3 first_mismatch=-1
9A 2 3000 30 mutation PASS altered=1 detected_at=1
10V 2 3000 30 TYPE 0.81337 0.67858 0.55126
11V 2 3000 30 MAJ 0.97729 0.98834 0.88527
12L 2 3000 30 WORD-MI 0.16166
13L 2 3000 30 GOLD-MI 0.69330
14C 2 10000 30 JT-V1 1000 0.53185 0.55463 0.42227
15C 2 10000 30 JT-V1 2000 0.55622 0.58562 0.43245
16C 2 10000 30 JT-V1 4000 0.65251 0.65746 0.40394
17C 2 10000 30 JT-V1 8000 0.74463 0.74153 0.62933
18C 2 10000 30 JT-V1 10000 0.77409 0.77141 0.74406
19V 2 10000 30 JT-V1 0.77409 0.77141 0.74406
20V 2 10000 30 JT-V1-COLL 0.77128 0.76893 0.72777
21L 2 10000 30 ATTN-V1 0.58379
22V 2 10000 30 TYPE-A 0.67791 0.72137 0.69111
23A 2 10000 30 replay PASS events=5 first_mismatch=-1
24A 2 10000 30 mutation PASS altered=2 detected_at=2
25V 2 10000 30 TYPE 0.90581 0.80019 0.60353
26V 2 10000 30 MAJ 0.97879 0.98923 0.89409
27L 2 10000 30 WORD-MI 0.26885
28L 2 10000 30 GOLD-MI 0.72498
29C 2 30000 30 JT-V1 1000 0.53185 0.55463 0.42227
30C 2 30000 30 JT-V1 2000 0.55622 0.58562 0.43245
31C 2 30000 30 JT-V1 4000 0.65251 0.65746 0.40394
32C 2 30000 30 JT-V1 8000 0.74463 0.74153 0.62933
33C 2 30000 30 JT-V1 16000 0.79642 0.78121 0.59470
34C 2 30000 30 JT-V1 30000 0.84617 0.83670 0.56008
35V 2 30000 30 JT-V1 0.84617 0.83670 0.56008
36V 2 30000 30 JT-V1-COLL 0.84473 0.83491 0.54107
37L 2 30000 30 ATTN-V1 0.62671
38V 2 30000 30 TYPE-A 0.79663 0.81006 0.50509
39A 2 30000 30 replay PASS events=6 first_mismatch=-1
40A 2 30000 30 mutation PASS altered=3 detected_at=3
41V 2 30000 30 TYPE 0.92451 0.82138 0.60421
42V 2 30000 30 MAJ 0.97775 0.98875 0.88934
43L 2 30000 30 WORD-MI 0.38086
44L 2 30000 30 GOLD-MI 0.73453
experiments/jigsaw-transformer/results-v2/1792358782/v1-seed-3.tsv +44 -0
……@@ -0,0 +1,44 @@
1C 3 3000 30 JT-V1 1000 0.53489 0.54102 0.27537
2C 3 3000 30 JT-V1 2000 0.56374 0.57191 0.32476
3C 3 3000 30 JT-V1 3000 0.61895 0.62236 0.49256
4V 3 3000 30 JT-V1 0.61895 0.62236 0.49256
5V 3 3000 30 JT-V1-COLL 0.62433 0.62215 0.49323
6L 3 3000 30 ATTN-V1 0.47409
7V 3 3000 30 TYPE-A 0.51535 0.56967 0.45061
8A 3 3000 30 replay PASS events=3 first_mismatch=-1
9A 3 3000 30 mutation PASS altered=1 detected_at=1
10V 3 3000 30 TYPE 0.79529 0.61468 0.75710
11V 3 3000 30 MAJ 0.97468 0.98651 0.86942
12L 3 3000 30 WORD-MI 0.15552
13L 3 3000 30 GOLD-MI 0.70371
14C 3 10000 30 JT-V1 1000 0.53489 0.54102 0.27537
15C 3 10000 30 JT-V1 2000 0.56374 0.57191 0.32476
16C 3 10000 30 JT-V1 4000 0.65698 0.66436 0.52233
17C 3 10000 30 JT-V1 8000 0.73919 0.73620 0.50744
18C 3 10000 30 JT-V1 10000 0.76423 0.74158 0.58390
19V 3 10000 30 JT-V1 0.76423 0.74158 0.58390
20V 3 10000 30 JT-V1-COLL 0.76545 0.74144 0.58051
21L 3 10000 30 ATTN-V1 0.56515
22V 3 10000 30 TYPE-A 0.67024 0.68435 0.55210
23A 3 10000 30 replay PASS events=5 first_mismatch=-1
24A 3 10000 30 mutation PASS altered=2 detected_at=2
25V 3 10000 30 TYPE 0.92204 0.84151 0.77470
26V 3 10000 30 MAJ 0.97641 0.98763 0.88024
27L 3 10000 30 WORD-MI 0.27664
28L 3 10000 30 GOLD-MI 0.74009
29C 3 30000 30 JT-V1 1000 0.53489 0.54102 0.27537
30C 3 30000 30 JT-V1 2000 0.56374 0.57191 0.32476
31C 3 30000 30 JT-V1 4000 0.65698 0.66436 0.52233
32C 3 30000 30 JT-V1 8000 0.73919 0.73620 0.50744
33C 3 30000 30 JT-V1 16000 0.80245 0.79720 0.65562
34C 3 30000 30 JT-V1 30000 0.83872 0.81670 0.75034
35V 3 30000 30 JT-V1 0.83872 0.81670 0.75034
36V 3 30000 30 JT-V1-COLL 0.83673 0.81461 0.74290
37L 3 30000 30 ATTN-V1 0.61535
38V 3 30000 30 TYPE-A 0.79603 0.79385 0.71854
39A 3 30000 30 replay PASS events=6 first_mismatch=-1
40A 3 30000 30 mutation PASS altered=3 detected_at=3
41V 3 30000 30 TYPE 0.92078 0.81209 0.56631
42V 3 30000 30 MAJ 0.97727 0.98819 0.88566
43L 3 30000 30 WORD-MI 0.37845
44L 3 30000 30 GOLD-MI 0.72961
experiments/jigsaw-transformer/results-v2/1792358782/v1-seed-4.tsv +44 -0
……@@ -0,0 +1,44 @@
1C 4 3000 30 JT-V1 1000 0.50291 0.53564 0.44068
2C 4 3000 30 JT-V1 2000 0.60121 0.62236 0.41492
3C 4 3000 30 JT-V1 3000 0.61743 0.62383 0.52475
4V 4 3000 30 JT-V1 0.61743 0.62383 0.52475
5V 4 3000 30 JT-V1-COLL 0.61927 0.62453 0.53831
6L 4 3000 30 ATTN-V1 0.48578
7V 4 3000 30 TYPE-A 0.51036 0.56611 0.47525
8A 4 3000 30 replay PASS events=3 first_mismatch=-1
9A 4 3000 30 mutation PASS altered=1 detected_at=1
10V 4 3000 30 TYPE 0.78722 0.61768 0.57763
11V 4 3000 30 MAJ 0.97339 0.98581 0.86237
12L 4 3000 30 WORD-MI 0.16105
13L 4 3000 30 GOLD-MI 0.71224
14C 4 10000 30 JT-V1 1000 0.50291 0.53564 0.44068
15C 4 10000 30 JT-V1 2000 0.60121 0.62236 0.41492
16C 4 10000 30 JT-V1 4000 0.65731 0.66806 0.55932
17C 4 10000 30 JT-V1 8000 0.75745 0.75856 0.51119
18C 4 10000 30 JT-V1 10000 0.76652 0.77456 0.50508
19V 4 10000 30 JT-V1 0.76652 0.77456 0.50508
20V 4 10000 30 JT-V1-COLL 0.76706 0.77324 0.49559
21L 4 10000 30 ATTN-V1 0.56231
22V 4 10000 30 TYPE-A 0.67055 0.71817 0.45288
23A 4 10000 30 replay PASS events=5 first_mismatch=-1
24A 4 10000 30 mutation PASS altered=2 detected_at=2
25V 4 10000 30 TYPE 0.77178 0.53529 0.55458
26V 4 10000 30 MAJ 0.97560 0.98707 0.87458
27L 4 10000 30 WORD-MI 0.27889
28L 4 10000 30 GOLD-MI 0.73639
29C 4 30000 30 JT-V1 1000 0.50291 0.53564 0.44068
30C 4 30000 30 JT-V1 2000 0.60121 0.62236 0.41492
31C 4 30000 30 JT-V1 4000 0.65731 0.66806 0.55932
32C 4 30000 30 JT-V1 8000 0.75745 0.75856 0.51119
33C 4 30000 30 JT-V1 16000 0.80220 0.79713 0.77220
34C 4 30000 30 JT-V1 30000 0.85019 0.84458 0.59254
35V 4 30000 30 JT-V1 0.85019 0.84458 0.59254
36V 4 30000 30 JT-V1-COLL 0.85004 0.84361 0.59254
37L 4 30000 30 ATTN-V1 0.62900
38V 4 30000 30 TYPE-A 0.80053 0.81824 0.55729
39A 4 30000 30 replay PASS events=6 first_mismatch=-1
40A 4 30000 30 mutation PASS altered=3 detected_at=3
41V 4 30000 30 TYPE 0.81074 0.54787 0.52203
42V 4 30000 30 MAJ 0.97478 0.98665 0.87051
43L 4 30000 30 WORD-MI 0.37526
44L 4 30000 30 GOLD-MI 0.73533
experiments/jigsaw-transformer/results-v2/1792358782/v1-seed-5.tsv +44 -0
……@@ -0,0 +1,44 @@
1C 5 3000 30 JT-V1 1000 0.50511 0.53814 0.38505
2C 5 3000 30 JT-V1 2000 0.56858 0.58961 0.44424
3C 5 3000 30 JT-V1 3000 0.61451 0.61246 0.53707
4V 5 3000 30 JT-V1 0.61451 0.61246 0.53707
5V 5 3000 30 JT-V1-COLL 0.61687 0.61142 0.53146
6L 5 3000 30 ATTN-V1 0.48871
7V 5 3000 30 TYPE-A 0.51737 0.56564 0.49657
8A 5 3000 30 replay PASS events=3 first_mismatch=-1
9A 5 3000 30 mutation PASS altered=1 detected_at=1
10V 5 3000 30 TYPE 0.87463 0.79210 0.75576
11V 5 3000 30 MAJ 0.97317 0.98562 0.87103
12L 5 3000 30 WORD-MI 0.16322
13L 5 3000 30 GOLD-MI 0.67959
14C 5 10000 30 JT-V1 1000 0.50511 0.53814 0.38505
15C 5 10000 30 JT-V1 2000 0.56858 0.58961 0.44424
16C 5 10000 30 JT-V1 4000 0.64889 0.66025 0.49844
17C 5 10000 30 JT-V1 8000 0.75895 0.77383 0.67913
18C 5 10000 30 JT-V1 10000 0.74958 0.74055 0.66417
19V 5 10000 30 JT-V1 0.74958 0.74055 0.66417
20V 5 10000 30 JT-V1-COLL 0.75033 0.73944 0.65483
21L 5 10000 30 ATTN-V1 0.57325
22V 5 10000 30 TYPE-A 0.65556 0.68318 0.59502
23A 5 10000 30 replay PASS events=5 first_mismatch=-1
24A 5 10000 30 mutation PASS altered=2 detected_at=2
25V 5 10000 30 TYPE 0.91059 0.81057 0.78069
26V 5 10000 30 MAJ 0.97412 0.98618 0.87601
27L 5 10000 30 WORD-MI 0.27297
28L 5 10000 30 GOLD-MI 0.73756
29C 5 30000 30 JT-V1 1000 0.50511 0.53814 0.38505
30C 5 30000 30 JT-V1 2000 0.56858 0.58961 0.44424
31C 5 30000 30 JT-V1 4000 0.64889 0.66025 0.49844
32C 5 30000 30 JT-V1 8000 0.75895 0.77383 0.67913
33C 5 30000 30 JT-V1 16000 0.81288 0.78730 0.52710
34C 5 30000 30 JT-V1 30000 0.83115 0.82662 0.54455
35V 5 30000 30 JT-V1 0.83115 0.82662 0.54455
36V 5 30000 30 JT-V1-COLL 0.83043 0.82544 0.53146
37L 5 30000 30 ATTN-V1 0.62343
38V 5 30000 30 TYPE-A 0.78598 0.80272 0.48847
39A 5 30000 30 replay PASS events=6 first_mismatch=-1
40A 5 30000 30 mutation PASS altered=3 detected_at=3
41V 5 30000 30 TYPE 0.92034 0.82919 0.75576
42V 5 30000 30 MAJ 0.97578 0.98729 0.88598
43L 5 30000 30 WORD-MI 0.38075
44L 5 30000 30 GOLD-MI 0.74128
experiments/jigsaw-transformer/results-v2/1792358782/v1-shuf-1.tsv +32 -0
……@@ -0,0 +1,32 @@
1C 1 3000 30 JT-V1-SHUF 1000 0.41373 0.45952 0.31824
2C 1 3000 30 JT-V1-SHUF 2000 0.47039 0.51630 0.34717
3C 1 3000 30 JT-V1-SHUF 3000 0.50072 0.53363 0.44403
4V 1 3000 30 JT-V1-SHUF 0.50072 0.53363 0.44403
5V 1 3000 30 JT-V1-COLL-SHUF 0.50621 0.53390 0.48176
6L 1 3000 30 ATTN-V1-SHUF 0.36698
7V 1 3000 30 TYPE-A-SHUF 0.45467 0.51144 0.46604
8A 1 3000 30 replay PASS events=3 first_mismatch=-1
9A 1 3000 30 mutation PASS altered=1 detected_at=1
10C 1 10000 30 JT-V1-SHUF 1000 0.41373 0.45952 0.31824
11C 1 10000 30 JT-V1-SHUF 2000 0.47039 0.51630 0.34717
12C 1 10000 30 JT-V1-SHUF 4000 0.51285 0.55068 0.51132
13C 1 10000 30 JT-V1-SHUF 8000 0.56313 0.59144 0.57233
14C 1 10000 30 JT-V1-SHUF 10000 0.61289 0.62760 0.36981
15V 1 10000 30 JT-V1-SHUF 0.61289 0.62760 0.36981
16V 1 10000 30 JT-V1-COLL-SHUF 0.61668 0.62890 0.38176
17L 1 10000 30 ATTN-V1-SHUF 0.37373
18V 1 10000 30 TYPE-A-SHUF 0.53248 0.58014 0.31698
19A 1 10000 30 replay PASS events=5 first_mismatch=-1
20A 1 10000 30 mutation PASS altered=2 detected_at=2
21C 1 30000 30 JT-V1-SHUF 1000 0.41373 0.45952 0.31824
22C 1 30000 30 JT-V1-SHUF 2000 0.47039 0.51630 0.34717
23C 1 30000 30 JT-V1-SHUF 4000 0.51285 0.55068 0.51132
24C 1 30000 30 JT-V1-SHUF 8000 0.56313 0.59144 0.57233
25C 1 30000 30 JT-V1-SHUF 16000 0.65941 0.67281 0.43648
26C 1 30000 30 JT-V1-SHUF 30000 0.71109 0.72205 0.63459
27V 1 30000 30 JT-V1-SHUF 0.71109 0.72205 0.63459
28V 1 30000 30 JT-V1-COLL-SHUF 0.71557 0.72205 0.63333
29L 1 30000 30 ATTN-V1-SHUF 0.36524
30V 1 30000 30 TYPE-A-SHUF 0.64847 0.68637 0.60189
31A 1 30000 30 replay PASS events=6 first_mismatch=-1
32A 1 30000 30 mutation PASS altered=3 detected_at=3
experiments/jigsaw-transformer/results-v2/1792358782/v1-shuf-2.tsv +32 -0
……@@ -0,0 +1,32 @@
1C 2 3000 30 JT-V1-SHUF 1000 0.43946 0.47954 0.39308
2C 2 3000 30 JT-V1-SHUF 2000 0.46175 0.48602 0.32587
3C 2 3000 30 JT-V1-SHUF 3000 0.49510 0.53965 0.48608
4V 2 3000 30 JT-V1-SHUF 0.49510 0.53965 0.48608
5V 2 3000 30 JT-V1-COLL-SHUF 0.50330 0.53965 0.48608
6L 2 3000 30 ATTN-V1-SHUF 0.37593
7V 2 3000 30 TYPE-A-SHUF 0.45275 0.52626 0.47794
8A 2 3000 30 replay PASS events=3 first_mismatch=-1
9A 2 3000 30 mutation PASS altered=1 detected_at=1
10C 2 10000 30 JT-V1-SHUF 1000 0.43946 0.47954 0.39308
11C 2 10000 30 JT-V1-SHUF 2000 0.46175 0.48602 0.32587
12C 2 10000 30 JT-V1-SHUF 4000 0.47883 0.51204 0.45485
13C 2 10000 30 JT-V1-SHUF 8000 0.54044 0.56809 0.34148
14C 2 10000 30 JT-V1-SHUF 10000 0.56113 0.59162 0.45418
15V 2 10000 30 JT-V1-SHUF 0.56113 0.59162 0.45418
16V 2 10000 30 JT-V1-COLL-SHUF 0.56514 0.58948 0.45146
17L 2 10000 30 ATTN-V1-SHUF 0.37769
18V 2 10000 30 TYPE-A-SHUF 0.49933 0.55270 0.46436
19A 2 10000 30 replay PASS events=5 first_mismatch=-1
20A 2 10000 30 mutation PASS altered=2 detected_at=2
21C 2 30000 30 JT-V1-SHUF 1000 0.43946 0.47954 0.39308
22C 2 30000 30 JT-V1-SHUF 2000 0.46175 0.48602 0.32587
23C 2 30000 30 JT-V1-SHUF 4000 0.47883 0.51204 0.45485
24C 2 30000 30 JT-V1-SHUF 8000 0.54044 0.56809 0.34148
25C 2 30000 30 JT-V1-SHUF 16000 0.63690 0.65505 0.42566
26C 2 30000 30 JT-V1-SHUF 30000 0.73193 0.73794 0.44399
27V 2 30000 30 JT-V1-SHUF 0.73193 0.73794 0.44399
28V 2 30000 30 JT-V1-COLL-SHUF 0.73248 0.73704 0.44060
29L 2 30000 30 ATTN-V1-SHUF 0.37553
30V 2 30000 30 TYPE-A-SHUF 0.65724 0.68983 0.39986
31A 2 30000 30 replay PASS events=6 first_mismatch=-1
32A 2 30000 30 mutation PASS altered=3 detected_at=3
experiments/jigsaw-transformer/results-v2/1792358782/v1-shuf-3.tsv +32 -0
……@@ -0,0 +1,32 @@
1C 3 3000 30 JT-V1-SHUF 1000 0.41258 0.39085 0.17321
2C 3 3000 30 JT-V1-SHUF 2000 0.45359 0.49029 0.35047
3C 3 3000 30 JT-V1-SHUF 3000 0.46840 0.50049 0.35724
4V 3 3000 30 JT-V1-SHUF 0.46840 0.50049 0.35724
5V 3 3000 30 JT-V1-COLL-SHUF 0.47517 0.50196 0.38295
6L 3 3000 30 ATTN-V1-SHUF 0.37230
7V 3 3000 30 TYPE-A-SHUF 0.42339 0.48106 0.35995
8A 3 3000 30 replay PASS events=3 first_mismatch=-1
9A 3 3000 30 mutation PASS altered=1 detected_at=1
10C 3 10000 30 JT-V1-SHUF 1000 0.41258 0.39085 0.17321
11C 3 10000 30 JT-V1-SHUF 2000 0.45359 0.49029 0.35047
12C 3 10000 30 JT-V1-SHUF 4000 0.49433 0.52313 0.30311
13C 3 10000 30 JT-V1-SHUF 8000 0.56200 0.58574 0.34844
14C 3 10000 30 JT-V1-SHUF 10000 0.58060 0.60594 0.48038
15V 3 10000 30 JT-V1-SHUF 0.58060 0.60594 0.48038
16V 3 10000 30 JT-V1-COLL-SHUF 0.58692 0.60755 0.48444
17L 3 10000 30 ATTN-V1-SHUF 0.37506
18V 3 10000 30 TYPE-A-SHUF 0.51740 0.56471 0.45061
19A 3 10000 30 replay PASS events=5 first_mismatch=-1
20A 3 10000 30 mutation PASS altered=2 detected_at=2
21C 3 30000 30 JT-V1-SHUF 1000 0.41258 0.39085 0.17321
22C 3 30000 30 JT-V1-SHUF 2000 0.45359 0.49029 0.35047
23C 3 30000 30 JT-V1-SHUF 4000 0.49433 0.52313 0.30311
24C 3 30000 30 JT-V1-SHUF 8000 0.56200 0.58574 0.34844
25C 3 30000 30 JT-V1-SHUF 16000 0.65097 0.66737 0.38160
26C 3 30000 30 JT-V1-SHUF 30000 0.69793 0.70594 0.50947
27V 3 30000 30 JT-V1-SHUF 0.69793 0.70594 0.50947
28V 3 30000 30 JT-V1-COLL-SHUF 0.70016 0.70727 0.50068
29L 3 30000 30 ATTN-V1-SHUF 0.38652
30V 3 30000 30 TYPE-A-SHUF 0.62813 0.66632 0.46211
31A 3 30000 30 replay PASS events=6 first_mismatch=-1
32A 3 30000 30 mutation PASS altered=3 detected_at=3
experiments/jigsaw-transformer/results-v2/1792358782/v1-shuf-4.tsv +32 -0
……@@ -0,0 +1,32 @@
1C 4 3000 30 JT-V1-SHUF 1000 0.46325 0.49672 0.38847
2C 4 3000 30 JT-V1-SHUF 2000 0.45417 0.49560 0.37627
3C 4 3000 30 JT-V1-SHUF 3000 0.46159 0.49092 0.30644
4V 4 3000 30 JT-V1-SHUF 0.46159 0.49092 0.30644
5V 4 3000 30 JT-V1-COLL-SHUF 0.46959 0.49057 0.31254
6L 4 3000 30 ATTN-V1-SHUF 0.37539
7V 4 3000 30 TYPE-A-SHUF 0.42244 0.47939 0.29356
8A 4 3000 30 replay PASS events=3 first_mismatch=-1
9A 4 3000 30 mutation PASS altered=1 detected_at=1
10C 4 10000 30 JT-V1-SHUF 1000 0.46325 0.49672 0.38847
11C 4 10000 30 JT-V1-SHUF 2000 0.45417 0.49560 0.37627
12C 4 10000 30 JT-V1-SHUF 4000 0.49922 0.54633 0.42915
13C 4 10000 30 JT-V1-SHUF 8000 0.56928 0.59287 0.33559
14C 4 10000 30 JT-V1-SHUF 10000 0.57176 0.60035 0.52203
15V 4 10000 30 JT-V1-SHUF 0.57176 0.60035 0.52203
16V 4 10000 30 JT-V1-COLL-SHUF 0.57415 0.60000 0.52000
17L 4 10000 30 ATTN-V1-SHUF 0.37929
18V 4 10000 30 TYPE-A-SHUF 0.51118 0.56660 0.49424
19A 4 10000 30 replay PASS events=5 first_mismatch=-1
20A 4 10000 30 mutation PASS altered=2 detected_at=2
21C 4 30000 30 JT-V1-SHUF 1000 0.46325 0.49672 0.38847
22C 4 30000 30 JT-V1-SHUF 2000 0.45417 0.49560 0.37627
23C 4 30000 30 JT-V1-SHUF 4000 0.49922 0.54633 0.42915
24C 4 30000 30 JT-V1-SHUF 8000 0.56928 0.59287 0.33559
25C 4 30000 30 JT-V1-SHUF 16000 0.64122 0.65234 0.46712
26C 4 30000 30 JT-V1-SHUF 30000 0.70106 0.72243 0.61831
27V 4 30000 30 JT-V1-SHUF 0.70106 0.72243 0.61831
28V 4 30000 30 JT-V1-COLL-SHUF 0.70405 0.72208 0.62034
29L 4 30000 30 ATTN-V1-SHUF 0.37734
30V 4 30000 30 TYPE-A-SHUF 0.62732 0.67862 0.55864
31A 4 30000 30 replay PASS events=6 first_mismatch=-1
32A 4 30000 30 mutation PASS altered=3 detected_at=3
experiments/jigsaw-transformer/results-v2/1792358782/v1-shuf-5.tsv +32 -0
……@@ -0,0 +1,32 @@
1C 5 3000 30 JT-V1-SHUF 1000 0.43279 0.47256 0.39377
2C 5 3000 30 JT-V1-SHUF 2000 0.46174 0.50069 0.38754
3C 5 3000 30 JT-V1-SHUF 3000 0.46241 0.49562 0.22555
4V 5 3000 30 JT-V1-SHUF 0.46241 0.49562 0.22555
5V 5 3000 30 JT-V1-COLL-SHUF 0.46958 0.49653 0.22617
6L 5 3000 30 ATTN-V1-SHUF 0.37020
7V 5 3000 30 TYPE-A-SHUF 0.42190 0.48104 0.22741
8A 5 3000 30 replay PASS events=3 first_mismatch=-1
9A 5 3000 30 mutation PASS altered=1 detected_at=1
10C 5 10000 30 JT-V1-SHUF 1000 0.43279 0.47256 0.39377
11C 5 10000 30 JT-V1-SHUF 2000 0.46174 0.50069 0.38754
12C 5 10000 30 JT-V1-SHUF 4000 0.49062 0.50611 0.35078
13C 5 10000 30 JT-V1-SHUF 8000 0.54986 0.58308 0.41994
14C 5 10000 30 JT-V1-SHUF 10000 0.58301 0.61066 0.40810
15V 5 10000 30 JT-V1-SHUF 0.58301 0.61066 0.40810
16V 5 10000 30 JT-V1-COLL-SHUF 0.58357 0.60941 0.40498
17L 5 10000 30 ATTN-V1-SHUF 0.37665
18V 5 10000 30 TYPE-A-SHUF 0.51517 0.57210 0.38567
19A 5 10000 30 replay PASS events=5 first_mismatch=-1
20A 5 10000 30 mutation PASS altered=2 detected_at=2
21C 5 30000 30 JT-V1-SHUF 1000 0.43279 0.47256 0.39377
22C 5 30000 30 JT-V1-SHUF 2000 0.46174 0.50069 0.38754
23C 5 30000 30 JT-V1-SHUF 4000 0.49062 0.50611 0.35078
24C 5 30000 30 JT-V1-SHUF 8000 0.54986 0.58308 0.41994
25C 5 30000 30 JT-V1-SHUF 16000 0.64654 0.65240 0.54829
26C 5 30000 30 JT-V1-SHUF 30000 0.70267 0.72020 0.54081
27V 5 30000 30 JT-V1-SHUF 0.70267 0.72020 0.54081
28V 5 30000 30 JT-V1-COLL-SHUF 0.70406 0.71992 0.53894
29L 5 30000 30 ATTN-V1-SHUF 0.37932
30V 5 30000 30 TYPE-A-SHUF 0.63816 0.67936 0.50156
31A 5 30000 30 replay PASS events=6 first_mismatch=-1
32A 5 30000 30 mutation PASS altered=3 detected_at=3
experiments/jigsaw-transformer/results-v2/1792358782/word-seed-1.tsv +44 -0
……@@ -0,0 +1,44 @@
1C 1 3000 30 JT-V2W 1000 0.49832 0.52842 0.49748
2C 1 3000 30 JT-V2W 2000 0.59350 0.60137 0.54340
3C 1 3000 30 JT-V2W 3000 0.59256 0.60041 0.44025
4V 1 3000 30 JT-V2W 0.59256 0.60041 0.44025
5V 1 3000 30 JT-V2W-COLL 0.59588 0.59877 0.43459
6L 1 3000 30 ATTN-V2W 0.48603
7V 1 3000 30 TYPE-A 0.49386 0.55137 0.45220
8A 1 3000 30 replay PASS events=3 first_mismatch=-1
9A 1 3000 30 mutation PASS altered=1 detected_at=1
10V 1 3000 30 TYPE 0.77499 0.61384 0.63396
11V 1 3000 30 MAJ 0.97629 0.98795 0.88931
12L 1 3000 30 WORD-MI 0.14961
13L 1 3000 30 GOLD-MI 0.70421
14C 1 10000 30 JT-V2W 1000 0.49832 0.52842 0.49748
15C 1 10000 30 JT-V2W 2000 0.59350 0.60137 0.54340
16C 1 10000 30 JT-V2W 4000 0.64957 0.64685 0.56792
17C 1 10000 30 JT-V2W 8000 0.74060 0.74110 0.65849
18C 1 10000 30 JT-V2W 10000 0.76367 0.76397 0.65346
19V 1 10000 30 JT-V2W 0.76367 0.76397 0.65346
20V 1 10000 30 JT-V2W-COLL 0.76621 0.76445 0.64717
21L 1 10000 30 ATTN-V2W 0.57706
22V 1 10000 30 TYPE-A 0.64821 0.69973 0.59371
23A 1 10000 30 replay PASS events=5 first_mismatch=-1
24A 1 10000 30 mutation PASS altered=2 detected_at=2
25V 1 10000 30 TYPE 0.90545 0.78986 0.76164
26V 1 10000 30 MAJ 0.97820 0.98904 0.89937
27L 1 10000 30 WORD-MI 0.26366
28L 1 10000 30 GOLD-MI 0.72903
29C 1 30000 30 JT-V2W 1000 0.49832 0.52842 0.49748
30C 1 30000 30 JT-V2W 2000 0.59350 0.60137 0.54340
31C 1 30000 30 JT-V2W 4000 0.64957 0.64685 0.56792
32C 1 30000 30 JT-V2W 8000 0.74060 0.74110 0.65849
33C 1 30000 30 JT-V2W 16000 0.81087 0.80473 0.75535
34C 1 30000 30 JT-V2W 30000 0.85236 0.82527 0.74025
35V 1 30000 30 JT-V2W 0.85236 0.82527 0.74025
36V 1 30000 30 JT-V2W-COLL 0.85022 0.82404 0.72767
37L 1 30000 30 ATTN-V2W 0.63302
38V 1 30000 30 TYPE-A 0.79801 0.79651 0.69811
39A 1 30000 30 replay PASS events=6 first_mismatch=-1
40A 1 30000 30 mutation PASS altered=3 detected_at=3
41V 1 30000 30 TYPE 0.92877 0.84308 0.76289
42V 1 30000 30 MAJ 0.97776 0.98870 0.89623
43L 1 30000 30 WORD-MI 0.37422
44L 1 30000 30 GOLD-MI 0.73022
experiments/jigsaw-transformer/results-v2/1792358782/word-seed-2.tsv +44 -0
……@@ -0,0 +1,44 @@
1C 2 3000 30 JT-V2W 1000 0.51172 0.54683 0.41819
2C 2 3000 30 JT-V2W 2000 0.55779 0.58858 0.43177
3C 2 3000 30 JT-V2W 3000 0.62003 0.64332 0.59606
4V 2 3000 30 JT-V2W 0.62003 0.64332 0.59606
5V 2 3000 30 JT-V2W-COLL 0.62285 0.64352 0.59470
6L 2 3000 30 ATTN-V2W 0.50012
7V 2 3000 30 TYPE-A 0.52169 0.58030 0.53360
8A 2 3000 30 replay PASS events=3 first_mismatch=-1
9A 2 3000 30 mutation PASS altered=1 detected_at=1
10V 2 3000 30 TYPE 0.81337 0.67858 0.55126
11V 2 3000 30 MAJ 0.97729 0.98834 0.88527
12L 2 3000 30 WORD-MI 0.16166
13L 2 3000 30 GOLD-MI 0.69330
14C 2 10000 30 JT-V2W 1000 0.51172 0.54683 0.41819
15C 2 10000 30 JT-V2W 2000 0.55779 0.58858 0.43177
16C 2 10000 30 JT-V2W 4000 0.65645 0.66485 0.40801
17C 2 10000 30 JT-V2W 8000 0.75432 0.75160 0.64358
18C 2 10000 30 JT-V2W 10000 0.78454 0.77900 0.76511
19V 2 10000 30 JT-V2W 0.78454 0.77900 0.76511
20V 2 10000 30 JT-V2W-COLL 0.78386 0.77707 0.75085
21L 2 10000 30 ATTN-V2W 0.58788
22V 2 10000 30 TYPE-A 0.67791 0.72137 0.69111
23A 2 10000 30 replay PASS events=5 first_mismatch=-1
24A 2 10000 30 mutation PASS altered=2 detected_at=2
25V 2 10000 30 TYPE 0.90581 0.80019 0.60353
26V 2 10000 30 MAJ 0.97879 0.98923 0.89409
27L 2 10000 30 WORD-MI 0.26885
28L 2 10000 30 GOLD-MI 0.72498
29C 2 30000 30 JT-V2W 1000 0.51172 0.54683 0.41819
30C 2 30000 30 JT-V2W 2000 0.55779 0.58858 0.43177
31C 2 30000 30 JT-V2W 4000 0.65645 0.66485 0.40801
32C 2 30000 30 JT-V2W 8000 0.75432 0.75160 0.64358
33C 2 30000 30 JT-V2W 16000 0.80848 0.78901 0.60760
34C 2 30000 30 JT-V2W 30000 0.85188 0.84105 0.57502
35V 2 30000 30 JT-V2W 0.85188 0.84105 0.57502
36V 2 30000 30 JT-V2W-COLL 0.85049 0.83884 0.55193
37L 2 30000 30 ATTN-V2W 0.62823
38V 2 30000 30 TYPE-A 0.79663 0.81006 0.50509
39A 2 30000 30 replay PASS events=6 first_mismatch=-1
40A 2 30000 30 mutation PASS altered=3 detected_at=3
41V 2 30000 30 TYPE 0.92451 0.82138 0.60421
42V 2 30000 30 MAJ 0.97775 0.98875 0.88934
43L 2 30000 30 WORD-MI 0.38086
44L 2 30000 30 GOLD-MI 0.73453
experiments/jigsaw-transformer/results-v2/1792358782/word-seed-3.tsv +44 -0
……@@ -0,0 +1,44 @@
1C 3 3000 30 JT-V2W 1000 0.51398 0.52676 0.27673
2C 3 3000 30 JT-V2W 2000 0.55278 0.56548 0.33829
3C 3 3000 30 JT-V2W 3000 0.60787 0.62292 0.48444
4V 3 3000 30 JT-V2W 0.60787 0.62292 0.48444
5V 3 3000 30 JT-V2W-COLL 0.61275 0.62180 0.47632
6L 3 3000 30 ATTN-V2W 0.47612
7V 3 3000 30 TYPE-A 0.51535 0.56967 0.45061
8A 3 3000 30 replay PASS events=3 first_mismatch=-1
9A 3 3000 30 mutation PASS altered=1 detected_at=1
10V 3 3000 30 TYPE 0.79529 0.61468 0.75710
11V 3 3000 30 MAJ 0.97468 0.98651 0.86942
12L 3 3000 30 WORD-MI 0.15552
13L 3 3000 30 GOLD-MI 0.70371
14C 3 10000 30 JT-V2W 1000 0.51398 0.52676 0.27673
15C 3 10000 30 JT-V2W 2000 0.55278 0.56548 0.33829
16C 3 10000 30 JT-V2W 4000 0.65987 0.67372 0.54939
17C 3 10000 30 JT-V2W 8000 0.74663 0.74326 0.50135
18C 3 10000 30 JT-V2W 10000 0.77552 0.74717 0.60690
19V 3 10000 30 JT-V2W 0.77552 0.74717 0.60690
20V 3 10000 30 JT-V2W-COLL 0.77620 0.74633 0.59946
21L 3 10000 30 ATTN-V2W 0.57685
22V 3 10000 30 TYPE-A 0.67024 0.68435 0.55210
23A 3 10000 30 replay PASS events=5 first_mismatch=-1
24A 3 10000 30 mutation PASS altered=2 detected_at=2
25V 3 10000 30 TYPE 0.92204 0.84151 0.77470
26V 3 10000 30 MAJ 0.97641 0.98763 0.88024
27L 3 10000 30 WORD-MI 0.27664
28L 3 10000 30 GOLD-MI 0.74009
29C 3 30000 30 JT-V2W 1000 0.51398 0.52676 0.27673
30C 3 30000 30 JT-V2W 2000 0.55278 0.56548 0.33829
31C 3 30000 30 JT-V2W 4000 0.65987 0.67372 0.54939
32C 3 30000 30 JT-V2W 8000 0.74663 0.74326 0.50135
33C 3 30000 30 JT-V2W 16000 0.81548 0.80664 0.64682
34C 3 30000 30 JT-V2W 30000 0.84895 0.82313 0.76793
35V 3 30000 30 JT-V2W 0.84895 0.82313 0.76793
36V 3 30000 30 JT-V2W-COLL 0.84644 0.82020 0.74628
37L 3 30000 30 ATTN-V2W 0.62275
38V 3 30000 30 TYPE-A 0.79603 0.79385 0.71854
39A 3 30000 30 replay PASS events=6 first_mismatch=-1
40A 3 30000 30 mutation PASS altered=3 detected_at=3
41V 3 30000 30 TYPE 0.92078 0.81209 0.56631
42V 3 30000 30 MAJ 0.97727 0.98819 0.88566
43L 3 30000 30 WORD-MI 0.37845
44L 3 30000 30 GOLD-MI 0.72961
experiments/jigsaw-transformer/results-v2/1792358782/word-seed-4.tsv +44 -0
……@@ -0,0 +1,44 @@
1C 4 3000 30 JT-V2W 1000 0.49278 0.52229 0.36881
2C 4 3000 30 JT-V2W 2000 0.58625 0.61349 0.41898
3C 4 3000 30 JT-V2W 3000 0.60837 0.62586 0.54441
4V 4 3000 30 JT-V2W 0.60837 0.62586 0.54441
5V 4 3000 30 JT-V2W-COLL 0.61241 0.62397 0.52542
6L 4 3000 30 ATTN-V2W 0.49074
7V 4 3000 30 TYPE-A 0.51036 0.56611 0.47525
8A 4 3000 30 replay PASS events=3 first_mismatch=-1
9A 4 3000 30 mutation PASS altered=1 detected_at=1
10V 4 3000 30 TYPE 0.78722 0.61768 0.57763
11V 4 3000 30 MAJ 0.97339 0.98581 0.86237
12L 4 3000 30 WORD-MI 0.16105
13L 4 3000 30 GOLD-MI 0.71224
14C 4 10000 30 JT-V2W 1000 0.49278 0.52229 0.36881
15C 4 10000 30 JT-V2W 2000 0.58625 0.61349 0.41898
16C 4 10000 30 JT-V2W 4000 0.65589 0.67345 0.57288
17C 4 10000 30 JT-V2W 8000 0.77226 0.76855 0.52136
18C 4 10000 30 JT-V2W 10000 0.76795 0.77554 0.51797
19V 4 10000 30 JT-V2W 0.76795 0.77554 0.51797
20V 4 10000 30 JT-V2W-COLL 0.77267 0.77617 0.51797
21L 4 10000 30 ATTN-V2W 0.56637
22V 4 10000 30 TYPE-A 0.67055 0.71817 0.45288
23A 4 10000 30 replay PASS events=5 first_mismatch=-1
24A 4 10000 30 mutation PASS altered=2 detected_at=2
25V 4 10000 30 TYPE 0.77178 0.53529 0.55458
26V 4 10000 30 MAJ 0.97560 0.98707 0.87458
27L 4 10000 30 WORD-MI 0.27889
28L 4 10000 30 GOLD-MI 0.73639
29C 4 30000 30 JT-V2W 1000 0.49278 0.52229 0.36881
30C 4 30000 30 JT-V2W 2000 0.58625 0.61349 0.41898
31C 4 30000 30 JT-V2W 4000 0.65589 0.67345 0.57288
32C 4 30000 30 JT-V2W 8000 0.77226 0.76855 0.52136
33C 4 30000 30 JT-V2W 16000 0.81278 0.80545 0.78034
34C 4 30000 30 JT-V2W 30000 0.85909 0.85017 0.61559
35V 4 30000 30 JT-V2W 0.85909 0.85017 0.61559
36V 4 30000 30 JT-V2W-COLL 0.85714 0.84857 0.58915
37L 4 30000 30 ATTN-V2W 0.64005
38V 4 30000 30 TYPE-A 0.80053 0.81824 0.55729
39A 4 30000 30 replay PASS events=6 first_mismatch=-1
40A 4 30000 30 mutation PASS altered=3 detected_at=3
41V 4 30000 30 TYPE 0.81074 0.54787 0.52203
42V 4 30000 30 MAJ 0.97478 0.98665 0.87051
43L 4 30000 30 WORD-MI 0.37526
44L 4 30000 30 GOLD-MI 0.73533
experiments/jigsaw-transformer/results-v2/1792358782/word-seed-5.tsv +44 -0
……@@ -0,0 +1,44 @@
1C 5 3000 30 JT-V2W 1000 0.48809 0.51980 0.37134
2C 5 3000 30 JT-V2W 2000 0.55975 0.58690 0.45421
3C 5 3000 30 JT-V2W 3000 0.60658 0.61211 0.49034
4V 5 3000 30 JT-V2W 0.60658 0.61211 0.49034
5V 5 3000 30 JT-V2W-COLL 0.60943 0.60941 0.48411
6L 5 3000 30 ATTN-V2W 0.49395
7V 5 3000 30 TYPE-A 0.51737 0.56564 0.49657
8A 5 3000 30 replay PASS events=3 first_mismatch=-1
9A 5 3000 30 mutation PASS altered=1 detected_at=1
10V 5 3000 30 TYPE 0.87463 0.79210 0.75576
11V 5 3000 30 MAJ 0.97317 0.98562 0.87103
12L 5 3000 30 WORD-MI 0.16322
13L 5 3000 30 GOLD-MI 0.67959
14C 5 10000 30 JT-V2W 1000 0.48809 0.51980 0.37134
15C 5 10000 30 JT-V2W 2000 0.55975 0.58690 0.45421
16C 5 10000 30 JT-V2W 4000 0.65252 0.66664 0.49470
17C 5 10000 30 JT-V2W 8000 0.76411 0.77911 0.70093
18C 5 10000 30 JT-V2W 10000 0.76336 0.75021 0.66978
19V 5 10000 30 JT-V2W 0.76336 0.75021 0.66978
20V 5 10000 30 JT-V2W-COLL 0.76574 0.74993 0.66355
21L 5 10000 30 ATTN-V2W 0.57656
22V 5 10000 30 TYPE-A 0.65556 0.68318 0.59502
23A 5 10000 30 replay PASS events=5 first_mismatch=-1
24A 5 10000 30 mutation PASS altered=2 detected_at=2
25V 5 10000 30 TYPE 0.91059 0.81057 0.78069
26V 5 10000 30 MAJ 0.97412 0.98618 0.87601
27L 5 10000 30 WORD-MI 0.27297
28L 5 10000 30 GOLD-MI 0.73756
29C 5 30000 30 JT-V2W 1000 0.48809 0.51980 0.37134
30C 5 30000 30 JT-V2W 2000 0.55975 0.58690 0.45421
31C 5 30000 30 JT-V2W 4000 0.65252 0.66664 0.49470
32C 5 30000 30 JT-V2W 8000 0.76411 0.77911 0.70093
33C 5 30000 30 JT-V2W 16000 0.82703 0.79814 0.53333
34C 5 30000 30 JT-V2W 30000 0.84048 0.83322 0.54704
35V 5 30000 30 JT-V2W 0.84048 0.83322 0.54704
36V 5 30000 30 JT-V2W-COLL 0.84010 0.83225 0.53146
37L 5 30000 30 ATTN-V2W 0.63101
38V 5 30000 30 TYPE-A 0.78598 0.80272 0.48847
39A 5 30000 30 replay PASS events=6 first_mismatch=-1
40A 5 30000 30 mutation PASS altered=3 detected_at=3
41V 5 30000 30 TYPE 0.92034 0.82919 0.75576
42V 5 30000 30 MAJ 0.97578 0.98729 0.88598
43L 5 30000 30 WORD-MI 0.38075
44L 5 30000 30 GOLD-MI 0.74128
experiments/jigsaw-transformer/results-v2/1792358782/word-shuf-1.tsv +32 -0
……@@ -0,0 +1,32 @@
1C 1 3000 30 JT-V2W-SHUF 1000 0.42351 0.46233 0.32327
2C 1 3000 30 JT-V2W-SHUF 2000 0.46528 0.50815 0.35849
3C 1 3000 30 JT-V2W-SHUF 3000 0.50094 0.53144 0.44340
4V 1 3000 30 JT-V2W-SHUF 0.50094 0.53144 0.44340
5V 1 3000 30 JT-V2W-COLL-SHUF 0.50487 0.53130 0.44340
6L 1 3000 30 ATTN-V2W-SHUF 0.36365
7V 1 3000 30 TYPE-A-SHUF 0.45467 0.51144 0.46604
8A 1 3000 30 replay PASS events=3 first_mismatch=-1
9A 1 3000 30 mutation PASS altered=1 detected_at=1
10C 1 10000 30 JT-V2W-SHUF 1000 0.42351 0.46233 0.32327
11C 1 10000 30 JT-V2W-SHUF 2000 0.46528 0.50815 0.35849
12C 1 10000 30 JT-V2W-SHUF 4000 0.51971 0.55466 0.54277
13C 1 10000 30 JT-V2W-SHUF 8000 0.59633 0.61740 0.56164
14C 1 10000 30 JT-V2W-SHUF 10000 0.63899 0.64596 0.42138
15V 1 10000 30 JT-V2W-SHUF 0.63899 0.64596 0.42138
16V 1 10000 30 JT-V2W-COLL-SHUF 0.64326 0.64719 0.43333
17L 1 10000 30 ATTN-V2W-SHUF 0.37611
18V 1 10000 30 TYPE-A-SHUF 0.53248 0.58014 0.31698
19A 1 10000 30 replay PASS events=5 first_mismatch=-1
20A 1 10000 30 mutation PASS altered=2 detected_at=2
21C 1 30000 30 JT-V2W-SHUF 1000 0.42351 0.46233 0.32327
22C 1 30000 30 JT-V2W-SHUF 2000 0.46528 0.50815 0.35849
23C 1 30000 30 JT-V2W-SHUF 4000 0.51971 0.55466 0.54277
24C 1 30000 30 JT-V2W-SHUF 8000 0.59633 0.61740 0.56164
25C 1 30000 30 JT-V2W-SHUF 16000 0.68351 0.69000 0.44654
26C 1 30000 30 JT-V2W-SHUF 30000 0.74391 0.74130 0.64969
27V 1 30000 30 JT-V2W-SHUF 0.74391 0.74130 0.64969
28V 1 30000 30 JT-V2W-COLL-SHUF 0.74711 0.74021 0.64403
29L 1 30000 30 ATTN-V2W-SHUF 0.37405
30V 1 30000 30 TYPE-A-SHUF 0.64847 0.68637 0.60189
31A 1 30000 30 replay PASS events=6 first_mismatch=-1
32A 1 30000 30 mutation PASS altered=3 detected_at=3
experiments/jigsaw-transformer/results-v2/1792358782/word-shuf-2.tsv +32 -0
……@@ -0,0 +1,32 @@
1C 2 3000 30 JT-V2W-SHUF 1000 0.45137 0.48030 0.38086
2C 2 3000 30 JT-V2W-SHUF 2000 0.45628 0.48485 0.28513
3C 2 3000 30 JT-V2W-SHUF 3000 0.49721 0.54503 0.50034
4V 2 3000 30 JT-V2W-SHUF 0.49721 0.54503 0.50034
5V 2 3000 30 JT-V2W-COLL-SHUF 0.50067 0.54462 0.49898
6L 2 3000 30 ATTN-V2W-SHUF 0.38258
7V 2 3000 30 TYPE-A-SHUF 0.45275 0.52626 0.47794
8A 2 3000 30 replay PASS events=3 first_mismatch=-1
9A 2 3000 30 mutation PASS altered=1 detected_at=1
10C 2 10000 30 JT-V2W-SHUF 1000 0.45137 0.48030 0.38086
11C 2 10000 30 JT-V2W-SHUF 2000 0.45628 0.48485 0.28513
12C 2 10000 30 JT-V2W-SHUF 4000 0.49910 0.52612 0.47997
13C 2 10000 30 JT-V2W-SHUF 8000 0.57699 0.59804 0.35845
14C 2 10000 30 JT-V2W-SHUF 10000 0.59938 0.61661 0.48405
15V 2 10000 30 JT-V2W-SHUF 0.59938 0.61661 0.48405
16V 2 10000 30 JT-V2W-COLL-SHUF 0.60221 0.61495 0.48133
17L 2 10000 30 ATTN-V2W-SHUF 0.37857
18V 2 10000 30 TYPE-A-SHUF 0.49933 0.55270 0.46436
19A 2 10000 30 replay PASS events=5 first_mismatch=-1
20A 2 10000 30 mutation PASS altered=2 detected_at=2
21C 2 30000 30 JT-V2W-SHUF 1000 0.45137 0.48030 0.38086
22C 2 30000 30 JT-V2W-SHUF 2000 0.45628 0.48485 0.28513
23C 2 30000 30 JT-V2W-SHUF 4000 0.49910 0.52612 0.47997
24C 2 30000 30 JT-V2W-SHUF 8000 0.57699 0.59804 0.35845
25C 2 30000 30 JT-V2W-SHUF 16000 0.66976 0.67527 0.43517
26C 2 30000 30 JT-V2W-SHUF 30000 0.76343 0.75933 0.44807
27V 2 30000 30 JT-V2W-SHUF 0.76343 0.75933 0.44807
28V 2 30000 30 JT-V2W-COLL-SHUF 0.76342 0.75720 0.43720
29L 2 30000 30 ATTN-V2W-SHUF 0.37441
30V 2 30000 30 TYPE-A-SHUF 0.65724 0.68983 0.39986
31A 2 30000 30 replay PASS events=6 first_mismatch=-1
32A 2 30000 30 mutation PASS altered=3 detected_at=3
experiments/jigsaw-transformer/results-v2/1792358782/word-shuf-3.tsv +32 -0
……@@ -0,0 +1,32 @@
1C 3 3000 30 JT-V2W-SHUF 1000 0.42250 0.40307 0.28281
2C 3 3000 30 JT-V2W-SHUF 2000 0.45058 0.48225 0.34574
3C 3 3000 30 JT-V2W-SHUF 3000 0.48131 0.50748 0.39310
4V 3 3000 30 JT-V2W-SHUF 0.48131 0.50748 0.39310
5V 3 3000 30 JT-V2W-COLL-SHUF 0.48611 0.50846 0.39107
6L 3 3000 30 ATTN-V2W-SHUF 0.37872
7V 3 3000 30 TYPE-A-SHUF 0.42339 0.48106 0.35995
8A 3 3000 30 replay PASS events=3 first_mismatch=-1
9A 3 3000 30 mutation PASS altered=1 detected_at=1
10C 3 10000 30 JT-V2W-SHUF 1000 0.42250 0.40307 0.28281
11C 3 10000 30 JT-V2W-SHUF 2000 0.45058 0.48225 0.34574
12C 3 10000 30 JT-V2W-SHUF 4000 0.50379 0.53767 0.34506
13C 3 10000 30 JT-V2W-SHUF 8000 0.58632 0.60671 0.46211
14C 3 10000 30 JT-V2W-SHUF 10000 0.61123 0.62823 0.49865
15V 3 10000 30 JT-V2W-SHUF 0.61123 0.62823 0.49865
16V 3 10000 30 JT-V2W-COLL-SHUF 0.61706 0.62760 0.48579
17L 3 10000 30 ATTN-V2W-SHUF 0.37547
18V 3 10000 30 TYPE-A-SHUF 0.51740 0.56471 0.45061
19A 3 10000 30 replay PASS events=5 first_mismatch=-1
20A 3 10000 30 mutation PASS altered=2 detected_at=2
21C 3 30000 30 JT-V2W-SHUF 1000 0.42250 0.40307 0.28281
22C 3 30000 30 JT-V2W-SHUF 2000 0.45058 0.48225 0.34574
23C 3 30000 30 JT-V2W-SHUF 4000 0.50379 0.53767 0.34506
24C 3 30000 30 JT-V2W-SHUF 8000 0.58632 0.60671 0.46211
25C 3 30000 30 JT-V2W-SHUF 16000 0.68262 0.68358 0.48173
26C 3 30000 30 JT-V2W-SHUF 30000 0.72866 0.72250 0.50947
27V 3 30000 30 JT-V2W-SHUF 0.72866 0.72250 0.50947
28V 3 30000 30 JT-V2W-COLL-SHUF 0.73088 0.72250 0.50677
29L 3 30000 30 ATTN-V2W-SHUF 0.38619
30V 3 30000 30 TYPE-A-SHUF 0.62813 0.66632 0.46211
31A 3 30000 30 replay PASS events=6 first_mismatch=-1
32A 3 30000 30 mutation PASS altered=3 detected_at=3
experiments/jigsaw-transformer/results-v2/1792358782/word-shuf-4.tsv +32 -0
……@@ -0,0 +1,32 @@
1C 4 3000 30 JT-V2W-SHUF 1000 0.45742 0.48379 0.36610
2C 4 3000 30 JT-V2W-SHUF 2000 0.46026 0.49259 0.37288
3C 4 3000 30 JT-V2W-SHUF 3000 0.46703 0.49916 0.30441
4V 4 3000 30 JT-V2W-SHUF 0.46703 0.49916 0.30441
5V 4 3000 30 JT-V2W-COLL-SHUF 0.47142 0.49972 0.30034
6L 4 3000 30 ATTN-V2W-SHUF 0.37831
7V 4 3000 30 TYPE-A-SHUF 0.42244 0.47939 0.29356
8A 4 3000 30 replay PASS events=3 first_mismatch=-1
9A 4 3000 30 mutation PASS altered=1 detected_at=1
10C 4 10000 30 JT-V2W-SHUF 1000 0.45742 0.48379 0.36610
11C 4 10000 30 JT-V2W-SHUF 2000 0.46026 0.49259 0.37288
12C 4 10000 30 JT-V2W-SHUF 4000 0.51553 0.55646 0.43525
13C 4 10000 30 JT-V2W-SHUF 8000 0.59886 0.60901 0.43051
14C 4 10000 30 JT-V2W-SHUF 10000 0.59960 0.62446 0.55864
15V 4 10000 30 JT-V2W-SHUF 0.59960 0.62446 0.55864
16V 4 10000 30 JT-V2W-COLL-SHUF 0.60859 0.62474 0.55864
17L 4 10000 30 ATTN-V2W-SHUF 0.38034
18V 4 10000 30 TYPE-A-SHUF 0.51118 0.56660 0.49424
19A 4 10000 30 replay PASS events=5 first_mismatch=-1
20A 4 10000 30 mutation PASS altered=2 detected_at=2
21C 4 30000 30 JT-V2W-SHUF 1000 0.45742 0.48379 0.36610
22C 4 30000 30 JT-V2W-SHUF 2000 0.46026 0.49259 0.37288
23C 4 30000 30 JT-V2W-SHUF 4000 0.51553 0.55646 0.43525
24C 4 30000 30 JT-V2W-SHUF 8000 0.59886 0.60901 0.43051
25C 4 30000 30 JT-V2W-SHUF 16000 0.66974 0.66778 0.51593
26C 4 30000 30 JT-V2W-SHUF 30000 0.73090 0.73760 0.62780
27V 4 30000 30 JT-V2W-SHUF 0.73090 0.73760 0.62780
28V 4 30000 30 JT-V2W-COLL-SHUF 0.73655 0.73788 0.62441
29L 4 30000 30 ATTN-V2W-SHUF 0.37815
30V 4 30000 30 TYPE-A-SHUF 0.62732 0.67862 0.55864
31A 4 30000 30 replay PASS events=6 first_mismatch=-1
32A 4 30000 30 mutation PASS altered=3 detected_at=3
experiments/jigsaw-transformer/results-v2/1792358782/word-shuf-5.tsv +32 -0
……@@ -0,0 +1,32 @@
1C 5 3000 30 JT-V2W-SHUF 1000 0.44301 0.46694 0.41059
2C 5 3000 30 JT-V2W-SHUF 2000 0.46000 0.49743 0.38692
3C 5 3000 30 JT-V2W-SHUF 3000 0.46527 0.49743 0.26417
4V 5 3000 30 JT-V2W-SHUF 0.46527 0.49743 0.26417
5V 5 3000 30 JT-V2W-COLL-SHUF 0.46849 0.49646 0.26604
6L 5 3000 30 ATTN-V2W-SHUF 0.37431
7V 5 3000 30 TYPE-A-SHUF 0.42190 0.48104 0.22741
8A 5 3000 30 replay PASS events=3 first_mismatch=-1
9A 5 3000 30 mutation PASS altered=1 detected_at=1
10C 5 10000 30 JT-V2W-SHUF 1000 0.44301 0.46694 0.41059
11C 5 10000 30 JT-V2W-SHUF 2000 0.46000 0.49743 0.38692
12C 5 10000 30 JT-V2W-SHUF 4000 0.49784 0.51521 0.38318
13C 5 10000 30 JT-V2W-SHUF 8000 0.58396 0.61163 0.47352
14C 5 10000 30 JT-V2W-SHUF 10000 0.61336 0.63122 0.42679
15V 5 10000 30 JT-V2W-SHUF 0.61336 0.63122 0.42679
16V 5 10000 30 JT-V2W-COLL-SHUF 0.61768 0.63129 0.42368
17L 5 10000 30 ATTN-V2W-SHUF 0.37665
18V 5 10000 30 TYPE-A-SHUF 0.51517 0.57210 0.38567
19A 5 10000 30 replay PASS events=5 first_mismatch=-1
20A 5 10000 30 mutation PASS altered=2 detected_at=2
21C 5 30000 30 JT-V2W-SHUF 1000 0.44301 0.46694 0.41059
22C 5 30000 30 JT-V2W-SHUF 2000 0.46000 0.49743 0.38692
23C 5 30000 30 JT-V2W-SHUF 4000 0.49784 0.51521 0.38318
24C 5 30000 30 JT-V2W-SHUF 8000 0.58396 0.61163 0.47352
25C 5 30000 30 JT-V2W-SHUF 16000 0.67893 0.67380 0.57321
26C 5 30000 30 JT-V2W-SHUF 30000 0.72388 0.73243 0.55327
27V 5 30000 30 JT-V2W-SHUF 0.72388 0.73243 0.55327
28V 5 30000 30 JT-V2W-COLL-SHUF 0.72690 0.73270 0.54704
29L 5 30000 30 ATTN-V2W-SHUF 0.37899
30V 5 30000 30 TYPE-A-SHUF 0.63816 0.67936 0.50156
31A 5 30000 30 replay PASS events=6 first_mismatch=-1
32A 5 30000 30 mutation PASS altered=3 detected_at=3