## Token classes (mean): V-measure / many-to-one / sense accuracy
tier 3000
JT-V1 0.614 / 0.619 / 0.523
JT-V1-COLL 0.616 / 0.619 / 0.525
JT-V2W 0.607 / 0.621 / 0.511
JT-V2W-COLL 0.611 / 0.619 / 0.503
JT-V2D 0.624 / 0.641 / 0.563
JT-V2D-COLL 0.665 / 0.659 / 0.592
JT-V1-SHUF 0.478 / 0.512 / 0.364
JT-V1-COLL-SHUF 0.485 / 0.513 / 0.378
JT-V2W-SHUF 0.482 / 0.516 / 0.381
JT-V2W-COLL-SHUF 0.486 / 0.516 / 0.380
JT-V2D-SHUF 0.370 / 0.471 / 0.329
JT-V2D-COLL-SHUF 0.464 / 0.508 / 0.375
TYPE-A 0.512 / 0.567 / 0.482
TYPE-A-SHUF 0.435 / 0.496 / 0.365
TYPE 0.809 / 0.663 / 0.655
MAJ 0.975 / 0.987 / 0.875
tier 10000
JT-V1 0.761 / 0.757 / 0.630
JT-V1-COLL 0.762 / 0.756 / 0.622
JT-V2W 0.771 / 0.763 / 0.643
JT-V2W-COLL 0.773 / 0.763 / 0.636
JT-V2D 0.806 / 0.786 / 0.631
JT-V2D-COLL 0.825 / 0.797 / 0.661
JT-V1-SHUF 0.582 / 0.607 / 0.447
JT-V1-COLL-SHUF 0.585 / 0.607 / 0.449
JT-V2W-SHUF 0.613 / 0.629 / 0.478
JT-V2W-COLL-SHUF 0.618 / 0.629 / 0.477
JT-V2D-SHUF 0.466 / 0.551 / 0.395
JT-V2D-COLL-SHUF 0.558 / 0.593 / 0.454
TYPE-A 0.664 / 0.701 / 0.577
TYPE-A-SHUF 0.515 / 0.567 / 0.422
TYPE 0.883 / 0.755 / 0.695
MAJ 0.977 / 0.988 / 0.885
tier 30000
JT-V1 0.842 / 0.829 / 0.637
JT-V1-COLL 0.841 / 0.828 / 0.628
JT-V2W 0.851 / 0.835 / 0.649
JT-V2W-COLL 0.849 / 0.833 / 0.629
JT-V2D 0.864 / 0.835 / 0.643
JT-V2D-COLL 0.869 / 0.837 / 0.630
JT-V1-SHUF 0.709 / 0.722 / 0.549
JT-V1-COLL-SHUF 0.711 / 0.722 / 0.547
JT-V2W-SHUF 0.738 / 0.739 / 0.558
JT-V2W-COLL-SHUF 0.741 / 0.738 / 0.552
JT-V2D-SHUF 0.614 / 0.671 / 0.490
JT-V2D-COLL-SHUF 0.687 / 0.709 / 0.529
TYPE-A 0.795 / 0.804 / 0.593
TYPE-A-SHUF 0.640 / 0.680 / 0.505
TYPE 0.901 / 0.771 / 0.642
MAJ 0.977 / 0.988 / 0.886
## Parse link F1 (mean)
3000: ATTN-V1 0.485 ATTN-V2W 0.489 ATTN-V2D 0.516 WORD-MI 0.158 GOLD-MI 0.699
10000: ATTN-V1 0.572 ATTN-V2W 0.577 ATTN-V2D 0.609 WORD-MI 0.272 GOLD-MI 0.734
30000: ATTN-V1 0.625 ATTN-V2W 0.631 ATTN-V2D 0.642 WORD-MI 0.378 GOLD-MI 0.734
## Continuous checkpoints, tier 30000 (mean V-measure)
V1 1000:0.517 2000:0.580 4000:0.653 8000:0.746 16000:0.803 30000:0.842
V2W 1000:0.501 2000:0.570 4000:0.655 8000:0.756 16000:0.815 30000:0.851
V2D 1000:0.453 2000:0.557 4000:0.693 8000:0.809 16000:0.844 30000:0.864
## Predictions (sense = sense accuracy on polysemous occurrences, at 30000)
K1 V2W: PASS — sense JT-V2W − JT-V2W-COLL +0.020 [+0.013, +0.027]
K2 V2W: FAIL — shuffled sense JT-V2W − COLL +0.006 [+0.002, +0.010]
K3 V2W: PASS — sense JT-V2W − JT-V1 +0.012 [+0.000, +0.024]
K4 V2W: FAIL — sense JT-V2W − MAJ -0.236 [-0.357, -0.116]
K5 V2W: PASS — V-measure JT-V2W − JT-V1 +0.008 [+0.006, +0.011]
K6 V2W: PASS — 60/60 audit checks PASS
K1 V2D: PASS — sense JT-V2D − JT-V2D-COLL +0.013 [+0.006, +0.021]
K2 V2D: PASS — shuffled sense JT-V2D − COLL -0.039 [-0.073, -0.005]
K3 V2D: FAIL — sense JT-V2D − JT-V1 +0.006 [-0.006, +0.017]
K4 V2D: FAIL — sense JT-V2D − MAJ -0.243 [-0.360, -0.125]
K5 V2D: PASS — V-measure JT-V2D − JT-V1 +0.021 [+0.011, +0.031]
K6 V2D: PASS — 60/60 audit checks PASS
## Scored, no prediction
V1 sense − V1-COLL: +0.010 [+0.001, +0.018]
V1 shuffled sense − COLL: +0.003 [-0.002, +0.008]
V2W sense − V2D: +0.006 [-0.012, +0.025]