R0: PASS — 10/10 seeds reproduce Test A exactly
C5-U: PASS — 10/10 seeds
## Link precision / recall / F1 (mean, held-out)
3000: G-FOREST 0.709/0.700/0.705 G-MST 0.719/0.719/0.719 G-MPG 0.549/0.854/0.668 L-FOREST 0.619/0.577/0.597 L-MST 0.626/0.626/0.626 L-MPG 0.527/0.786/0.631 R-FOREST 0.383/0.063/0.107 R-MST 0.382/0.382/0.382 R-MPG 0.387/0.480/0.428
10000: G-FOREST 0.735/0.732/0.734 G-MST 0.733/0.733/0.733 G-MPG 0.552/0.857/0.672 L-FOREST 0.691/0.673/0.681 L-MST 0.689/0.689/0.689 L-MPG 0.553/0.846/0.668 R-FOREST 0.388/0.149/0.214 R-MST 0.374/0.374/0.374 R-MPG 0.379/0.472/0.421
30000: G-FOREST 0.737/0.734/0.735 G-MST 0.735/0.735/0.735 G-MPG 0.555/0.858/0.674 L-FOREST 0.701/0.694/0.698 L-MST 0.700/0.700/0.700 L-MPG 0.560/0.865/0.680 R-FOREST 0.395/0.220/0.282 R-MST 0.385/0.385/0.385 R-MPG 0.389/0.481/0.430
## Means: types / instance recall / type recall / candidate precision
tier 3000
C-G 10570 1.000 0.979 0.117
M-G-comp 2072 0.306 0.297 0.245
M-G-uconn-FOREST 5843 0.697 0.685 0.124
M-G-ucomp-FOREST 2831 0.558 0.547 0.270
M-G-uconn-MST 6414 0.703 0.690 0.122
M-G-ucomp-MST 2904 0.571 0.558 0.280
M-G-uconn-MPG 9177 0.952 0.930 0.118
M-G-ucomp-MPG 919 0.070 0.064 0.170
M-L-uconn-FOREST 3675 0.577 0.558 0.110
M-L-ucomp-FOREST 2041 0.432 0.415 0.211
M-L-uconn-MST 5247 0.628 0.591 0.111
M-L-ucomp-MST 2559 0.452 0.423 0.229
M-L-uconn-MPG 7414 0.939 0.892 0.121
M-L-ucomp-MPG 871 0.094 0.080 0.202
M-R-uconn-FOREST 499 0.110 0.106 0.220
M-R-ucomp-FOREST 452 0.095 0.092 0.225
M-R-uconn-MST 15462 0.546 0.432 0.186
M-R-ucomp-MST 7088 0.242 0.179 0.230
M-R-uconn-MPG 20379 0.683 0.545 0.174
M-R-ucomp-MPG 4149 0.115 0.077 0.247
tier 10000
C-G 17434 1.000 0.995 0.113
M-G-comp 3614 0.336 0.333 0.258
M-G-uconn-FOREST 9701 0.728 0.724 0.124
M-G-ucomp-FOREST 4760 0.593 0.589 0.277
M-G-uconn-MST 10291 0.728 0.724 0.123
M-G-ucomp-MST 4868 0.588 0.584 0.275
M-G-uconn-MPG 14529 0.951 0.946 0.115
M-G-ucomp-MPG 1624 0.076 0.074 0.167
M-L-uconn-FOREST 7606 0.686 0.678 0.121
M-L-ucomp-FOREST 4278 0.521 0.513 0.239
M-L-uconn-MST 9845 0.705 0.690 0.120
M-L-ucomp-MST 5152 0.525 0.511 0.242
M-L-uconn-MPG 14142 0.967 0.950 0.117
M-L-ucomp-MPG 1813 0.067 0.063 0.154
M-R-uconn-FOREST 5808 0.269 0.256 0.229
M-R-ucomp-FOREST 4624 0.200 0.189 0.250
M-R-uconn-MST 42503 0.576 0.493 0.188
M-R-ucomp-MST 20259 0.245 0.200 0.223
M-R-uconn-MPG 56205 0.710 0.610 0.170
M-R-ucomp-MPG 12298 0.108 0.081 0.229
tier 30000
C-G 23859 1.000 0.999 0.112
M-G-comp 5187 0.343 0.343 0.258
M-G-uconn-FOREST 12920 0.732 0.732 0.123
M-G-ucomp-FOREST 6505 0.597 0.597 0.273
M-G-uconn-MST 13678 0.734 0.733 0.122
M-G-ucomp-MST 6697 0.594 0.594 0.271
M-G-uconn-MPG 19017 0.952 0.951 0.113
M-G-ucomp-MPG 2336 0.090 0.089 0.184
M-L-uconn-FOREST 12605 0.712 0.709 0.120
M-L-ucomp-FOREST 7058 0.547 0.543 0.247
M-L-uconn-MST 15363 0.718 0.713 0.120
M-L-ucomp-MST 8193 0.546 0.541 0.247
M-L-uconn-MPG 21731 0.983 0.977 0.117
M-L-ucomp-MPG 3030 0.079 0.077 0.173
M-R-uconn-FOREST 30930 0.343 0.326 0.191
M-R-ucomp-FOREST 20935 0.219 0.206 0.219
M-R-uconn-MST 104866 0.551 0.494 0.170
M-R-ucomp-MST 50760 0.241 0.209 0.206
M-R-uconn-MPG 134823 0.678 0.612 0.155
M-R-ucomp-MPG 32673 0.109 0.092 0.206
## Predictions (at 30000 unless stated)
U1: PASS — type recall M-G-ucomp-FOREST − M-G-comp +0.254 [+0.249, +0.260]
U2 G: FAIL — link F1 MST − FOREST -0.000 [-0.001, +0.000]
U2 L: PASS — link F1 MST − FOREST +0.002 [+0.002, +0.003]
U3: FAIL — M-G-ucomp-FOREST: type recall 0.597 x C-G (bar >= 0.7), types 0.273 x C-G (bar <= 0.10)
U4: PASS — type recall M-x-ucomp-MST − M-R-ucomp-MST 3000 G: +0.379 [+0.363]; 3000 L: +0.244 [+0.235]; 10000 G: +0.384 [+0.368]; 10000 L: +0.311 [+0.277]; 30000 G: +0.384 [+0.372]; 30000 L: +0.332 [+0.303]
U5: PASS — precision M-G-ucomp-MST − C-G 3000: +0.163 [+0.159, +0.167]; 10000: +0.162 [+0.160, +0.164]; 30000: +0.160 [+0.157, +0.163]