## Level-1 link F1 (mean)
tier W G S1 S2 BACKOFF
3000 0.156 0.705 0.668 0.516 0.711
10000 0.272 0.734 0.674 0.510 0.716
30000 0.380 0.735 0.672 0.511 0.714
## Level-2 type recall at budget (mean)
tier 3000: budget 300 1000 3000 10000 all
C-G 0.722 0.857 0.942 0.977 0.979 (list 10570)
HARD 0.673 0.763 0.795 0.795 0.795 (list 2072)
GRADED 0.751 0.849 0.884 0.978 0.979 (list 10570)
GRADED-G 0.691 0.782 0.817 0.978 0.979 (list 10570)
GRADED-R 0.719 0.822 0.927 0.977 0.979 (list 10570)
tier 10000: budget 300 1000 3000 10000 all
C-G 0.717 0.853 0.947 0.984 0.995 (list 17434)
HARD 0.711 0.827 0.878 0.881 0.881 (list 3614)
GRADED 0.750 0.876 0.910 0.929 0.995 (list 17434)
GRADED-G 0.743 0.850 0.879 0.977 0.995 (list 17434)
GRADED-R 0.717 0.871 0.946 0.983 0.995 (list 17434)
tier 30000: budget 300 1000 3000 10000 all
C-G 0.706 0.848 0.950 0.986 0.999 (list 23859)
HARD 0.704 0.834 0.883 0.899 0.899 (list 5187)
GRADED 0.734 0.870 0.922 0.948 0.999 (list 23859)
GRADED-G 0.744 0.850 0.890 0.910 0.999 (list 23859)
GRADED-R 0.706 0.857 0.955 0.987 0.999 (list 23859)
## Predictions
F1: FAIL — 3000: BACKOFF-G +0.006[-0.003]; 10000: BACKOFF-G -0.018[-0.021]; 30000: BACKOFF-G -0.021
F2: PASS — 3000:0.979 10000:0.995 30000:0.999 | HARD 3000:0.795 10000:0.881 30000:0.899
F3a: FAIL — 3000: GRADED-C-G@1K -0.008[-0.019]; 10000: GRADED-C-G@1K +0.023[+0.019]; 30000: GRADED-C-G@1K +0.022[+0.017]
F3b (GRADED >= HARD at B = |HARD list|): see note — compared at the largest common budget below each HARD list size:
3000: |HARD| 2072, budget 1000: GRADED 0.849 vs HARD 0.763
10000: |HARD| 3614, budget 3000: GRADED 0.910 vs HARD 0.878
30000: |HARD| 5187, budget 3000: GRADED 0.922 vs HARD 0.883
F4: PASS — GRADED-GRADED-G@1K 3000: +0.067[+0.052]; 10000: +0.025[+0.019]
F5: FAIL — GRADED-R-C-G@1K 3000: -0.035[-0.069]; 10000: +0.018[+0.015]; 30000: +0.009[+0.007]