5d99c5c · atomspace-layer-discovery

Changed files (15) +1043 -0

experiments/fractal/RESULTS.md +32 -0

…	…	@@ -0,0 +1,32 @@
	1	# Graded connectors — results
	2
	3	Run `results/1792353740` (harness `8dce47f`, 10 seeds, tiers 3K/10K/30K). Scored against PREREG.md (`53f1644`, committed before
	4	the code) by `summarize.py`; full output in `results/1792353740/summary.txt`.
	5
	6	\| \| prediction \| verdict \|
	7	\|---\|---\|---\|
	8	\| F1 \| finest-level-first backoff parses better than any single level \| FAIL: tied with the class level at 3K (+0.006), worse at 10K/30K (−0.018, −0.021) \|
	9	\| F2 \| never deleting keeps ≥ 95% of true phrase types \| PASS (0.979–0.999; HARD 0.80–0.90) — true by construction of "never delete" \|
	10	\| F3 \| graded order beats frequency order at 1K candidates \| FAIL at 3K (−0.008); passes at 10K/30K (+0.023, +0.022). Graded beats HARD at comparable budgets (0.85 vs 0.76, 0.91 vs 0.88, 0.92 vs 0.88) \|
	11	\| F4 \| the multi-scale parse beats the class-only parse for ranking \| PASS (+0.067, +0.025) \|
	12	\| F5 \| a random hierarchy gives no gain over frequency order \| FAIL: +0.018 and +0.009 at 10K/30K \|
	13
	14	## What it says
	15
	16	1. "Finest level wins" is the wrong way to combine levels. Word-level evidence that just passes the gate
	17	overrides stronger class-level evidence, so the backoff parse is no better (and later worse) than the
	18	class-level parse. Combining levels needs a rule based on evidence strength, not on fineness.
	19	2. Never deleting does keep the answers (trivially), and graded ranking finds more true phrases per
	20	candidate than the hard rule — but only ~2 points more than plain frequency order at 1K, from 10K
	21	sentences up.
	22	3. Much of that gain is not the hierarchy. A random hierarchy (true words, random classes) recovers about
	23	half of it, so the word level plus "is this span an assembly" carries much of the signal; the true class
	24	hierarchy adds little on top.
	25	4. Tiering by structure is not selective for phrases. Complete assemblies include verb subtrees and
	26	clauses (not in the noun/prepositional-phrase gold), which push true phrases back at middle budgets
	27	(3K-candidate recall 0.88–0.92 vs frequency's 0.94–0.95).
	28
	29	## Not supported
	30
	31	Graded connectors as implemented here (finest-first backoff, three-tier ranking) do not show that
	32	multi-scale connectors make the search substantially better. The combination rule is the likely weak point.

experiments/fractal/results/1792353740/commit.txt +1 -0

…	…	@@ -0,0 +1 @@
	1	8dce47fea17903e2a2974bc6d90736dfdea86dc3

experiments/fractal/results/1792353740/seed-1.tsv +90 -0

…	…	@@ -0,0 +1,90 @@
	1	P 1 3000 W 0.14961 0.48373 0.08849
	2	P 1 3000 G 0.70421 0.70986 0.69865
	3	P 1 3000 S1 0.67100 0.67499 0.66706
	4	P 1 3000 S2 0.51381 0.51381 0.51381
	5	P 1 3000 BACKOFF 0.71302 0.71302 0.71302
	6	R 1 3000 C-G 300 0.71503 10644
	7	R 1 3000 C-G 1000 0.86632 10644
	8	R 1 3000 C-G 3000 0.94756 10644
	9	R 1 3000 C-G 10000 0.98031 10644
	10	R 1 3000 C-G -1 0.98093 10644
	11	R 1 3000 HARD 300 0.66839 2112
	12	R 1 3000 HARD 1000 0.76290 2112
	13	R 1 3000 HARD 3000 0.80249 2112
	14	R 1 3000 HARD 10000 0.80249 2112
	15	R 1 3000 HARD -1 0.80249 2112
	16	R 1 3000 GRADED 300 0.75212 10644
	17	R 1 3000 GRADED 1000 0.86363 10644
	18	R 1 3000 GRADED 3000 0.89409 10644
	19	R 1 3000 GRADED 10000 0.98093 10644
	20	R 1 3000 GRADED -1 0.98093 10644
	21	R 1 3000 GRADED-G 300 0.69720 10644
	22	R 1 3000 GRADED-G 1000 0.78922 10644
	23	R 1 3000 GRADED-G 3000 0.81741 10644
	24	R 1 3000 GRADED-G 10000 0.98093 10644
	25	R 1 3000 GRADED-G -1 0.98093 10644
	26	R 1 3000 GRADED-R 300 0.71275 10644
	27	R 1 3000 GRADED-R 1000 0.83461 10644
	28	R 1 3000 GRADED-R 3000 0.91689 10644
	29	R 1 3000 GRADED-R 10000 0.98073 10644
	30	R 1 3000 GRADED-R -1 0.98093 10644
	31	P 1 10000 W 0.26366 0.50178 0.17881
	32	P 1 10000 G 0.72903 0.73036 0.72770
	33	P 1 10000 S1 0.67176 0.67394 0.66960
	34	P 1 10000 S2 0.51310 0.51310 0.51310
	35	P 1 10000 BACKOFF 0.71111 0.71111 0.71111
	36	R 1 10000 C-G 300 0.70466 17415
	37	R 1 10000 C-G 1000 0.85886 17415
	38	R 1 10000 C-G 3000 0.95275 17415
	39	R 1 10000 C-G 10000 0.98694 17415
	40	R 1 10000 C-G -1 0.99565 17415
	41	R 1 10000 HARD 300 0.71026 3625
	42	R 1 10000 HARD 1000 0.83088 3625
	43	R 1 10000 HARD 3000 0.88435 3625
	44	R 1 10000 HARD 10000 0.88705 3625
	45	R 1 10000 HARD -1 0.88705 3625
	46	R 1 10000 GRADED 300 0.74114 17415
	47	R 1 10000 GRADED 1000 0.87648 17415
	48	R 1 10000 GRADED 3000 0.91026 17415
	49	R 1 10000 GRADED 10000 0.93057 17415
	50	R 1 10000 GRADED -1 0.99565 17415
	51	R 1 10000 GRADED-G 300 0.74860 17415
	52	R 1 10000 GRADED-G 1000 0.85451 17415
	53	R 1 10000 GRADED-G 3000 0.88352 17415
	54	R 1 10000 GRADED-G 10000 0.98321 17415
	55	R 1 10000 GRADED-G -1 0.99565 17415
	56	R 1 10000 GRADED-R 300 0.70466 17415
	57	R 1 10000 GRADED-R 1000 0.87834 17415
	58	R 1 10000 GRADED-R 3000 0.95109 17415
	59	R 1 10000 GRADED-R 10000 0.98674 17415
	60	R 1 10000 GRADED-R -1 0.99565 17415
	61	P 1 30000 W 0.37422 0.52270 0.29143
	62	P 1 30000 G 0.73022 0.73156 0.72889
	63	P 1 30000 S1 0.66651 0.66866 0.66437
	64	P 1 30000 S2 0.51349 0.51349 0.51349
	65	P 1 30000 BACKOFF 0.71056 0.71056 0.71056
	66	R 1 30000 C-G 300 0.69347 23862
	67	R 1 30000 C-G 1000 0.85720 23862
	68	R 1 30000 C-G 3000 0.95668 23862
	69	R 1 30000 C-G 10000 0.98736 23862
	70	R 1 30000 C-G -1 0.99959 23862
	71	R 1 30000 HARD 300 0.70611 5254
	72	R 1 30000 HARD 1000 0.84207 5254
	73	R 1 30000 HARD 3000 0.88870 5254
	74	R 1 30000 HARD 10000 0.90010 5254
	75	R 1 30000 HARD -1 0.90010 5254
	76	R 1 30000 GRADED 300 0.74176 23862
	77	R 1 30000 GRADED 1000 0.88021 23862
	78	R 1 30000 GRADED 3000 0.92974 23862
	79	R 1 30000 GRADED 10000 0.95689 23862
	80	R 1 30000 GRADED -1 0.99959 23862
	81	R 1 30000 GRADED-G 300 0.74860 23862
	82	R 1 30000 GRADED-G 1000 0.85098 23862
	83	R 1 30000 GRADED-G 3000 0.89202 23862
	84	R 1 30000 GRADED-G 10000 0.90943 23862
	85	R 1 30000 GRADED-G -1 0.99959 23862
	86	R 1 30000 GRADED-R 300 0.69347 23862
	87	R 1 30000 GRADED-R 1000 0.86301 23862
	88	R 1 30000 GRADED-R 3000 0.95959 23862
	89	R 1 30000 GRADED-R 10000 0.98819 23862
	90	R 1 30000 GRADED-R -1 0.99959 23862

experiments/fractal/results/1792353740/seed-10.tsv +90 -0

…	…	@@ -0,0 +1,90 @@
	1	P 10 3000 W 0.16146 0.49347 0.09652
	2	P 10 3000 G 0.70595 0.70802 0.70389
	3	P 10 3000 S1 0.65674 0.65914 0.65436
	4	P 10 3000 S2 0.51324 0.51324 0.51324
	5	P 10 3000 BACKOFF 0.69791 0.69791 0.69791
	6	R 10 3000 C-G 300 0.72255 10496
	7	R 10 3000 C-G 1000 0.85212 10496
	8	R 10 3000 C-G 3000 0.94676 10496
	9	R 10 3000 C-G 10000 0.97816 10496
	10	R 10 3000 C-G -1 0.98003 10496
	11	R 10 3000 HARD 300 0.64809 2055
	12	R 10 3000 HARD 1000 0.74418 2055
	13	R 10 3000 HARD 3000 0.77641 2055
	14	R 10 3000 HARD 10000 0.77641 2055
	15	R 10 3000 HARD -1 0.77641 2055
	16	R 10 3000 GRADED 300 0.77059 10496
	17	R 10 3000 GRADED 1000 0.85982 10496
	18	R 10 3000 GRADED 3000 0.89351 10496
	19	R 10 3000 GRADED 10000 0.97920 10496
	20	R 10 3000 GRADED -1 0.98003 10496
	21	R 10 3000 GRADED-G 300 0.67325 10496
	22	R 10 3000 GRADED-G 1000 0.76498 10496
	23	R 10 3000 GRADED-G 3000 0.80553 10496
	24	R 10 3000 GRADED-G 10000 0.97941 10496
	25	R 10 3000 GRADED-G -1 0.98003 10496
	26	R 10 3000 GRADED-R 300 0.72504 10496
	27	R 10 3000 GRADED-R 1000 0.87521 10496
	28	R 10 3000 GRADED-R 3000 0.94301 10496
	29	R 10 3000 GRADED-R 10000 0.97879 10496
	30	R 10 3000 GRADED-R -1 0.98003 10496
	31	P 10 10000 W 0.27641 0.52287 0.18786
	32	P 10 10000 G 0.72493 0.72626 0.72360
	33	P 10 10000 S1 0.66011 0.66180 0.65842
	34	P 10 10000 S2 0.50933 0.50933 0.50933
	35	P 10 10000 BACKOFF 0.69935 0.69935 0.69935
	36	R 10 10000 C-G 300 0.71880 17287
	37	R 10 10000 C-G 1000 0.85129 17287
	38	R 10 10000 C-G 3000 0.95216 17287
	39	R 10 10000 C-G 10000 0.98357 17287
	40	R 10 10000 C-G -1 0.99418 17287
	41	R 10 10000 HARD 300 0.70778 3654
	42	R 10 10000 HARD 1000 0.83195 3654
	43	R 10 10000 HARD 3000 0.88644 3654
	44	R 10 10000 HARD 10000 0.88894 3654
	45	R 10 10000 HARD -1 0.88894 3654
	46	R 10 10000 GRADED 300 0.75395 17287
	47	R 10 10000 GRADED 1000 0.88540 17287
	48	R 10 10000 GRADED 3000 0.91244 17287
	49	R 10 10000 GRADED 10000 0.92970 17287
	50	R 10 10000 GRADED -1 0.99418 17287
	51	R 10 10000 GRADED-G 300 0.74938 17287
	52	R 10 10000 GRADED-G 1000 0.86148 17287
	53	R 10 10000 GRADED-G 3000 0.88686 17287
	54	R 10 10000 GRADED-G 10000 0.97920 17287
	55	R 10 10000 GRADED-G -1 0.99418 17287
	56	R 10 10000 GRADED-R 300 0.71880 17287
	57	R 10 10000 GRADED-R 1000 0.86730 17287
	58	R 10 10000 GRADED-R 3000 0.94842 17287
	59	R 10 10000 GRADED-R 10000 0.98295 17287
	60	R 10 10000 GRADED-R -1 0.99418 17287
	61	P 10 30000 W 0.37988 0.53180 0.29547
	62	P 10 30000 G 0.72892 0.73026 0.72758
	63	P 10 30000 S1 0.65835 0.66004 0.65667
	64	P 10 30000 S2 0.50120 0.50120 0.50120
	65	P 10 30000 BACKOFF 0.70740 0.70740 0.70740
	66	R 10 30000 C-G 300 0.70611 23686
	67	R 10 30000 C-G 1000 0.84838 23686
	68	R 10 30000 C-G 3000 0.95300 23686
	69	R 10 30000 C-G 10000 0.98565 23686
	70	R 10 30000 C-G -1 0.99834 23686
	71	R 10 30000 HARD 300 0.71007 5184
	72	R 10 30000 HARD 1000 0.82862 5184
	73	R 10 30000 HARD 3000 0.88769 5184
	74	R 10 30000 HARD 10000 0.90391 5184
	75	R 10 30000 HARD -1 0.90391 5184
	76	R 10 30000 GRADED 300 0.72171 23686
	77	R 10 30000 GRADED 1000 0.87084 23686
	78	R 10 30000 GRADED 3000 0.91993 23686
	79	R 10 30000 GRADED 10000 0.93989 23686
	80	R 10 30000 GRADED -1 0.99834 23686
	81	R 10 30000 GRADED-G 300 0.74792 23686
	82	R 10 30000 GRADED-G 1000 0.85691 23686
	83	R 10 30000 GRADED-G 3000 0.89580 23686
	84	R 10 30000 GRADED-G 10000 0.91223 23686
	85	R 10 30000 GRADED-G -1 0.99834 23686
	86	R 10 30000 GRADED-R 300 0.70611 23686
	87	R 10 30000 GRADED-R 1000 0.85379 23686
	88	R 10 30000 GRADED-R 3000 0.95695 23686
	89	R 10 30000 GRADED-R 10000 0.98648 23686
	90	R 10 30000 GRADED-R -1 0.99834 23686

experiments/fractal/results/1792353740/seed-2.tsv +90 -0

…	…	@@ -0,0 +1,90 @@
	1	P 2 3000 W 0.16166 0.49609 0.09656
	2	P 2 3000 G 0.69330 0.69731 0.68933
	3	P 2 3000 S1 0.65694 0.65933 0.65458
	4	P 2 3000 S2 0.51317 0.51317 0.51317
	5	P 2 3000 BACKOFF 0.69349 0.69349 0.69349
	6	R 2 3000 C-G 300 0.72705 10659
	7	R 2 3000 C-G 1000 0.83943 10659
	8	R 2 3000 C-G 3000 0.93685 10659
	9	R 2 3000 C-G 10000 0.97424 10659
	10	R 2 3000 C-G -1 0.97653 10659
	11	R 2 3000 HARD 300 0.67075 2107
	12	R 2 3000 HARD 1000 0.77815 2107
	13	R 2 3000 HARD 3000 0.80910 2107
	14	R 2 3000 HARD 10000 0.80910 2107
	15	R 2 3000 HARD -1 0.80910 2107
	16	R 2 3000 GRADED 300 0.74948 10659
	17	R 2 3000 GRADED 1000 0.84483 10659
	18	R 2 3000 GRADED 3000 0.88388 10659
	19	R 2 3000 GRADED 10000 0.97570 10659
	20	R 2 3000 GRADED -1 0.97653 10659
	21	R 2 3000 GRADED-G 300 0.69443 10659
	22	R 2 3000 GRADED-G 1000 0.79290 10659
	23	R 2 3000 GRADED-G 3000 0.81678 10659
	24	R 2 3000 GRADED-G 10000 0.97570 10659
	25	R 2 3000 GRADED-G -1 0.97653 10659
	26	R 2 3000 GRADED-R 300 0.71645 10659
	27	R 2 3000 GRADED-R 1000 0.79767 10659
	28	R 2 3000 GRADED-R 3000 0.90008 10659
	29	R 2 3000 GRADED-R 10000 0.97486 10659
	30	R 2 3000 GRADED-R -1 0.97653 10659
	31	P 2 10000 W 0.26885 0.50609 0.18304
	32	P 2 10000 G 0.72498 0.72629 0.72368
	33	P 2 10000 S1 0.66602 0.66763 0.66442
	34	P 2 10000 S2 0.50212 0.50212 0.50212
	35	P 2 10000 BACKOFF 0.70734 0.70734 0.70734
	36	R 2 10000 C-G 300 0.71624 17440
	37	R 2 10000 C-G 1000 0.83797 17440
	38	R 2 10000 C-G 3000 0.94142 17440
	39	R 2 10000 C-G 10000 0.98193 17440
	40	R 2 10000 C-G -1 0.99460 17440
	41	R 2 10000 HARD 300 0.69942 3593
	42	R 2 10000 HARD 1000 0.81803 3593
	43	R 2 10000 HARD 3000 0.87703 3593
	44	R 2 10000 HARD 10000 0.88056 3593
	45	R 2 10000 HARD -1 0.88056 3593
	46	R 2 10000 GRADED 300 0.74179 17440
	47	R 2 10000 GRADED 1000 0.85833 17440
	48	R 2 10000 GRADED 3000 0.90133 17440
	49	R 2 10000 GRADED 10000 0.92397 17440
	50	R 2 10000 GRADED -1 0.99460 17440
	51	R 2 10000 GRADED-G 300 0.73099 17440
	52	R 2 10000 GRADED-G 1000 0.84171 17440
	53	R 2 10000 GRADED-G 3000 0.87661 17440
	54	R 2 10000 GRADED-G 10000 0.97424 17440
	55	R 2 10000 GRADED-G -1 0.99460 17440
	56	R 2 10000 GRADED-R 300 0.71624 17440
	57	R 2 10000 GRADED-R 1000 0.86020 17440
	58	R 2 10000 GRADED-R 3000 0.93041 17440
	59	R 2 10000 GRADED-R 10000 0.98400 17440
	60	R 2 10000 GRADED-R -1 0.99460 17440
	61	P 2 30000 W 0.38086 0.53077 0.29698
	62	P 2 30000 G 0.73453 0.73586 0.73321
	63	P 2 30000 S1 0.67261 0.67423 0.67099
	64	P 2 30000 S2 0.50420 0.50420 0.50420
	65	P 2 30000 BACKOFF 0.71247 0.71247 0.71247
	66	R 2 30000 C-G 300 0.70129 23976
	67	R 2 30000 C-G 1000 0.83278 23976
	68	R 2 30000 C-G 3000 0.94246 23976
	69	R 2 30000 C-G 10000 0.98400 23976
	70	R 2 30000 C-G -1 0.99958 23976
	71	R 2 30000 HARD 300 0.67865 5229
	72	R 2 30000 HARD 1000 0.82219 5229
	73	R 2 30000 HARD 3000 0.87391 5229
	74	R 2 30000 HARD 10000 0.89676 5229
	75	R 2 30000 HARD -1 0.89676 5229
	76	R 2 30000 GRADED 300 0.73432 23976
	77	R 2 30000 GRADED 1000 0.86394 23976
	78	R 2 30000 GRADED 3000 0.92169 23976
	79	R 2 30000 GRADED 10000 0.95555 23976
	80	R 2 30000 GRADED -1 0.99958 23976
	81	R 2 30000 GRADED-G 300 0.72518 23976
	82	R 2 30000 GRADED-G 1000 0.83673 23976
	83	R 2 30000 GRADED-G 3000 0.88658 23976
	84	R 2 30000 GRADED-G 10000 0.90798 23976
	85	R 2 30000 GRADED-G -1 0.99958 23976
	86	R 2 30000 GRADED-R 300 0.70129 23976
	87	R 2 30000 GRADED-R 1000 0.83901 23976
	88	R 2 30000 GRADED-R 3000 0.94807 23976
	89	R 2 30000 GRADED-R 10000 0.98567 23976
	90	R 2 30000 GRADED-R -1 0.99958 23976

experiments/fractal/results/1792353740/seed-3.tsv +90 -0

…	…	@@ -0,0 +1,90 @@
	1	P 3 3000 W 0.15552 0.49413 0.09228
	2	P 3 3000 G 0.70371 0.70581 0.70162
	3	P 3 3000 S1 0.66509 0.66691 0.66328
	4	P 3 3000 S2 0.51828 0.51828 0.51828
	5	P 3 3000 BACKOFF 0.69082 0.69082 0.69082
	6	R 3 3000 C-G 300 0.72159 10526
	7	R 3 3000 C-G 1000 0.87617 10526
	8	R 3 3000 C-G 3000 0.94423 10526
	9	R 3 3000 C-G 10000 0.97774 10526
	10	R 3 3000 C-G -1 0.97943 10526
	11	R 3 3000 HARD 300 0.67154 2062
	12	R 3 3000 HARD 1000 0.75657 2062
	13	R 3 3000 HARD 3000 0.78520 2062
	14	R 3 3000 HARD 10000 0.78520 2062
	15	R 3 3000 HARD -1 0.78520 2062
	16	R 3 3000 GRADED 300 0.75212 10526
	17	R 3 3000 GRADED 1000 0.84436 10526
	18	R 3 3000 GRADED 3000 0.88232 10526
	19	R 3 3000 GRADED 10000 0.97901 10526
	20	R 3 3000 GRADED -1 0.97943 10526
	21	R 3 3000 GRADED-G 300 0.69084 10526
	22	R 3 3000 GRADED-G 1000 0.77248 10526
	23	R 3 3000 GRADED-G 3000 0.81170 10526
	24	R 3 3000 GRADED-G 10000 0.97901 10526
	25	R 3 3000 GRADED-G -1 0.97943 10526
	26	R 3 3000 GRADED-R 300 0.69254 10526
	27	R 3 3000 GRADED-R 1000 0.75997 10526
	28	R 3 3000 GRADED-R 3000 0.94105 10526
	29	R 3 3000 GRADED-R 10000 0.97795 10526
	30	R 3 3000 GRADED-R -1 0.97943 10526
	31	P 3 10000 W 0.27664 0.52233 0.18814
	32	P 3 10000 G 0.74009 0.74136 0.73883
	33	P 3 10000 S1 0.67771 0.67923 0.67620
	34	P 3 10000 S2 0.51072 0.51072 0.51072
	35	P 3 10000 BACKOFF 0.72226 0.72226 0.72226
	36	R 3 10000 C-G 300 0.71459 17331
	37	R 3 10000 C-G 1000 0.86302 17331
	38	R 3 10000 C-G 3000 0.95271 17331
	39	R 3 10000 C-G 10000 0.98304 17331
	40	R 3 10000 C-G -1 0.99406 17331
	41	R 3 10000 HARD 300 0.72180 3536
	42	R 3 10000 HARD 1000 0.82570 3536
	43	R 3 10000 HARD 3000 0.87468 3536
	44	R 3 10000 HARD 10000 0.87617 3536
	45	R 3 10000 HARD -1 0.87617 3536
	46	R 3 10000 GRADED 300 0.76293 17331
	47	R 3 10000 GRADED 1000 0.88274 17331
	48	R 3 10000 GRADED 3000 0.91327 17331
	49	R 3 10000 GRADED 10000 0.93003 17331
	50	R 3 10000 GRADED -1 0.99406 17331
	51	R 3 10000 GRADED-G 300 0.75254 17331
	52	R 3 10000 GRADED-G 1000 0.84839 17331
	53	R 3 10000 GRADED-G 3000 0.87489 17331
	54	R 3 10000 GRADED-G 10000 0.98007 17331
	55	R 3 10000 GRADED-G -1 0.99406 17331
	56	R 3 10000 GRADED-R 300 0.71459 17331
	57	R 3 10000 GRADED-R 1000 0.87913 17331
	58	R 3 10000 GRADED-R 3000 0.95441 17331
	59	R 3 10000 GRADED-R 10000 0.98092 17331
	60	R 3 10000 GRADED-R -1 0.99406 17331
	61	P 3 30000 W 0.37845 0.52967 0.29439
	62	P 3 30000 G 0.72961 0.73071 0.72851
	63	P 3 30000 S1 0.67682 0.67834 0.67530
	64	P 3 30000 S2 0.50016 0.50016 0.50016
	65	P 3 30000 BACKOFF 0.70991 0.70991 0.70991
	66	R 3 30000 C-G 300 0.70950 23826
	67	R 3 30000 C-G 1000 0.86069 23826
	68	R 3 30000 C-G 3000 0.95441 23826
	69	R 3 30000 C-G 10000 0.98452 23826
	70	R 3 30000 C-G -1 1.00000 23826
	71	R 3 30000 HARD 300 0.72053 5199
	72	R 3 30000 HARD 1000 0.84097 5199
	73	R 3 30000 HARD 3000 0.88486 5199
	74	R 3 30000 HARD 10000 0.89864 5199
	75	R 3 30000 HARD -1 0.89864 5199
	76	R 3 30000 GRADED 300 0.75000 23826
	77	R 3 30000 GRADED 1000 0.88910 23826
	78	R 3 30000 GRADED 3000 0.93321 23826
	79	R 3 30000 GRADED 10000 0.95632 23826
	80	R 3 30000 GRADED -1 1.00000 23826
	81	R 3 30000 GRADED-G 300 0.75615 23826
	82	R 3 30000 GRADED-G 1000 0.86196 23826
	83	R 3 30000 GRADED-G 3000 0.88974 23826
	84	R 3 30000 GRADED-G 10000 0.90691 23826
	85	R 3 30000 GRADED-G -1 1.00000 23826
	86	R 3 30000 GRADED-R 300 0.70950 23826
	87	R 3 30000 GRADED-R 1000 0.87065 23826
	88	R 3 30000 GRADED-R 3000 0.95738 23826
	89	R 3 30000 GRADED-R 10000 0.98813 23826
	90	R 3 30000 GRADED-R -1 1.00000 23826

experiments/fractal/results/1792353740/seed-4.tsv +90 -0

…	…	@@ -0,0 +1,90 @@
	1	P 4 3000 W 0.16105 0.50579 0.09578
	2	P 4 3000 G 0.71224 0.71782 0.70674
	3	P 4 3000 S1 0.66974 0.67223 0.66726
	4	P 4 3000 S2 0.50699 0.50699 0.50699
	5	P 4 3000 BACKOFF 0.72583 0.72583 0.72583
	6	R 4 3000 C-G 300 0.70462 10459
	7	R 4 3000 C-G 1000 0.84966 10459
	8	R 4 3000 C-G 3000 0.94105 10459
	9	R 4 3000 C-G 10000 0.97837 10459
	10	R 4 3000 C-G -1 0.98070 10459
	11	R 4 3000 HARD 300 0.69084 2115
	12	R 4 3000 HARD 1000 0.78456 2115
	13	R 4 3000 HARD 3000 0.80980 2115
	14	R 4 3000 HARD 10000 0.80980 2115
	15	R 4 3000 HARD -1 0.80980 2115
	16	R 4 3000 GRADED 300 0.73155 10459
	17	R 4 3000 GRADED 1000 0.86217 10459
	18	R 4 3000 GRADED 3000 0.89546 10459
	19	R 4 3000 GRADED 10000 0.97964 10459
	20	R 4 3000 GRADED -1 0.98070 10459
	21	R 4 3000 GRADED-G 300 0.69720 10459
	22	R 4 3000 GRADED-G 1000 0.79729 10459
	23	R 4 3000 GRADED-G 3000 0.82252 10459
	24	R 4 3000 GRADED-G 10000 0.98028 10459
	25	R 4 3000 GRADED-G -1 0.98070 10459
	26	R 4 3000 GRADED-R 300 0.66200 10459
	27	R 4 3000 GRADED-R 1000 0.72752 10459
	28	R 4 3000 GRADED-R 3000 0.93660 10459
	29	R 4 3000 GRADED-R 10000 0.97837 10459
	30	R 4 3000 GRADED-R -1 0.98070 10459
	31	P 4 10000 W 0.27889 0.51613 0.19106
	32	P 4 10000 G 0.73639 0.73834 0.73444
	33	P 4 10000 S1 0.68711 0.68885 0.68538
	34	P 4 10000 S2 0.50203 0.50203 0.50203
	35	P 4 10000 BACKOFF 0.72681 0.72681 0.72681
	36	R 4 10000 C-G 300 0.70314 17450
	37	R 4 10000 C-G 1000 0.85560 17450
	38	R 4 10000 C-G 3000 0.94254 17450
	39	R 4 10000 C-G 10000 0.98537 17450
	40	R 4 10000 C-G -1 0.99364 17450
	41	R 4 10000 HARD 300 0.70759 3637
	42	R 4 10000 HARD 1000 0.82909 3637
	43	R 4 10000 HARD 3000 0.87680 3637
	44	R 4 10000 HARD 10000 0.87977 3637
	45	R 4 10000 HARD -1 0.87977 3637
	46	R 4 10000 GRADED 300 0.74534 17450
	47	R 4 10000 GRADED 1000 0.87383 17450
	48	R 4 10000 GRADED 3000 0.90946 17450
	49	R 4 10000 GRADED 10000 0.93003 17450
	50	R 4 10000 GRADED -1 0.99364 17450
	51	R 4 10000 GRADED-G 300 0.73770 17450
	52	R 4 10000 GRADED-G 1000 0.85008 17450
	53	R 4 10000 GRADED-G 3000 0.87786 17450
	54	R 4 10000 GRADED-G 10000 0.97243 17450
	55	R 4 10000 GRADED-G -1 0.99364 17450
	56	R 4 10000 GRADED-R 300 0.70314 17450
	57	R 4 10000 GRADED-R 1000 0.86853 17450
	58	R 4 10000 GRADED-R 3000 0.93851 17450
	59	R 4 10000 GRADED-R 10000 0.98219 17450
	60	R 4 10000 GRADED-R -1 0.99364 17450
	61	P 4 30000 W 0.37526 0.52800 0.29106
	62	P 4 30000 G 0.73533 0.73647 0.73420
	63	P 4 30000 S1 0.67372 0.67540 0.67206
	64	P 4 30000 S2 0.51145 0.51145 0.51145
	65	P 4 30000 BACKOFF 0.71552 0.71552 0.71552
	66	R 4 30000 C-G 300 0.69847 23798
	67	R 4 30000 C-G 1000 0.84500 23798
	68	R 4 30000 C-G 3000 0.94572 23798
	69	R 4 30000 C-G 10000 0.98579 23798
	70	R 4 30000 C-G -1 0.99915 23798
	71	R 4 30000 HARD 300 0.68109 5191
	72	R 4 30000 HARD 1000 0.82909 5191
	73	R 4 30000 HARD 3000 0.87553 5191
	74	R 4 30000 HARD 10000 0.89483 5191
	75	R 4 30000 HARD -1 0.89483 5191
	76	R 4 30000 GRADED 300 0.73049 23798
	77	R 4 30000 GRADED 1000 0.86260 23798
	78	R 4 30000 GRADED 3000 0.92006 23798
	79	R 4 30000 GRADED 10000 0.95802 23798
	80	R 4 30000 GRADED -1 0.99915 23798
	81	R 4 30000 GRADED-G 300 0.73770 23798
	82	R 4 30000 GRADED-G 1000 0.84712 23798
	83	R 4 30000 GRADED-G 3000 0.88740 23798
	84	R 4 30000 GRADED-G 10000 0.91137 23798
	85	R 4 30000 GRADED-G -1 0.99915 23798
	86	R 4 30000 GRADED-R 300 0.69847 23798
	87	R 4 30000 GRADED-R 1000 0.85517 23798
	88	R 4 30000 GRADED-R 3000 0.95059 23798
	89	R 4 30000 GRADED-R 10000 0.98410 23798
	90	R 4 30000 GRADED-R -1 0.99915 23798

experiments/fractal/results/1792353740/seed-5.tsv +90 -0

…	…	@@ -0,0 +1,90 @@
	1	P 5 3000 W 0.16322 0.53423 0.09632
	2	P 5 3000 G 0.67959 0.68494 0.67433
	3	P 5 3000 S1 0.65658 0.66207 0.65118
	4	P 5 3000 S2 0.51186 0.51186 0.51186
	5	P 5 3000 BACKOFF 0.70152 0.70152 0.70152
	6	R 5 3000 C-G 300 0.73513 10378
	7	R 5 3000 C-G 1000 0.85600 10378
	8	R 5 3000 C-G 3000 0.93967 10378
	9	R 5 3000 C-G 10000 0.97540 10378
	10	R 5 3000 C-G -1 0.97730 10378
	11	R 5 3000 HARD 300 0.64011 2061
	12	R 5 3000 HARD 1000 0.72251 2061
	13	R 5 3000 HARD 3000 0.75321 2061
	14	R 5 3000 HARD 10000 0.75321 2061
	15	R 5 3000 HARD -1 0.75321 2061
	16	R 5 3000 GRADED 300 0.73050 10378
	17	R 5 3000 GRADED 1000 0.82405 10378
	18	R 5 3000 GRADED 3000 0.86315 10378
	19	R 5 3000 GRADED 10000 0.97604 10378
	20	R 5 3000 GRADED -1 0.97730 10378
	21	R 5 3000 GRADED-G 300 0.65777 10378
	22	R 5 3000 GRADED-G 1000 0.74101 10378
	23	R 5 3000 GRADED-G 3000 0.80912 10378
	24	R 5 3000 GRADED-G 10000 0.97625 10378
	25	R 5 3000 GRADED-G -1 0.97730 10378
	26	R 5 3000 GRADED-R 300 0.73513 10378
	27	R 5 3000 GRADED-R 1000 0.84864 10378
	28	R 5 3000 GRADED-R 3000 0.92180 10378
	29	R 5 3000 GRADED-R 10000 0.97583 10378
	30	R 5 3000 GRADED-R -1 0.97730 10378
	31	P 5 10000 W 0.27297 0.51935 0.18514
	32	P 5 10000 G 0.73756 0.73908 0.73604
	33	P 5 10000 S1 0.67972 0.68190 0.67756
	34	P 5 10000 S2 0.50290 0.50290 0.50290
	35	P 5 10000 BACKOFF 0.71886 0.71886 0.71886
	36	R 5 10000 C-G 300 0.71873 17483
	37	R 5 10000 C-G 1000 0.85432 17483
	38	R 5 10000 C-G 3000 0.94555 17483
	39	R 5 10000 C-G 10000 0.98213 17483
	40	R 5 10000 C-G -1 0.99495 17483
	41	R 5 10000 HARD 300 0.71158 3586
	42	R 5 10000 HARD 1000 0.82657 3586
	43	R 5 10000 HARD 3000 0.87786 3586
	44	R 5 10000 HARD 10000 0.88123 3586
	45	R 5 10000 HARD -1 0.88123 3586
	46	R 5 10000 GRADED 300 0.75152 17483
	47	R 5 10000 GRADED 1000 0.87408 17483
	48	R 5 10000 GRADED 3000 0.90393 17483
	49	R 5 10000 GRADED 10000 0.92243 17483
	50	R 5 10000 GRADED -1 0.99495 17483
	51	R 5 10000 GRADED-G 300 0.73828 17483
	52	R 5 10000 GRADED-G 1000 0.85201 17483
	53	R 5 10000 GRADED-G 3000 0.87871 17483
	54	R 5 10000 GRADED-G 10000 0.97730 17483
	55	R 5 10000 GRADED-G -1 0.99495 17483
	56	R 5 10000 GRADED-R 300 0.72336 17483
	57	R 5 10000 GRADED-R 1000 0.87723 17483
	58	R 5 10000 GRADED-R 3000 0.94618 17483
	59	R 5 10000 GRADED-R 10000 0.98045 17483
	60	R 5 10000 GRADED-R -1 0.99495 17483
	61	P 5 30000 W 0.38075 0.53489 0.29558
	62	P 5 30000 G 0.74128 0.74281 0.73975
	63	P 5 30000 S1 0.67228 0.67443 0.67014
	64	P 5 30000 S2 0.51267 0.51267 0.51267
	65	P 5 30000 BACKOFF 0.71676 0.71676 0.71676
	66	R 5 30000 C-G 300 0.70843 24161
	67	R 5 30000 C-G 1000 0.84654 24161
	68	R 5 30000 C-G 3000 0.94534 24161
	69	R 5 30000 C-G 10000 0.98718 24161
	70	R 5 30000 C-G -1 0.99958 24161
	71	R 5 30000 HARD 300 0.70759 5226
	72	R 5 30000 HARD 1000 0.83414 5226
	73	R 5 30000 HARD 3000 0.87618 5226
	74	R 5 30000 HARD 10000 0.89426 5226
	75	R 5 30000 HARD -1 0.89426 5226
	76	R 5 30000 GRADED 300 0.72777 24161
	77	R 5 30000 GRADED 1000 0.85958 24161
	78	R 5 30000 GRADED 3000 0.91234 24161
	79	R 5 30000 GRADED 10000 0.93462 24161
	80	R 5 30000 GRADED -1 0.99958 24161
	81	R 5 30000 GRADED-G 300 0.73576 24161
	82	R 5 30000 GRADED-G 1000 0.84717 24161
	83	R 5 30000 GRADED-G 3000 0.88690 24161
	84	R 5 30000 GRADED-G 10000 0.90708 24161
	85	R 5 30000 GRADED-G -1 0.99958 24161
	86	R 5 30000 GRADED-R 300 0.70843 24161
	87	R 5 30000 GRADED-R 1000 0.85873 24161
	88	R 5 30000 GRADED-R 3000 0.95354 24161
	89	R 5 30000 GRADED-R 10000 0.98486 24161
	90	R 5 30000 GRADED-R -1 0.99958 24161

experiments/fractal/results/1792353740/seed-6.tsv +90 -0

…	…	@@ -0,0 +1,90 @@
	1	P 6 3000 W 0.14721 0.47416 0.08713
	2	P 6 3000 G 0.70396 0.70733 0.70062
	3	P 6 3000 S1 0.68320 0.68502 0.68139
	4	P 6 3000 S2 0.51456 0.51456 0.51456
	5	P 6 3000 BACKOFF 0.69967 0.69967 0.69967
	6	R 6 3000 C-G 300 0.72680 10698
	7	R 6 3000 C-G 1000 0.86123 10698
	8	R 6 3000 C-G 3000 0.94014 10698
	9	R 6 3000 C-G 10000 0.97825 10698
	10	R 6 3000 C-G -1 0.98053 10698
	11	R 6 3000 HARD 300 0.66363 2035
	12	R 6 3000 HARD 1000 0.75249 2035
	13	R 6 3000 HARD 3000 0.79743 2035
	14	R 6 3000 HARD 10000 0.79743 2035
	15	R 6 3000 HARD -1 0.79743 2035
	16	R 6 3000 GRADED 300 0.75414 10698
	17	R 6 3000 GRADED 1000 0.84238 10698
	18	R 6 3000 GRADED 3000 0.89354 10698
	19	R 6 3000 GRADED 10000 0.97970 10698
	20	R 6 3000 GRADED -1 0.98053 10698
	21	R 6 3000 GRADED-G 300 0.68641 10698
	22	R 6 3000 GRADED-G 1000 0.78107 10698
	23	R 6 3000 GRADED-G 3000 0.80882 10698
	24	R 6 3000 GRADED-G 10000 0.97991 10698
	25	R 6 3000 GRADED-G -1 0.98053 10698
	26	R 6 3000 GRADED-R 300 0.73053 10698
	27	R 6 3000 GRADED-R 1000 0.84051 10698
	28	R 6 3000 GRADED-R 3000 0.89996 10698
	29	R 6 3000 GRADED-R 10000 0.97887 10698
	30	R 6 3000 GRADED-R -1 0.98053 10698
	31	P 6 10000 W 0.26077 0.50237 0.17608
	32	P 6 10000 G 0.72774 0.72907 0.72642
	33	P 6 10000 S1 0.67042 0.67221 0.66865
	34	P 6 10000 S2 0.51274 0.51274 0.51274
	35	P 6 10000 BACKOFF 0.71280 0.71280 0.71280
	36	R 6 10000 C-G 300 0.71935 17582
	37	R 6 10000 C-G 1000 0.84963 17582
	38	R 6 10000 C-G 3000 0.94905 17582
	39	R 6 10000 C-G 10000 0.98467 17582
	40	R 6 10000 C-G -1 0.99648 17582
	41	R 6 10000 HARD 300 0.69905 3622
	42	R 6 10000 HARD 1000 0.80841 3622
	43	R 6 10000 HARD 3000 0.86060 3622
	44	R 6 10000 HARD 10000 0.86350 3622
	45	R 6 10000 HARD -1 0.86350 3622
	46	R 6 10000 GRADED 300 0.74171 17582
	47	R 6 10000 GRADED 1000 0.87386 17582
	48	R 6 10000 GRADED 3000 0.91342 17582
	49	R 6 10000 GRADED 10000 0.93123 17582
	50	R 6 10000 GRADED -1 0.99648 17582
	51	R 6 10000 GRADED-G 300 0.72432 17582
	52	R 6 10000 GRADED-G 1000 0.82995 17582
	53	R 6 10000 GRADED-G 3000 0.86123 17582
	54	R 6 10000 GRADED-G 10000 0.97577 17582
	55	R 6 10000 GRADED-G -1 0.99648 17582
	56	R 6 10000 GRADED-R 300 0.72328 17582
	57	R 6 10000 GRADED-R 1000 0.86309 17582
	58	R 6 10000 GRADED-R 3000 0.95464 17582
	59	R 6 10000 GRADED-R 10000 0.98737 17582
	60	R 6 10000 GRADED-R -1 0.99648 17582
	61	P 6 30000 W 0.37723 0.52798 0.29345
	62	P 6 30000 G 0.73329 0.73463 0.73196
	63	P 6 30000 S1 0.67082 0.67261 0.66904
	64	P 6 30000 S2 0.51361 0.51361 0.51361
	65	P 6 30000 BACKOFF 0.71462 0.71462 0.71462
	66	R 6 30000 C-G 300 0.70816 23768
	67	R 6 30000 C-G 1000 0.84466 23768
	68	R 6 30000 C-G 3000 0.95278 23768
	69	R 6 30000 C-G 10000 0.98840 23768
	70	R 6 30000 C-G -1 0.99959 23768
	71	R 6 30000 HARD 300 0.71375 5170
	72	R 6 30000 HARD 1000 0.83534 5170
	73	R 6 30000 HARD 3000 0.88442 5170
	74	R 6 30000 HARD 10000 0.89934 5170
	75	R 6 30000 HARD -1 0.89934 5170
	76	R 6 30000 GRADED 300 0.74606 23768
	77	R 6 30000 GRADED 1000 0.87365 23768
	78	R 6 30000 GRADED 3000 0.93621 23768
	79	R 6 30000 GRADED 10000 0.95505 23768
	80	R 6 30000 GRADED -1 0.99959 23768
	81	R 6 30000 GRADED-G 300 0.74275 23768
	82	R 6 30000 GRADED-G 1000 0.84486 23768
	83	R 6 30000 GRADED-G 3000 0.89354 23768
	84	R 6 30000 GRADED-G 10000 0.91218 23768
	85	R 6 30000 GRADED-G -1 0.99959 23768
	86	R 6 30000 GRADED-R 300 0.70816 23768
	87	R 6 30000 GRADED-R 1000 0.85294 23768
	88	R 6 30000 GRADED-R 3000 0.95878 23768
	89	R 6 30000 GRADED-R 10000 0.99068 23768
	90	R 6 30000 GRADED-R -1 0.99959 23768

experiments/fractal/results/1792353740/seed-7.tsv +90 -0

…	…	@@ -0,0 +1,90 @@
	1	P 7 3000 W 0.14431 0.47357 0.08513
	2	P 7 3000 G 0.71937 0.72537 0.71346
	3	P 7 3000 S1 0.66356 0.66784 0.65934
	4	P 7 3000 S2 0.51413 0.51413 0.51413
	5	P 7 3000 BACKOFF 0.74567 0.74567 0.74567
	6	R 7 3000 C-G 300 0.71012 10473
	7	R 7 3000 C-G 1000 0.85076 10473
	8	R 7 3000 C-G 3000 0.93997 10473
	9	R 7 3000 C-G 10000 0.97481 10473
	10	R 7 3000 C-G -1 0.97670 10473
	11	R 7 3000 HARD 300 0.67338 2027
	12	R 7 3000 HARD 1000 0.75378 2027
	13	R 7 3000 HARD 3000 0.78610 2027
	14	R 7 3000 HARD 10000 0.78610 2027
	15	R 7 3000 HARD -1 0.78610 2027
	16	R 7 3000 GRADED 300 0.74118 10473
	17	R 7 3000 GRADED 1000 0.84215 10473
	18	R 7 3000 GRADED 3000 0.87091 10473
	19	R 7 3000 GRADED 10000 0.97628 10473
	20	R 7 3000 GRADED -1 0.97670 10473
	21	R 7 3000 GRADED-G 300 0.69647 10473
	22	R 7 3000 GRADED-G 1000 0.77561 10473
	23	R 7 3000 GRADED-G 3000 0.80688 10473
	24	R 7 3000 GRADED-G 10000 0.97670 10473
	25	R 7 3000 GRADED-G -1 0.97670 10473
	26	R 7 3000 GRADED-R 300 0.72838 10473
	27	R 7 3000 GRADED-R 1000 0.83417 10473
	28	R 7 3000 GRADED-R 3000 0.93829 10473
	29	R 7 3000 GRADED-R 10000 0.97502 10473
	30	R 7 3000 GRADED-R -1 0.97670 10473
	31	P 7 10000 W 0.27579 0.52705 0.18676
	32	P 7 10000 G 0.73902 0.74042 0.73762
	33	P 7 10000 S1 0.67587 0.67759 0.67416
	34	P 7 10000 S2 0.50930 0.50930 0.50930
	35	P 7 10000 BACKOFF 0.72127 0.72127 0.72127
	36	R 7 10000 C-G 300 0.71725 17516
	37	R 7 10000 C-G 1000 0.84824 17516
	38	R 7 10000 C-G 3000 0.94521 17516
	39	R 7 10000 C-G 10000 0.98237 17516
	40	R 7 10000 C-G -1 0.99559 17516
	41	R 7 10000 HARD 300 0.71390 3635
	42	R 7 10000 HARD 1000 0.82473 3635
	43	R 7 10000 HARD 3000 0.87448 3635
	44	R 7 10000 HARD 10000 0.87636 3635
	45	R 7 10000 HARD -1 0.87636 3635
	46	R 7 10000 GRADED 300 0.74769 17516
	47	R 7 10000 GRADED 1000 0.87510 17516
	48	R 7 10000 GRADED 3000 0.90470 17516
	49	R 7 10000 GRADED 10000 0.92296 17516
	50	R 7 10000 GRADED -1 0.99559 17516
	51	R 7 10000 GRADED-G 300 0.73804 17516
	52	R 7 10000 GRADED-G 1000 0.85055 17516
	53	R 7 10000 GRADED-G 3000 0.87427 17516
	54	R 7 10000 GRADED-G 10000 0.97544 17516
	55	R 7 10000 GRADED-G -1 0.99559 17516
	56	R 7 10000 GRADED-R 300 0.71725 17516
	57	R 7 10000 GRADED-R 1000 0.86461 17516
	58	R 7 10000 GRADED-R 3000 0.94479 17516
	59	R 7 10000 GRADED-R 10000 0.98489 17516
	60	R 7 10000 GRADED-R -1 0.99559 17516
	61	P 7 30000 W 0.38347 0.53698 0.29822
	62	P 7 30000 G 0.74109 0.74256 0.73963
	63	P 7 30000 S1 0.67587 0.67759 0.67416
	64	P 7 30000 S2 0.51140 0.51140 0.51140
	65	P 7 30000 BACKOFF 0.71483 0.71483 0.71483
	66	R 7 30000 C-G 300 0.70802 23815
	67	R 7 30000 C-G 1000 0.84950 23815
	68	R 7 30000 C-G 3000 0.94773 23815
	69	R 7 30000 C-G 10000 0.98489 23815
	70	R 7 30000 C-G -1 0.99937 23815
	71	R 7 30000 HARD 300 0.70676 5081
	72	R 7 30000 HARD 1000 0.82641 5081
	73	R 7 30000 HARD 3000 0.87741 5081
	74	R 7 30000 HARD 10000 0.89232 5081
	75	R 7 30000 HARD -1 0.89232 5081
	76	R 7 30000 GRADED 300 0.72061 23815
	77	R 7 30000 GRADED 1000 0.86566 23815
	78	R 7 30000 GRADED 3000 0.90911 23815
	79	R 7 30000 GRADED 10000 0.93682 23815
	80	R 7 30000 GRADED -1 0.99937 23815
	81	R 7 30000 GRADED-G 300 0.73552 23815
	82	R 7 30000 GRADED-G 1000 0.85013 23815
	83	R 7 30000 GRADED-G 3000 0.88308 23815
	84	R 7 30000 GRADED-G 10000 0.90638 23815
	85	R 7 30000 GRADED-G -1 0.99937 23815
	86	R 7 30000 GRADED-R 300 0.70802 23815
	87	R 7 30000 GRADED-R 1000 0.86083 23815
	88	R 7 30000 GRADED-R 3000 0.95151 23815
	89	R 7 30000 GRADED-R 10000 0.98573 23815
	90	R 7 30000 GRADED-R -1 0.99937 23815

experiments/fractal/results/1792353740/seed-8.tsv +90 -0

…	…	@@ -0,0 +1,90 @@
	1	P 8 3000 W 0.15559 0.52131 0.09144
	2	P 8 3000 G 0.70116 0.70773 0.69471
	3	P 8 3000 S1 0.66986 0.67288 0.66686
	4	P 8 3000 S2 0.54058 0.54058 0.54058
	5	P 8 3000 BACKOFF 0.71172 0.71172 0.71172
	6	R 8 3000 C-G 300 0.72817 10644
	7	R 8 3000 C-G 1000 0.86012 10644
	8	R 8 3000 C-G 3000 0.94551 10644
	9	R 8 3000 C-G 10000 0.97597 10644
	10	R 8 3000 C-G -1 0.97833 10644
	11	R 8 3000 HARD 300 0.67604 2034
	12	R 8 3000 HARD 1000 0.75692 2034
	13	R 8 3000 HARD 3000 0.78181 2034
	14	R 8 3000 HARD 10000 0.78181 2034
	15	R 8 3000 HARD -1 0.78181 2034
	16	R 8 3000 GRADED 300 0.75756 10644
	17	R 8 3000 GRADED 1000 0.85068 10644
	18	R 8 3000 GRADED 3000 0.88844 10644
	19	R 8 3000 GRADED 10000 0.97790 10644
	20	R 8 3000 GRADED -1 0.97833 10644
	21	R 8 3000 GRADED-G 300 0.67904 10644
	22	R 8 3000 GRADED-G 1000 0.77044 10644
	23	R 8 3000 GRADED-G 3000 0.81356 10644
	24	R 8 3000 GRADED-G 10000 0.97790 10644
	25	R 8 3000 GRADED-G -1 0.97833 10644
	26	R 8 3000 GRADED-R 300 0.73482 10644
	27	R 8 3000 GRADED-R 1000 0.86119 10644
	28	R 8 3000 GRADED-R 3000 0.93156 10644
	29	R 8 3000 GRADED-R 10000 0.97726 10644
	30	R 8 3000 GRADED-R -1 0.97833 10644
	31	P 8 10000 W 0.28046 0.51827 0.19224
	32	P 8 10000 G 0.73494 0.73639 0.73349
	33	P 8 10000 S1 0.66620 0.66868 0.66374
	34	P 8 10000 S2 0.52202 0.52202 0.52202
	35	P 8 10000 BACKOFF 0.71237 0.71237 0.71237
	36	R 8 10000 C-G 300 0.72946 17275
	37	R 8 10000 C-G 1000 0.85368 17275
	38	R 8 10000 C-G 3000 0.94744 17275
	39	R 8 10000 C-G 10000 0.98584 17275
	40	R 8 10000 C-G -1 0.99549 17275
	41	R 8 10000 HARD 300 0.72195 3601
	42	R 8 10000 HARD 1000 0.83652 3601
	43	R 8 10000 HARD 3000 0.88736 3601
	44	R 8 10000 HARD 10000 0.88972 3601
	45	R 8 10000 HARD -1 0.88972 3601
	46	R 8 10000 GRADED 300 0.74941 17275
	47	R 8 10000 GRADED 1000 0.88243 17275
	48	R 8 10000 GRADED 3000 0.91418 17275
	49	R 8 10000 GRADED 10000 0.93263 17275
	50	R 8 10000 GRADED -1 0.99549 17275
	51	R 8 10000 GRADED-G 300 0.75585 17275
	52	R 8 10000 GRADED-G 1000 0.86419 17275
	53	R 8 10000 GRADED-G 3000 0.88822 17275
	54	R 8 10000 GRADED-G 10000 0.97661 17275
	55	R 8 10000 GRADED-G -1 0.99549 17275
	56	R 8 10000 GRADED-R 300 0.72946 17275
	57	R 8 10000 GRADED-R 1000 0.87320 17275
	58	R 8 10000 GRADED-R 3000 0.94958 17275
	59	R 8 10000 GRADED-R 10000 0.98434 17275
	60	R 8 10000 GRADED-R -1 0.99549 17275
	61	P 8 30000 W 0.38439 0.54193 0.29781
	62	P 8 30000 G 0.73066 0.73210 0.72921
	63	P 8 30000 S1 0.66702 0.66951 0.66456
	64	P 8 30000 S2 0.52037 0.52037 0.52037
	65	P 8 30000 BACKOFF 0.71467 0.71467 0.71467
	66	R 8 30000 C-G 300 0.71959 23804
	67	R 8 30000 C-G 1000 0.84510 23804
	68	R 8 30000 C-G 3000 0.95044 23804
	69	R 8 30000 C-G 10000 0.98584 23804
	70	R 8 30000 C-G -1 1.00000 23804
	71	R 8 30000 HARD 300 0.71937 5148
	72	R 8 30000 HARD 1000 0.84317 5148
	73	R 8 30000 HARD 3000 0.89359 5148
	74	R 8 30000 HARD 10000 0.90624 5148
	75	R 8 30000 HARD -1 0.90624 5148
	76	R 8 30000 GRADED 300 0.74040 23804
	77	R 8 30000 GRADED 1000 0.86913 23804
	78	R 8 30000 GRADED 3000 0.92405 23804
	79	R 8 30000 GRADED 10000 0.94357 23804
	80	R 8 30000 GRADED -1 1.00000 23804
	81	R 8 30000 GRADED-G 300 0.75585 23804
	82	R 8 30000 GRADED-G 1000 0.85647 23804
	83	R 8 30000 GRADED-G 3000 0.89766 23804
	84	R 8 30000 GRADED-G 10000 0.91761 23804
	85	R 8 30000 GRADED-G -1 1.00000 23804
	86	R 8 30000 GRADED-R 300 0.71959 23804
	87	R 8 30000 GRADED-R 1000 0.86290 23804
	88	R 8 30000 GRADED-R 3000 0.95559 23804
	89	R 8 30000 GRADED-R 10000 0.98970 23804
	90	R 8 30000 GRADED-R -1 1.00000 23804

experiments/fractal/results/1792353740/seed-9.tsv +90 -0

…	…	@@ -0,0 +1,90 @@
	1	P 9 3000 W 0.15741 0.49574 0.09356
	2	P 9 3000 G 0.72358 0.72648 0.72070
	3	P 9 3000 S1 0.68553 0.68697 0.68410
	4	P 9 3000 S2 0.51669 0.51669 0.51669
	5	P 9 3000 BACKOFF 0.72818 0.72818 0.72818
	6	R 9 3000 C-G 300 0.73125 10725
	7	R 9 3000 C-G 1000 0.86021 10725
	8	R 9 3000 C-G 3000 0.93708 10725
	9	R 9 3000 C-G 10000 0.97562 10725
	10	R 9 3000 C-G -1 0.97792 10725
	11	R 9 3000 HARD 300 0.72396 2115
	12	R 9 3000 HARD 1000 0.81958 2115
	13	R 9 3000 HARD 3000 0.84833 2115
	14	R 9 3000 HARD 10000 0.84833 2115
	15	R 9 3000 HARD -1 0.84833 2115
	16	R 9 3000 GRADED 300 0.76708 10725
	17	R 9 3000 GRADED 1000 0.85354 10725
	18	R 9 3000 GRADED 3000 0.87708 10725
	19	R 9 3000 GRADED 10000 0.97750 10725
	20	R 9 3000 GRADED -1 0.97792 10725
	21	R 9 3000 GRADED-G 300 0.74125 10725
	22	R 9 3000 GRADED-G 1000 0.83479 10725
	23	R 9 3000 GRADED-G 3000 0.85521 10725
	24	R 9 3000 GRADED-G 10000 0.97792 10725
	25	R 9 3000 GRADED-G -1 0.97792 10725
	26	R 9 3000 GRADED-R 300 0.75167 10725
	27	R 9 3000 GRADED-R 1000 0.84125 10725
	28	R 9 3000 GRADED-R 3000 0.93708 10725
	29	R 9 3000 GRADED-R 10000 0.97646 10725
	30	R 9 3000 GRADED-R -1 0.97792 10725
	31	P 9 10000 W 0.27007 0.51388 0.18317
	32	P 9 10000 G 0.74383 0.74476 0.74290
	33	P 9 10000 S1 0.68537 0.68681 0.68394
	34	P 9 10000 S2 0.51999 0.51999 0.51999
	35	P 9 10000 BACKOFF 0.72729 0.72729 0.72729
	36	R 9 10000 C-G 300 0.72417 17566
	37	R 9 10000 C-G 1000 0.85708 17566
	38	R 9 10000 C-G 3000 0.94583 17566
	39	R 9 10000 C-G 10000 0.98062 17566
	40	R 9 10000 C-G -1 0.99479 17566
	41	R 9 10000 HARD 300 0.71667 3651
	42	R 9 10000 HARD 1000 0.83458 3651
	43	R 9 10000 HARD 3000 0.88417 3651
	44	R 9 10000 HARD 10000 0.88875 3651
	45	R 9 10000 HARD -1 0.88875 3651
	46	R 9 10000 GRADED 300 0.76500 17566
	47	R 9 10000 GRADED 1000 0.87521 17566
	48	R 9 10000 GRADED 3000 0.91271 17566
	49	R 9 10000 GRADED 10000 0.93542 17566
	50	R 9 10000 GRADED -1 0.99479 17566
	51	R 9 10000 GRADED-G 300 0.75313 17566
	52	R 9 10000 GRADED-G 1000 0.85208 17566
	53	R 9 10000 GRADED-G 3000 0.88479 17566
	54	R 9 10000 GRADED-G 10000 0.97229 17566
	55	R 9 10000 GRADED-G -1 0.99479 17566
	56	R 9 10000 GRADED-R 300 0.72417 17566
	57	R 9 10000 GRADED-R 1000 0.87542 17566
	58	R 9 10000 GRADED-R 3000 0.94667 17566
	59	R 9 10000 GRADED-R 10000 0.98083 17566
	60	R 9 10000 GRADED-R -1 0.99479 17566
	61	P 9 30000 W 0.38092 0.53275 0.29644
	62	P 9 30000 G 0.74939 0.75032 0.74845
	63	P 9 30000 S1 0.68577 0.68721 0.68434
	64	P 9 30000 S2 0.51830 0.51830 0.51830
	65	P 9 30000 BACKOFF 0.72593 0.72593 0.72593
	66	R 9 30000 C-G 300 0.71146 23892
	67	R 9 30000 C-G 1000 0.85104 23892
	68	R 9 30000 C-G 3000 0.94750 23892
	69	R 9 30000 C-G 10000 0.98583 23892
	70	R 9 30000 C-G -1 0.99938 23892
	71	R 9 30000 HARD 300 0.70000 5192
	72	R 9 30000 HARD 1000 0.83417 5192
	73	R 9 30000 HARD 3000 0.89021 5192
	74	R 9 30000 HARD 10000 0.90208 5192
	75	R 9 30000 HARD -1 0.90208 5192
	76	R 9 30000 GRADED 300 0.73104 23892
	77	R 9 30000 GRADED 1000 0.86458 23892
	78	R 9 30000 GRADED 3000 0.91479 23892
	79	R 9 30000 GRADED 10000 0.94125 23892
	80	R 9 30000 GRADED -1 0.99938 23892
	81	R 9 30000 GRADED-G 300 0.75313 23892
	82	R 9 30000 GRADED-G 1000 0.84667 23892
	83	R 9 30000 GRADED-G 3000 0.89146 23892
	84	R 9 30000 GRADED-G 10000 0.91250 23892
	85	R 9 30000 GRADED-G -1 0.99938 23892
	86	R 9 30000 GRADED-R 300 0.71146 23892
	87	R 9 30000 GRADED-R 1000 0.85771 23892
	88	R 9 30000 GRADED-R 3000 0.95500 23892
	89	R 9 30000 GRADED-R 10000 0.98771 23892
	90	R 9 30000 GRADED-R -1 0.99938 23892

experiments/fractal/results/1792353740/src.sha256 +2 -0

…	…	@@ -0,0 +1,2 @@
	1	6c4a0249d7fcb7be072722932fbd4861fa1dc9891a57023535fd85edf5d2821f fractal.c
	2	2f4343faaa8bd25592fdadbc5bd38807f63d6823bcf85aff9eaeda50e96b8396 ../across-cluster/ac.c

experiments/fractal/results/1792353740/summary.txt +36 -0

…	…	@@ -0,0 +1,36 @@
	1	## Level-1 link F1 (mean)
	2	tier W G S1 S2 BACKOFF
	3	3000 0.156 0.705 0.668 0.516 0.711
	4	10000 0.272 0.734 0.674 0.510 0.716
	5	30000 0.380 0.735 0.672 0.511 0.714
	6
	7	## Level-2 type recall at budget (mean)
	8	tier 3000: budget 300 1000 3000 10000 all
	9	C-G 0.722 0.857 0.942 0.977 0.979 (list 10570)
	10	HARD 0.673 0.763 0.795 0.795 0.795 (list 2072)
	11	GRADED 0.751 0.849 0.884 0.978 0.979 (list 10570)
	12	GRADED-G 0.691 0.782 0.817 0.978 0.979 (list 10570)
	13	GRADED-R 0.719 0.822 0.927 0.977 0.979 (list 10570)
	14	tier 10000: budget 300 1000 3000 10000 all
	15	C-G 0.717 0.853 0.947 0.984 0.995 (list 17434)
	16	HARD 0.711 0.827 0.878 0.881 0.881 (list 3614)
	17	GRADED 0.750 0.876 0.910 0.929 0.995 (list 17434)
	18	GRADED-G 0.743 0.850 0.879 0.977 0.995 (list 17434)
	19	GRADED-R 0.717 0.871 0.946 0.983 0.995 (list 17434)
	20	tier 30000: budget 300 1000 3000 10000 all
	21	C-G 0.706 0.848 0.950 0.986 0.999 (list 23859)
	22	HARD 0.704 0.834 0.883 0.899 0.899 (list 5187)
	23	GRADED 0.734 0.870 0.922 0.948 0.999 (list 23859)
	24	GRADED-G 0.744 0.850 0.890 0.910 0.999 (list 23859)
	25	GRADED-R 0.706 0.857 0.955 0.987 0.999 (list 23859)
	26
	27	## Predictions
	28	F1: FAIL — 3000: BACKOFF-G +0.006[-0.003]; 10000: BACKOFF-G -0.018[-0.021]; 30000: BACKOFF-G -0.021
	29	F2: PASS — 3000:0.979 10000:0.995 30000:0.999 \| HARD 3000:0.795 10000:0.881 30000:0.899
	30	F3a: FAIL — 3000: GRADED-C-G@1K -0.008[-0.019]; 10000: GRADED-C-G@1K +0.023[+0.019]; 30000: GRADED-C-G@1K +0.022[+0.017]
	31	F3b (GRADED >= HARD at B = \|HARD list\|): see note — compared at the largest common budget below each HARD list size:
	32	3000: \|HARD\| 2072, budget 1000: GRADED 0.849 vs HARD 0.763
	33	10000: \|HARD\| 3614, budget 3000: GRADED 0.910 vs HARD 0.878
	34	30000: \|HARD\| 5187, budget 3000: GRADED 0.922 vs HARD 0.883
	35	F4: PASS — GRADED-GRADED-G@1K 3000: +0.067[+0.052]; 10000: +0.025[+0.019]
	36	F5: FAIL — GRADED-R-C-G@1K 3000: -0.035[-0.069]; 10000: +0.018[+0.015]; 30000: +0.009[+0.007]

experiments/fractal/summarize.py +72 -0

…	…	@@ -0,0 +1,72 @@
	1	#!/usr/bin/env python3
	2	# Score a results/<stamp>/ directory against PREREG.md. Usage: summarize.py results/<stamp>
	3	import glob, math, os, sys
	4
	5	T975 = {9: 2.262}
	6	def mean(x): return sum(x) / len(x)
	7	def ci(d):
	8	n = len(d); m = mean(d); sd = math.sqrt(sum((v - m) ** 2 for v in d) / (n - 1))
	9	h = T975.get(n - 1, 1.96) * sd / math.sqrt(n)
	10	return m, m - h, m + h
	11
	12	P, R, L = {}, {}, {}
	13	for f in sorted(glob.glob(os.path.join(sys.argv[1], 'seed-*.tsv'))):
	14	for line in open(f):
	15	p = line.rstrip('\n').split('\t')
	16	if p[0] == 'P': P[(int(p[2]), p[3], int(p[1]))] = float(p[4])
	17	elif p[0] == 'R':
	18	R[(int(p[2]), p[3], int(p[4]), int(p[1]))] = float(p[5]); L[(int(p[2]), p[3], int(p[1]))] = int(p[6])
	19	seeds = sorted({k[2] for k in P}); tiers = sorted({k[0] for k in P})
	20	out = []
	21	def say(s=''): out.append(s); print(s)
	22	def pv(t, a): return [P[(t, a, s)] for s in seeds]
	23	def rv(t, a, b): return [R[(t, a, b, s)] for s in seeds]
	24
	25	say('## Level-1 link F1 (mean)')
	26	say('tier\t' + '\t'.join(['W', 'G', 'S1', 'S2', 'BACKOFF']))
	27	for t in tiers: say(f'{t}\t' + '\t'.join(f'{mean(pv(t, a)):.3f}' for a in ['W', 'G', 'S1', 'S2', 'BACKOFF']))
	28	say('\n## Level-2 type recall at budget (mean)')
	29	arms = ['C-G', 'HARD', 'GRADED', 'GRADED-G', 'GRADED-R']
	30	for t in tiers:
	31	say(f'tier {t}: budget\t300\t1000\t3000\t10000\tall')
	32	for a in arms:
	33	say(f' {a}\t' + '\t'.join(f'{mean(rv(t, a, b)):.3f}' for b in [300, 1000, 3000, 10000, -1]) + f'\t(list {mean([L[(t, a, s)] for s in seeds]):.0f})')
	34
	35	say('\n## Predictions')
	36	def verdict(n, ok, d): say(f'{n}: {"PASS" if ok else "FAIL"} — {d}')
	37	cells = []; ok = True
	38	for t in tiers:
	39	best = max(['W', 'G', 'S1', 'S2'], key=lambda a: mean(pv(t, a)))
	40	if t <= 10000:
	41	for a in ['W', 'G', 'S1', 'S2']:
	42	c = ci([x - y for x, y in zip(pv(t, 'BACKOFF'), pv(t, a))])
	43	ok &= c[1] > 0
	44	if a == best: cells.append(f'{t}: BACKOFF-{a} {c[0]:+.3f}[{c[1]:+.3f}]')
	45	else:
	46	d = mean(pv(t, 'BACKOFF')) - mean(pv(t, best)); ok &= d >= -0.01
	47	cells.append(f'{t}: BACKOFF-{best} {d:+.3f}')
	48	verdict('F1', ok, '; '.join(cells))
	49	g = [(t, mean(rv(t, 'GRADED', -1))) for t in tiers]
	50	verdict('F2', all(x >= 0.95 for _, x in g), ' '.join(f'{t}:{x:.3f}' for t, x in g) + ' \| HARD ' + ' '.join(f'{t}:{mean(rv(t, "HARD", -1)):.3f}' for t in tiers))
	51	ok = True; cells = []
	52	for t in tiers:
	53	c = ci([x - y for x, y in zip(rv(t, 'GRADED', 1000), rv(t, 'C-G', 1000))]); ok &= c[1] > 0
	54	cells.append(f'{t}: GRADED-C-G@1K {c[0]:+.3f}[{c[1]:+.3f}]')
	55	# GRADED at B = \|HARD list\| vs HARD (interpolate GRADED at the nearest budget not exceeding the HARD list)
	56	verdict('F3a', ok, '; '.join(cells))
	57	say('F3b (GRADED >= HARD at B = \|HARD list\|): see note — compared at the largest common budget below each HARD list size:')
	58	for t in tiers:
	59	hl = mean([L[(t, 'HARD', s)] for s in seeds])
	60	b = max([x for x in [300, 1000, 3000, 10000] if x <= hl] or [300])
	61	say(f' {t}: \|HARD\| {hl:.0f}, budget {b}: GRADED {mean(rv(t, "GRADED", b)):.3f} vs HARD {mean(rv(t, "HARD", b)):.3f}')
	62	ok = True; cells = []
	63	for t in [x for x in tiers if x <= 10000]:
	64	c = ci([x - y for x, y in zip(rv(t, 'GRADED', 1000), rv(t, 'GRADED-G', 1000))]); ok &= c[1] > 0
	65	cells.append(f'{t}: {c[0]:+.3f}[{c[1]:+.3f}]')
	66	verdict('F4', ok, 'GRADED-GRADED-G@1K ' + '; '.join(cells))
	67	ok = True; cells = []
	68	for t in tiers:
	69	c = ci([x - y for x, y in zip(rv(t, 'GRADED-R', 1000), rv(t, 'C-G', 1000))]); ok &= not c[1] > 0
	70	cells.append(f'{t}: {c[0]:+.3f}[{c[1]:+.3f}]')
	71	verdict('F5', ok, 'GRADED-R-C-G@1K ' + '; '.join(cells))
	72	open(os.path.join(sys.argv[1], 'summary.txt'), 'w').write('\n'.join(out) + '\n')

atomspace-layer-discovery commits / 5d99c5c

Changed files (15) +1043 -0