OverviewFilesCommitsIssuesDocsReleasesPeers

terrain-lookdev commits / e6454b7

lookdev: diffusion throughput matrix — 14x faster, 3.6x less VRAM
bench_diffusion.py runs each config in its OWN server on a scratch port, at fresh coordinates, so the always-on service is never touched and a cached tile (0.04 s) cannot fake a result. RESULT, 512 tile = 236 km2, all measured on an idle GPU fp32 baseline (what we had shipped) 10.16 s 23 km2/s 3522 MiB + fp16 6.91 s - + onestep_latent 4.17 s 57 km2/s 1560 MiB + decoder_tile_size 256 2.11 s 112 km2/s 946 MiB + torch.compile 0.76 s 309 km2/s 976 MiB Live service, confirmed after switching: 512 in 1.04 s, 1024 in 1.79 s (527 km2/s), 976 MiB. 1024 tile went 25.07 s -> 1.79 s. 14x. WHAT MATTERED, AND WHAT DID NOT onestep_latent drops the intermediate latent pass (world_pipeline:1253). A STEP-COUNT win: fp16 alone is only 1.08x. decoder_tile 256 faster AND lighter, which is counter-intuitive. 1024 is catastrophic (0.18x, 4818 MiB) -- it spills on an 8 GB card. torch.compile 3x, but only in combination. On its own earlier it was a LOSS (5.36 s vs 3.89 s). batch-size both 2 and 8 are worse than the default 1,4. reduce-overhead WORSE than default compile (1.45 s vs 0.76 s) and +40% VRAM. The CUDA-graph hypothesis from the literature did not survive contact with this workload; graphs pin buffers. AN INSTRUMENT BUG THIS RUN FIXED The first matrix reported an identical 3524 MiB for every config, and I wrote that up as "VRAM did not move, none of these levers touch it". Wrong. gpu_mem() took the max across ALL compute apps and waited 3 s after SIGTERM, so every row was reading the DYING previous server's fp32 footprint. It now queries the server's own pid and waits for the GPU to actually clear. VRAM in fact drops 3522 -> 976 MiB, which was the constraint I had said was untouchable. Third time this session a stale process produced a confident wrong number. CAVEAT: fp16, onestep_latent and compile all change numerics, so this terrain is not bit-identical to the fp32 world. Quality is UNMEASURED. dtype is in ignore_for_config upstream, so a cache does NOT invalidate on a dtype change -- switch precision only with a fresh process. Also adds a TD_COMPILE_MODE hook to the local terrain-diffusion clone (upstream passes no mode); unset reproduces upstream exactly. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
e6454b74db82b3413f8ab41b937f61cb2108c24d
by plankatron <noreply@anthropic.com> · 2026-08-31T18:14:19.000Z

Changed files (1) +146 -0

bench_diffusion.py +146 -0
……@@ -0,0 +1,146 @@
1#!/usr/bin/env python3
2"""
3DIFFUSION THROUGHPUT MATRIX — what actually makes tile generation cheaper.
4
5hyper-games #46. Streaming terrain is bounded by two costs measured earlier:
6~4.5 s fixed per request plus area, and 3.5 GB of resident VRAM with a 5 s
785-92% GPU burst per tile. This finds which knobs move those.
8
9The levers, all read from source rather than guessed:
10 --dtype fp16 Turing (sm_75) has fp16 tensor cores; bf16 needs Ampere, so
11 it is not tested here.
12 onestep_latent world_pipeline.py:1253 -- the detail stage runs an initial
13 latent pass plus one intermediate (T_INTER). This drops the
14 intermediate. Roughly half the latent work.
15 --compile torch.compile, ON by default in the CLI and which I had
16 been disabling. Pays a large one-off cost, so the harness
17 warms up before timing.
18 --batch-size latent batch.
19
20Each config runs in its OWN server on a scratch port, so the service the lookdev
21app uses is never touched. Every tile is at fresh coordinates -- a cached tile
22returns in 0.04 s and would silently fake any result here.
23"""
24import json, os, subprocess, sys, time, urllib.request, signal
25
26PORT = 8819
27MODEL = "xandergos/terrain-diffusion-30m"
28VENV = "/storage/projects/code/godot/terrain-diffusion/.venv/bin/python"
29CWD = "/storage/projects/code/godot/terrain-diffusion"
30ORIGIN = 21_000_000 # far from anything this session has touched
31
32
33def wait_up(proc, timeout=600):
34 t0 = time.time()
35 while time.time() - t0 < timeout:
36 if proc.poll() is not None:
37 return False
38 try:
39 urllib.request.urlopen(f"http://127.0.0.1:{PORT}/health", timeout=2).read()
40 return True
41 except Exception:
42 time.sleep(2)
43 return False
44
45
46def tile(i1, j1, n):
47 u = (f"http://127.0.0.1:{PORT}/terrain?i1={i1}&j1={j1}"
48 f"&i2={i1+n}&j2={j1+n}&scale=1")
49 t0 = time.time()
50 with urllib.request.urlopen(u, timeout=3600) as r:
51 r.read()
52 return time.time() - t0
53
54
55def gpu_apps():
56 """{pid: MiB} for every process holding a CUDA context."""
57 try:
58 out = subprocess.run(["nvidia-smi", "--query-compute-apps=pid,used_memory",
59 "--format=csv,noheader,nounits"],
60 capture_output=True, text=True, timeout=20).stdout
61 d = {}
62 for line in out.strip().splitlines():
63 a, b = [x.strip() for x in line.split(",")[:2]]
64 if a.isdigit() and b.isdigit():
65 d[int(a)] = int(b)
66 return d
67 except Exception:
68 return {}
69
70
71def gpu_mem_of(pid):
72 """This server's OWN footprint. Taking the max across all apps is what made
73 every row of the first run report the dying previous server's fp32 3524 MiB."""
74 apps = gpu_apps()
75 if pid in apps:
76 return apps[pid]
77 # the server may be a child of the process group we launched
78 return max((m for p, m in apps.items() if abs(p - pid) < 8), default=0)
79
80
81def wait_gpu_clear(exclude, timeout=90):
82 """Do not start the next config until the previous context is really gone."""
83 t0 = time.time()
84 while time.time() - t0 < timeout:
85 if not {p: m for p, m in gpu_apps().items() if p not in exclude}:
86 return True
87 time.sleep(2)
88 return False
89
90
91def run(label, extra, n=512, reps=3, offset=0):
92 cmd = [VENV, "-m", "terrain_diffusion.inference.api", MODEL,
93 "--seed", "1234", "--host", "127.0.0.1", "--port", str(PORT),
94 "--log-mode", "info"] + extra
95 log = open(f"/tmp/bench-{label.replace(' ','_')}.log", "w")
96 p = subprocess.Popen(cmd, cwd=CWD, stdout=log, stderr=subprocess.STDOUT,
97 preexec_fn=os.setsid)
98 try:
99 if not wait_up(p):
100 print(f" {label:<28} FAILED TO START (see /tmp/bench-{label.replace(' ','_')}.log)")
101 return None
102 base = ORIGIN + offset * 200_000
103 warm = tile(base, base, 256) # pay compile / lazy init here
104 ts = [tile(base + 20_000*(k+1), base + 20_000*(k+1), n) for k in range(reps)]
105 mem = gpu_mem_of(p.pid)
106 best, avg = min(ts), sum(ts)/len(ts)
107 km2 = (n*30/1000)**2
108 print(f" {label:<28} warm {warm:6.1f}s tile {avg:6.2f}s (best {best:5.2f})"
109 f" {km2/avg:6.1f} km2/s {mem:5d} MiB")
110 return dict(label=label, warm=warm, avg=avg, best=best, mem=mem)
111 finally:
112 try: os.killpg(os.getpgid(p.pid), signal.SIGTERM)
113 except Exception: pass
114 p.wait(timeout=60)
115 wait_gpu_clear(exclude=set(BASELINE_PIDS))
116
117
118BASELINE_PIDS = set(gpu_apps().keys()) # the always-on service, not ours to wait on
119
120if __name__ == "__main__":
121 print(f"512 tile = 15.36 km across = 236 km2. Fresh coordinates every time.\n")
122 print(f" {'config':<28} {'model load':>11} {'per tile':>22} {'throughput':>12} {'VRAM':>9}")
123 rows = []
124 F16 = ["--no-compile", "--dtype", "fp16", "--kwarg", "onestep_latent=True"]
125 D = lambda n: F16 + ["--kwarg", f"decoder_tile_size={n}"]
126 C = lambda n: ["--compile", "--dtype", "fp16", "--kwarg", "onestep_latent=True",
127 "--kwarg", f"decoder_tile_size={n}"]
128 for i, (label, extra) in enumerate([
129 ("decoder_tile 256 (winner)", D(256)),
130 ("decoder_tile 192", D(192)),
131 ("decoder_tile 128", D(128)),
132 ("decoder_tile 64", D(64)),
133 ("256 + compile default", C(256)),
134 ("256 + compile reduce-ovh", C(256)),
135 ]):
136 if "reduce-ovh" in label:
137 os.environ["TD_COMPILE_MODE"] = "reduce-overhead"
138 else:
139 os.environ.pop("TD_COMPILE_MODE", None)
140 r = run(label, extra, offset=i)
141 if r: rows.append(r)
142 if rows:
143 b = rows[0]["avg"]
144 print("\n speedup vs baseline:")
145 for r in rows:
146 print(f" {r['label']:<28} {b/r['avg']:5.2f}x (VRAM {r['mem']} MiB)")