OverviewFilesCommitsIssuesDocsReleasesPeers

ifsrenderer-linux commits / 8b3b45f

Split density from colour, half the colour's resolution: 512 now fits
The bake grid was four interleaved channels per voxel (RGB + density) at one resolution: 16 bytes per voxel, 268 MB at N=256, and 2.1 GB at 512 which the docstring recorded as "will not fit". Colour is the channel that can afford to be coarser. Density carries the structure the fractal's detail lives in; hue varies smoothly across a flame, so halving colour's resolution is invisible where halving density's would not be. Density is now N^3 single-channel and colour a separate 3 x (N/2)^3 grid — 5.5 bytes per voxel instead of 16. N=384 peak RSS 4.27 -> 1.47 GB bake 20.0 -> 8.6s (2.9x, as predicted) N=512 did not fit -> 2.75 GB, 12.8s OpenVDB grids carry independent transforms, so the colour grid is written at twice the voxel size and Blender samples it correctly with no upsampling. That was VERIFIED first — a probe loading a mixed-resolution VDB and reading back both transforms — because the whole design collapses without it. Colour arrives premultiplied by opacity, so dividing it back out needs the density of the SAME colour voxel: the sum of its 2x2x2 density block, computed host-side. That is what keeps the colour grid to three channels rather than four. Occupancy, emission scale and the density grid are identical at every resolution, so geometry is untouched — and the suites that would catch otherwise all pass: volume_axes 5/5, auto_bounds 3/3, bake_orientation 7/7, bake_follows_document 3/3. test_volume_axes needed real thought rather than a signature bump: its colour check reads an INDEX, and colour's index space halved. Updated to MARK//2, and its guard-the-guard test now also asserts the HALVED indices stay distinct — a marker like 4,5,6 would halve to 2,2,3 and leave the colour check unable to see a transpose while still looking like a test. NOT done, and not for want of trying: making emission_scale resolution-invariant. It drifts 10.1 -> 18.2 across N=128..384. I attempted a fix (multiply the column sum by voxel size, making it a distance integral) on the reasoning that a ray crosses ~N occupied voxels; measured, it made the spread WORSE (179 -> 968), because these attractors are SHEETS and a ray crosses a fixed number of voxels regardless of N. Reverted. The real cause is peak-normalisation interacting with how a sheet resolves, and fixing it means changing how the VDB is normalised — a design change to a tuned path, not a tweak. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
8b3b45f607ae6e49fb2c71b2eefd48af8f1cb16e
by plankatron <fleeky@gmail.com> · 2026-08-18T15:58:21.000Z

Changed files (5) +121 -70

blender/ifs_layout.py +34 -13
……@@ -335,26 +335,47 @@ _PLOT_VOLUME = """ vec4 color = vec4(getPaletteColor(p.color_index), selected_i
335335 accumulate_voxel(p.pos.xyz, color);"""
336336
337337_VOXEL_HELPER = """
338// Four contiguous texels per voxel, mirroring the 2D histogram's layout so the
339// same coalescing argument applies: one cache line per voxel.
338// Density at FULL resolution, colour at HALF.
339//
340// The grid used to be four interleaved channels per voxel (RGB + density) at one
341// resolution, which cost 16 bytes per voxel: 268 MB at N=256 and 2.1 GB at N=512.
342// Splitting them costs 5.5 bytes per voxel instead — a 2.9x saving — because
343// colour is the channel that can afford it. Density carries the structure and is
344// what the fractal's detail lives in; hue varies smoothly across a flame, so
345// halving its resolution is invisible where halving density's would not be.
346//
347// OpenVDB grids each carry their own transform, so the half-res colour grid is
348// written with twice the voxel size and Blender samples it correctly with no
349// upsampling on our side. Verified before this was built, not assumed.
350//
351// Colour arrives premultiplied by opacity (see _PLOT_VOLUME), so recovering it
352// means dividing by the density in the SAME colour voxel — which is the sum of
353// the 2x2x2 density block, computed host-side rather than accumulated here. That
354// is what keeps this to three channels instead of four.
340355void accumulate_voxel(vec3 pos, vec4 color)
341356{
342 vec3 span = max(volume_max - volume_min, vec3(1e-9));
343 vec3 t = (pos - volume_min) / span;
344 if (any(lessThan(t, vec3(0.0))) || any(greaterThanEqual(t, vec3(1.0))))
345 return; // outside the bake bounds; the chaos game is unbounded
346 ivec3 v = ivec3(t * vec3(volume_size));
347 ivec3 base = ivec3(v.x * 4, v.y, v.z);
348 imageAtomicAdd(volume_img, base + ivec3(0, 0, 0), uint(max(color.r, 0.0) * HIST_SCALE));
349 imageAtomicAdd(volume_img, base + ivec3(1, 0, 0), uint(max(color.g, 0.0) * HIST_SCALE));
350 imageAtomicAdd(volume_img, base + ivec3(2, 0, 0), uint(max(color.b, 0.0) * HIST_SCALE));
351 imageAtomicAdd(volume_img, base + ivec3(3, 0, 0), uint(max(color.w, 0.0) * HIST_SCALE));
357\tvec3 span = max(volume_max - volume_min, vec3(1e-9));
358\tvec3 t = (pos - volume_min) / span;
359\tif (any(lessThan(t, vec3(0.0))) || any(greaterThanEqual(t, vec3(1.0))))
360\t\treturn; // outside the bake bounds; the chaos game is unbounded
361\timageAtomicAdd(volume_img, ivec3(t * vec3(volume_size)),
362\t uint(max(color.w, 0.0) * HIST_SCALE));
363
364\t// Three contiguous texels per colour voxel, the same coalescing argument the
365\t// old four-channel layout made.
366\tivec3 c = ivec3(t * vec3(volume_size >> 1));
367\tivec3 cbase = ivec3(c.x * 3, c.y, c.z);
368\timageAtomicAdd(colour_img, cbase + ivec3(0, 0, 0), uint(max(color.r, 0.0) * HIST_SCALE));
369\timageAtomicAdd(colour_img, cbase + ivec3(1, 0, 0), uint(max(color.g, 0.0) * HIST_SCALE));
370\timageAtomicAdd(colour_img, cbase + ivec3(2, 0, 0), uint(max(color.b, 0.0) * HIST_SCALE));
352371}
353372"""
354373
355374VOLUME_LAYOUT: tuple[Resource, ...] = (
356375 Resource("image", 0, "volume_img", "UINT_3D",
357 "R32UI, 4N x N x N, fixed point. imageAtomicAdd target."),
376 "R32UI, N x N x N density, fixed point. imageAtomicAdd target."),
377 Resource("image", 2, "colour_img", "UINT_3D",
378 "R32UI, 3(N/2) x N/2 x N/2 premultiplied colour, HALF resolution."),
358379 Resource("image", 1, "state_img", "FLOAT_2D", "as the 2D path"),
359380 Resource("sampler", 0, "alias_tables_tex", "FLOAT_2D", ""),
360381 Resource("uniform_buf", 0, "frame_ubo", "FrameParams", "", instance="frame"),
blender/ifs_volume.py +54 -41
……@@ -65,6 +65,7 @@ def build_shader(hist_scale: float = HIST_SCALE):
6565 rw = {'NO_RESTRICT', 'READ', 'WRITE'}
6666 info.image(0, 'R32UI', 'UINT_3D', 'volume_img', qualifiers=rw)
6767 info.image(1, 'RGBA32F', 'FLOAT_2D', 'state_img', qualifiers=rw)
68 info.image(2, 'R32UI', 'UINT_3D', 'colour_img', qualifiers=rw)
6869 info.sampler(0, 'FLOAT_2D', 'alias_tables_tex')
6970 info.push_constant('VEC3', 'volume_min')
7071 info.push_constant('VEC3', 'volume_max')
……@@ -96,10 +97,20 @@ def _document_buffers(doc):
9697
9798def accumulate(shader, doc, ubos, alias_tex, size, bounds, dispatches,
9899 invocation_iters=500):
99 """Runs the chaos game into a fresh 4N x N x N grid and reads it back."""
100 """
101 Runs the chaos game into a fresh density grid plus a HALF-resolution colour
102 grid, and reads both back.
103
104 Returns `(density, colour)` — density (z, y, x) float32, colour
105 (z/2, y/2, x/2, 3) float32 and still premultiplied by opacity. Splitting them
106 costs 5.5 bytes per voxel against the old interleaved 16.
107 """
100108 lo, hi = bounds
101 volume = gpu.types.GPUTexture((size * 4, size, size), format='R32UI')
109 half = max(1, size // 2)
110 volume = gpu.types.GPUTexture((size, size, size), format='R32UI')
102111 volume.clear(format='UINT', value=(0, 0, 0, 0))
112 colour_tex = gpu.types.GPUTexture((half * 3, half, half), format='R32UI')
113 colour_tex.clear(format='UINT', value=(0, 0, 0, 0))
103114 rows = (ifs_render.INVOCATIONS + ifs_render.STATE_W - 1) // ifs_render.STATE_W
104115 state = gpu.types.GPUTexture((ifs_render.STATE_W * 2, rows), format='RGBA32F')
105116 state.clear(format='FLOAT', value=(0.0, 0.0, 0.0, 0.0))
……@@ -111,6 +122,7 @@ def accumulate(shader, doc, ubos, alias_tex, size, bounds, dispatches,
111122 invocation_iters=invocation_iters))
112123 shader.bind()
113124 shader.image('volume_img', volume)
125 shader.image('colour_img', colour_tex)
114126 shader.image('state_img', state)
115127 shader.uniform_sampler('alias_tables_tex', alias_tex)
116128 shader.uniform_float('volume_min', lo)
……@@ -121,18 +133,17 @@ def accumulate(shader, doc, ubos, alias_tex, size, bounds, dispatches,
121133 shader.uniform_block(name, ubo)
122134 gpu.compute.dispatch(shader, ifs_render.WORKGROUPS, 1, 1)
123135
136 # ONE allocation each, not two. `astype(float32) / HIST_SCALE` materialises the
137 # float copy AND the quotient; np.divide with an explicit dtype does both in a
138 # single pass, which is most of a 2.4x peak-RSS overshoot.
124139 raw = np.array(volume.read(), dtype=np.uint32)
125 # (z, y, 4x) as read back; split the interleaved channels out.
126 #
127 # ONE allocation, not two. `raw.astype(np.float32) / HIST_SCALE` materialises
128 # the float copy AND the quotient — at N=384 that is two 906 MB arrays where
129 # one will do, and measured peak RSS was 2.4x the grid size because of chains
130 # like this. np.divide with an explicit dtype converts and scales in a single
131 # pass.
132 grid = np.divide(raw.reshape(size, size, size, 4), HIST_SCALE,
133 dtype=np.float32)
140 density = np.divide(raw.reshape(size, size, size), HIST_SCALE, dtype=np.float32)
134141 del raw
135 return grid
142 raw_colour = np.array(colour_tex.read(), dtype=np.uint32)
143 colour = np.divide(raw_colour.reshape(half, half, half, 3), HIST_SCALE,
144 dtype=np.float32)
145 del raw_colour
146 return density, colour
136147
137148
138149def auto_bounds(shader, doc, ubos, alias_tex, probe_size=64, dispatches=8,
……@@ -156,8 +167,8 @@ def auto_bounds(shader, doc, ubos, alias_tex, probe_size=64, dispatches=8,
156167 extent = start_extent
157168 while True:
158169 box = ((-extent, -extent, -extent), (extent, extent, extent))
159 coarse = accumulate(shader, doc, ubos, alias_tex, probe_size, box, dispatches)
160 density = coarse[:, :, :, 3]
170 density, _coarse_colour = accumulate(
171 shader, doc, ubos, alias_tex, probe_size, box, dispatches)
161172 hit = np.argwhere(density > 0)
162173 if hit.size == 0:
163174 if extent >= max_extent:
……@@ -222,9 +233,8 @@ def auto_bounds(shader, doc, ubos, alias_tex, probe_size=64, dispatches=8,
222233 probe_lo = tuple(float(lo_w[a] - pad[a]) for a in range(3))
223234 probe_hi = tuple(float(hi_w[a] + pad[a]) for a in range(3))
224235
225 finer = accumulate(shader, doc, ubos, alias_tex, probe_size,
226 (probe_lo, probe_hi), dispatches)
227 finer_density = finer[:, :, :, 3]
236 finer_density, _finer_colour = accumulate(
237 shader, doc, ubos, alias_tex, probe_size, (probe_lo, probe_hi), dispatches)
228238 if not (finer_density > 0).any():
229239 break # the tighter box found nothing; keep what we had
230240
……@@ -286,8 +296,8 @@ def _fit(density, box, probe_size, percentile, margin):
286296 return lo_w - span * margin, hi_w + span * margin
287297
288298
289def write_vdb(grid: np.ndarray, bounds, path: str, name: str = "density",
290 gamma: float = 1.0):
299def write_vdb(density_grid: np.ndarray, colour_grid: np.ndarray, bounds,
300 path: str, name: str = "density", gamma: float = 1.0):
291301 """
292302 Writes density and colour grids to a `.vdb`.
293303
……@@ -296,7 +306,7 @@ def write_vdb(grid: np.ndarray, bounds, path: str, name: str = "density",
296306 """
297307 import openvdb
298308
299 size = grid.shape[0]
309 size = density_grid.shape[0]
300310 lo, hi = bounds
301311
302312 # Per-axis scale AND a translation to `lo`. Both matter and both were missing:
……@@ -323,9 +333,9 @@ def write_vdb(grid: np.ndarray, bounds, path: str, name: str = "density",
323333 # against the same convention it was written with, so it agreed with itself.
324334 # test_bake_orientation measures the baked centroid against an analytically
325335 # known one instead, where it showed up as a clean -0.5 on all three axes.
326 def _transform():
336 def _transform(scale=1.0):
327337 """A fresh Transform per grid; sharing one between grids is asking for it."""
328 sx, sy, sz = ((hi[a] - lo[a]) / size for a in range(3))
338 sx, sy, sz = ((hi[a] - lo[a]) / size * scale for a in range(3))
329339 return openvdb.createLinearTransform(matrix=[
330340 [sx, 0.0, 0.0, 0.0],
331341 [0.0, sy, 0.0, 0.0],
……@@ -357,7 +367,7 @@ def write_vdb(grid: np.ndarray, bounds, path: str, name: str = "density",
357367 # Applied to density ONLY, and only after colour has been divided by the ORIGINAL
358368 # density below - colour is accumulated weighted by density, so dividing it by a
359369 # gamma-lifted density collapses it to black. That mistake renders an empty volume.
360 density = grid[:, :, :, 3]
370 density = density_grid
361371 peak = float(density.max())
362372 if peak <= 0:
363373 raise RuntimeError("nothing was accumulated - the bounds may exclude the fractal")
……@@ -377,26 +387,29 @@ def write_vdb(grid: np.ndarray, bounds, path: str, name: str = "density",
377387 grids.append(dgrid)
378388
379389 # Colour is accumulated weighted by density, so divide it back out.
390 # Colour is HALF resolution and still premultiplied by opacity, so dividing it
391 # back out needs the density of the SAME colour voxel — the sum of its 2x2x2
392 # density block, not a single voxel. Getting that wrong tints the fractal in a
393 # way that looks like a palette bug rather than an indexing one.
394 half = colour_grid.shape[0]
395 usable = half * 2
396 block = density_grid[:usable, :usable, :usable].reshape(
397 half, 2, half, 2, half, 2).sum(axis=(1, 3, 5))
398 safe = np.maximum(block[:, :, :, None], 1e-9)
399
380400 # Same transpose as density - the spatial axes only, leaving the component axis
381401 # last. A colour grid transposed differently from density would tint the wrong
382402 # parts of the fractal, which is harder to spot than a whole-volume rotation.
383 #
384 # Built per SLAB. The whole-array form (divide -> clip -> transpose -> astype)
385 # peaked at four simultaneous copies of a 3-channel volume; slabs bound that to
386 # one plane at a time while producing byte-identical output.
387 colour = np.empty((size, size, size, 3), dtype=np.float32)
388 for z in range(size):
389 plane = grid[z] # (y, x, 4)
390 safe = np.maximum(plane[:, :, 3:4], 1e-9)
391 tinted = np.clip(plane[:, :, :3] / safe, 0.0, 1.0)
392 # grid is [z][y][x]; the target is [x][y][z], so this plane becomes a
393 # column across the first axis.
394 colour[:, :, z, :] = tinted.transpose(1, 0, 2)
403 colour = np.ascontiguousarray(
404 np.clip(colour_grid / safe, 0.0, 1.0).transpose(2, 1, 0, 3))
405
395406 try:
396407 cgrid = openvdb.Vec3SGrid()
397408 cgrid.copyFromArray(colour)
398409 cgrid.name = "color"
399 cgrid.transform = _transform()
410 # Twice the voxel size, because it is half the resolution. VDB grids carry
411 # independent transforms and Blender honours them, so no upsampling.
412 cgrid.transform = _transform(scale=2.0)
400413 grids.append(cgrid)
401414 except Exception:
402415 # A density-only volume is still useful; do not lose the bake over colour.
……@@ -438,10 +451,10 @@ def bake_document(doc, out_path, size=256, dispatches=200, bounds=None,
438451 lo, hi, fit = auto_bounds(shader, doc, ubos, alias_tex)
439452 bounds = (lo, hi)
440453
441 grid = accumulate(shader, doc, ubos, alias_tex, size, bounds, dispatches,
442 invocation_iters)
443 path, peak = write_vdb(grid, bounds, out_path, gamma=gamma)
444 occupied = int((grid[:, :, :, 3] > 0).sum())
454 grid, colour_grid = accumulate(shader, doc, ubos, alias_tex, size, bounds,
455 dispatches, invocation_iters)
456 path, peak = write_vdb(grid, colour_grid, bounds, out_path, gamma=gamma)
457 occupied = int((grid > 0).sum())
445458
446459 # How bright to drive emission, DERIVED rather than guessed. A fixed number cannot
447460 # work: measured across two shipped templates the required scale differs by ~200x
……@@ -458,7 +471,7 @@ def bake_document(doc, out_path, size=256, dispatches=200, bounds=None,
458471 # probe voxels thick - the fixed axis is the thin direction, so the statistic
459472 # reports the shortest possible path and derives an exposure far too dark. The
460473 # camera can look from anywhere, so the metric has to be view-independent.
461 normalised = grid[:, :, :, 3] / peak if peak > 0 else grid[:, :, :, 3]
474 normalised = grid / peak if peak > 0 else grid
462475 per_axis = []
463476 for axis in range(3):
464477 columns = normalised.sum(axis=axis)
blender/test_auto_bounds.py +3 -3
……@@ -73,9 +73,9 @@ class AutoBoundsCentring(unittest.TestCase):
7373 data=gpu.types.Buffer('FLOAT', len(alias), alias))
7474 lo, hi, _fit = ifs_volume.auto_bounds(
7575 self.shader, doc, ubos, tex, refine_passes=refine_passes)
76 grid = ifs_volume.accumulate(self.shader, doc, ubos, tex, SIZE,
77 (lo, hi), DISPATCHES)
78 hit = np.argwhere(grid[:, :, :, 3] > 0) # columns are (z, y, x)
76 grid, _colour = ifs_volume.accumulate(self.shader, doc, ubos, tex, SIZE,
77 (lo, hi), DISPATCHES)
78 hit = np.argwhere(grid > 0) # columns are (z, y, x)
7979 self.assertGreater(hit.size, 0, f"{template} produced an empty bake")
8080 z = hit[:, 0]
8181 return int(z.min()), int(z.max())
blender/test_bake_orientation.py +3 -4
……@@ -437,11 +437,10 @@ class Orientation(unittest.TestCase):
437437 grid.copyToArray(good)
438438 # `write_vdb` expects [z][y][x] and transposes; handing it the array that
439439 # is ALREADY [x][y][z] reproduces exactly the bug this file guards.
440 swapped = np.zeros((VOXELS, VOXELS, VOXELS, 4), dtype=np.float32)
441 swapped[:, :, :, 3] = good
442 swapped[:, :, :, :3] = good[:, :, :, None]
440 half = VOXELS // 2
441 swapped_colour = np.zeros((half, half, half, 3), dtype=np.float32)
443442 path = os.path.join(self.tmp, "transposed.vdb")
444 ifs_volume.write_vdb(swapped, BOUNDS, path)
443 ifs_volume.write_vdb(good, swapped_colour, BOUNDS, path)
445444
446445 bad = openvdb.read(path, "density")
447446 array = np.zeros((VOXELS, VOXELS, VOXELS), dtype=np.float32)
blender/test_volume_axes.py +27 -9
……@@ -43,12 +43,19 @@ NONCUBIC = ((-1.0, -2.0, -3.0), (1.0, 2.0, 3.0))
4343
4444
4545def _grid_with_marker():
46 """A (z, y, x, 4) accumulator with one lit voxel, as `accumulate` returns."""
46 """
47 The pair `accumulate` returns: full-res density and HALF-res colour.
48
49 Colour is premultiplied by density, as the kernel accumulates it, so the
50 marker's colour voxel carries density x hue and write_vdb divides it back out
51 using the 2x2x2 density block.
52 """
4753 import numpy as np
48 grid = np.zeros((S, S, S, 4), dtype=np.float32)
49 grid[MARK_Z, MARK_Y, MARK_X, 3] = 1.0 # density
50 grid[MARK_Z, MARK_Y, MARK_X, :3] = (1.0, 0.0, 0.0) # colour, pre-divided by density
51 return grid
54 density = np.zeros((S, S, S), dtype=np.float32)
55 density[MARK_Z, MARK_Y, MARK_X] = 1.0
56 colour = np.zeros((S // 2, S // 2, S // 2, 3), dtype=np.float32)
57 colour[MARK_Z // 2, MARK_Y // 2, MARK_X // 2] = (1.0, 0.0, 0.0)
58 return density, colour
5259
5360
5461class BakeAxes(unittest.TestCase):
……@@ -66,7 +73,7 @@ class BakeAxes(unittest.TestCase):
6673 import ifs_volume
6774 tmp = tempfile.mkdtemp()
6875 path = os.path.join(tmp, "axes.vdb")
69 ifs_volume.write_vdb(_grid_with_marker(), BOUNDS, path)
76 ifs_volume.write_vdb(*_grid_with_marker(), BOUNDS, path)
7077 grids = openvdb.readAllGridMetadata(path)
7178 names = [g.name for g in grids]
7279 out = {}
……@@ -97,14 +104,25 @@ class BakeAxes(unittest.TestCase):
97104 grids = self._write_and_read()
98105 if "color" not in grids:
99106 self.skipTest("colour grid not written (openvdb Vec3SGrid unavailable)")
107 # Colour is HALF resolution, so the same world point is at half the index.
108 # The axis ORDER is what this checks, and halving preserves it.
100109 acc = grids["color"].getConstAccessor()
101 value = acc.getValue((MARK_X, MARK_Y, MARK_Z))
110 value = acc.getValue((MARK_X // 2, MARK_Y // 2, MARK_Z // 2))
102111 self.assertGreater(value[0], 0.5,
103112 "colour grid disagrees with the density grid's axes")
104113
105114 def test_the_marker_is_asymmetric_enough_to_detect_a_swap(self):
106 """Guards the guard: equal indices would make a transpose invisible."""
115 """
116 Guards the guard: equal indices would make a transpose invisible.
117
118 Checked at BOTH resolutions. Colour is half-res, so a marker whose indices
119 are distinct but collide when halved (say 4, 5, 6 -> 2, 2, 3) would leave
120 the colour check unable to see a swap while still looking like a test.
121 """
107122 self.assertEqual(3, len({MARK_X, MARK_Y, MARK_Z}))
123 self.assertEqual(3, len({MARK_X // 2, MARK_Y // 2, MARK_Z // 2}),
124 "the halved marker indices collide, so the half-res "
125 "colour grid cannot detect a transpose")
108126
109127 # ------------------------------------------------------- placement and scale
110128
……@@ -115,7 +133,7 @@ class BakeAxes(unittest.TestCase):
115133 import ifs_volume
116134 tmp = tempfile.mkdtemp()
117135 path = os.path.join(tmp, "placed.vdb")
118 ifs_volume.write_vdb(_grid_with_marker(), NONCUBIC, path)
136 ifs_volume.write_vdb(*_grid_with_marker(), NONCUBIC, path)
119137 return openvdb.read(path, "density")
120138
121139 def test_voxel_lands_at_its_world_position_not_at_the_origin(self):