| … | … | @@ -4,6 +4,61 @@ Working notes from reading the accessible implementations, 2026-08-18. The |
| 4 | 4 | question was "which existing addon should we take notes from", and the useful |
| 5 | 5 | answer turned out to be "not an addon". |
| 6 | 6 | |
| 7 | --- | |
| 8 | ||
| 9 | ## Correction (2026-08-18, later the same day) — the attribute contract | |
| 10 | ||
| 11 | An audit of Blender's own branches found that **design task #159470's prose and | |
| 12 | its implementation disagree**, and the recommendation below was written against | |
| 13 | the prose. What the code actually reads, on branch `gsplat` in `blender/blender` | |
| 14 | (`source/blender/io/common/intern/gsplat.cc`): | |
| 15 | ||
| 16 | | attribute | type | contents | | |
| 17 | |---|---|---| | |
| 18 | | `color` | float4 | `.rgb` = the **raw** degree-0 SH coefficient (`f_dc`); `.a` = opacity, **sigmoid already applied** | | |
| 19 | | `scale` | float3 | **activated** — `exp()` already applied, i.e. the real sigma | | |
| 20 | | `rotation` | quaternion | w-first | | |
| 21 | | `sh_0` … `sh_14` | float3 | degrees 1–3, flat running index | | |
| 22 | ||
| 23 | **There is no `opacity` attribute and no `sh_0_0`.** Degree 0 lives entirely in | |
| 24 | `color`, and the shader does the `0.5 + C0 * f_dc` mapping — EEVEE evaluates | |
| 25 | `srgb_to_linear(0.282094792 * color.rgb + ... + 0.5)`. | |
| 26 | ||
| 27 | So the "degree-0 SH is not raw colour" claim below is right, and Blender's answer | |
| 28 | is that the attribute holds the coefficient rather than the colour. | |
| 29 | ||
| 30 | Two of our three attributes were already correct by luck: `scale` is written | |
| 31 | unlogged (activated) and `rotation` is w-first identity. The mismatch is opacity. | |
| 32 | ||
| 33 | **Why `to_pointcloud` still does not write `color` by default.** That name is the | |
| 34 | one this project has already been burned by — as a POINT attribute it renders | |
| 35 | BLACK, which is why colour lives in `ifs_color` and why a test forbids the name. | |
| 36 | Blender's splat path requires exactly that name, so the two contracts collide. | |
| 37 | Nothing that ships reads the splat one (it is 5.3-alpha, on a branch, no | |
| 38 | milestone, no buildbot builds), so it is available as | |
| 39 | `to_pointcloud(..., splat_contract=True)` rather than imposed on the rendering | |
| 40 | that works today. | |
| 41 | ||
| 42 | **Other corrections from the same audit:** | |
| 43 | ||
| 44 | - The PoC named in the task (`tmp_gsplat_io_poc`) is stale. Work moved to the | |
| 45 | official **`gsplat`** branch, plus `gsplat_cycles_v2` (Sergey) and | |
| 46 | `local_gsplatting_shenanigans` (Mark van de Ruit, EEVEE). | |
| 47 | - **Neither Blender renderer sorts either.** They use dithered / stochastic | |
| 48 | opacity — a third valid answer alongside our "emission commutes, so no sort". | |
| 49 | - Cycles' path is **not** camera-aligned billboards: it is a true ray-traced | |
| 50 | ellipsoid primitive with stochastic transparency, and `intern/cycles/` is | |
| 51 | **Apache-2.0**, so that part is permissively reusable. | |
| 52 | - `render_as` is a **datablock property**, not an attribute — RNA-settable | |
| 53 | (`pointcloud.render_as ∈ {'POINTS','SPLATS'}`) but *not* settable from inside a | |
| 54 | Geometry Nodes tree. | |
| 55 | - **BlendSplat** (codeberg.org/soerensc/BlendSplat-Library, CC-BY-4.0) was missed | |
| 56 | below entirely: pure Geometry Nodes + shader assets, Blender 5.1+, EEVEE *and* | |
| 57 | Cycles, no binaries, and it has a node category for *generating* splats. It is | |
| 58 | the closest thing to a shipping answer that works on 5.2 **today**. | |
| 59 | ||
| 60 | --- | |
| 61 | ||
| 7 | 62 | ## What we read, and the verdicts |
| 8 | 63 | |
| 9 | 64 | ### splatviz-blender (xy-gao) — accessible, wrong reference |
| … | … | @@ -126,9 +181,36 @@ it is safe. |
| 126 | 181 | 2. **A 3DGS PLY writer** from `ifs_points`, using the table above. Zero |
| 127 | 182 | dependencies, ships no native code, and lets any external viewer confirm our |
| 128 | 183 | representation is right before we commit to any addon. |
| 129 | 3. Keep the PointCloud attributes (`scale`, `rotation`, `opacity`) aligned with | |
| 130 | Blender design task #159470, so the native path renders our clouds when it | |
| 131 | ships. | |
| 184 | 3. ~~Keep the PointCloud attributes (`scale`, `rotation`, `opacity`) aligned with | |
| 185 | Blender design task #159470~~ — **superseded, see the Correction above.** | |
| 186 | `scale` and `rotation` are already right; opacity belongs in `color.a` with the | |
| 187 | colour as a raw SH coefficient, and that shape is written only under | |
| 188 | `splat_contract=True` because `color` renders black for points. | |
| 189 | ||
| 190 | ## If we implement more of this, read these three files | |
| 191 | ||
| 192 | From the audit, ranked. All public, all readable today: | |
| 193 | ||
| 194 | - `not_mark/blender:local_gsplatting_shenanigans` → | |
| 195 | `source/blender/draw/intern/shaders/draw_gsplat_lib.bsl.hh` — the cleanest | |
| 196 | camera-aligned billboard construction available, by a Blender GPU dev, with | |
| 197 | every paper equation cited by number. It credits `antimatter15/splat` for the | |
| 198 | eigendecomposition, i.e. the same technique we already chose. GPL-2.0-or-later. | |
| 199 | - `.../draw_cache_impl_gsplat.cc` — per-point attribute plumbing at production | |
| 200 | scale: reading `color`/`scale`/`rotation`/`sh_N` off a PointCloud, quantising, | |
| 201 | and uploading as buffer textures. The same no-SSBO problem we solve with | |
| 202 | `texelFetch`. | |
| 203 | - `Sergey/blender:gsplat_cycles_v2` → | |
| 204 | `intern/cycles/kernel/geom/gsplat_intersect.h` — ~180 lines, **Apache-2.0**, | |
| 205 | gaussians as a real ray-traced primitive. Only relevant if we ever want F12 | |
| 206 | rather than viewport. | |
| 207 | ||
| 208 | KIRI's addon is worth 30 minutes for one thing only: `viewport_render.py`, | |
| 209 | `shader_system.py` and its `vert.glsl`/`frag.glsl` are a complete, working, | |
| 210 | pure-`gpu`-module `SpaceView3D` draw-handler splat renderer — the exact shape of | |
| 211 | `ifs_viewport`. Ignore the other 127 Python files and the 738 MB release. Note its | |
| 212 | licence is self-contradictory (Apache in `LICENSE`, GPL in the manifest), so | |
| 213 | resolve that before copying a line. | |
| 132 | 214 | |
| 133 | 215 | ## Sources |
| 134 | 216 |
| … | … | @@ -63,6 +63,32 @@ POINTS_WIDTH = 4096 |
| 63 | 63 | # volume grid. Both halves of that cost a wrong conclusion during the volume work. |
| 64 | 64 | COLOR_ATTRIBUTE = "ifs_color" |
| 65 | 65 | |
| 66 | # The degree-0 spherical-harmonics basis constant, 1/(2*sqrt(pi)). | |
| 67 | # | |
| 68 | # Verified against THREE primary sources that agree: graphdeco-inria's | |
| 69 | # `gaussian_model.py`, antimatter15/splat's `convert.py`, and Blender's own EEVEE | |
| 70 | # splat shader on the `gsplat` branch, which evaluates | |
| 71 | # `srgb_to_linear(0.282094792 * color.rgb + ... + 0.5)`. | |
| 72 | SH_C0 = 0.28209479177387814 | |
| 73 | ||
| 74 | ||
| 75 | def encode_sh_dc(colour): | |
| 76 | """ | |
| 77 | Colour -> the RAW degree-0 SH coefficient that splat renderers store. | |
| 78 | ||
| 79 | Degree-0 SH is not colour, and this is the mapping that catches people out: | |
| 80 | the renderer computes `0.5 + C0 * coefficient`, so going the other way needs | |
| 81 | BOTH the offset and the constant. An encoder that drops the 0.5 still round | |
| 82 | trips against its own decoder while rendering everything washed toward grey, | |
| 83 | which is why the test checks a hand-computed value as well as a round trip. | |
| 84 | """ | |
| 85 | return (np.asarray(colour, dtype=np.float32) - 0.5) / SH_C0 | |
| 86 | ||
| 87 | ||
| 88 | def decode_sh_dc(coefficient): | |
| 89 | """The inverse of `encode_sh_dc`, i.e. what the shader does.""" | |
| 90 | return 0.5 + SH_C0 * np.asarray(coefficient, dtype=np.float32) | |
| 91 | ||
| 66 | 92 | |
| 67 | 93 | def build_shader(): |
| 68 | 94 | """Compiles the points variant of the kernel.""" |
| … | … | @@ -319,7 +345,8 @@ def emit_from_file(document_path, capacity=1_000_000, max_dispatches=400, |
| 319 | 345 | shader=shader) |
| 320 | 346 | |
| 321 | 347 | |
| 322 | def to_pointcloud(positions, colors, radius, name="IFS Points"): | |
| 348 | def to_pointcloud(positions, colors, radius, name="IFS Points", | |
| 349 | splat_contract=False): | |
| 323 | 350 | """ |
| 324 | 351 | Builds a Blender PointCloud from the emitted arrays. |
| 325 | 352 | |
| … | … | @@ -395,4 +422,36 @@ def to_pointcloud(positions, colors, radius, name="IFS Points"): |
| 395 | 422 | rotation[:, 0] = 1.0 |
| 396 | 423 | attribute('rotation', 'QUATERNION').data.foreach_set('value', rotation.ravel()) |
| 397 | 424 | |
| 425 | # ── Blender's implemented splat contract, opt-in ──────────────────────── | |
| 426 | # | |
| 427 | # Read from the `gsplat` branch (2026-08-18) rather than from design task | |
| 428 | # #159470's prose, which they DISAGREE on. What both its EEVEE and Cycles | |
| 429 | # paths actually read is: | |
| 430 | # | |
| 431 | # color float4 .rgb = the RAW degree-0 SH coefficient (f_dc), | |
| 432 | # .a = opacity, sigmoid ALREADY applied | |
| 433 | # scale float3 activated (exp already applied) <- ours already is | |
| 434 | # rotation quat w-first <- ours already is | |
| 435 | # sh_0..14 float3 degrees 1-3, flat running index | |
| 436 | # | |
| 437 | # There is NO `opacity` attribute and no `sh_0_0` in the implementation. | |
| 438 | # | |
| 439 | # WHY THIS IS OPT-IN. `color` is the one attribute name this project has | |
| 440 | # already been burned by: as a POINT attribute it renders BLACK, which is why | |
| 441 | # the colour lives in `ifs_color` and why a test forbids the name outright. | |
| 442 | # Blender's splat path nevertheless requires exactly that name. The two | |
| 443 | # contracts collide, and nothing that ships today reads the splat one — it is | |
| 444 | # 5.3-alpha, on a branch, with no milestone — so imposing it by default would | |
| 445 | # risk the rendering that works now to satisfy a renderer nobody can run. | |
| 446 | # Ask for it when targeting that path. | |
| 447 | # | |
| 448 | # `opacity` is kept alongside regardless: Blender does not read it, but it is | |
| 449 | # OUR per-point data surfaced under an obvious name, and it costs one array. | |
| 450 | if splat_contract: | |
| 451 | splat_colour = np.empty((count, 4), dtype=np.float32) | |
| 452 | splat_colour[:, :3] = encode_sh_dc(colors[:, :3]) | |
| 453 | splat_colour[:, 3] = opacity | |
| 454 | attribute('color', 'FLOAT_COLOR').data.foreach_set( | |
| 455 | 'color', splat_colour.ravel()) | |
| 456 | ||
| 398 | 457 | return cloud |
| … | … | @@ -192,6 +192,66 @@ class SplatAttributes(unittest.TestCase): |
| 192 | 192 | self.assertLess(float(np.abs(norms - 1.0).max()), 1e-6, 'quaternion is not unit length') |
| 193 | 193 | |
| 194 | 194 | |
| 195 | # ------------------------------------- Blender's own splat contract (opt-in) | |
| 196 | ||
| 197 | def test_the_sh_encoding_round_trips(self): | |
| 198 | """ | |
| 199 | Degree-0 SH is NOT raw colour. Blender's EEVEE splat shader evaluates | |
| 200 | `0.282094792 * color.rgb + 0.5` (and the reference PLY writer agrees), so a | |
| 201 | desired colour has to be encoded as `(c - 0.5) / C0` going in. Asserted as a | |
| 202 | round trip plus one hand-checked value, because an encoder that forgot the | |
| 203 | 0.5 offset still round-trips against its own decoder. | |
| 204 | """ | |
| 205 | import numpy as np | |
| 206 | import ifs_points | |
| 207 | self.assertAlmostEqual(0.28209479177387814, ifs_points.SH_C0, places=12) | |
| 208 | for c in (0.0, 0.25, 0.5, 1.0): | |
| 209 | self.assertAlmostEqual( | |
| 210 | c, ifs_points.decode_sh_dc(ifs_points.encode_sh_dc(c)), places=6) | |
| 211 | # 0.5 is the neutral point: it encodes to exactly zero. | |
| 212 | self.assertAlmostEqual(0.0, float(ifs_points.encode_sh_dc(0.5)), places=9) | |
| 213 | ||
| 214 | def test_the_splat_contract_is_off_by_default(self): | |
| 215 | """ | |
| 216 | `color` is POISONED as a point attribute — it renders black, which is why | |
| 217 | the colour lives in `ifs_color`. Blender's splat path nonetheless requires | |
| 218 | that exact name, so the two contracts collide and the splat one must be | |
| 219 | asked for rather than imposed on the rendering that works today. | |
| 220 | """ | |
| 221 | cloud, _pos, _col = self._cloud() | |
| 222 | self.assertNotIn('color', [a.name for a in cloud.attributes], | |
| 223 | 'the splat contract wrote `color` without being asked — ' | |
| 224 | 'that name renders black for points') | |
| 225 | ||
| 226 | def test_the_splat_contract_writes_colour_and_opacity_together(self): | |
| 227 | """ | |
| 228 | Blender's implemented contract has NO `opacity` attribute: degree-0 SH is in | |
| 229 | `color.rgb` as the RAW coefficient and opacity is `color.a`, already | |
| 230 | sigmoid-activated. This is the shape both its EEVEE and Cycles paths read. | |
| 231 | """ | |
| 232 | import numpy as np | |
| 233 | import ifs_points | |
| 234 | positions, colors = _sample_arrays() | |
| 235 | cloud = ifs_points.to_pointcloud(positions, colors, radius=0.01, | |
| 236 | name='splat-contract', splat_contract=True) | |
| 237 | self.assertIn('color', cloud.attributes) | |
| 238 | got = _read(cloud, 'color', 'color', 4) | |
| 239 | want_rgb = ifs_points.encode_sh_dc(colors[:, :3]) | |
| 240 | self.assertLess(float(np.abs(got[:, :3] - want_rgb).max()), 1e-5, | |
| 241 | 'color.rgb is not the raw degree-0 SH coefficient') | |
| 242 | self.assertLess(float(np.abs(got[:, 3] - colors[:, 3]).max()), 1e-6, | |
| 243 | 'color.a is not the activated opacity') | |
| 244 | ||
| 245 | def test_the_splat_contract_leaves_the_points_path_intact(self): | |
| 246 | """Opting in must ADD to what renders today, not replace it.""" | |
| 247 | import ifs_points | |
| 248 | positions, colors = _sample_arrays() | |
| 249 | cloud = ifs_points.to_pointcloud(positions, colors, radius=0.01, | |
| 250 | name='splat-both', splat_contract=True) | |
| 251 | for name in (ifs_points.COLOR_ATTRIBUTE, 'radius', 'scale', 'rotation'): | |
| 252 | self.assertIn(name, cloud.attributes, f'{name} was lost') | |
| 253 | ||
| 254 | ||
| 195 | 255 | if __name__ == "__main__": |
| 196 | 256 | result = unittest.main(argv=sys.argv[:1], exit=False, verbosity=2).result |
| 197 | 257 | sys.exit(0 if result.wasSuccessful() else 1) |