OverviewFilesCommitsIssuesDocsReleasesPeers

dc-terrain commits / 839766c

bvh: parallel build, and two hypotheses that did not pay
Measured on the bridged voronoi mountain, 5,160,000 triangles, 16 cores. The SDF's narrow band had already taken the QUERIES to 1.2 s, which is what made the build visible as the remaining cost of Mesh to SDF Grid. build, serial 10543 ms build, parallel 7405 ms 1.42x + centroids precomputed 7438 ms no change PARALLEL BELOW THE TOP LEVELS. Depth < 4 spawns a thread for the left subtree and runs the right inline, so up to 16 subtrees build at once. Safe because the two sides own DISJOINT ranges of tri_indices_ after nth_element has partitioned about mid, triangles_ is immutable after load_arrays, and nodes now come from an atomic bump into an array that cannot move -- bvh_nodes_ is RESIZED to its worst case (2n-1) up front rather than grown by push_back, because a reallocation would move the array under a thread reading it. Shrunk to what was used afterwards. ⛔ 1.42x, NOT the ~10x sixteen threads suggest. So the build is memory bound. ⛔ AND THE OBVIOUS CAUSE WAS NOT THE CAUSE. nth_element's comparator recomputed a centroid from the Triangle on every comparison -- n log n random dereferences into 5.16M 36-byte structs. Precomputing them into a flat float array, which should have been the fix, changed the time by 0.4%. Kept because it is plainly better code, but it is not where the time is. What is left, by elimination, is the per-node AABB pass: ~20 levels each touching every triangle in the node through the PERMUTED tri_indices_, so 100M+ random reads of a 36-byte struct to min/max nine floats. That wants a different builder -- Morton-code/LBVH, which sorts once and builds bottom-up from contiguous order -- not another micro-fix to this one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
839766c22ac20174872134ec260cc8630661296b
by plankatron <fleeky@gmail.com> · 2026-09-08T17:31:56.000Z

Changed files (2) +70 -19

libdc/include/dc_generator_mesh_sdf.h +15 -0
……@@ -2,6 +2,7 @@
22#define DC_GENERATOR_MESH_SDF_H
33
44#include "dc_generator.h"
5#include <atomic>
56#include <vector>
67#include <string>
78#include <cstdint>
……@@ -70,6 +71,20 @@ private:
7071 std::vector<Triangle> triangles_;
7172 std::vector<int> tri_indices_; // for BVH reordering
7273 std::vector<BVHNode> bvh_nodes_;
74 // Nodes are handed out by an ATOMIC BUMP, not push_back: the build is
75 // parallel below the top few levels and push_back mutates the vector's
76 // size. bvh_nodes_ is resized to its worst case (2n-1 for n triangles)
77 // before the build so no allocation can move it under a running thread,
78 // then shrunk to what was used.
79 std::atomic<int> bvh_alloc_{0};
80 // Triangle centroids, precomputed once, 3 floats per triangle.
81 //
82 // nth_element's comparator used to recompute a centroid from the Triangle
83 // on EVERY comparison -- n log n random dereferences into a 5.16M-element
84 // array of 9-float structs, which is why threading the build only bought
85 // 1.42x: it is memory bound, not compute bound. Comparing floats in a flat
86 // array instead keeps the sort in cache.
87 std::vector<float> tri_centroids_;
7388 float bounds_min_[3] = {};
7489 float bounds_max_[3] = {};
7590 bool loaded_ = false;
libdc/src/generators/dc_generator_mesh_sdf.cpp +55 -19
……@@ -1,4 +1,5 @@
11#include "dc_generator_mesh_sdf.h"
2#include <thread>
23
34#include <fstream>
45#include <sstream>
……@@ -223,11 +224,30 @@ void DCGeneratorMeshSDF::build_bvh() {
223224 tri_indices_.resize(n);
224225 std::iota(tri_indices_.begin(), tri_indices_.end(), 0);
225226
227 // WORST CASE UP FRONT. A binary tree whose leaves hold at least one
228 // triangle has at most n leaves and n-1 interior nodes, so 2n-1 covers any
229 // mesh. Resized rather than reserved because the parallel build hands out
230 // nodes by atomic bump and never touches the vector's size -- if it could
231 // grow, a reallocation would move the array under a thread reading it.
232 // Centroids once, not once per comparison. See the note on tri_centroids_.
233 tri_centroids_.resize((size_t)n * 3);
234 for (int i = 0; i < n; i++) {
235 const Triangle& t = triangles_[i];
236 tri_centroids_[(size_t)i * 3 + 0] = (t.v0.x + t.v1.x + t.v2.x) / 3.0f;
237 tri_centroids_[(size_t)i * 3 + 1] = (t.v0.y + t.v1.y + t.v2.y) / 3.0f;
238 tri_centroids_[(size_t)i * 3 + 2] = (t.v0.z + t.v1.z + t.v2.z) / 3.0f;
239 }
240
226241 bvh_nodes_.clear();
227 bvh_nodes_.reserve(2 * n);
242 bvh_nodes_.resize((size_t)(2 * n > 0 ? 2 * n : 1));
243 bvh_alloc_.store(1);
228244
229 bvh_nodes_.push_back(BVHNode());
230245 build_bvh_recursive(0, 0, n, 0);
246
247 // Give back what the split did not need. The bound is generous by design;
248 // a median split usually lands nearer n/2 nodes than 2n.
249 bvh_nodes_.resize((size_t)bvh_alloc_.load());
250 bvh_nodes_.shrink_to_fit();
231251}
232252
233253void DCGeneratorMeshSDF::build_bvh_recursive(int node_idx, int start, int count, int depth) {
……@@ -270,20 +290,13 @@ void DCGeneratorMeshSDF::build_bvh_recursive(int node_idx, int start, int count,
270290 if (extent[1] > extent[axis]) axis = 1;
271291 if (extent[2] > extent[axis]) axis = 2;
272292
273 auto centroid_component = [&](int tri_idx, int ax) -> float {
274 const Triangle& t = triangles_[tri_idx];
275 const float v[3][3] = {{t.v0.x, t.v0.y, t.v0.z},
276 {t.v1.x, t.v1.y, t.v1.z},
277 {t.v2.x, t.v2.y, t.v2.z}};
278 return (v[0][ax] + v[1][ax] + v[2][ax]) / 3.0f;
279 };
280
293 const float* cent = tri_centroids_.data();
281294 int mid = start + count / 2;
282295 std::nth_element(tri_indices_.begin() + start,
283296 tri_indices_.begin() + mid,
284297 tri_indices_.begin() + start + count,
285 [&](int a, int b) {
286 return centroid_component(a, axis) < centroid_component(b, axis);
298 [cent, axis](int a, int b) {
299 return cent[(size_t)a * 3 + axis] < cent[(size_t)b * 3 + axis];
287300 });
288301
289302 int left_count = mid - start;
……@@ -293,18 +306,41 @@ void DCGeneratorMeshSDF::build_bvh_recursive(int node_idx, int start, int count,
293306 return; // Can't split, keep as leaf
294307 }
295308
296 int left_idx = (int)bvh_nodes_.size();
297 bvh_nodes_.push_back(BVHNode());
298 int right_idx = (int)bvh_nodes_.size();
299 bvh_nodes_.push_back(BVHNode());
309 const int left_idx = bvh_alloc_.fetch_add(2);
310 const int right_idx = left_idx + 1;
300311
301 // Re-fetch after potential reallocation
302312 bvh_nodes_[node_idx].left = left_idx;
303313 bvh_nodes_[node_idx].right = right_idx;
304314 bvh_nodes_[node_idx].tri_count = 0; // interior node
305315
306 build_bvh_recursive(left_idx, start, left_count, depth + 1);
307 build_bvh_recursive(right_idx, mid, right_count, depth + 1);
316 // ── PARALLEL BELOW THE TOP LEVELS ───────────────────────────────────────
317 //
318 // Measured 2026-09-08 on the bridged voronoi mountain: this build is 10.5 s
319 // of a 12.7 s Mesh to SDF Grid, over 5.16M triangles, on one core. The
320 // narrow band had already taken the QUERIES to 1.2 s, which is what made
321 // the build visible as the remaining cost.
322 //
323 // Safe because the two subtrees own DISJOINT ranges of tri_indices_ --
324 // nth_element has already partitioned [start, start+count) about mid, and
325 // neither side touches the other's half again -- triangles_ is immutable
326 // after load_arrays, and nodes come from the atomic bump above into an
327 // array that cannot move.
328 //
329 // Only the first few levels spawn: depth 4 is up to 16 subtrees, which
330 // matches a typical core count, and below that a thread costs more than
331 // the subtree does. The count guard keeps small meshes entirely serial.
332 const int PARALLEL_DEPTH = 4;
333 const int PARALLEL_MIN_TRIS = 8192;
334 if (depth < PARALLEL_DEPTH && count > PARALLEL_MIN_TRIS) {
335 std::thread left_thread([this, left_idx, start, left_count, depth]() {
336 build_bvh_recursive(left_idx, start, left_count, depth + 1);
337 });
338 build_bvh_recursive(right_idx, mid, right_count, depth + 1);
339 left_thread.join();
340 } else {
341 build_bvh_recursive(left_idx, start, left_count, depth + 1);
342 build_bvh_recursive(right_idx, mid, right_count, depth + 1);
343 }
308344}
309345
310346// ---------------------------------------------------------------------------