OverviewFilesCommitsIssuesDocsReleasesPeers

hyperswarm-peer commits / 96bcaea

test: verify the holder path against a real testnet, and assert the token
§16 said test_self_announce's 10/10 proved less than it looked like: with no DHT nothing answers an announce, self_holder_count() is 0 throughout, and the recording, the ping round and the verdict were exercised only in their EMPTY form. Every claim about the populated path was untested. test_self_announce_live.cpp, driven by run_self_announce_live.js against an 8-node hyperdht/testnet: holders=7 tokens=7 ping_rounds=4 announces=1 5/5 Seven real nodes accepted the announce and were recorded from their replies; the ping round ran against them; they answered, so the failure path did not fire and there was one announce rather than a storm. ⛔ The assertion that actually distinguishes the fix is the TOKEN, and the first version of this gate did not have it. The routing-table guess this replaced ALSO populated the node list — from table().closest() — so "holders > 0" passes either way. It wrote memset(token, 0, 32). tokens == holders is the only externally visible difference between recording real committers and guessing at neighbours. Adds HyperDht::self_holder_tokens() for that assertion. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
96bcaea75a4a49952e2db5a78c8a88016858b7ca
by plankatron <fleeky@gmail.com> · 2026-09-10T16:53:16.000Z

Changed files (7) +263 -0

docs/PARITY-hyperswarm.md +26 -0
……@@ -1030,3 +1030,29 @@ named here so nobody reads 10/10 as more than it is.
10301030abort.
10311031
10321032Suite: 13 binaries green.
1033
1034### 16.1 The populated-holder path, verified against a REAL testnet
1035
1036§16 flagged that `test_self_announce.cpp`'s 10/10 proved less than it looked
1037like: with no DHT, nothing answers an announce, `self_holder_count()` is 0
1038throughout, and the recording, ping round and verdict were exercised only in
1039their EMPTY form. That debt is now paid.
1040
1041`test/test_self_announce_live.cpp`, driven by `test/run_self_announce_live.js`
1042against an 8-node `hyperdht/testnet`:
1043
1044 holders=7 tokens=7 ping_rounds=4 announces=1
1045 5/5 passed
1046
1047- 7 real nodes accepted the announce and were recorded FROM THEIR REPLIES
1048- the ping round ran against those nodes
1049- they answered, so the failure path did NOT fire — 1 announce, no storm
1050
1051⛔ **The assertion that actually distinguishes the fix is the TOKEN.** The
1052routing-table guess this replaced also populated the node list — from
1053`table().closest()` — so `holders > 0` passes either way. It wrote
1054`memset(token, 0, 32)`. `tokens == holders` is the only externally visible
1055difference between recording real committers and guessing at neighbours, and
1056the first version of this gate did not assert it. Added.
1057
1058Suite: 14 binaries.
src/hyperdht.cpp +11 -0
……@@ -379,6 +379,17 @@ int HyperDht::self_holder_count() const {
379379 return it == _announced_topics.end() ? 0 : it->second.node_count;
380380}
381381
382int HyperDht::self_holder_tokens() const {
383 auto it = _announced_topics.find(_topic_hex(_self_target));
384 if (it == _announced_topics.end()) return 0;
385 int n = 0;
386 for (int i = 0; i < it->second.node_count; i++) {
387 static const uint8_t zero[32] = { 0 };
388 if (memcmp(it->second.nodes[i].token, zero, 32) != 0) n++;
389 }
390 return n;
391}
392
382393void HyperDht::_ping_self_holders(uint64_t now_ms) {
383394 auto it = _announced_topics.find(_topic_hex(_self_target));
384395 if (it == _announced_topics.end() || it->second.node_count == 0) return;
src/hyperdht.h +5 -0
……@@ -189,6 +189,11 @@ public:
189189 uint64_t self_announce_count() const { return _self_announce_count; }
190190 uint64_t self_ping_rounds() const { return _self_ping_rounds; }
191191 int self_holder_count() const;
192 // How many recorded holders carry a NON-ZERO token. The routing-table guess
193 // this replaced wrote `memset(token, 0, 32)`, so a zero token is the
194 // signature of the old behaviour and the only thing that tells the two
195 // apart from outside. See PARITY-hyperswarm.md §15.
196 int self_holder_tokens() const;
192197
193198 // Process time-based events.
194199 void tick(uint64_t now_ms);
test/build_tests.sh +10 -0
……@@ -279,3 +279,13 @@ g++ -o test_peer_topics \
279279 $OBJ/dht_connector.o $OBJ/swarm_manager.o \
280280 $UDX_OBJS $LDFLAGS
281281echo " OK: test_peer_topics"
282
283# ---- test_self_announce_live ----
284# Needs a real testnet: run via `node run_self_announce_live.js`, not directly.
285echo "Building test_self_announce_live..."
286g++ $CXXFLAGS -o $OBJ/test_self_announce_live.o test_self_announce_live.cpp
287g++ -o test_self_announce_live \
288 $OBJ/test_self_announce_live.o $OBJ/noise_ik.o $OBJ/noise_handshake.o \
289 $OBJ/noise_stream.o $OBJ/dht_rpc.o $OBJ/holepuncher.o $OBJ/hyperdht.o \
290 $UDX_OBJS $LDFLAGS
291echo " OK: test_self_announce_live"
test/run_self_announce_live.js +31 -0
……@@ -0,0 +1,31 @@
1#!/usr/bin/env node
2// Stands up a real JS DHT testnet and runs test_self_announce_live against it.
3//
4// node run_self_announce_live.js
5//
6// The point is real RESPONDERS. test_self_announce.cpp has no DHT, so nothing
7// answers an announce and every claim about the populated-holder path went
8// untested. See docs/PARITY-hyperswarm.md §16.
9
10const { spawn } = require('child_process')
11const path = require('path')
12const createTestnet = require('hyperdht/testnet')
13
14const BIN = path.join(__dirname, 'test_self_announce_live')
15
16async function main () {
17 const testnet = await createTestnet(8, { teardown: () => {} })
18 const bootstrap = testnet.bootstrap.map(b => `${b.host}:${b.port}`).join(',')
19 console.log(`testnet up (${testnet.nodes.length} nodes): ${bootstrap}\n`)
20
21 const code = await new Promise((resolve) => {
22 const p = spawn(BIN, [bootstrap], { stdio: 'inherit' })
23 p.on('exit', resolve)
24 })
25
26 await testnet.destroy()
27 console.log(`\ntest_self_announce_live exit=${code}`)
28 process.exit(code)
29}
30
31main().catch((err) => { console.error(err); process.exit(1) })
test/test_self_announce_live +0 -0
(no content changes — binary or rename)
test/test_self_announce_live.cpp +180 -0
……@@ -0,0 +1,180 @@
1// Gate: against a REAL testnet, a listening peer records the nodes that took its
2// announce and pings them.
3//
4// test_self_announce_live <host:port[,host:port...]>
5// driven by test/run_self_announce_live.js, which stands up the testnet
6//
7// ⛔ WHY THIS EXISTS. test_self_announce.cpp is 10/10 and proves less than it
8// looks like it does: it has no DHT, so nothing ever answers an announce,
9// self_holder_count() is 0 throughout, and the recording, the ping round and the
10// verdict are all exercised only in their EMPTY form. Every claim about the
11// POPULATED path was untested. See docs/PARITY-hyperswarm.md §16.
12//
13// What this asserts, all of which requires real responders:
14// - holders are recorded from real commit replies (§15: they used to be
15// guessed from table().closest() with a zeroed token)
16// - the recorded token is NOT zero
17// - the ping round actually runs against them
18// - the holders answer, so the re-announce is NOT triggered spuriously
19
20#include <cstdio>
21#include <cstdlib>
22#include <cstring>
23#include <string>
24#include <vector>
25
26#include <sodium.h>
27
28#include "dht_rpc.h"
29#include "hyperdht.h"
30
31extern "C" {
32#include <udx.h>
33#include <uv.h>
34}
35
36struct Ctx {
37 uv_loop_t loop;
38 udx_t udx;
39 udx_socket_t socket;
40 uv_timer_t timer;
41 uv_timer_t deadline;
42 DhtRpc rpc;
43 HyperDht dht;
44 bool listened = false;
45 int holders_seen = 0;
46 bool token_nonzero = false;
47 uint64_t rounds_at_first_holder = 0;
48 uint64_t announces_at_first_holder = 0;
49 uint64_t stop_at = 0;
50};
51
52static void on_socket_recv(udx_socket_t *sock, ssize_t read_len, const uv_buf_t *buf,
53 const struct sockaddr *from) {
54 Ctx *ctx = static_cast<Ctx *>(sock->data);
55 if (!ctx || read_len <= 0) return;
56 const uint8_t *data = (const uint8_t *)buf->base;
57 if ((size_t)read_len >= 4 && (data[0] == DHT_REQUEST_ID || data[0] == DHT_RESPONSE_ID)) {
58 ctx->rpc.on_message(data, (size_t)read_len, (const struct sockaddr_in *)from);
59 }
60}
61
62static void on_tick(uv_timer_t *h) {
63 Ctx *ctx = static_cast<Ctx *>(h->data);
64 const uint64_t now = uv_now(&ctx->loop);
65 ctx->rpc.tick(now);
66 ctx->dht.tick(now);
67
68 if (!ctx->listened && ctx->rpc.is_bootstrapped()) {
69 ctx->listened = true;
70 printf(" bootstrapped, announcing self\n");
71 ctx->dht.listen();
72 }
73
74 const int holders = ctx->dht.self_holder_count();
75 if (holders > ctx->holders_seen) {
76 if (ctx->holders_seen == 0) {
77 ctx->rounds_at_first_holder = ctx->dht.self_ping_rounds();
78 ctx->announces_at_first_holder = ctx->dht.self_announce_count();
79 // Give the ping loop several rounds with holders present.
80 ctx->stop_at = now + 12000;
81 printf(" first holder recorded at %llums\n", (unsigned long long)now);
82 }
83 ctx->holders_seen = holders;
84 }
85
86 if (ctx->stop_at && now >= ctx->stop_at) uv_stop(&ctx->loop);
87}
88
89static void on_deadline(uv_timer_t *h) {
90 uv_stop(&static_cast<Ctx *>(h->data)->loop);
91}
92
93int main(int argc, char **argv) {
94 if (sodium_init() < 0) return 1;
95 if (argc < 2) { printf("usage: %s <host:port,...>\n", argv[0]); return 2; }
96
97 std::vector<std::string> boots;
98 {
99 std::string all(argv[1]);
100 size_t p = 0;
101 while (p <= all.size()) {
102 size_t c = all.find(',', p);
103 if (c == std::string::npos) c = all.size();
104 if (c > p) boots.push_back(all.substr(p, c - p));
105 p = c + 1;
106 }
107 }
108 std::vector<const char *> bootv;
109 for (auto &b : boots) bootv.push_back(b.c_str());
110 printf(" bootstrap: %s\n", argv[1]);
111
112 Ctx *ctx = new Ctx();
113 uv_loop_init(&ctx->loop);
114 udx_init(&ctx->loop, &ctx->udx, nullptr);
115 udx_socket_init(&ctx->udx, &ctx->socket, nullptr);
116 ctx->socket.data = ctx;
117
118 struct sockaddr_in bind_addr;
119 uv_ip4_addr("0.0.0.0", 0, &bind_addr);
120 udx_socket_bind(&ctx->socket, (struct sockaddr *)&bind_addr, 0);
121 udx_socket_recv_start(&ctx->socket, on_socket_recv);
122
123 ctx->rpc.init(&ctx->socket, &ctx->loop);
124 ctx->rpc.set_bootstrap(bootv.data(), (int)bootv.size());
125
126 struct sockaddr_in local_addr;
127 int alen = sizeof(local_addr);
128 udx_socket_getsockname(&ctx->socket, (struct sockaddr *)&local_addr, &alen);
129 IPv4Address local;
130 memcpy(local.ip, &local_addr.sin_addr.s_addr, 4);
131 local.port = ntohs(local_addr.sin_port);
132 ctx->rpc.set_local_address(local);
133
134 Ed25519KeyPair kp;
135 crypto_sign_keypair(kp.public_key, kp.secret_key);
136 ctx->dht.init(&ctx->rpc, kp);
137
138 ctx->rpc.bootstrap();
139
140 uv_timer_init(&ctx->loop, &ctx->timer);
141 ctx->timer.data = ctx;
142 uv_timer_start(&ctx->timer, on_tick, 0, 20);
143
144 uv_timer_init(&ctx->loop, &ctx->deadline);
145 ctx->deadline.data = ctx;
146 uv_timer_start(&ctx->deadline, on_deadline, 45000, 0);
147
148 uv_run(&ctx->loop, UV_RUN_DEFAULT);
149
150 int checks = 0, failures = 0;
151 auto check = [&](bool c, const char *what) {
152 checks++;
153 if (c) { printf(" PASS: %s\n", what); return; }
154 failures++;
155 printf(" FAIL: %s\n", what);
156 };
157
158 printf("\n");
159 check(ctx->listened, "reached the testnet and announced");
160 check(ctx->holders_seen > 0,
161 "recorded at least one node that ACCEPTED the announce");
162 // ⛔ THE ONE THAT DISTINGUISHES THE FIX. The routing-table guess this
163 // replaced also populated the node list — from table().closest() — so
164 // "holders > 0" passes either way. It wrote `memset(token, 0, 32)`, so a
165 // real token is the only externally visible difference.
166 check(ctx->dht.self_holder_tokens() == ctx->holders_seen,
167 "every recorded holder carries a REAL token, not a zeroed one");
168 check(ctx->dht.self_ping_rounds() > ctx->rounds_at_first_holder,
169 "ping rounds ran against the recorded holders");
170 // Holders on a healthy testnet answer, so the failure path must NOT fire.
171 check(ctx->dht.self_announce_count() <= ctx->announces_at_first_holder + 1,
172 "live holders answered, so no spurious re-announce storm");
173
174 printf("\n holders=%d tokens=%d ping_rounds=%llu announces=%llu\n",
175 ctx->holders_seen, ctx->dht.self_holder_tokens(),
176 (unsigned long long)ctx->dht.self_ping_rounds(),
177 (unsigned long long)ctx->dht.self_announce_count());
178 printf("\n%d/%d passed\n", checks - failures, checks);
179 return failures ? 1 : 0;
180}