5974dc0 · hyperswarm-peer

Changed files (5) +229 -3

docs/PARITY-hyperswarm.md +64 -0

…	…	@@ -793,3 +793,67 @@ timeout, so a fix that merely fired twice would not pass.
793	793	⚠ Not addressed: the explicit-error abort path (§11.1), where JS aborts
794	794	immediately on a relay error and we still wait the timeout. Separate change,
795	795	separate gate.
	796
	797	## 12. Slice 8 — defect B, first half: the self-announce happened ONCE (FIXED)
	798
	799	§9 described B as "no Announcer, no liveness ping" and framed it as a missing
	800	component. Reading `server.js` found something sharper and more damaging first.
	801
	802	⛔ THE ANNOUNCER IS PER-SERVER, NOT PER-TOPIC. `server.js:173-175`:
	803
	804	this.target = unslabbedHash(keyPair.publicKey)
	805	this._announcer = new Announcer(this.dht, keyPair, this.target, opts)
	806
	807	So JS has TWO separate announce mechanisms and I had been treating them as one:
	808
	809	\| \| keyed by \| cadence \|
	810	\|---\|---\|---\|
	811	\| hyperdht `Announcer` \| the server's OWN public key \| 3s ping loop, ~5 min re-announce \|
	812	\| hyperswarm `PeerDiscovery` \| a swarm topic \| 10 min refresh \|
	813
	814	Ours had the topic one and, for the server one:
	815
	816	void HyperDht::listen() {
	817	_listening = true;
	818	crypto_generichash(self_target, 32, _keypair.public_key, 32, nullptr, 0);
	819	announce(self_target); // ONCE. Nothing ever refreshed it.
	820	}
	821
	822	The self-announce goes straight to `announce()` and is not a `TopicDiscovery`,
	823	so `_tick_topics` never saw it. DHT records expire. When ours did,
	824	`find_peer(our_pk)` found nothing and the peer was unreachable until the process
	825	restarted — while its TOPIC records, which ARE refreshed, kept it looking
	826	discoverable. That is the "still discoverable, no longer reachable" symptom
	827	stated exactly, and it is a plainer cause than the missing ping loop.
	828
	829	Fix: `_tick_self_announce()` re-announces every `SELF_ANNOUNCE_UPDATE_MS`
	830	(300000 — JS's `100 x 3s` background loop).
	831
	832	Gate: `test/test_self_announce.cpp`, 7/7. Asserts the first announce, the
	833	schedule, no announce before the interval, one after it, that it KEEPS going
	834	(the defect was "once, ever"), and that many ticks inside one interval do not
	835	multiply announces — a record refreshed 100x/second would be a different defect.
	836
	837	fixed 7/7
	838	mutated 5/7 (_tick_self_announce commented out of tick)
	839
	840	Suite: 12 binaries green.
	841
	842	### 12.1 Second half — the liveness ping — NOT built
	843
	844	JS pings every node holding its record every 3s and re-announces the moment
	845	fewer than `MIN_ACTIVE = 3` answer (`announcer.js:104-125`). We now re-announce
	846	on a timer but never check whether the holders are alive, so losing them costs
	847	up to `SELF_ANNOUNCE_UPDATE_MS` instead of ~3 seconds.
	848
	849	`DhtRpc::ping(addr, tag)` exists, so this is buildable. What it needs:
	850
	851	- keep the nodes that took each announce (the `AnnouncedTopic` struct already
	852	stores address + token; it needs to be populated for the SELF target too)
	853	- a ping round on a 3s schedule with tag→response accounting
	854	- re-announce immediately when answered < min(held, 3)
	855
	856	⏳ Also still absent: the three-generation `_serverRelays` cycling and
	857	`refresh-chain.js`'s reverse hash chain (we always send `refresh` empty).
	858	Neither is required for the record to stay alive; both are needed for full
	859	parity.

src/hyperdht.cpp +30 -3

…	…	@@ -350,9 +350,32 @@ void HyperDht::listen() {
350	350	// Self-announce under BLAKE2b(own_public_key).
351	351	// Matches JS: Server.listen() → dht.announce(hash(keyPair.publicKey), keyPair, ...)
352	352	// This allows connect_to_peer(pk) → find_peer(BLAKE2b(pk)) to find us.
353		uint8_t self_target[32];
354		crypto_generichash(self_target, 32, _keypair.public_key, 32, nullptr, 0);
355		announce(self_target);
	353	crypto_generichash(_self_target, 32, _keypair.public_key, 32, nullptr, 0);
	354	announce(_self_target);
	355	_self_announce_count++;
	356
	357	// ⛔ AND SCHEDULE THE NEXT ONE. This used to announce once and stop.
	358	//
	359	// The self-announce is not a TopicDiscovery — it goes straight to announce()
	360	// — so _tick_topics never saw it and nothing refreshed it. DHT records
	361	// expire. When ours did, find_peer(our public key) found nothing and this
	362	// peer was unreachable until the process restarted, while its topic records
	363	// (which ARE refreshed) kept it looking discoverable.
	364	//
	365	// JS keeps this alive in a dedicated component, hyperdht's Announcer, one
	366	// per server, targeting hash(keyPair.publicKey) — see SELF_ANNOUNCE_UPDATE_MS.
	367	_self_announce_at = _last_tick_ms + SELF_ANNOUNCE_UPDATE_MS;
	368	}
	369
	370	void HyperDht::_tick_self_announce(uint64_t now_ms) {
	371	if (!_listening \|\| _self_announce_at == 0) return;
	372	if (now_ms < _self_announce_at) return;
	373
	374	DHT_DISC("[HyperDht] re-announcing self %02x%02x%02x%02x...\n",
	375	_self_target[0], _self_target[1], _self_target[2], _self_target[3]);
	376	announce(_self_target);
	377	_self_announce_count++;
	378	_self_announce_at = now_ms + SELF_ANNOUNCE_UPDATE_MS;
356	379	}
357	380
358	381	// ---- Tick ----
…	…	@@ -370,6 +393,10 @@ void HyperDht::tick(uint64_t now_ms) {
370	393
371	394	_last_tick_ms = now_ms;
372	395
	396	// Keep our own record alive. Without this a listening peer is announced
	397	// exactly once and becomes unreachable when that record expires.
	398	_tick_self_announce(now_ms);
	399
373	400	// Reclaim stale handshakes. Without this the pool is one-way and a
374	401	// long-lived responder eventually stops answering anyone (§8).
375	402	_sweep_handshakes(now_ms);

src/hyperdht.h +24 -0

…	…	@@ -24,6 +24,18 @@ static constexpr int HYPERDHT_MAX_QUERIES = 16;
24	24	static constexpr int HYPERDHT_MAX_PEERS = 64;
25	25
26	26	// Topic refresh timing — matches JS peer-discovery.js.
	27	// ⛔ THE SERVER'S OWN RECORD MUST BE KEPT ALIVE, and it was announced once.
	28	// listen() announced BLAKE2b(our public key) and nothing ever refreshed it: the
	29	// self-announce is not a TopicDiscovery, so _tick_topics never saw it. DHT
	30	// records expire, and when ours did, find_peer(our_pk) found nothing and the
	31	// peer was unreachable until it restarted.
	32	//
	33	// JS gives this its own component — hyperdht's Announcer (lib/announcer.js),
	34	// constructed per SERVER against hash(keyPair.publicKey) (server.js:173-175),
	35	// distinct from hyperswarm's topic-level PeerDiscovery. Its _background loop is
	36	// 100 iterations of a 3s sleep, so a full re-announce every ~5 minutes.
	37	static constexpr uint64_t SELF_ANNOUNCE_UPDATE_MS = 300000; // 5 min, = JS 100 x 3s.
	38
27	39	static constexpr uint64_t TOPIC_REFRESH_INTERVAL_MS = 600000; // 10 minutes.
28	40	static constexpr uint64_t TOPIC_REFRESH_JITTER_MS = 120000; // 0–2 minutes random.
29	41	static constexpr uint64_t TOPIC_REFRESH_GRACE_MS = 30000; // 30s grace (laptop wake).
…	…	@@ -159,6 +171,12 @@ public:
159	171	// Start listening for incoming handshakes.
160	172	void listen();
161	173
	174	// When the self-announce is next due, or 0 when not listening. For gates:
	175	// a record that is never refreshed is invisible from the outside until it
	176	// expires, which is why this is observable at all.
	177	uint64_t self_announce_due_at() const { return _self_announce_at; }
	178	uint64_t self_announce_count() const { return _self_announce_count; }
	179
162	180	// Process time-based events.
163	181	void tick(uint64_t now_ms);
164	182
…	…	@@ -325,6 +343,12 @@ private:
325	343	// synthetic value — and a fix that cannot be driven on demand cannot be
326	344	// proven. 0 until the first tick; the sweep adopts those (see the .cpp).
327	345	uint64_t _last_tick_ms = 0;
	346
	347	// Self-announce (see SELF_ANNOUNCE_UPDATE_MS).
	348	uint8_t _self_target[32] = { 0 };
	349	uint64_t _self_announce_at = 0; // 0 = not listening
	350	uint64_t _self_announce_count = 0;
	351	void _tick_self_announce(uint64_t now_ms);
328	352	void _finalize_handshake(DhtHandshake *hs);
329	353
330	354	// Respond to any non-internal query with closer nodes + token + optional value.

test/build_tests.sh +11 -0

…	…	@@ -256,3 +256,14 @@ g++ -o test_find_peer_exhausted \
256	256	$OBJ/dht_connector.o \
257	257	$UDX_OBJS $LDFLAGS
258	258	echo " OK: test_find_peer_exhausted"
	259
	260	# ---- test_self_announce ----
	261	# A listening peer must keep re-announcing its OWN key. See
	262	# docs/PARITY-hyperswarm.md §9.
	263	echo "Building test_self_announce..."
	264	g++ $CXXFLAGS -o $OBJ/test_self_announce.o test_self_announce.cpp
	265	g++ -o test_self_announce \
	266	$OBJ/test_self_announce.o $OBJ/noise_ik.o $OBJ/noise_handshake.o \
	267	$OBJ/noise_stream.o $OBJ/dht_rpc.o $OBJ/holepuncher.o $OBJ/hyperdht.o \
	268	$UDX_OBJS $LDFLAGS
	269	echo " OK: test_self_announce"

test/test_self_announce.cpp +100 -0

…	…	@@ -0,0 +1,100 @@
	1	// Gate: a listening peer must keep re-announcing its OWN key.
	2	//
	3	// docs/PARITY-hyperswarm.md §9. listen() announced BLAKE2b(our public key)
	4	// exactly once, and nothing refreshed it — the self-announce is not a
	5	// TopicDiscovery, so _tick_topics never saw it. DHT records expire; when ours
	6	// did, find_peer(our_pk) found nothing and the peer was unreachable until
	7	// restart, while its TOPIC records (which are refreshed) kept it looking
	8	// discoverable. That is the "still discoverable, no longer reachable" symptom.
	9	//
	10	// JS keeps this alive in hyperdht's Announcer, one per SERVER, targeting
	11	// hash(keyPair.publicKey) (server.js:173-175) — a component distinct from
	12	// hyperswarm's topic-level PeerDiscovery, and one we do not have.
	13	//
	14	// RED before the fix: self_announce_count() stays at 1 for ever.
	15
	16	#include <cstdio>
	17	#include <cstring>
	18
	19	#include <sodium.h>
	20
	21	#include "hyperdht.h"
	22
	23	extern "C" {
	24	#include <udx.h>
	25	#include <uv.h>
	26	}
	27
	28	static int checks = 0, failures = 0;
	29	static void check(bool cond, const char *what) {
	30	checks++;
	31	if (cond) { printf(" PASS: %s\n", what); return; }
	32	failures++;
	33	printf(" FAIL: %s\n", what);
	34	}
	35
	36	int main() {
	37	if (sodium_init() < 0) { printf("sodium_init failed\n"); return 1; }
	38
	39	uv_loop_t loop;
	40	uv_loop_init(&loop);
	41	udx_t udx;
	42	udx_init(&loop, &udx, nullptr);
	43	udx_socket_t sock;
	44	udx_socket_init(&udx, &sock, nullptr);
	45	struct sockaddr_in addr;
	46	uv_ip4_addr("127.0.0.1", 0, &addr);
	47	udx_socket_bind(&sock, (struct sockaddr *)&addr, 0);
	48
	49	DhtRpc rpc;
	50	rpc.init(&sock, &loop);
	51
	52	Ed25519KeyPair kp;
	53	crypto_sign_keypair(kp.public_key, kp.secret_key);
	54
	55	HyperDht dht;
	56	dht.init(&rpc, kp);
	57
	58	uint64_t now = 1000;
	59	dht.tick(now); // establish the clock before listening
	60
	61	check(dht.self_announce_count() == 0, "nothing announced before listen()");
	62
	63	dht.listen();
	64	check(dht.self_announce_count() == 1, "listen() announces our own key");
	65	check(dht.self_announce_due_at() > now, "and schedules the next one");
	66
	67	// Well short of the interval: nothing more should happen.
	68	now += SELF_ANNOUNCE_UPDATE_MS / 2;
	69	dht.tick(now);
	70	check(dht.self_announce_count() == 1,
	71	"no re-announce before the interval elapses");
	72
	73	// Past it: exactly one more.
	74	now += SELF_ANNOUNCE_UPDATE_MS;
	75	dht.tick(now);
	76	check(dht.self_announce_count() == 2,
	77	"re-announced once the interval elapsed");
	78
	79	// And it keeps going — the defect was that it happened once, ever.
	80	for (int i = 0; i < 5; i++) {
	81	now += SELF_ANNOUNCE_UPDATE_MS;
	82	dht.tick(now);
	83	}
	84	check(dht.self_announce_count() == 7,
	85	"keeps re-announcing on every interval, not just once");
	86
	87	// Ticking repeatedly inside one interval must not multiply announces:
	88	// a record refreshed 100x a second would be a different defect.
	89	uint64_t before = dht.self_announce_count();
	90	for (int i = 0; i < 50; i++) dht.tick(now + i);
	91	check(dht.self_announce_count() == before,
	92	"many ticks inside one interval produce no extra announces");
	93
	94	printf("\n%d/%d passed\n", checks - failures, checks);
	95
	96	udx_socket_close(&sock);
	97	uv_run(&loop, UV_RUN_NOWAIT);
	98	uv_loop_close(&loop);
	99	return failures ? 1 : 0;
	100	}

hyperswarm-peer commits / 5974dc0

Changed files (5) +229 -3