8e904e0 · hyper-social

Changed files (5) +329 -9

docs/MVP.md +58 -9

…	…	@@ -82,14 +82,62 @@ The MVP is small on purpose, so the things we will want next must be **additions
82	82	\| more media levels \| the pyramid may be partial; a level is added to a pointer \|
83	83	\| new room kinds \| `maxReach` gains a row; deny-by-default means an unknown kind is already safe \|
84	84
85		⛔ Would be a rewrite — so decide before the MVP ships:
86
87		1. The address form. `favela://<room>/<position>` vs keeping a type segment. Records are
88		immutable in their addressing once they exist (§8 forbids rewriting).
89		2. Interactions: frontmatter-only records, or pointer-valued log entries? Changes what is on
90		disk for every edge ever written.
91		3. The facet shape. If it ships bsky-shaped it leaks forever, because every consumer will have
92		been written against it.
	85	⛔ Would be a rewrite — so decide before the MVP ships. Status 2026-09-06, after
	86	hyper-social #61: one settled and shipped, two reopened by measurement.
	87
	88	1. The address form — ⚠ REOPENED, and more deeply than it was asked.
	89	It was briefly settled as `favela://<roomKey>/<collection>/<position>`. Both halves failed the same
	90	day:
	91
	92	- `roomKey` derives from the DID — `device-node.js:127`, `crypto.hash([NS('room-key'), did])`.
	93	Not from the identity root, as `interaction.md` §3.4 claimed. A DID rotation would change every
	94	address in the room: precisely what the scheme exists to prevent.
	95	- `position` is a view coordinate, not an address. keet resolves cross-references by `oplog`
	96	(`chat.js:30,31,62`); `index` at `:49` is a per-thread counter. A counter assigned inside `apply()`
	97	is a function of the linearization, and apply is replayed on every reorg — so it is not final until
	98	its block is indexed, which under N=2 quorum may be never.
	99	- And §4 of #61 removes the premise under both: the room is not a primitive.
	100
	101	Candidate, unbuilt: authority = the feed's identity root; last segment = `oplog` — collision-free
	102	by construction, known at append time, stable under reorg, and needing no assign-in-apply.
	103
	104	*2. Interactions — ⚠ REOPENED, in the direction of less* mechanism.**
	105	Bodiless-edge-as-log-entry was decided because edge volume in a popular room is unbounded. With
	106	interactions written into the author's own feed (#61 §4), volume is bounded by that author's
	107	activity and the optimisation stops being forced. Prefer one representation — every record is a file
	108	— until a measurement says otherwise.
	109
	110	⛔ One thing from that work is not reopened, because it is a measured live defect rather than a
	111	decision: an inline pointer-set value carries no `$oid`, so `_resolve` → `ensureLocal` returns null
	112	on every peer but its author (`overlay-repo-store.js:149,176`; loser-club, `pointer-substrate.js:120-133`).
	113	`library.md` §10.1.1.
	114
	115	3. The facet shape — ✅ SETTLED AND SHIPPED 2026-09-06.
	116	`strip()` emits `{ byteStart, byteEnd, type, value }`, flat, `type ∈ link\|mention\|tag`. Test-first:
	117	the harness was flipped and went red for the right reason, then `lib/strip.js`. **41 tests / 2,324
	118	asserts** green across five suites.
	119
	120	Cost measured before deciding: favela-markdown has zero consumers, and the 36 vectors were
	121	already in the native form — the harness carried an `expandFacet()` building bsky shape from them.
	122	So the change was three emitter functions, the sort rank, the overlap loop, and deleting the expander;
	123	the vectors did not change.
	124
	125	What it removed: a mention's target is a `favela://` URI and it travelled in a field named `did`
	126	(`markdown-strip.md` §3.4 had flagged this and left it). `project(rec,'bsky')` now owns the bsky
	127	shape and is the only place that can resolve a mention to a DID, so that field finally holds one. A
	128	test greps `strip()`'s output across all 36 vectors for `$type`, `app.bsky`, `richtext`, `features`,
	129	`"did"`.
	130
	131	## 4.1 ⚠ And §1's component list is itself under revision
	132
	133	#61 §4 proposes feed and document as the only primitives, with a room as the emergent name
	134	for a dense cluster of matings — which removes the writer-set scaling wall by construction and makes
	135	counts per-observer. §1 and §3 of this document are written against the room-as-container model and
	136	have not been reconciled.
	137
	138	⛔ And §1's acceptance criterion is already violated: the substrate is
	139	`atproto-store/lib/pointer-substrate.js` and `atproto-room/lib/state-base.js`. Packaging, not
	140	architecture — but it fails today as written.
93	141
94	142	## 5. Not in the MVP, deliberately
95	143
…	…	@@ -102,4 +150,5 @@ by anything above — which is the point of keeping the core small.
102	150	- It does not delete the atproto stack. It stops loading it. ~17,900 lines stay where they are.
103	151	- It does not make the core faster or leaner as a goal — leanness is a consequence, and by itself a
104	152	bad reason (hyper-social #28: plumbing must not outrank the product).
105		- It does not settle §4's three rewrite-class questions. Those need a decision, not a plan.
	153	- It does not settle §4's three rewrite-class questions. **One is settled and shipped (the facet
	154	shape); two were reopened by measurement on 2026-09-06** — see hyper-social #61.

docs/notes/FOR-LINAS.md +91 -0

…	…	@@ -0,0 +1,91 @@
	1	# A social network as sections and connectors — where we'd want your read
	2
	3	## What we have
	4
	5	A peer-to-peer stack on Hypercore: append-only signed logs, multi-writer logs with a
	6	deterministic `apply()`, content-addressed bytes. No servers, no global index, no relay.
	7
	8	A user's posts are a log. Their devices are its writers; identity is a lookup inside `apply()`,
	9	so N devices act as one member without sharing a key. A post is a markdown file; the log holds a
	10	pointer, the bytes live content-addressed and any peer holding them can serve them.
	11
	12	That much runs today.
	13
	14	## Where we got stuck, and why we think it's your problem
	15
	16	We kept trying to make a "room" — a shared log with a membership list — carry the social structure:
	17	audience, addressing, discovery, moderation. Every one of those broke, and they broke in the same
	18	direction. Membership is retroactive (admit someone, they get the whole history); leaving revokes
	19	nothing (they already replicated); and interacting inside someone's room means becoming a writer in
	20	it, so a popular post makes every liker a hypercore.
	21
	22	The thing that survived the demolition is that a connection is not an object, it's an event. A
	23	join is two typed half-edges mating. Membership is the receipt. We had been modelling the receipt
	24	and calling it a primitive.
	25
	26	So the current shape is:
	27
	28	- germ — a log (a person's feed, a co-authored document)
	29	- connector — a typed, directed half-edge: `writes-to+`, `readers-welcome−`, `lists−`, `bridges-to+`
	30	- mating — an actual join/follow/listing/bridge event, which is the only thing that crosses a boundary
	31	- disjunct — the multiset of connector types a germ has actually mated with
	32
	33	And then the claim we'd like shot at: **a germ's type should be inferred from its disjunct, after
	34	the fact, never declared.** Two feeds are the same kind of thing if their connector patterns
	35	co-occur — even with no members in common.
	36
	37	That last clause is the whole bet. Our existing design computes MI between two rooms from
	38	membership overlap, which can only ever find rooms that already intersect. It cannot tell a
	39	dev community from a social one across a partition. Disjunct statistics should be able to.
	40
	41	## Our one data point that this transfers
	42
	43	We ran your formalism on a non-language domain — carving Quake level geometry into reusable pieces.
	44	It sat at 1.0 germs/disjunct for nine attempts: every piece unique, no structure found. The
	45	change that moved it was not the statistics, it was what the connector is typed by — geometry
	46	(the shape of the opening) → element (what the opening is). That gave 1.78 germs/disjunct, and
	47	pieces reused ≥5 times went from 0% to 32.3%.
	48
	49	So we believe the method works outside language, and we believe the typing decision is the whole
	50	game. We do not know what to type a social connector by.
	51
	52	## Where we think it differs from your corpus work, and this is the real question
	53
	54	Your statistics come from a large shared corpus. Ours cannot: **each peer only ever observes the
	55	matings it participates in or replicates.** Every peer computes a different, partial, biased sample.
	56	There is no global view and we don't want one — that's the privacy property, not a limitation we're
	57	trying to remove. Each peer is a local section; global consistency is not required and probably not
	58	achievable.
	59
	60	Concretely:
	61
	62	1. Does (germ, disjunct) MI survive being computed on a per-observer partial sample? Or does the
	63	sheaf-of-local-samples destroy the statistic you rely on? We can measure agreement between peers;
	64	we don't know what agreement level would mean the structure is real rather than an artifact of
	65	each peer's vantage point.
	66
	67	2. What should a social connector be typed by? Our failure mode in Quake was typing by the
	68	observable surface instead of by role. The social analogue of "geometry" is probably membership;
	69	the analogue of "element" is probably something like the kind of act the mating was. We can't
	70	tell from the inside.
	71
	72	3. Jaccard or MI at small N? You found Jaccard beat MI. Our N is small by construction — a peer
	73	might see 10³ matings, not 10⁸. Is there a regime where one is clearly right?
	74
	75	4. How many observations before disjuncts stabilise? If the answer is 10⁵ we have the right idea
	76	at the wrong scale, and we'd rather know that now.
	77
	78	5. A structural question. Kleinberg's group-structure model says greedy local routing reaches any
	79	node in O(log²n) hops if long-range links follow 1/(size of smallest common group). Nested
	80	communities are exactly that structure. Is it legitimate to use the inferred community hierarchy
	81	as the routing metric — i.e. can the thing the statistics discover also be the thing you navigate
	82	by, or is that circular in a way that will bite us?
	83
	84	## What we're not asking
	85
	86	We're not asking for architecture. The storage layer is settled and boring. The question is whether
	87	the sparse-symbolic connector formalism is the right lens for structure that is **observed
	88	per-peer, incomplete, and never aggregated** — and if so, what the connector should be typed by.
	89
	90	If the answer is "your sample is too thin, this needs a corpus", that's the most useful thing you
	91	could tell us, and we'd rather hear it before we build it.

docs/notes/FOR-P2P-FOLKS.md +124 -0

…	…	@@ -0,0 +1,124 @@
	1	# A p2p social network with no relay — the short version
	2
	3	For people who know Hypercore, SSB, atproto or ActivityPub. No new maths, no jargon.
	4
	5	## The two primitives
	6
	7	A feed. One person's log. Their devices are the writers — an autobase, so a phone and a laptop
	8	both append and it stays one identity, no shared key. Posts are markdown files: the log holds a
	9	pointer, the bytes are content-addressed in an overlay and any peer that has them can serve them.
	10
	11	A document. A log with several deliberate writers and a key. A wiki page, a DM, a private group.
	12	You create it and invite people.
	13
	14	That's it. There is no third thing.
	15
	16	## What we removed, and why
	17
	18	We spent a long time trying to make "rooms" the container — a shared log with a member list that
	19	carries privacy, addressing, and discovery all at once. Three things killed it, and if you've built
	20	on replicated logs you'll recognise all three:
	21
	22	1. Adding a member gives them the entire history. The log is append-only and it replicates whole.
	23	That's fine for a chat group and completely wrong as a privacy model for a feed.
	24	2. Removing a member revokes nothing. They already have the blocks. "Enforced by replication"
	25	only ever meant future writes.
	26	3. Interacting in someone's room means becoming a writer in it. A popular post therefore turns
	27	every liker into a hypercore that every replicator has to carry. That's a wall, not a slope.
	28
	29	The fix was to stop calling it a room. Call it a shared document and none of the above is
	30	surprising — everyone who has it has all of it, you don't enter it, you obtain a copy.
	31
	32	## So where do likes and replies go?
	33
	34	In your own feed. You like Ana's post by writing a record in your log that names hers.
	35
	36	- No writer-set growth. Ana's feed still has exactly her devices in it.
	37	- Ana sees your like if she replicates you — i.e. if she follows you back — or via a doorbell (below).
	38	- Counts are per-observer and we don't hide that. Your client shows what it can see. It never
	39	invents a global total, because there isn't one.
	40
	41	If you want a real comment thread with agreed ordering and agreed counts, that's a document, made
	42	on purpose, with a bounded set of writers. Which is what multi-writer logs are actually good at.
	43
	44	## Following
	45
	46	Following is subscribing to a feed. You replicate the log, and pull post bytes by hash when you
	47	actually open something — so you pay for what you read, not for everything they ever wrote.
	48
	49	Your timeline is a local projection over the feeds you hold. There is no relay, no AppView, no
	50	indexer. Two people legitimately see different timelines, and that's the design, not a bug.
	51
	52	## Strangers
	53
	54	A stranger has nothing you've subscribed to, so they can't reach you by replication. They ring a
	55	doorbell: an optimistic append to your feed's topic that your device either accepts or drops. Keet
	56	already does exactly this for join requests, so it's existing, tested code. A human accepting is the
	57	spam gate.
	58
	59	## Discovery, honestly
	60
	61	Bootstrap is an invite link. Same as keet, same as every p2p system that actually onboards
	62	people. No infrastructure required.
	63
	64	Then a registry. A registry is just a feed whose posts are listings of other feeds. You follow
	65	one and you have a directory. Anyone can run one. You can be in several.
	66
	67	Two things kept deliberately separate:
	68
	69	- Subscribing to a registry reveals nothing about you.
	70	- Being listed in one publishes your key. That has to be your choice — a registry that could
	71	list you unilaterally would make discoverability opt-out.
	72
	73	And two honest limits:
	74
	75	- Delisting stops propagation, it doesn't retract. The listing already replicated.
	76	- Unlisted is not private. Your feed announces on a DHT topic derived from its key, so anyone
	77	with the key finds you. A registry is a phone book, not a gate. Private means encrypted —
	78	a document with a key. The UI must never let those two look alike.
	79
	80	Search in three tiers: local full-text over what you already replicate (works offline, ships
	81	first); search over registry listings you subscribe to (metadata only, public by design); global
	82	search — not built, and we're not pretending otherwise.
	83
	84	## The part that's speculative
	85
	86	Nothing above needs it, but it's where we think this goes.
	87
	88	Every join, follow, listing and bridge is an event where two things fit together. Over time each
	89	feed has a pattern of what it fits with. Feeds with similar patterns are probably similar feeds —
	90	and crucially, that can be true with no members in common, which membership-overlap clustering can
	91	never detect.
	92
	93	If that works, communities don't get created, they get noticed: your client says "these twelve
	94	feeds clearly form a thing", and a person names it. The naming is what makes it real; we're
	95	explicit that the software must not auto-name clusters, because naming is a political act.
	96
	97	And the same structure that makes communities visible is what makes the network navigable — you can
	98	route a query hop by hop toward a target using only local knowledge, provided some people are members
	99	of far-apart communities. Which means the cross-community folks aren't noise, they're the road
	100	network.
	101
	102	⚠ All of that is unproven and we've written down what would falsify it: if independent peers don't
	103	converge on the same clusters from their different vantage points, then "emergent community" is a
	104	story and not a mechanism. That test runs against our existing data before any new code.
	105
	106	## How it compares
	107
	108	- vs SSB — closest relative. Feeds you replicate by subscription, no global index. Differences:
	109	content-addressed bytes so media survives its origin, sparse replication so you don't carry
	110	everything, and multi-device without a shared key.
	111	- vs atproto — atproto is effectively one giant public room: every repo public, one relay
	112	firehosing everything, filtering done afterwards by labelers and feed generators. The subscribable,
	113	stackable filtering is genuinely good and worth stealing. The single global aggregator isn't
	114	something we can or want to reproduce.
	115	- vs ActivityPub — AP has real addressing (`to`/`cc`), which we map onto cleanly. What it lacks
	116	is history and content addressing: an object is an HTTPS URL and dies with its host.
	117	- Both are projections here, not the substrate. A post is a markdown file; the atproto record and
	118	the AS2 object are derived from it. Adding a network never changes the record.
	119
	120	## Status
	121
	122	The storage layer runs. The one new thing is a post record whose body is a pointer to a markdown
	123	file. Following, likes, replies, multi-device identity and the doorbell are all existing keet
	124	machinery being used as-is.

hyper-feed/node_modules +1 -0

…	…	@@ -0,0 +1 @@
	1	/storage/projects/code/hypercore-kb/hyper-social/hyper-board/node_modules

scripts/atproto-handle-resolution-test.sh +55 -0

…	…	@@ -0,0 +1,55 @@
	1	#!/usr/bin/env bash
	2	# atproto-handle-resolution-test.sh — empirically validate how bsky resolves a
	3	# handle, and prove whether a TLS cert on the handle hostname is required.
	4	#
	5	# Thesis under test (issue #85): a subdomain handle can be verified by a
	6	# `_atproto.<handle>` DNS TXT record ALONE — no A record, no web server, no TLS
	7	# cert on the subdomain. Only the shared PDS host (named in the DID doc) needs
	8	# TLS. So wildcard DNS + apex-only cert is sufficient for unlimited pretty
	9	# verified handles.
	10	#
	11	# Usage: ./atproto-handle-resolution-test.sh <handle> [<handle> ...]
	12	set -u
	13	DNS=1.1.1.1
	14	APPVIEW=https://public.api.bsky.app
	15	PLC=https://plc.directory
	16
	17	check () {
	18	h="$1"; echo "══════════════════════════════════════════════"; echo "HANDLE: $h"
	19	# 1. What bsky's OWN resolver returns (this is the arbiter)
	20	did=$(curl -s --max-time 8 "$APPVIEW/xrpc/com.atproto.identity.resolveHandle?handle=$h" \
	21	\| sed -n 's/."did":"$[^"]$".*/\1/p')
	22	echo " [1] bsky resolveHandle → ${did:-<FAILED>}"
	23	[ -z "$did" ] && { echo " (bsky does not resolve this handle)"; return; }
	24
	25	# 2. Which mechanism backs it?
	26	txt=$(dig +short TXT _atproto.$h @$DNS 2>/dev/null \| tr -d '"' \| tr '\n' ' ')
	27	a=$(dig +short A $h @$DNS 2>/dev/null \| tr '\n' ' ')
	28	echo " [2] _atproto.$h TXT → [${txt:-<none>}]"
	29	echo " $h A record → [${a:-<NONE — hostname has no web endpoint>}]"
	30
	31	# 3. Is the HTTPS well-known / TLS path even usable?
	32	wk=$(curl -sS --max-time 8 "https://$h/.well-known/atproto-did" 2>&1 \| head -c 80)
	33	echo " [3] https://$h/.well-known/atproto-did → ${wk:-<empty>}"
	34
	35	# 4. Bidirectional half: does the DID doc claim this handle back?
	36	aka=$(curl -s --max-time 8 "$PLC/$did" \| sed -n 's/."alsoKnownAs":\[$[^]]$\].*/\1/p')
	37	echo " [4] DID doc alsoKnownAs → ${aka:-<none>}"
	38
	39	# 5. Where is the actual data (PDS)? — should be a SHARED host, not this handle
	40	pds=$(curl -s --max-time 8 "$PLC/$did" \
	41	\| sed -n 's/."serviceEndpoint":"$[^"]$".*/\1/p' \| head -1)
	42	echo " [5] PDS serviceEndpoint → ${pds:-<none>}"
	43
	44	# verdict
	45	if [ -n "$txt" ] && [ -z "$a" ]; then
	46	echo " ⇒ VERDICT: verified by DNS TXT ONLY. No A record, no server, NO TLS cert on '$h'."
	47	elif [ -n "$txt" ]; then
	48	echo " ⇒ VERDICT: DNS TXT present (authoritative; resolver tries it first). Cert on '$h' not required."
	49	else
	50	echo " ⇒ VERDICT: no TXT — this handle uses the HTTPS well-known method (needs a cert on '$h')."
	51	fi
	52	}
	53
	54	for h in "$@"; do check "$h"; done
	55	echo "══════════════════════════════════════════════"

hyper-social commits / 8e904e0

Changed files (5) +329 -9