0f76a32 · favela-markdown

favela-markdown commits / 0f76a32

README: show the record and the code, not a table of exports

Yesterday's rewrite fixed the stale "Nothing is built" but left a file that described
the package instead of teaching it. A reader got an export table and a test count and
still had no idea what a record looks like or how to call anything.

It opens with the record now — the actual bytes, fence and all — because "a post is a
markdown file" is the entire thesis and one example carries it better than a paragraph.
Then a worked snippet: parse, render, strip, with real output.

Every line of that output was produced by running it, not written from memory. Two
things came out of doing that rather than assuming. A `favela://<root>/` with a
trailing slash parses as nothing (parts length 2 matches no form), so a plausible-looking
example would have shipped broken. And `serialize(parse(x)) === x` is false for
hand-written input — the law holds for CANONICAL input, and the useful phrasing is that
anything else gets normalised and is byte-stable forever after. The README now says
that, which is the version a caller can act on.

Added the favela:// scheme, which had no mention at all despite being the thing most
likely to be got wrong: three forms told apart by shape, and an authority that is an
identity root and never a feed key. Both are 64 hex characters so the bytes cannot tell
you which you were handed, and recordUri refuses at mint time because a URI written
into a record can never be corrected.

Kept the two governing properties and the why-this-exists argument, moved below the
usage. Kept the note that three consumers still do not import this package, since a
single codec with three private reimplementations downstream is not yet a single codec.

77/77 tests, 2432 asserts.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

0f76a32303b9403f1ea75895b4fad34c7136dc11

by plankatron <fleeky@gmail.com> · 2026-09-09T17:57:23.000Z

Changed files (1) +105 -44

README.md +105 -44

…	…	@@ -1,70 +1,131 @@
1	1	# favela-markdown
2	2
3		A post is a markdown document. atproto and ActivityPub are projections derived from it
4		on read, not the thing itself.
	3	A post is a markdown file. Not a signed CBOR record that happens to contain text — a file, with
	4	a JSON header and a body, that a person could open in an editor and still recognise.
5	5
6		This exists because atproto became favela's canonical P2P path by default rather than by
7		decision — importing MST derivation, DAG-CBOR byte-exactness, lexicon field ceilings and
8		blob CIDs sealed inside signed records into a network that only ever wanted to interface
9		with atproto. That machinery is useful at the bridge/relay layer. It should not sit
10		underneath every full-mode peer.
	6	atproto and ActivityPub are projections derived from it when someone reads it, not the thing
	7	itself. This library is the codec: parse a record, render it, strip it down to plain text, check it,
	8	and read the `favela://` links inside it. No storage, no network, no keys.
11	9
12		## The two governing properties
	10	## A record
13	11
14		- Mutable — a post is a file that can be rewritten.
15		- Extensible — new extensions never migrate old records, because projections are
16		derived on read.
	12	```
	13	---
	14	{
	15	"createdAt": "2026-09-09T12:00:00.000Z",
	16	"langs": [
	17	"en"
	18	]
	19	}
	20	---
17	21
18		Consequence: the content hash changes on every edit, so **the rkey is the stable identity,
19		never the hash**.
	22	Hello.
	23	```
20	24
21		## What a full peer needs
	25	JSON in the fence rather than YAML, because `audience: no` parsing as `false` is a privacy bug
	26	rather than a curiosity. One file rather than a sidecar, because one durable human-readable
	27	artifact is the property worth keeping.
22	28
23		Parse markdown, one file per record, replication. **No MST, no DAG-CBOR, no byte-exact
24		CIDs.** Those are the bridge's problem.
	29	## Using it
25	30
26		## The surface
27
28		⚠ There is no `main` and no `exports` — *consumers deep-import `favela-markdown/lib/.js`**, and
	31	There is no `main` and no `exports` — *consumers deep-import `favela-markdown/lib/.js`**, and
29	32	that path is the contract.
30	33
31		\| module \| exports \|
	34	```js
	35	import { parse, serialize } from 'favela-markdown/lib/record.js'
	36	import { render } from 'favela-markdown/lib/render.js'
	37	import { strip } from 'favela-markdown/lib/strip.js'
	38
	39	const rec = parse(md) // { frontmatter, body, publishable, errors } — never throws
	40	rec.frontmatter // { createdAt: '2026-09-09T12:00:00.000Z', langs: ['en'] }
	41
	42	render(rec.body)
	43	// <p>Hello <span class="mention" data-target="favela://aaaa…">@alice</span> — see the notes.</p>
	44
	45	strip(rec.body)
	46	// { text: 'Hello @alice — see the notes.',
	47	// facets: [ { byteStart: 6, byteEnd: 12, type: 'mention', value: 'favela://aaaa…' } ] }
	48	```
	49
	50	`parse` is total: anything unparseable degrades into `errors` rather than throwing, because a
	51	malformed record someone else wrote must not take down your reader.
	52
	53	Round-tripping. `serialize(parse(x)) === x` holds for canonical input. Anything else gets
	54	normalised — key order, indentation, the blank line, the trailing newline are all pinned — so a
	55	record that has been through the codec once is byte-stable forever after. That is the useful
	56	direction, and it is what makes an in-place rewrite safe.
	57
	58	\| module \| what it's for \|
32	59	\|---\|---\|
33		\| `lib/record.js` \| `parse`, `serialize`, `CANONICAL` — frontmatter + body, both directions \|
34		\| `lib/strip.js` \| `strip` — the projection down to plain text + facets \|
	60	\| `lib/record.js` \| `parse`, `serialize` — the file, both directions \|
35	61	\| `lib/render.js` \| `render`, `embeds` — HTML, and the embeds a record names \|
36		\| `lib/uri.js` \| `parseUri`, `recordUri`, `userUri`, `blobUri` and their `is*` guards \|
37		\| `lib/dialect.js` \| the pinned micromark extension set, `isMention`, `isBlob`, `blobOid` \|
38		\| `lib/reach.js` \| `NETWORKS`, `maxReach` — how far a record is allowed to travel \|
	62	\| `lib/strip.js` \| `strip` — plain text plus facets, the projection everything else derives from \|
	63	\| `lib/uri.js` \| the `favela://` scheme: `parseUri`, `recordUri`, `userUri`, `blobUri` \|
	64	\| `lib/dialect.js` \| the pinned markdown extension set \|
	65	\| `lib/reach.js` \| `NETWORKS`, `maxReach` — how far a record may travel \|
39	66	\| `lib/lint.js` \| `lint` \|
40	67
41		`bin/favela-markdown` exposes `explain`, `lint`, `project` and `strip`.
	68	And a CLI for looking at one by hand: `favela-markdown explain \| lint \| project \| strip`.
42	69
43		## Status
	70	## The two properties that govern everything
	71
	72	Mutable — a post is a file that can be rewritten. So the content hash changes on every edit,
	73	and the rkey is the stable identity, never the hash. Anything that addresses a post by its
	74	hash breaks the first time someone fixes a typo.
	75
	76	Extensible — a new extension never migrates old records, because projections are derived on
	77	read. Unknown frontmatter keys are preserved and sorted after the pinned ones, so adding a field
	78	does not change the bytes of any record that predates it.
44	79
45		Working, 0.1.6 — the design below shipped.
	80	## The `favela://` scheme
	81
	82	Three forms, told apart by shape rather than by the order you test them in:
	83
	84	```
	85	favela://blob/<oid> a blob
	86	favela://<root> a user
	87	favela://<root>/<collection>/<name> a record
	88	```
	89
	90	⛔ The authority is an identity ROOT, never a feed key. Both are 64 hex characters, so nothing
	91	in the bytes distinguishes them and a reader cannot tell which it was handed. A feed key says where
	92	bytes are; a root says who someone is, and only the second survives rotation. `recordUri` refuses
	93	anything that is not a 64-hex root, because once a URI is written into a record it cannot be
	94	corrected — you may not rewrite a record you did not author.
	95
	96	`parseUri` is deliberately more permissive: it has to read bytes that were already written.
	97
	98	## Why this exists
	99
	100	atproto became favela's canonical P2P path by default rather than by decision. That pulled MST
	101	derivation, DAG-CBOR byte-exactness, lexicon field ceilings and blob CIDs sealed inside signed
	102	records into a network that only ever wanted to interface with atproto. All of that machinery is
	103	genuinely useful at the bridge layer. None of it belongs underneath every full peer.
	104
	105	So a full peer needs: parse markdown, one file per record, replicate. **No MST, no DAG-CBOR, no
	106	byte-exact CIDs.** Those are the bridge's problem.
	107
	108	## Status
46	109
47		- `npm test` — 77/77 tests, 2432 asserts pass (brittle-bare, run 2026-09-09), including the
48		36 `markdown-strip` conformance vectors and a strip fuzzer.
49		- Real consumers today are `favela-peer` (`record`, `render`, `strip`, `reach`) and
50		`favela-surface` (`render`, `uri`).
	110	Working. `npm test` runs 77 tests — 2432 asserts, all passing — including the 36 `markdown-strip`
	111	conformance vectors and a fuzzer.
51	112
52		⚠ The three consumers this file used to name — `favela-web`, `atproto-room/lib/signer.js`
53		`materialize()` and `ap-bridge/lib/as2-project.js` — do not import this package. Those two
54		projections still derive their own way; wiring them to this codec is outstanding work, not a
55		description of today.
	113	In use by `favela-peer` (`record`, `render`, `strip`, `reach`) and `favela-surface` (`render`,
	114	`uri`).
56	115
57		⚠ `spec/design.md` still carries `Status: design. Nothing built.` in its header. That line is
58		stale in the same way this file was.
	116	⚠ Three other projections still derive their own way and do not import this package:
	117	`favela-web`, `atproto-room/lib/signer.js` `materialize()`, and `ap-bridge/lib/as2-project.js`.
	118	Wiring them to this codec is outstanding work — the point of a single codec is lost while three
	119	consumers each keep their own.
59	120
60	121	## Specs
61	122
62		- `spec/design.md` — the decision and what it changes
63		- `spec/library.md` — this package's surface, its guarantees, and the sync obligations it inherits
	123	- `spec/design.md` — the decision and what it changes ⚠ its header still reads
	124	"Status: design. Nothing built."
	125	- `spec/library.md` — the surface, its guarantees, and the sync obligations it inherits
64	126	- `spec/record-vocabulary.md` — the `net.favela.post` vocabulary
65	127	- `spec/interaction.md`
66		- `spec/markdown-strip.md` + `spec/vectors/markdown-strip.json` — the projection down to
67		plain text + facets, with 36 conformance vectors
	128	- `spec/markdown-strip.md` + `spec/vectors/markdown-strip.json` — the strip contract, 36 vectors
68	129
69		The vocabulary and strip specs moved here from `favela-web/spec/` on 2026-09-02: they
70		describe the record, not the web app.
	130	The vocabulary and strip specs moved here from `favela-web/spec/` on 2026-09-02: they describe the
	131	record, not the web app.