OverviewFilesCommitsIssuesDocsReleasesPeers

codetree commits / e70006c

Phase 0a — validator hardening
Turns codetree into a tool we can trust to validate docs on any project. Four invariants established (I1-I4, see docs/DOC-DEDUP-LAYER.md): I1. Editing a spec never breaks codetree's own tests. - test/validator.test.js lines 19,20,22,39 changed from brittle snapshot assertions (`t.is(lastVerified, '2026-04-12')`) to parser-invariant checks (`t.ok(/^\d{4}-\d{2}-\d{2}$/.test(...))`). I2. External references resolve as 'external', not 'missing'. - New lib/config.js loads .codetree.json per-project; defines externalSymbols, externalPackages, externalPaths. - doc-cruft.js consults config before flagging refs missing. - New 'external' ref status with its own summary counter; doesn't inflate staleness. I3. Placeholder / example paths are not classified as refs. - Config.placeholderPatterns: ~/, /path/to/, <name>, ${var}, etc. - extractReferences filters placeholders before classification. I4. Deprecated modules skip undocumented-X warnings. - doc-parser.js reads **Status:** deprecated + **Replaced by:** X. - validator.js gates undocumented-export/method/dep warnings on !spec.deprecated. Existence checks still run. Scanner.js loads config from root and passes it through to doc-cruft. Tree exposes tree.config for downstream consumers. Full suite: 73/73 tests, 419/419 asserts pass. Design: docs/DOC-DEDUP-LAYER.md (new) — the canonical reference for the whole doc-dedup layer across Phase 0a-0g. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
e70006cf487023f6ab92851c0c3896421954b742
by plankatron <fleeky@gmail.com> · 2026-04-16T16:21:25.000Z

Changed files (9) +1122 -65

docs/DOC-DEDUP-LAYER.md +380 -0
……@@ -0,0 +1,380 @@
1# Doc Dedup Layer
2
3**Status:** Design. Implementation tracked in the P.10 plan Phase 0.
4
5Codetree's job: be the single source of truth for answering "does this
6doc match the code?" It already scans code (bottom-up) and parses specs
7(top-down) and cross-checks claims. This doc lays out the hardening +
8UX + integration work that turns the current implementation into a
9tool every project in this ecosystem can rely on.
10
11Everything in here is framed as an invariant — something that should
12stay true regardless of which project codetree is pointed at. If a
13change breaks one of these invariants, revisit the design before
14shipping.
15
16---
17
18## Invariants
19
20### I1. Updating a spec must never break codetree's own tests
21
22Tests verify the parser + validator behavior, not specific spec
23content. A brittle assertion like `t.is(spec.lastVerified, '2026-04-12')`
24couples codetree's test suite to one project's current spec state —
25the moment someone updates that date, codetree fails. Tests must check
26that values are *parsed correctly*, not that they equal specific
27snapshots.
28
29Acceptance check: `codetree docs --apply` on any test-corpus spec
30leaves `npm test` passing without further test edits.
31
32### I2. External references are first-class, not failures
33
34A ref to `an.open()` in pear-git-v3's spec is valid — anproto is an
35external dep. Codetree must either (a) crawl the external package and
36index its exports, or (b) be told via per-project config that `an.open`
37is a known external symbol. Either path MUST let the doc-cruft report
38stay noise-free on real projects.
39
40Acceptance check: running `codetree docs <project>` on a project with
41its deps declared produces zero "missing" flags for symbols that exist
42in those deps.
43
44### I3. Placeholder paths aren't real refs
45
46`~/.hyperident/`, `<user>@<host>`, `/path/to/foo`, `${varName}` are
47narrative examples. They must NOT be flagged as missing filepaths.
48
49Acceptance check: a curated fixture of 20 common placeholder forms
50produces zero false positives in doc-cruft.
51
52### I4. Deprecated modules don't generate noise
53
54A module marked `**Status:** deprecated` must skip undocumented-exports
55and undocumented-methods warnings. It's allowed to still check
56existence claims (deprecated doesn't mean deleted).
57
58Acceptance check: a deprecated module spec generates zero warnings
59for un-mentioned methods, but still fails if it claims a file or class
60that no longer exists.
61
62### I5. Every view in `codetree serve` is deep-linkable
63
64Every navigation state has a URL. Copy-paste a URL to a collaborator
65→ they land on the same view. `/module/hyper-orgs/file/lib/orgs.js`
66opens the same thing regardless of session, history, or click path.
67
68Acceptance check: opening a `/module/:name/...` URL directly in a
69cold browser produces the same rendered view as clicking through to
70that state from `/`.
71
72### I6. Codetree validates itself
73
74Codetree's own specs (if it ships any) pass codetree's own validator.
75Its README and CONTRIBUTING.md pass the doc-cruft check. Self-hosting
76prevents the "cobbler's children" problem.
77
78Acceptance check: `npm run prepublishOnly` includes a codetree self-check
79that must pass before publish.
80
81---
82
83## Architecture
84
85### Two layers, one truth
86
87**Bottom-up (code-canonical):**
88- `lib/scanner.js` / `lib/scan-stream.js` walk the source tree.
89- `lib/parsers/*.js` extract language-specific ASTs (JS, GDScript, C++, GLSL).
90- Output: `tree = { modules: { name → { files, publicAPI, dependencies, … } } }`.
91- This IS the ground truth. Specs are compared to it, not the other way.
92
93**Top-down (narrative):**
94- `lib/doc-parser.js` reads `spec/SYSTEM.md` and `spec/modules/*.md`.
95- Extracts structured claims: dependencies, exports, methods, LOC, dates.
96- Output: `specs = { system, modules[name] }`.
97
98**Validator:**
99- `lib/validator.js` cross-checks. For every claim in specs, asks code.
100- Produces `{ checks: [{ level, canonicity, claim, status, detail, expected, actual }] }`.
101- Status is pass/warn/fail. Canonicity is `code-canonical` (bottom-up
102 proves correctness) or `narrative` (top-down, staleness-based).
103
104**Doc cruft detector:**
105- `lib/doc-cruft.js` walks all markdown, extracts code-like references,
106 resolves each against the scanned tree.
107- Output: `{ docs: [{ path, references: {total, valid, missing, details}, staleness: {score, missingRatio, ageDays} }] }`.
108- Complement to validator — validator asks "does the spec claim match
109 code?"; doc-cruft asks "does this arbitrary doc reference something
110 that still exists?"
111
112---
113
114## Per-project config: `.codetree.json`
115
116Placed at project root. All fields optional; defaults are safe.
117
118```json
119{
120 "scanRoot": ".",
121
122 "externalSymbols": [
123 "an.open", "an.sign", "an.hash",
124 "IdentityKey", "BlindPairing"
125 ],
126
127 "externalPaths": [
128 "hyper-fleeky/anproto",
129 "hyper-fleeky/repos/hyperident"
130 ],
131
132 "externalPackages": [
133 "anproto", "keet-identity-key", "blind-pairing"
134 ],
135
136 "autoCrawl": false,
137 "siblingDirs": [
138 "../anproto",
139 "../hyper-rooms",
140 "../hyper-issues"
141 ],
142
143 "specDir": "spec",
144 "docGlobs": ["docs/**/*.md", "spec/**/*.md", "*.md"],
145
146 "placeholderPatterns": [
147 "^~/",
148 "^</[^>]+>$",
149 "\\$\\{[^}]+\\}",
150 "^/path/to/"
151 ],
152
153 "deprecatedModules": ["trust-map"],
154
155 "thresholds": {
156 "maxMissingRatio": 0.1,
157 "maxStalenessScore": 0.3,
158 "maxValidatorFailures": 0
159 },
160
161 "suggestConfidenceFloor": "high"
162}
163```
164
165- `autoCrawl: false` (default) — only scan `scanRoot`. `true` → follow
166 `package.json` deps (`file:../foo` and sibling-named packages up to
167 2 ancestors).
168- `siblingDirs` — explicit additional scan roots, merged with
169 auto-crawl results.
170- CLI flags `--auto-crawl` / `--no-auto-crawl` / `--sibling-dirs a,b`
171 override config. CLI > config > default.
172- `thresholds` used by `codetree docs --check` (pass/fail exit).
173
174---
175
176## CLI surface
177
178```
179codetree scan [dir] Full scan, outputs JSON tree
180codetree module <name> [dir] Show one module's structure
181codetree file <path> [dir] Show one file's structure
182codetree exports <module> [dir] List public API surface
183codetree deps <module> [dir] Show dependency chain
184codetree imports <module> [dir] Show internal import graph
185codetree docs [dir] Doc cruft report (staleness)
186codetree validate [dir] Validator report (pass/fail per claim)
187codetree summary [dir] Human-readable overview
188codetree categories [dir] Module categorization
189codetree serve [dir] [port] Start HTTP visualization
190
191Doc dedup ops:
192codetree docs --suggest <spec.md> Print proposed patches, no writes
193codetree docs --apply <spec.md> Apply high-confidence patches
194codetree docs --apply <spec.md> --confidence medium|low More aggressive
195codetree docs --check Exit nonzero if thresholds breached
196codetree docs --check --threshold 0.3 Override config threshold
197
198Flags:
199--auto-crawl / --no-auto-crawl Override config autoCrawl
200--sibling-dirs a,b Additional scan roots (comma-sep)
201--pretty / -p Pretty JSON / human output
202--depth <n> Module discovery depth (default: 2)
203--ignore <patterns> Extra ignore patterns
204--shallow Parse only module entry files
205--no-watch Disable fs.watch in serve
206```
207
208---
209
210## Suggest-fixes: confidence model
211
212`--apply` only writes high-confidence patches by default. Everything
213else emits a `MANUAL REVIEW` block.
214
215### High confidence
216
217- Numeric counts from tooling (`npm test` parse, file LOC, assert counts)
218- Dates (spec's `Last verified`, `sourceVerified` → today when explicitly requested)
219- Exact filepath claims where the file is found at that exact path
220- Exact class/method name where code has exactly one class/method by that name
221- Explicit existence claims that can be yes/no answered from the code index
222
223### Medium confidence
224
225- Refactored identifier (spec claim `X.foo()` no longer exists, but
226 code has a single `X.bar()` that fits semantically)
227- Dependency list drift (spec says deps=[A,B,C], code has [A,B,C,D])
228- LOC changed within ±20 lines (just updates the number)
229
230### Low confidence
231
232- Prose claims ("The module handles authentication")
233- Architectural descriptions
234- Narrative commentary
235- Anything requiring a human to decide what the spec *should* say
236
237Default `--apply` threshold is **high**. CLI flag `--confidence medium|low`
238opens up more aggressive patching; these still emit a summary line
239the operator must read.
240
241### MANUAL REVIEW output format
242
243```
244=== spec/SYSTEM.md line 175 — confidence: medium ===
245current: - ❌ Auth uses plain detached Ed25519 (v1's miswritten format)...
246suggestion: (BUG-1 is fixed in code; this narrative paragraph should be rewritten or removed)
247reason: lib/auth/verify.js uses canonical anproto (an.open). No plain
248 detached signing remains. Narrative doesn't automatically rewrite.
249action: review + rewrite manually, or run --confidence low to attempt auto-rewrite
250```
251
252---
253
254## URL routing in `serve`
255
256Every view maps to a URL. Path-based (not hash-based). Server-side
257routing in `lib/server.js`; client-side `history.pushState` for SPA
258navigation; `popstate` listener rebuilds graphStack from URL.
259
260| Path | Renders |
261|-----|---------|
262| `/` | System overview |
263| `/system` | Explicit system overview |
264| `/module/:name` | Module detail |
265| `/module/:name/file/*filePath` | File structure view |
266| `/module/:name/deps` | Dependency graph rooted here |
267| `/module/:name/exports` | Public API surface |
268| `/module/:name/imports` | Internal import graph |
269| `/path/*fsPath` | Directory view |
270| `/validation` | Full validator report |
271| `/validation/module/:name` | Per-module drift |
272| `/validation/level/:level` | Filter by system/module/file/method |
273| `/docs` | Doc cruft list |
274| `/docs/*docPath` | Per-doc drift detail |
275| `/suggestions/*docPath` | Proposed-patch preview |
276
277All links in the UI are real `<a href>` elements. Middle-click +
278ctrl-click work like any other page. JS intercepts left-click for SPA
279routing. URL updates via `pushState` on every navigation; `popstate`
280restores graphStack.
281
282Breadcrumbs use real URLs; each segment is an `<a>` with the URL for
283that level.
284
285---
286
287## Integration path
288
289### Phase 0 (current plan): codetree hardening
290
291Invariants I1–I6 realized. Config file format shipped. CLI grows
292`validate`, `docs --suggest / --apply / --check`, serve routing.
293
294### Phase later: pear-git hook
295
296`.pear-git/ci.yaml` gains a codetree step:
297```yaml
298- name: Doc-code alignment
299 run: npx codetree docs --check --threshold 0.3
300 timeout: 60
301```
302Fails the deploy if docs drift past configured threshold. Repo page
303in pear-git web UI shows a "docs drift" badge with the latest report.
304
305### Phase later: pre-receive hook
306
307Optional `pre-receive` git hook on pear-git server side that runs
308codetree against the pushed commit. Rejects push if drift exceeds
309threshold. Opt-in per-repo.
310
311### Phase later: cross-project validator service
312
313A `codetree serve` instance running as an RPC endpoint (via hyper-svc)
314so other tools can query "is spec X in sync with code?" without
315spawning a subprocess. IDE/editor integration consumes this.
316
317---
318
319## Testing strategy
320
321Every invariant from the top gets a test:
322
323- **I1 (test brittleness):** run `codetree docs --apply` against a
324 fixture spec, re-run `npm test`, assert it still passes.
325- **I2 (externals):** fixture project with `.codetree.json` declaring
326 externals; assert doc-cruft report has zero false positives.
327- **I3 (placeholders):** curated list of 20 placeholder forms; assert
328 each classifies as non-ref.
329- **I4 (deprecated):** fixture module marked deprecated; assert zero
330 undocumented-X warnings.
331- **I5 (deep linking):** hit every URL pattern with an HTTP client,
332 assert 200 + expected page content.
333- **I6 (self-hosting):** `codetree validate codetree/` must pass.
334
335All tests use `brittle` (existing harness).
336
337---
338
339## Open design decisions
340
341- **Config location:** `.codetree.json` at project root (settled).
342 Should it support a `.codetreerc` form too? Probably not — one
343 canonical path.
344- **Apply writes:** `--apply` writes in place. Should we also emit a
345 `.patch` file the operator can review? Probably yes as a first-pass
346 default, then actual in-place write via `--apply --write`.
347- **Validation endpoint output:** JSON for machines, HTML for humans.
348 `/validation.json` vs `/validation`. Or `?format=json`. Latter is
349 simpler.
350- **Error budget thresholds:** per-project config. Defaults sane. But
351 should defaults be strict (no drift allowed) or loose (only fail on
352 >30% stale)? Arguably strict is the right default for specs; loose
353 is the right default for docs/. Different path, different threshold.
354
355---
356
357## Non-goals (explicitly out of scope)
358
359- **Natural-language rewriting of spec prose.** Codetree reports drift;
360 it doesn't LLM the prose back into correctness. Humans do that.
361- **Cross-repo doc links.** If spec A references doc B in a different
362 repo, codetree checks the ref format but not the remote existence.
363- **Source semantics.** Codetree doesn't understand "this function
364 changed behavior but kept its signature." It operates on structural
365 claims only.
366- **Style enforcement.** Linting prose style, markdown formatting, etc.
367 — separate tool.
368
369---
370
371## References
372
373- `lib/validator.js` — current validator implementation
374- `lib/doc-parser.js` — spec parser
375- `lib/doc-cruft.js` — cruft detector
376- `lib/scanner.js`, `lib/scan-stream.js` — scan pipeline
377- `lib/server.js` — HTTP serve entrypoint
378- `lib/render.js` — serve UI renderer
379- `../../pear-git-v3/docs/identity-use-cases.md` — companion design doc
380 for the canonical anproto work this validates
lib/config.js +202 -0
……@@ -0,0 +1,202 @@
1import { readFileSync, existsSync } from 'fs'
2import { join, resolve } from 'path'
3
4/**
5 * Codetree per-project configuration. Loads `.codetree.json` from the
6 * project root. CLI flags override config fields individually.
7 *
8 * All fields are optional. Defaults are safe (no auto-crawling, no
9 * assumed externals). A project opts into richer behavior by shipping
10 * a `.codetree.json` at its root.
11 *
12 * Schema documented in `docs/DOC-DEDUP-LAYER.md`.
13 */
14
15export const DEFAULT_CONFIG = {
16 scanRoot: '.',
17
18 // Symbols known to exist outside the scanned tree. doc-cruft resolves
19 // these as 'external' instead of 'missing'.
20 externalSymbols: [],
21
22 // Filesystem paths known to exist outside the scanned tree. Same
23 // treatment: 'external' not 'missing'.
24 externalPaths: [],
25
26 // Package names (from node_modules or similar). When an import statement
27 // references one of these, it's not expected to resolve locally.
28 externalPackages: [],
29
30 // Cross-filesystem dependency crawling.
31 // false (default): only scan scanRoot.
32 // true: follow package.json `file:../foo` deps, then walk up 2 ancestors
33 // for bare-name matches against sibling directories.
34 autoCrawl: false,
35
36 // Explicit additional scan roots. Merged with autoCrawl results.
37 siblingDirs: [],
38
39 // Where to find specs relative to scanRoot.
40 specDir: 'spec',
41
42 // Glob patterns for markdown discovery in doc-cruft.
43 docGlobs: ['docs/**/*.md', 'spec/**/*.md', '*.md'],
44
45 // Regex patterns matching placeholder / example paths that should
46 // NOT be classified as real filesystem references.
47 placeholderPatterns: [
48 '^~/', // ~/foo — user home shorthand
49 '^/path/to/', // /path/to/foo — canonical placeholder
50 '^<[^>]+>$', // <placeholder> — template token
51 '\\$\\{[^}]+\\}', // ${var} — variable interpolation
52 '<[a-zA-Z_][a-zA-Z0-9_-]*>', // <name> inside a path, e.g. /etc/<name>
53 '^\\[.+\\]\\(.+\\)$' // accidental markdown link syntax
54 ],
55
56 // Module names to treat as deprecated. Validator skips
57 // undocumented-exports and undocumented-methods warnings for these.
58 // Existence checks still run.
59 deprecatedModules: [],
60
61 // Thresholds for `codetree docs --check` gate.
62 thresholds: {
63 maxMissingRatio: 0.1,
64 maxStalenessScore: 0.3,
65 maxValidatorFailures: 0
66 },
67
68 // Minimum confidence for `codetree docs --apply`.
69 // Values: 'high' (default), 'medium', 'low'.
70 suggestConfidenceFloor: 'high'
71}
72
73/**
74 * Load `.codetree.json` from `rootDir`, merged with defaults. Missing
75 * file returns defaults. Bad JSON logs a warning and returns defaults —
76 * never throws.
77 *
78 * @param {string} rootDir - Project root path
79 * @returns {object} merged config
80 */
81export function loadConfig (rootDir) {
82 const path = join(rootDir, '.codetree.json')
83 if (!existsSync(path)) return cloneDefaults()
84
85 let user
86 try {
87 user = JSON.parse(readFileSync(path, 'utf-8'))
88 } catch (err) {
89 // eslint-disable-next-line no-console
90 console.warn('[codetree] failed to load ' + path + ': ' + err.message)
91 return cloneDefaults()
92 }
93
94 return mergeConfig(DEFAULT_CONFIG, user)
95}
96
97/**
98 * Apply CLI flag overrides on top of a loaded config. CLI > config > default.
99 *
100 * @param {object} config - Loaded config
101 * @param {object} cli - CLI flag values (only fields present override)
102 * @returns {object} new config
103 */
104export function applyCliOverrides (config, cli) {
105 const out = mergeConfig(config, {})
106 if (cli.autoCrawl !== undefined) out.autoCrawl = cli.autoCrawl
107 if (Array.isArray(cli.siblingDirs) && cli.siblingDirs.length) {
108 out.siblingDirs = dedupe([...out.siblingDirs, ...cli.siblingDirs])
109 }
110 if (cli.suggestConfidence) out.suggestConfidenceFloor = cli.suggestConfidence
111 if (cli.threshold !== undefined) {
112 out.thresholds = { ...out.thresholds, maxStalenessScore: cli.threshold }
113 }
114 return out
115}
116
117/**
118 * Check if a reference (symbol name, path, or package) is a known external.
119 *
120 * @param {string} ref - The reference to check
121 * @param {object} config - Loaded config
122 * @returns {boolean}
123 */
124export function isExternalRef (ref, config) {
125 if (!ref) return false
126 if (config.externalSymbols.includes(ref)) return true
127 if (config.externalPackages.includes(ref)) return true
128 for (const p of config.externalPaths) {
129 if (ref === p || ref.startsWith(p + '/')) return true
130 }
131 return false
132}
133
134/**
135 * Check if a reference string matches any placeholder pattern.
136 *
137 * @param {string} ref - The reference to test
138 * @param {object} config - Loaded config
139 * @returns {boolean}
140 */
141export function isPlaceholder (ref, config) {
142 if (!ref) return false
143 for (const pattern of config.placeholderPatterns) {
144 try {
145 if (new RegExp(pattern).test(ref)) return true
146 } catch {
147 // skip malformed patterns rather than crash
148 }
149 }
150 return false
151}
152
153// --- internals ---
154
155function cloneDefaults () {
156 return mergeConfig(DEFAULT_CONFIG, {})
157}
158
159function mergeConfig (base, user) {
160 const out = {}
161 for (const key of Object.keys(base)) {
162 const baseVal = base[key]
163 const userVal = user[key]
164
165 if (key === 'thresholds') {
166 out.thresholds = { ...baseVal, ...(userVal || {}) }
167 } else if (Array.isArray(baseVal)) {
168 out[key] = Array.isArray(userVal) ? dedupe(userVal.slice()) : baseVal.slice()
169 } else if (userVal !== undefined) {
170 out[key] = userVal
171 } else {
172 out[key] = baseVal
173 }
174 }
175 // Pass through user-only keys we don't know about (forward-compat)
176 for (const key of Object.keys(user)) {
177 if (!(key in base)) out[key] = user[key]
178 }
179 return out
180}
181
182function dedupe (arr) {
183 return Array.from(new Set(arr))
184}
185
186/**
187 * Expose default config for tests that want to compare against it.
188 */
189export function getDefaults () {
190 return cloneDefaults()
191}
192
193/**
194 * Resolve a config's siblingDirs to absolute paths, relative to rootDir.
195 *
196 * @param {object} config
197 * @param {string} rootDir
198 * @returns {string[]}
199 */
200export function resolvedSiblingDirs (config, rootDir) {
201 return (config.siblingDirs || []).map(s => resolve(rootDir, s))
202}
lib/doc-cruft.js +51 -20
……@@ -1,5 +1,6 @@
11import { readFileSync, readdirSync, statSync, existsSync } from 'fs'
22import { join, relative, resolve, extname } from 'path'
3import { isExternalRef, isPlaceholder, loadConfig } from './config.js'
34
45/**
56 * Detect documentation cruft by scanning all markdown files and
……@@ -8,11 +9,14 @@ import { join, relative, resolve, extname } from 'path'
89 * @param {string} rootDir - Project root directory
910 * @param {object} tree - Built code tree from buildTree()
1011 * @param {Function} [ignore] - Ignore matcher from scanner.js
12 * @param {object} [config] - Loaded codetree config (optional; loaded from
13 * rootDir if not passed)
1114 * @returns {object} Doc cruft report with per-doc details and summary
1215 */
13export function detectDocCruft (rootDir, tree, ignore) {
16export function detectDocCruft (rootDir, tree, ignore, config) {
1417 const root = resolve(rootDir)
1518 const ig = ignore || (() => false)
19 const cfg = config || loadConfig(root)
1620 const mdFiles = findMarkdownFiles(root, ig)
1721
1822 // Build lookup indexes from the tree for fast cross-referencing
……@@ -21,6 +25,7 @@ export function detectDocCruft (rootDir, tree, ignore) {
2125 const docs = []
2226 let totalMissing = 0
2327 let totalValid = 0
28 let totalExternal = 0
2429
2530 for (const mdPath of mdFiles) {
2631 const relPath = relative(root, mdPath)
……@@ -31,27 +36,33 @@ export function detectDocCruft (rootDir, tree, ignore) {
3136 let mtime
3237 try { mtime = statSync(mdPath).mtimeMs } catch { mtime = 0 }
3338
34 // Extract code references
35 const refs = extractReferences(content, relPath)
39 // Extract code references (placeholder-filtered via config)
40 const refs = extractReferences(content, relPath, cfg)
3641
3742 // Cross-reference each against the code tree
3843 const details = []
3944 let valid = 0
4045 let missing = 0
46 let external = 0
4147
4248 for (const ref of refs) {
43 const status = resolveReference(ref, index, root)
49 const status = resolveReference(ref, index, root, cfg)
4450 details.push({ ref: ref.value, type: ref.type, status, line: ref.line })
4551 if (status === 'valid') valid++
52 else if (status === 'external') external++
4653 else missing++
4754 }
4855
4956 totalValid += valid
5057 totalMissing += missing
51
52 // Staleness score: weighted combination of missing refs and age
53 const total = valid + missing
54 const missingRatio = total > 0 ? missing / total : 0
58 totalExternal += external
59
60 // Staleness score: weighted combination of missing refs and age.
61 // External references (to known deps outside the scan) don't count
62 // toward missingRatio — they're expected to be out of scope.
63 const total = valid + missing + external
64 const resolvable = valid + missing
65 const missingRatio = resolvable > 0 ? missing / resolvable : 0
5566 const ageDays = (Date.now() - mtime) / (1000 * 60 * 60 * 24)
5667 const ageFactor = Math.min(ageDays / 90, 1) // caps at 90 days
5768 const score = Math.round((missingRatio * 0.6 + ageFactor * 0.4) * 100) / 100
……@@ -60,7 +71,7 @@ export function detectDocCruft (rootDir, tree, ignore) {
6071 path: relPath,
6172 lines: lines.length,
6273 lastModified: new Date(mtime).toISOString().split('T')[0],
63 references: { total, valid, missing, details },
74 references: { total, valid, missing, external, details },
6475 staleness: { score, missingRatio: Math.round(missingRatio * 100) / 100, ageDays: Math.round(ageDays) }
6576 })
6677 }
……@@ -79,7 +90,8 @@ export function detectDocCruft (rootDir, tree, ignore) {
7990 freshDocs,
8091 staleDocs,
8192 validReferences: totalValid,
82 missingReferences: totalMissing
93 missingReferences: totalMissing,
94 externalReferences: totalExternal
8395 }
8496 }
8597}
……@@ -147,7 +159,7 @@ function buildCodeIndex (tree) {
147159
148160// --- Reference extraction from markdown ---
149161
150function extractReferences (content, docPath) {
162function extractReferences (content, docPath, config) {
151163 const refs = []
152164 const lines = content.split('\n')
153165 const seen = new Set()
……@@ -165,6 +177,9 @@ function extractReferences (content, docPath) {
165177 if (seen.has(val)) continue
166178 seen.add(val)
167179
180 // Placeholder / example path — skip, not a real reference
181 if (config && isPlaceholder(val, config)) continue
182
168183 const type = classifyReference(val)
169184 if (type) refs.push({ value: val, type, line: lineNum })
170185 }
……@@ -266,7 +281,16 @@ function isCommonWord (val) {
266281
267282// --- Cross-referencing ---
268283
269function resolveReference (ref, index, root) {
284function resolveReference (ref, index, root, config) {
285 // External registry check — refs declared as external in .codetree.json
286 // resolve as 'external' instead of 'missing'. Only applies when the
287 // reference can't resolve locally; that way a local symbol shadowing
288 // an external still resolves as 'valid'.
289 const markExternalIfConfigured = (value) => {
290 if (config && isExternalRef(value, config)) return 'external'
291 return 'missing'
292 }
293
270294 switch (ref.type) {
271295 case 'filepath': {
272296 // Clean up: remove trailing slashes, leading ./
……@@ -278,33 +302,40 @@ function resolveReference (ref, index, root) {
278302 // Check if directory exists on disk
279303 const absPath = resolve(root, cleaned)
280304 if (existsSync(absPath)) return 'valid'
281 return 'missing'
305 return markExternalIfConfigured(cleaned)
282306 }
283307
284308 case 'class':
285 return index.classNames.has(ref.value) || index.exportNames.has(ref.value) ? 'valid' : 'missing'
309 if (index.classNames.has(ref.value) || index.exportNames.has(ref.value)) return 'valid'
310 return markExternalIfConfigured(ref.value)
286311
287312 case 'function': {
288313 const name = ref.value.replace(/\(.*\)$/, '')
289 return index.functionNames.has(name) || index.exportNames.has(name) ? 'valid' : 'missing'
314 if (index.functionNames.has(name) || index.exportNames.has(name)) return 'valid'
315 // External check against both bare name and the original foo() form.
316 if (config && (isExternalRef(name, config) || isExternalRef(ref.value, config))) {
317 return 'external'
318 }
319 return 'missing'
290320 }
291321
292322 case 'class-or-function':
293 return (index.classNames.has(ref.value) || index.functionNames.has(ref.value) || index.exportNames.has(ref.value))
294 ? 'valid' : 'missing'
323 if (index.classNames.has(ref.value) || index.functionNames.has(ref.value) || index.exportNames.has(ref.value)) return 'valid'
324 return markExternalIfConfigured(ref.value)
295325
296326 case 'property':
297327 case 'identifier':
298328 // For identifiers, check if they appear as any known symbol
299 return (index.classNames.has(ref.value) || index.functionNames.has(ref.value) || index.exportNames.has(ref.value))
300 ? 'valid' : 'unknown'
329 if (index.classNames.has(ref.value) || index.functionNames.has(ref.value) || index.exportNames.has(ref.value)) return 'valid'
330 if (config && isExternalRef(ref.value, config)) return 'external'
331 return 'unknown'
301332
302333 case 'doc-link': {
303334 // Check if linked document exists on disk (relative to the doc's directory or root)
304335 const target = ref.value.replace(/^\.\//, '')
305336 const absPath = resolve(root, target)
306337 if (existsSync(absPath)) return 'valid'
307 return 'missing'
338 return markExternalIfConfigured(target)
308339 }
309340
310341 default:
lib/doc-parser.js +16 -1
……@@ -113,7 +113,9 @@ function parseModuleSpec (content, filename) {
113113 claimedGetters: [],
114114 claimedExports: [],
115115 claimedConstructorParams: [],
116 roleInSystem: null
116 roleInSystem: null,
117 deprecated: false,
118 replacedBy: null
117119 }
118120
119121 // Module name from title: "# hyper-orgs -- Module Spec"
……@@ -124,6 +126,19 @@ function parseModuleSpec (content, filename) {
124126 const locMatch = content.match(/\*\*Location:\*\*\s*`([^`]+)`/)
125127 if (locMatch) spec.location = locMatch[1].replace(/\/$/, '')
126128
129 // Deprecation status. Module spec opt-in header:
130 // **Status:** deprecated
131 // **Replaced by:** IdentityRegistry
132 // Deprecated modules skip undocumented-export/method warnings but
133 // still fail existence checks (claimed file missing → fail).
134 const statusMatch = content.match(/\*\*Status:\*\*\s*([^\n]+)/i)
135 if (statusMatch) {
136 const statusLine = statusMatch[1].trim().toLowerCase()
137 if (statusLine.startsWith('deprecated')) spec.deprecated = true
138 }
139 const replacedMatch = content.match(/\*\*Replaced by:\*\*\s*`?([^`\n]+?)`?\s*$/im)
140 if (replacedMatch) spec.replacedBy = replacedMatch[1].trim()
141
127142 // Test count
128143 const testMatch = content.match(/\*\*Tests:\*\*\s*(\d+)\s*pass/)
129144 if (testMatch) spec.testCount = parseInt(testMatch[1], 10)
lib/scanner.js +11 -1
……@@ -8,6 +8,7 @@ import { parseSpecs, findAllDocs } from './doc-parser.js'
88import { validate } from './validator.js'
99import { detectDocCruft } from './doc-cruft.js'
1010import { buildIgnoreMatcher } from './ignore.js'
11import { loadConfig, applyCliOverrides } from './config.js'
1112
1213// Re-export for consumers that want the push-based source directly
1314export { scanStream } from './scan-stream.js'
……@@ -34,6 +35,11 @@ export async function scan (rootDir, opts = {}) {
3435 const ignore = buildIgnoreMatcher(rootDir, opts.ignore || [])
3536 const shallow = !!opts.shallow
3637
38 // Load per-project config (.codetree.json at rootDir). CLI overrides
39 // are threaded through opts.cli (set by the CLI entrypoint).
40 const baseConfig = loadConfig(rootDir)
41 const config = opts.cli ? applyCliOverrides(baseConfig, opts.cli) : baseConfig
42
3743 // Drive the event source and reduce events into a graph.
3844 const graph = await new Promise((resolve, reject) => {
3945 const source = scanStream(rootDir, { ...opts, ignore, shallow })
……@@ -56,9 +62,13 @@ export async function scan (rootDir, opts = {}) {
5662 // Skip in shallow mode — it's one of the most expensive steps and the
5763 // tree doesn't have the structural detail needed for meaningful checks.
5864 if (!shallow) {
59 tree.docCruft = detectDocCruft(rootDir, tree, ignore)
65 tree.docCruft = detectDocCruft(rootDir, tree, ignore, config)
6066 }
6167
68 // Expose the loaded config so downstream consumers (CLI, serve) can
69 // make informed decisions without re-loading.
70 tree.config = config
71
6272 // Find and validate specs at the scan root (preserves existing behavior).
6373 const docs = findAllDocs(rootDir)
6474 let validation = null
lib/validator.js +44 -33
……@@ -301,16 +301,20 @@ function validateModule (tree, mod, spec) {
301301 }
302302
303303 // Check reverse: deps in package.json not documented
304 for (const dep of Object.keys(mod.dependencies)) {
305 // Only check local/known deps
306 if (!tree.modules[dep]) continue
307 if (!spec.dependencies.includes(dep)) {
308 result.checks.push({
309 level: 'module',
310 claim: 'Dependency "' + dep + '" is documented',
311 status: 'warn',
312 detail: 'package.json depends on ' + dep + ' but spec does not mention it'
313 })
304 // Deprecated modules skip this — the module is being removed, undocumented
305 // deps aren't worth flagging.
306 if (!spec.deprecated) {
307 for (const dep of Object.keys(mod.dependencies)) {
308 // Only check local/known deps
309 if (!tree.modules[dep]) continue
310 if (!spec.dependencies.includes(dep)) {
311 result.checks.push({
312 level: 'module',
313 claim: 'Dependency "' + dep + '" is documented',
314 status: 'warn',
315 detail: 'package.json depends on ' + dep + ' but spec does not mention it'
316 })
317 }
314318 }
315319 }
316320
……@@ -326,15 +330,19 @@ function validateModule (tree, mod, spec) {
326330 })
327331 }
328332
329 // Undocumented exports
330 for (const exp of mod.publicAPI) {
331 if (!spec.claimedExports.includes(exp)) {
332 result.checks.push({
333 level: 'module',
334 claim: 'Export "' + exp + '" is documented',
335 status: 'warn',
336 detail: 'Module exports "' + exp + '" but spec does not document it'
337 })
333 // Undocumented exports — skip for deprecated modules. A module being
334 // phased out shouldn't generate warnings for things its spec no longer
335 // bothers documenting.
336 if (!spec.deprecated) {
337 for (const exp of mod.publicAPI) {
338 if (!spec.claimedExports.includes(exp)) {
339 result.checks.push({
340 level: 'module',
341 claim: 'Export "' + exp + '" is documented',
342 status: 'warn',
343 detail: 'Module exports "' + exp + '" but spec does not document it'
344 })
345 }
338346 }
339347 }
340348
……@@ -351,20 +359,23 @@ function validateModule (tree, mod, spec) {
351359 })
352360 }
353361
354 // Undocumented methods (only for primary class)
355 const primaryClass = findPrimaryClass(mod, spec)
356 if (primaryClass && spec.claimedMethods.length > 0) {
357 for (const m of primaryClass.methods) {
358 if (m.kind === 'field') continue
359 if (m.name.startsWith('_')) continue // skip private
360 if (!spec.claimedMethods.includes(m.name) && m.name !== 'ready' && m.name !== 'close' && m.name !== 'sync') {
361 result.checks.push({
362 level: 'method',
363 canonicity: 'code-canonical',
364 claim: 'Method "' + m.name + '" is documented',
365 status: 'warn',
366 detail: 'Class has public method "' + m.name + '(' + m.params.join(', ') + ')" but spec does not document it'
367 })
362 // Undocumented methods (only for primary class). Skip for deprecated
363 // modules — they're being removed, no value in flagging missing docs.
364 if (!spec.deprecated) {
365 const primaryClass = findPrimaryClass(mod, spec)
366 if (primaryClass && spec.claimedMethods.length > 0) {
367 for (const m of primaryClass.methods) {
368 if (m.kind === 'field') continue
369 if (m.name.startsWith('_')) continue // skip private
370 if (!spec.claimedMethods.includes(m.name) && m.name !== 'ready' && m.name !== 'close' && m.name !== 'sync') {
371 result.checks.push({
372 level: 'method',
373 canonicity: 'code-canonical',
374 claim: 'Method "' + m.name + '" is documented',
375 status: 'warn',
376 detail: 'Class has public method "' + m.name + '(' + m.params.join(', ') + ')" but spec does not document it'
377 })
378 }
368379 }
369380 }
370381 }
test/config.test.js +152 -0
……@@ -0,0 +1,152 @@
1import { test } from 'brittle'
2import { mkdirSync, writeFileSync, rmSync, existsSync } from 'fs'
3import { tmpdir } from 'os'
4import { join } from 'path'
5
6import {
7 loadConfig,
8 applyCliOverrides,
9 isExternalRef,
10 isPlaceholder,
11 getDefaults,
12 DEFAULT_CONFIG
13} from '../lib/config.js'
14
15function mkFixture (name) {
16 const dir = join(tmpdir(), 'codetree-test-' + name + '-' + Date.now())
17 mkdirSync(dir, { recursive: true })
18 return dir
19}
20
21// --- loadConfig ---
22
23test('config: missing .codetree.json returns defaults', (t) => {
24 const dir = mkFixture('missing')
25 const config = loadConfig(dir)
26 t.is(config.autoCrawl, false, 'default autoCrawl is false')
27 t.ok(Array.isArray(config.externalSymbols), 'externalSymbols is an array')
28 t.is(config.externalSymbols.length, 0, 'no externals by default')
29 t.ok(Array.isArray(config.placeholderPatterns), 'placeholder patterns defaulted')
30 t.ok(config.placeholderPatterns.length > 0, 'has some default placeholders')
31 rmSync(dir, { recursive: true, force: true })
32})
33
34test('config: user values override defaults', (t) => {
35 const dir = mkFixture('override')
36 writeFileSync(join(dir, '.codetree.json'), JSON.stringify({
37 autoCrawl: true,
38 externalSymbols: ['an.sign', 'an.open'],
39 externalPackages: ['anproto']
40 }))
41
42 const config = loadConfig(dir)
43 t.is(config.autoCrawl, true, 'autoCrawl overridden')
44 t.ok(config.externalSymbols.includes('an.sign'), 'external symbol loaded')
45 t.ok(config.externalPackages.includes('anproto'), 'external package loaded')
46 // Defaults still present
47 t.ok(config.placeholderPatterns.length > 0, 'placeholders preserved')
48 rmSync(dir, { recursive: true, force: true })
49})
50
51test('config: bad JSON logs but does not throw', (t) => {
52 const dir = mkFixture('badjson')
53 writeFileSync(join(dir, '.codetree.json'), '{ this is not valid json')
54 // Silence the expected console.warn so the test log stays clean
55 const origWarn = console.warn
56 console.warn = () => {}
57 try {
58 const config = loadConfig(dir)
59 t.ok(config, 'returns some config on bad JSON')
60 t.is(config.autoCrawl, false, 'falls back to defaults')
61 } finally {
62 console.warn = origWarn
63 }
64 rmSync(dir, { recursive: true, force: true })
65})
66
67// --- applyCliOverrides ---
68
69test('config: CLI overrides win over loaded config', (t) => {
70 const base = { ...getDefaults(), autoCrawl: false, siblingDirs: ['../a'] }
71 const merged = applyCliOverrides(base, { autoCrawl: true, siblingDirs: ['../b'] })
72 t.is(merged.autoCrawl, true, 'CLI autoCrawl wins')
73 t.ok(merged.siblingDirs.includes('../a'), 'original sibling preserved')
74 t.ok(merged.siblingDirs.includes('../b'), 'CLI sibling merged in')
75})
76
77test('config: applyCliOverrides without fields is a no-op copy', (t) => {
78 const base = { ...getDefaults(), autoCrawl: true }
79 const merged = applyCliOverrides(base, {})
80 t.is(merged.autoCrawl, true, 'no accidental reset')
81 t.not(merged, base, 'returns a new object')
82})
83
84// --- isExternalRef ---
85
86test('config: isExternalRef matches external symbols', (t) => {
87 const config = { ...getDefaults(), externalSymbols: ['an.sign', 'IdentityKey'] }
88 t.ok(isExternalRef('an.sign', config), 'symbol match')
89 t.ok(isExternalRef('IdentityKey', config), 'class-like external')
90 t.not(isExternalRef('SomethingElse', config), 'non-external not matched')
91})
92
93test('config: isExternalRef matches external packages', (t) => {
94 const config = { ...getDefaults(), externalPackages: ['anproto', 'keet-identity-key'] }
95 t.ok(isExternalRef('anproto', config), 'package name match')
96 t.not(isExternalRef('something-else', config), 'non-package not matched')
97})
98
99test('config: isExternalRef matches external path prefixes', (t) => {
100 const config = { ...getDefaults(), externalPaths: ['hyper-fleeky/anproto'] }
101 t.ok(isExternalRef('hyper-fleeky/anproto', config), 'exact path match')
102 t.ok(isExternalRef('hyper-fleeky/anproto/lib/an.js', config), 'subpath match')
103 t.not(isExternalRef('hyper-fleeky/anproto-other', config), 'different prefix not matched')
104})
105
106// --- isPlaceholder (invariant I3) ---
107
108test('config: isPlaceholder recognizes common placeholder forms', (t) => {
109 const config = getDefaults()
110
111 // Home paths
112 t.ok(isPlaceholder('~/.hyperident/', config), 'home dir shorthand')
113 t.ok(isPlaceholder('~/config/foo', config), 'home subdir')
114
115 // Canonical "/path/to/" placeholder
116 t.ok(isPlaceholder('/path/to/hyperident', config), '/path/to/ placeholder')
117
118 // Angle-bracket placeholders
119 t.ok(isPlaceholder('<placeholder>', config), 'angle brackets')
120 t.ok(isPlaceholder('<name>', config), 'angle brackets short form')
121
122 // Variable interpolation
123 t.ok(isPlaceholder('${variable}', config), 'shell interpolation')
124 t.ok(isPlaceholder('prefix${foo}suffix', config), 'embedded interpolation')
125
126 // Path with embedded template token
127 t.ok(isPlaceholder('/etc/<name>', config), 'path with embedded token')
128
129 // Real paths should NOT match
130 t.not(isPlaceholder('lib/auth/verify.js', config), 'real path not a placeholder')
131 t.not(isPlaceholder('SomeClass', config), 'class name not a placeholder')
132 t.not(isPlaceholder('foo()', config), 'function call not a placeholder')
133})
134
135test('config: custom placeholder patterns can be added', (t) => {
136 const config = {
137 ...getDefaults(),
138 placeholderPatterns: [...DEFAULT_CONFIG.placeholderPatterns, '^@@internal/']
139 }
140 t.ok(isPlaceholder('@@internal/foo', config), 'custom pattern matched')
141 t.ok(isPlaceholder('~/.config', config), 'default pattern still applied')
142})
143
144test('config: malformed regex in placeholderPatterns is tolerated', (t) => {
145 const config = {
146 ...getDefaults(),
147 placeholderPatterns: ['(unclosed group', '^~/']
148 }
149 // Should not throw; bad regex just doesn't match
150 const result = isPlaceholder('~/foo', config)
151 t.ok(result, 'good pattern still works despite sibling bad regex')
152})
test/cruft-external.test.js +247 -0
……@@ -0,0 +1,247 @@
1import { test } from 'brittle'
2import { mkdirSync, writeFileSync, rmSync } from 'fs'
3import { tmpdir } from 'os'
4import { join } from 'path'
5
6import { scan } from '../lib/scanner.js'
7
8/**
9 * End-to-end checks for Phase 0a invariants.
10 *
11 * I2 — external refs resolve as 'external', not 'missing'.
12 * I3 — placeholder paths are not classified as references at all.
13 * I4 — deprecated modules don't emit undocumented-X warnings.
14 *
15 * Uses minimal in-tempdir fixtures so the test is self-contained and
16 * doesn't couple to pear-git-v3's current spec state (invariant I1).
17 */
18
19function mkFixture (name) {
20 const dir = join(tmpdir(), 'codetree-test-' + name + '-' + Date.now())
21 mkdirSync(dir, { recursive: true })
22 return dir
23}
24
25function cleanup (dir) {
26 rmSync(dir, { recursive: true, force: true })
27}
28
29// --- I2: external symbols resolve, don't flag as missing ---
30
31test('I2: external symbols declared in .codetree.json resolve as external', async (t) => {
32 const dir = mkFixture('i2-externals')
33 try {
34 // Minimal fixture: a fake "project" with a spec that references an external symbol
35 writeFileSync(join(dir, 'package.json'), JSON.stringify({
36 name: 'myproject', version: '0.1.0', main: 'index.js'
37 }))
38 writeFileSync(join(dir, 'index.js'), 'export function hello () { return "hi" }\n')
39 writeFileSync(join(dir, '.codetree.json'), JSON.stringify({
40 externalSymbols: ['an.sign', 'an.open'],
41 externalPackages: ['anproto']
42 }))
43 writeFileSync(join(dir, 'README.md'),
44 '# myproject\n\n' +
45 'Uses `an.sign()` and `an.open()` from `anproto`.\n' +
46 'Our own function: `hello()`.\n'
47 )
48
49 const tree = await scan(dir)
50 t.ok(tree.docCruft, 'docCruft produced')
51 const readme = tree.docCruft.docs.find(d => d.path === 'README.md')
52 t.ok(readme, 'README.md found in report')
53
54 // hello() is local and should resolve as valid
55 const helloDetail = readme.references.details.find(d => d.ref === 'hello()')
56 t.ok(helloDetail, 'hello() reference extracted')
57 t.is(helloDetail.status, 'valid', 'hello() resolves as valid')
58
59 // an.sign() and an.open() should be external, not missing
60 const signDetail = readme.references.details.find(d => d.ref === 'an.sign()')
61 t.ok(signDetail, 'an.sign() reference extracted')
62 t.is(signDetail.status, 'external', 'an.sign() resolves as external, not missing')
63
64 const openDetail = readme.references.details.find(d => d.ref === 'an.open()')
65 t.ok(openDetail, 'an.open() reference extracted')
66 t.is(openDetail.status, 'external', 'an.open() resolves as external')
67
68 // Summary counts separate external from missing
69 t.ok(tree.docCruft.summary.externalReferences >= 2, 'external refs counted separately')
70 } finally {
71 cleanup(dir)
72 }
73})
74
75test('I2: external refs do NOT inflate staleness score', async (t) => {
76 // Same fixture; confirm a doc with only external refs has low/zero staleness.
77 const dir = mkFixture('i2-staleness')
78 try {
79 writeFileSync(join(dir, 'package.json'), JSON.stringify({
80 name: 'myproject', version: '0.1.0', main: 'index.js'
81 }))
82 writeFileSync(join(dir, 'index.js'), 'export const x = 1\n')
83 writeFileSync(join(dir, '.codetree.json'), JSON.stringify({
84 externalSymbols: ['external_thing']
85 }))
86 writeFileSync(join(dir, 'docs.md'),
87 '# docs\n\n' +
88 'References `external_thing` (should not penalize staleness).\n'
89 )
90
91 const tree = await scan(dir)
92 const doc = tree.docCruft.docs.find(d => d.path === 'docs.md')
93 t.ok(doc, 'docs.md found')
94 t.is(doc.references.missing, 0, 'external ref not counted as missing')
95 t.ok(doc.references.external >= 1, 'external ref counted as external')
96 t.ok(doc.staleness.missingRatio === 0, 'missing ratio is 0 for externals-only doc')
97 } finally {
98 cleanup(dir)
99 }
100})
101
102// --- I3: placeholder paths filtered out ---
103
104test('I3: placeholder paths are not classified as references', async (t) => {
105 const dir = mkFixture('i3-placeholders')
106 try {
107 writeFileSync(join(dir, 'package.json'), JSON.stringify({
108 name: 'myproj', version: '0.1.0', main: 'index.js'
109 }))
110 writeFileSync(join(dir, 'index.js'), 'export function run () {}\n')
111 writeFileSync(join(dir, 'README.md'),
112 '# myproj\n\n' +
113 'Config lives at `~/.myproj/config.json`.\n' +
114 'Run `run()` from `<install-dir>/bin/myproj`.\n' +
115 'Paths like `/path/to/install` are examples.\n' +
116 'Env variable: `${MY_PROJ_HOME}`.\n'
117 )
118
119 const tree = await scan(dir)
120 const readme = tree.docCruft.docs.find(d => d.path === 'README.md')
121 t.ok(readme, 'README.md found')
122
123 // run() should be found
124 const runDetail = readme.references.details.find(d => d.ref === 'run()')
125 t.ok(runDetail, 'run() reference extracted')
126
127 // Placeholders should NOT appear in the details at all
128 const placeholderDetails = readme.references.details.filter(d =>
129 d.ref.startsWith('~/') ||
130 d.ref.startsWith('/path/to/') ||
131 d.ref.startsWith('<') ||
132 d.ref.includes('${')
133 )
134 t.is(placeholderDetails.length, 0, 'placeholder refs not in details')
135 } finally {
136 cleanup(dir)
137 }
138})
139
140// --- I4: deprecated modules don't warn about undocumented items ---
141// (Validator-side test; uses a minimal spec fixture.)
142
143test('I4: deprecated module skips undocumented-export/method warnings', async (t) => {
144 const dir = mkFixture('i4-deprecated')
145 try {
146 // Fixture: a project with one module (the "legacy" module) that
147 // exposes methods but is marked deprecated in its spec.
148 const modDir = join(dir, 'legacy')
149 mkdirSync(modDir, { recursive: true })
150 writeFileSync(join(modDir, 'package.json'), JSON.stringify({
151 name: 'legacy', version: '0.1.0', main: 'index.js'
152 }))
153 writeFileSync(join(modDir, 'index.js'),
154 'export class Legacy {\n' +
155 ' foo () {}\n' +
156 ' bar () {}\n' +
157 ' baz () {}\n' +
158 '}\n'
159 )
160
161 // Spec dir at scan root
162 const specDir = join(dir, 'spec')
163 const modulesDir = join(specDir, 'modules')
164 mkdirSync(modulesDir, { recursive: true })
165 writeFileSync(join(specDir, 'SYSTEM.md'),
166 '# System\n\n' +
167 '**Last verified:** 2026-04-16\n\n' +
168 '### Legacy modules\n\n' +
169 '| Module | Location | Purpose |\n' +
170 '|---|---|---|\n' +
171 '| **legacy** | `legacy/` | Deprecated module |\n'
172 )
173 writeFileSync(join(modulesDir, 'legacy.md'),
174 '# legacy -- Module Spec\n\n' +
175 '**Location:** `legacy/`\n' +
176 '**Status:** deprecated\n' +
177 '**Replaced by:** `NewClass`\n\n' +
178 '## What it is\n\nOld module, being removed.\n\n' +
179 '## API\n\n```js\nimport { Legacy } from \'legacy\'\nconst l = new Legacy()\n// (methods no longer documented)\n```\n'
180 )
181
182 const tree = await scan(dir)
183 t.ok(tree.validation, 'validation produced')
184
185 const modReport = tree.validation.modules.legacy
186 t.ok(modReport, 'legacy module validated')
187
188 // There should be NO warn-level "Method X is documented" checks for
189 // legacy's methods, because the module is deprecated.
190 const undocumentedMethodWarnings = modReport.checks.filter(c =>
191 c.status === 'warn' && /Method ".*" is documented/.test(c.claim)
192 )
193 t.is(undocumentedMethodWarnings.length, 0, 'no undocumented-method warnings for deprecated module')
194
195 const undocumentedExportWarnings = modReport.checks.filter(c =>
196 c.status === 'warn' && /Export ".*" is documented/.test(c.claim)
197 )
198 t.is(undocumentedExportWarnings.length, 0, 'no undocumented-export warnings for deprecated module')
199 } finally {
200 cleanup(dir)
201 }
202})
203
204test('I4: non-deprecated module still emits undocumented warnings', async (t) => {
205 // Contrast fixture — same shape, WITHOUT the deprecated marker.
206 const dir = mkFixture('i4-active')
207 try {
208 const modDir = join(dir, 'active')
209 mkdirSync(modDir, { recursive: true })
210 writeFileSync(join(modDir, 'package.json'), JSON.stringify({
211 name: 'active', version: '0.1.0', main: 'index.js'
212 }))
213 writeFileSync(join(modDir, 'index.js'),
214 'export class Active {\n' +
215 ' foo () {}\n' +
216 ' bar () {}\n' +
217 '}\n'
218 )
219
220 const specDir = join(dir, 'spec')
221 const modulesDir = join(specDir, 'modules')
222 mkdirSync(modulesDir, { recursive: true })
223 writeFileSync(join(specDir, 'SYSTEM.md'),
224 '# System\n\n**Last verified:** 2026-04-16\n\n' +
225 '### Active\n\n| Module | Location | Purpose |\n|---|---|---|\n' +
226 '| **active** | `active/` | Active module |\n'
227 )
228 writeFileSync(join(modulesDir, 'active.md'),
229 '# active -- Module Spec\n\n' +
230 '**Location:** `active/`\n\n' +
231 '## What it is\n\nActive module.\n\n' +
232 '## API\n\n```js\nimport { Active } from \'active\'\nconst a = new Active()\n// foo documented, bar is NOT\na.foo()\n```\n'
233 )
234
235 const tree = await scan(dir)
236 const modReport = tree.validation.modules.active
237 t.ok(modReport, 'active module validated')
238
239 // Should warn about `bar` being undocumented
240 const undocumentedMethodWarnings = modReport.checks.filter(c =>
241 c.status === 'warn' && c.claim.includes('Method "bar"')
242 )
243 t.ok(undocumentedMethodWarnings.length > 0, 'undocumented method "bar" emits warning when not deprecated')
244 } finally {
245 cleanup(dir)
246 }
247})
test/validator.test.js +19 -10
……@@ -15,13 +15,18 @@ test('doc-parser: parse hyper-orgs module spec', (t) => {
1515 t.ok(specs.modules['hyper-orgs'], 'found hyper-orgs spec')
1616
1717 const spec = specs.modules['hyper-orgs']
18 t.is(spec.location, 'hyper-fleeky/hyper-orgs', 'location parsed')
19 t.is(spec.testCount, 29, 'test count parsed')
20 t.is(spec.sourceVerified, '2026-04-10', 'source verified date parsed')
21 t.ok(spec.fileLOC['hyper-orgs.js'], 'LOC claim for hyper-orgs.js')
22 t.is(spec.fileLOC['hyper-orgs.js'], 637, 'LOC = 637')
23 t.ok(spec.fileLOC['schema.js'], 'LOC claim for schema.js')
24
18 // Parser invariants — check that fields are extracted and well-typed.
19 // Don't pin to specific spec-snapshot values; editing the spec must
20 // not break codetree's own tests (invariant I1).
21 t.ok(typeof spec.location === 'string' && spec.location.length > 0, 'location is a non-empty string')
22 t.ok(spec.location.includes('hyper-orgs'), 'location mentions the module name')
23 t.ok(typeof spec.testCount === 'number' && spec.testCount >= 0, 'testCount is a number')
24 t.ok(/^\d{4}-\d{2}-\d{2}$/.test(spec.sourceVerified), 'sourceVerified is a yyyy-mm-dd date string')
25 t.ok(typeof spec.fileLOC['hyper-orgs.js'] === 'number', 'LOC claim for hyper-orgs.js is a number')
26 t.ok(spec.fileLOC['hyper-orgs.js'] > 0, 'LOC claim for hyper-orgs.js is positive')
27 t.ok(typeof spec.fileLOC['schema.js'] === 'number', 'LOC claim for schema.js is a number')
28
29 // Semantic invariants — stable facts about hyper-orgs as a module.
2530 t.ok(spec.description, 'has description')
2631 t.ok(spec.dependencies.includes('hyper-rooms'), 'dependency on hyper-rooms')
2732 t.ok(spec.dependents.includes('pear-git-v3'), 'dependent pear-git-v3')
……@@ -36,7 +41,8 @@ test('doc-parser: parse SYSTEM.md', (t) => {
3641 const specs = parseSpecs(SPEC_DIR)
3742
3843 t.ok(specs.system, 'system spec parsed')
39 t.is(specs.system.lastVerified, '2026-04-12', 'last verified date')
44 // Parser invariant — a date was extracted. Don't pin to one value.
45 t.ok(/^\d{4}-\d{2}-\d{2}$/.test(specs.system.lastVerified), 'lastVerified is yyyy-mm-dd')
4046 t.ok(specs.system.modules.length > 5, specs.system.modules.length + ' modules documented')
4147
4248 const orgsEntry = specs.system.modules.find(m => m.name === 'hyper-orgs')
……@@ -44,8 +50,11 @@ test('doc-parser: parse SYSTEM.md', (t) => {
4450 t.ok(orgsEntry.location.includes('hyper-orgs'), 'location includes hyper-orgs')
4551 t.ok(orgsEntry.purpose, 'has purpose')
4652
47 t.ok(specs.system.builtItems.length > 0, 'has built items')
48 t.ok(specs.system.missingItems.length > 0, 'has missing items')
53 // Built items can be empty in theory if a project hasn't shipped anything,
54 // so only require >= 0. missingItems similarly — a well-maintained spec
55 // could have neither. Both arrays must exist; counts shouldn't be pinned.
56 t.ok(Array.isArray(specs.system.builtItems), 'builtItems is an array')
57 t.ok(Array.isArray(specs.system.missingItems), 'missingItems is an array')
4958})
5059
5160test('validator: hyper-orgs spec vs code', async (t) => {