v3.6.0 - reading the corpus back #239

Merged
john merged 69 commits from cadence/v3.6.0 into main 2026-08-23 18:24:25 +00:00
Owner

Three phases, 66 commits off v3.5.9. /cad-why walks the record join, cite-count measures whether recall was read, and /cad-task leaves a record behind it. Audit PASS on both arms, 5 of 5 requirements traced, 21 of 21 acceptance criteria covered.

Closes #190, #191, #192.

Three phases, 66 commits off v3.5.9. /cad-why walks the record join, cite-count measures whether recall was read, and /cad-task leaves a record behind it. Audit PASS on both arms, 5 of 5 requirements traced, 21 of 21 acceptance criteria covered. Closes #190, #191, #192.
john added 69 commits 2026-08-23 18:24:21 +00:00
/cad-why goes first because RBK-01 cannot read its own number without it. The
read-back gate counts how often a plan cites the corpus, and a near-zero count
has two readings that need opposite fixes: the planner ignored a good record, or
it correctly ignored a useless one. Nothing in the count separates them, so the
corpus gets read by hand first.
Pure module: split a positional <path>[:<line>] argument into a path and an
optional line with five distinct named refusals, build the bare-path and
line-scoped git log argv arrays (--follow/-L are mutually exclusive so they
stay separate builders, both pin -M and a fixed --format), and classify a
git result into ok/not-in-history/line-past-end/git-failed without carrying
any stderr bytes onward.
Pure module: sort a chain newest-first by commit date then full sha (both
descending, so a tied date and a reversed input array render identically),
carry the full sha with an 8-char abbreviation on the commit line, render a
fixed stated-absence line for each of the five join fields an entry lacks,
and cap the rendered chain at 10 entries with the untruncated total riding
the envelope and a truncation note inside the text itself.
New top-level cadence-core/bin/why.mjs: runs the explicit not-in-history
probe before either chain query, dispatches the bare-path or line-scoped
git invocation from lib/why-query.mjs, classifies the result through the
same module, and emits one JSON object carrying the resolved path/line, the
chain entries and lib/why-render.mjs's pre-rendered text. The declared flag
door (--dir, --top) runs before the positional-query refusal so the
adoption census can prove each declared refusal by flag name with no
positional argument at all. Adds the why.mjs row to arg-contract.mjs's
CONTRACTS table, and extends why-render.mjs's renderChain to return its
sorted, capped entries alongside the rendered text.
The seam lands as a discoverable skill (D-14): a self-contained
skills/cad-why/SKILL.md that runs why.mjs and relays its text field
verbatim, allowed-tools Bash alone so this phase can neither write nor
dispatch. Its prescribed invocation single-quotes the caller-supplied
query, because a double-quoted $(...) or backtick in a path runs before
Node starts.

COMMANDS.md gains the Support-cluster row skills/cad-help renders, and
weight-budgets.json gains the skill's row and re-pins COMMANDS.md, which
the new row grows past its old ceiling. README's skill count moves 27 to
28, the figure prose-agreement.test.mjs counts on disk.

arg-contract.test.mjs's literal table counts move 168 to 170 and 16 to 17
to match the why.mjs CONTRACTS row.
phaseDirsIn contains the phase-directory walk against a symlink escaping
the planning root, and why-corpus.mjs's header says so. Every caller then
joins a NAME onto a contained directory, and statSync following that
name's own symlink put the escape back one level down: a SUMMARY.md
symlinked outside the tree passes isFile() and its bytes reach a seam
whose stdout is the place EXP-01 was about.

The resolved path must now stay inside the resolved directory it was
joined onto. A non-symlink resolves to itself and passes trivially; a
symlink pointing out is refused EESCAPE before it is opened, which also
closes the check-to-use race on the same line, since a path a symlink
names cannot be swapped for one outside the directory after a check it
never passed.
Seventeen plan commits plus one orchestrator fix deliver /cad-why: a
queried path's commits joined to six record edges across the live
phases/ tier, the _archive-v<ver>/ tier and pruned milestones recovered
out of git history, with the gap named in words where no tier answers.

risk_surface fired once per plan on each committed range and passed all
three: plan 1 raised 2 and kept none, plan 2 raised 3 and its one
survivor is fixed in b5f49bad, plan 3 raised 1 and kept none. CONTEXT
D-10 and D-13 are corrected in place where plan deviations refuted them.
PLAN-1.md carries the lease widening approved at plan 1's structural
checkpoint.
One `planning.mjs` subcommand joins the two readers per item: the surfaced
set (bounded `results`, own-phase rows dropped, four kinds) against the
plan's D-NN mentions, scoped by phase. It REPORTS and never gates.

`cited_by_kind` carries all four arms always, with capture, deviation and
uat marked `unjoinable` rather than `cited: 0` - those artifacts carry no
identifier to join on, and absence and silence are different answers.

The `cite-count` row in CONTRACTS is one word, `--point` is refused against
its two values in the seam's own `bad-args` vocabulary, and the flag-entry
census moves 170 -> 173 for the three new rows.
The handler reads the effective backend across the config layers, the same
read `cmdRecall` makes with the same `builtin` schema default. On `none`
the envelope carries `backend: 'none'` and `--payload` is neither required
nor read - there is no envelope to hand over, because the workflow skips
the call that would have produced one, and `cmdRecall` ignores its query on
the same arm.

That is what makes three runs separable by their recorded fields alone:
`backend: 'none'`, a `surfaced.count` of 0 without it, and a non-empty
`surfaced` with `cited.count` 0. No fourth field restates any of it, and
the envelope shape is identical on both paths.

A torn config layer reads the key as absent and would default a deliberate
`none` back to `builtin`, so `warnings[]` rides the envelope when non-empty.
`cmdCiteCount` inlined its own `mergeLayers` + `?? 'builtin'` pair beside
`cmdRecall`'s, which is exactly how the off switch comes to mean one thing
at the seam that produces the surfaced set and another at the seam that
counts it. Both now read `memoryBackend(dir)`.

It also keeps the check-12 census at fifteen callsites over nine files.
That census exists so a new callsite cannot be added without choosing an
arm; the arm chosen here is to add no callsite.

`warnings` is returned rather than surfaced - the reader has no envelope -
and both callers keep putting it on theirs unchanged.
Five cases over scratch fixtures, spawning the real binary: the envelope's
two sides per item with the three unjoinable arms asserted as unjoinable
rather than as zero; the own-phase exclusion proved in BOTH directions on
one payload, so a rule that dropped both rows or kept both fails; two runs
byte-compared on raw stdout; the fixture's trace.jsonl read back and proved
to hold no lifecycle/dispatch, seeded with an anchor so an empty file
cannot pass it vacuously; and the three memory.backend states separated by
their recorded fields alone.

Falsifier run: deleting the own-phase exclusion from lib/cite-surfaced.mjs
fails exactly one of the five, the own-phase case.
Every site that echoes a `mktemp -d` directory for a later Bash invocation to
paste back as a literal now refuses the path at creation unless it is drawn
from `[A-Za-z0-9._/-]`. The path is built from `$TMPDIR`, which the operator
does not always own - a cloned repo's `.envrc`, a devcontainer, a CI runner -
and a single `"` in it closes the argument at the use site and runs the rest of
the directory name as commands.

Refused at creation rather than quoted defensively at each use, because the
carried literal is unquoted by construction: the workflow prose tells the model
to paste the echoed value, so there is no single place a quoting rule could
hold. The class is deliberately narrow and a `TMPDIR` holding a space is
refused too, which is one `export` away from fixed, where a path that executes
is not.

All three sites, not just the one the gate fired on: `plan.md` had inherited
the pattern from `review-triggers.md`, and `triage-gate.md` carries a third
copy.
A new lib/task-record.mjs owns four facts the writer, the recall walk and
/cad-why's task tier would otherwise each hold their own version of: the
tasks/ directory and RECORD.md names, the slug predicate, the guarded lister
and the pure renderer.

The slug predicate REFUSES rather than sanitises - the VAL-01 lesson from
milestone-prune --label, which was only trimmed before being joined onto a
directory path. The lister contains its walk the way phaseDirsIn does and one
level further in: RECORD.md itself is resolved and required to land inside the
planning root, because the recall tier reads snippets straight out of the path
it returns.

The renderer's sections are the corpus's own grammar - ## What shipped for the
recall tier, a three-column ## Commits table parseCommitRows reads back, and a
last ## Files section taskDeclaredFiles reads back - so the record joins to
shipped readers instead of needing new ones.
One-word `task-record` on planning.mjs, with a declared CONTRACTS row for its
five flags and the pinned entry count moved 173 -> 178 in the same commit, since
that assertion is what makes the contract walk non-vacuous.

Every figure is DERIVED from the range: one `git log` for the commits table and
one `git diff --name-only` for the declared-files line, both over the ids
resolveRange resolved. Nothing is retyped onto a flag, which is the
transcription surface D-07 names, and it is why a re-run over an unchanged range
rewrites byte-identical bytes.

A slug that is not one path segment is refused with nothing written. A tree with
no planning root gets neither one nor a tasks/ directory - written:false with a
reason, ok:true - and the outcome event lands before the envelope on every path
past argument validation, so a refusal still says a record was attempted.

The redactUrl census in planning.test.mjs answered the classification: the git
catch is wrapped, the mkdir/atomicWrite catch is not, because that detail is an
fs error over the path --dir just named.
parseTaskRecordSnippets is its OWN reader beside parseSummarySnippets, not a
call to it: that one indexes ## Deviations and ## Open items alone, which is the
measured reason a task record was invisible to recall even in a SUMMARY shape -
a query naming exactly what a shipped task did returned five hits over 59 and
none from .planning/tasks/.

cmdRecall's walk gains the tier LAST, on the identical argument the ARCHIVE.md
walk makes for its own position: search() orders by score then corpus position,
so appending leaves every existing index where it was and a tree with no tasks/
emits the bytes it emitted before this walk existed - asserted against an empty
tasks/ directory rather than assumed.

No phase key on a task row. references/recall.md states phase is optional and
that a reader must never substitute an inferred one, and phase: 0 would be
exactly that.
Produced by RUNNING task-record over the bound-plan-size run's own committed
range (093408c9^..6ed57a7a), never hand-written: a hand-written file proves
nothing about the writer and is the model-holding-Write shape D-07 rejects.

The indexed text is the record's own words, drawn from that task's ## Outcome
section, so the corpus carries what shipped rather than a fresh claim about it.
recall "plan size ceiling max_plan_tasks task count" now returns
tasks/bound-plan-size/RECORD.md, which makes AC1's check a statement about the
corpus rather than about a fixture.
`isTaskSlug` refuses a slug that traverses; it cannot see a
`tasks/<slug>` that already IS a symlink, and git carries symlinks, so a
cloned planning tree ships one. `mkdirSync(recursive)` follows it and
`atomicWrite` does not catch it: it lstats its own temp path, which
refuses a symlinked destination file and says nothing about a symlinked
parent directory. The record landed in a tree `taskRecordsIn` would then
refuse to read it back from.

The lister's containment rule becomes `insideRoot` and the writer answers
it too, so reader and writer agree about where a record may live.
taskRecordsIn contained every path it walked but judged no NAME, so a
tasks/ entry called `a\n## Commits` or one carrying a terminal escape
resolved inside the planning root and reached /cad-why's rendered output
and the recall index verbatim. The lister now applies isTaskSlug - the
same predicate the writer is held to - before joining the name onto
anything.
PLAN-3.md's Notes recorded both re-readings at plan time and nothing
carried them back: AC3's third clause contradicted its own first, and
AC5's grep already failed against a file this phase never touched on
those lines. CONTEXT.md now states what was contracted and SUMMARY.md
records the two as deviations.
Twelve stale claims found by a docs-verify sweep of the root docs and
docs/, each re-checked against the code before the edit:

- command and skill counts (README, MANIFESTO, LINEAGE) - 28 commands,
  34 skill dirs of which 6 are contracts, 6 agent roles across 19 rungs
- config.mjs's subcommand list gained get and lost the pruned mode key,
  and the schema is 78 leaves rather than ~50 (DESIGN)
- git-segments.mjs is eighty-five lines, as README and INTERNALS
  already said (DESIGN)
- the install URL is the self-hosted origin (DESIGN)
- the canonical .planning/ set now names what actually ships, including
  v3.6.0's tasks/<slug>/RECORD.md (DESIGN)
- risk_surface fires once per plan only in /cad-execute, once per run
  elsewhere, and the ephemeral-target drop needs an empty git ls-files
  only for a DIRECTORY target (docs/WORKFLOW)
- route-table's own note said five triggers where the schema defines four
- suite timings, which had roughly doubled (CONTRIBUTING)
docs: close v3.6.0, open v3.6.1 over the gaps it named
Some checks failed
test / node-test (git, 22) (pull_request) Successful in 30s
test / node-test (git, 24) (pull_request) Successful in 34s
test / node-test (other, 22) (pull_request) Failing after 1m17s
test / node-test (other, 24) (pull_request) Failing after 1m11s
test / node-test (planning, 22) (pull_request) Successful in 1m40s
test / node-test (planning, 24) (pull_request) Successful in 1m51s
test / node-test (prose, 22) (pull_request) Successful in 31s
test / node-test (prose, 24) (pull_request) Successful in 32s
test / node-test (review, 22) (pull_request) Successful in 17s
test / node-test (review, 24) (pull_request) Successful in 17s
test / node-test (routing, 22) (pull_request) Successful in 34s
test / node-test (routing, 24) (pull_request) Successful in 36s
test / self-verify (pull_request) Successful in 12s
test / typecheck (pull_request) Successful in 16s
e7dddc3bf0
john merged commit 085130a412 into main 2026-08-23 18:24:25 +00:00
Sign in to join this conversation.
No description provided.