v0.1.0: the full feature set #1
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "phase-2-ingest-at-scale"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Eight phases, 209 commits: archive core, ingest at scale, recall, hooks and
install, context injection, the feedback loop, observations, and retention
and lifecycle.
All 64 requirements are Complete and verified. The traceability audit passes
64/64 requirement -> phase -> plan -> verified, and 57/57 acceptance criteria
are covered by a UAT item.
cargo test --workspace --all-features: 61 suites, 732 tests, 0 failures.
The cycle's planning artifacts are archived under .planning/_archive-v0.1.0/,
with the per-phase narrative in .planning/ARCHIVE.md.
Adds the fourth subcommand (ING-09). Every number comes out of the store and none of it out of a log file, because there is no log file by design: runs is where a pass's failures live, so status prints the store path and its on-disk footprint, session and turn counts, how many transcripts carry a watermark and how many bytes those cover, and the last runs row with its start time, duration, file counts, bytes, turns and its error in full when non-null. It takes no ingest lock. Reading while a pass runs must work - that is what WAL is on for - and a status command that blocked behind an ingest would be useless exactly when it is wanted. An empty store is exit 0 with zeroes rather than an error: that is what a machine looks like before the first hook has ever fired. No --json flag, because RCL-06's stable shapes are phase 3 and an unadvertised flag accepted now becomes a shape to keep; any argument at all is misuse, exit 2, empty stdout. rusqlite is deliberately not named in the command: the binary crate does not depend on it, and a read command is not a reason to put a second SQL dependency in the hook path's build. Also drops a duplicated path from the one channel a user reads. Error::Io already renders as "{path}: {source}", so prefixing every reason with its path printed the path twice in runs.error and on the pass's stderr.Four synthetic transcripts shaped to what this phase must prove, registered in testkit::TRANSCRIPT_FIXTURES and documented in the fixtures README. session-recall.jsonl carries AC1's two probes - a SearchManager turn and an src/worker/S.ts turn - and AC3's structured-versus-prose pair: a Read tool_use whose file_path is docs/RETRY.md against a separate assistant turn naming that same path in prose and nothing else. session-errors-a/b.jsonl are two sessions holding the two stderr pairs AC2 is about. One pair differs only in a :line:col suffix, a 0x address, an ISO timestamp and a UUID; the other only in a bare integer, which D-04 measured must NOT be stripped. The a-file ends with an interrupted result carrying an empty stderr and no error flag, so 'an interruption is not an error' has a control. subagents/agent-echo.jsonl repeats session-recall.jsonl's last turn word for word, which is what gives D-07's 'sorts below at equal score' two genuinely equal BM25 scores rather than a tie that never happens. All four carry {{ROOT}} where a real transcript carries an absolute cwd, over two projects; testkit::copy_rooted_fixture_into rewrites it to a root the test owns and creates the project directory. The phase 1 and 2 fixtures hardcode /data/code/verbatim, which makes a project-scoped assertion true only on a checkout at that path and true for the wrong reason on that one.`path_words` split a command line on whitespace only, so the punctuation around a path was stored as part of the key: `cd /a/b; make` wrote `/a/b;`, `cargo test 2>/dev/null` wrote `2>/dev/null`, and a lookup for either real path missed the turn that named it. Measured over a 400-file sample, 9,816 of 21,061 path-shaped words (46.6%) carried a shell control character, so this was the common case rather than the tail - and `paths` is the table RCL-02 exists for. Every word is now split again on the shell's own control characters (`;&|<>()`), and quotes are trimmed from either end whether or not the partner survived the split. A surviving word is dropped when it carries an expansion, a glob or a URL scheme: `${ROOT}/x`, `src/*.rs` and `https://example.com/x` name something other than one file on disk. That is not RCL-04's forbidden rejection-for-being-common - a glob is not a path that happens to be popular, it is not a path. A substitution opened in one word and closed in the next stays past what a split can see (151 of the 21,061 measured words). The test states that outcome rather than asserting it away; separating it needs a parser.RCL-06's envelope: {command, ok, reason, data}, built as serde_json values and serialized, never assembled with format!. An excerpt cut out of a transcript carries quotes, backslashes and control bytes, and a hand-rolled writer fails on exactly the turns most worth reading while validating fine on the ones that are not. D-24 retrofits the flag onto the three commands phase 1 and 2 shipped, in the phase that owns the contract rather than beside install in phase 4 where something is already scripted against a shape. `no_more_arguments` gives way to `cmd::json_flag`: a subcommand still rejects every argument it was not written for, and --json is now one it was. Plain output is unchanged - verify and reindex still write nothing to stdout on a clean store, status still prints its prose - and the commentary stays on stderr in JSON mode so `verbatim verify --json | jq` works while a warning prints. Exit codes do not move. verify --json still exits 1 with the failing session ids in the document, and reindex --json reports a held lock as ok:false rather than an empty stdout a caller could not tell from a crash. The emitter and its first consumers land together: serde_json is named in the binary crate here (one Cargo.lock line, no newly compiled crate, since verbatim-core already links it), and a module nothing calls is dead code the lint gate refuses.`Json::entry` takes the first member with a key. Every real JSON consumer, `JSON.parse` included and so Claude Code, takes the last. Given `{"hooks":{}, "hooks":{...}}` install would have written its four entries into the object Claude Code ignores, reported success, and left no hook firing at all. There is no reading of a duplicate under which rewriting the whole file is safe, so a repeated key at any depth is an operational refusal that names where it is - `.hooks`, `.hooks.SessionStart`, `.hooks.SessionStart[0].matcher` - and says why. Nothing is written: the check runs inside `Document::read`, which is before install places the binary or touches either file.A `capture` module holding the elision itself: one record's JSON line in, the bytes to store out. Under `full` the line comes straight back and is never parsed, which is what keeps the default byte-identical and free. Under `lean` and `minimal` the top-level `toolUseResult` and `attachment` values are replaced by `{"verbatimElided": <bytes>}` - the key names the elision, its value is what stood there - and the line is re-serialized. `minimal` elides both unconditionally; `lean` elides one only when its serialized value passes 8 KB, the breakpoint the phase measured and the reading of the brief's "elide LARGE bodies". Without a threshold the two modes would store the same bytes. This is elision, not redaction: a whole named subtree goes and a mark says so. Nothing is rewritten in place and no value is partly kept. Nothing else in the line moves either - `type`, `uuid`, `timestamp`, `sessionId`, `cwd`, `subtype` and `compactMetadata` are what classifies a record, so an elided record is still the same turn, of the same type, at the same ordinal. A line carrying neither key, and a line that is not a JSON object at all, come back untouched and are never re-serialized, so only the records that shrink pay the round trip. `tests/fixtures/session-capture.jsonl` covers every arm: a tool result over the threshold and one under, an attachment over and one under, and a plain prompt and assistant turn with neither. It stays out of `TRANSCRIPT_FIXTURES`, whose members are ingested wholesale by tests that assert counts.