traceability: give CONTEXT acceptance criteria IDs so an audit can prove every criterion reached the checklist #76

New issue

Closed

opened 2026-07-28 19:12:39 +00:00 by crenshawdev · 1 comment

crenshawdev commented

2026-07-28 19:12:39 +00:00

(Migrated from github.com)

Split out of #70, which was rejected on its own kill condition. This is the defect that measurement actually found.

What happened

Two CONTEXT acceptance criteria in v1.4.0 phase 1 got no UAT item at all when the checklist was built:

AC4: files: ["a\"b.md", "c\"d.md"] produces a diagnostic rather than issues: []
AC5: plan-overlap reports src/x(1) and a backtick-bearing path as overlapping byte-exact

Round 1 of the checklist (95ddced, 15 items) contains neither. They appear only in round 2 (9330e9c, 23 items) as items 16 and 17. They were recovered because a second /cad-verify pass happened to run, not because anything detected the omission.

2 of 122 acceptance criteria across the shipped history, so 1.6%, silently dropped at checklist-build time.

Why this is not #70

#70 proposed IDs on truths and UAT items to stop wording drift breaking a merge. The measurement found zero instances of that across 139 merged findings, for a structural reason: k already serves as the ID on the one boundary where a merge happens.

This failure is a different shape. Nothing drifted and no merge mismatched. A criterion simply never became an item, and no artifact records that it should have. IDs on truths and UAT items would not have caught it. An ID on the acceptance criterion would, because then an audit can assert a total function: every AC-ID in CONTEXT has a UAT item pointing at it.

What you want Cadence to do

Give CONTEXT acceptance criteria stable IDs, and have /cad-audit assert coverage in both directions: every AC has at least one UAT item, and every UAT item traces to an AC or is explicitly marked as a verifier-added gap.

The pattern is already proven in-repo rather than imported. REQUIREMENTS.md assigns IDs, plan frontmatter carries requirements, and /cad-audit traces by ID. This is the same mechanism applied one artifact further down.

What it costs

Ordinals are not IDs, and the data says so. UAT k tracks the AC ordinal in only 98 of 122 cases (80%); the rest are offsets where an extra item got interleaved. So k cannot be reused as the AC identifier, and a new stable ID is needed rather than a positional convention.

Renumbering. /cad-phase already handles phase renumbering on insert or remove; AC IDs would need the same care, and an ID that renumbers under the user is worse than none.

Scope discipline, carried over from #70. Extend IDs only where an audit consumes them. Any artifact that gains an ID nothing reads is bookkeeping the user pays for.

The honest counter-argument. n is 2. One phase, one cycle, and a second verify pass recovered both. If the fix costs more than a second /cad-verify pass, it is not worth it. What tips it is that the recovery was luck rather than detection: nothing reported the gap, and on a phase where no second pass runs, the criteria stay dropped and the phase reads as verified.

#70, rejected, where this surfaced.

Split out of #70, which was rejected on its own kill condition. This is the defect that measurement actually found. ## What happened Two CONTEXT acceptance criteria in v1.4.0 phase 1 got no UAT item at all when the checklist was built: - AC4: ``files: ["a\"b.md", "c\"d.md"]`` produces a diagnostic rather than `issues: []` - AC5: `plan-overlap` reports `src/x(1)` and a backtick-bearing path as overlapping byte-exact Round 1 of the checklist (`95ddced`, 15 items) contains neither. They appear only in round 2 (`9330e9c`, 23 items) as items 16 and 17. They were recovered because a second `/cad-verify` pass happened to run, not because anything detected the omission. **2 of 122 acceptance criteria across the shipped history, so 1.6%, silently dropped at checklist-build time.** ## Why this is not #70 #70 proposed IDs on truths and UAT items to stop wording drift breaking a merge. The measurement found zero instances of that across 139 merged findings, for a structural reason: `k` already serves as the ID on the one boundary where a merge happens. This failure is a different shape. Nothing drifted and no merge mismatched. A criterion simply never became an item, and no artifact records that it should have. **IDs on truths and UAT items would not have caught it.** An ID on the acceptance criterion would, because then an audit can assert a total function: every AC-ID in CONTEXT has a UAT item pointing at it. ## What you want Cadence to do Give CONTEXT acceptance criteria stable IDs, and have `/cad-audit` assert coverage in both directions: every AC has at least one UAT item, and every UAT item traces to an AC or is explicitly marked as a verifier-added gap. The pattern is already proven in-repo rather than imported. `REQUIREMENTS.md` assigns IDs, plan frontmatter carries `requirements`, and `/cad-audit` traces by ID. This is the same mechanism applied one artifact further down. ## What it costs **Ordinals are not IDs, and the data says so.** UAT `k` tracks the AC ordinal in only 98 of 122 cases (80%); the rest are offsets where an extra item got interleaved. So `k` cannot be reused as the AC identifier, and a new stable ID is needed rather than a positional convention. **Renumbering.** `/cad-phase` already handles phase renumbering on insert or remove; AC IDs would need the same care, and an ID that renumbers under the user is worse than none. **Scope discipline, carried over from #70.** Extend IDs only where an audit consumes them. Any artifact that gains an ID nothing reads is bookkeeping the user pays for. **The honest counter-argument.** n is 2. One phase, one cycle, and a second verify pass recovered both. If the fix costs more than a second `/cad-verify` pass, it is not worth it. What tips it is that the recovery was luck rather than detection: nothing reported the gap, and on a phase where no second pass runs, the criteria stay dropped and the phase reads as verified. ## Related - #70, rejected, where this surfaced.

john referenced this issue from a commit

2026-07-30 04:10:56 +00:00

docs: open v2.0.0 - Stakes, not spend

john commented

2026-07-30 04:51:48 +00:00

Owner

Shipped in v2.0.0 as ACR-01.

CONTEXT acceptance criteria carry AC<N> ids, UAT items carry a criterion: field (cadence-core/templates/UAT.md), the grammar is written down in cadence-core/references/acceptance-criteria.md, and planning.mjs criteria-coverage backs a /cad-audit arm that FAILs on a criterion which reached no UAT item, and reports an item tracing to no criterion unless it is marked verifier-added.

One residue worth knowing, carried into v2.0.1 rather than left here: a checklist with no fields_version frontmatter marker is treated as legacy, which drops every one of its criteria out of the coverage domain silently. That is how v2.0.0's own closing audit counted 36 criteria where its six phases declared 45, and still reported a green gate.

Shipped in v2.0.0 as ACR-01. CONTEXT acceptance criteria carry `AC<N>` ids, UAT items carry a `criterion:` field (`cadence-core/templates/UAT.md`), the grammar is written down in `cadence-core/references/acceptance-criteria.md`, and `planning.mjs criteria-coverage` backs a `/cad-audit` arm that FAILs on a criterion which reached no UAT item, and reports an item tracing to no criterion unless it is marked verifier-added. One residue worth knowing, carried into v2.0.1 rather than left here: a checklist with no `fields_version` frontmatter marker is treated as `legacy`, which drops every one of its criteria out of the coverage domain silently. That is how v2.0.0's own closing audit counted 36 criteria where its six phases declared 45, and still reported a green gate.

john closed this issue

2026-07-30 04:51:55 +00:00