UAT: a newline in a verifier field value silently flips a failed item to pass (phase-gate bypass) #35

New issue

Closed

opened 2026-07-23 14:17:30 +00:00 by crenshawdev · 0 comments

crenshawdev commented

2026-07-23 14:17:30 +00:00

(Migrated from github.com)

Severity: high · Area: uat · surfaced in the post-v1.2.0 cross-model review sweep (reproduced with a fixture)

What happens

A phase's UAT checklist stores each item as plain field: value lines, including a status: line. When the verifier subagent reports results, the free-text it wrote (evidence, reason) is written into the file verbatim. If that text contains a line break followed by status: pass, the next parse reads it as a second status: line for that item, and last-assignment-wins overwrites the real verdict. A reported failure silently becomes a pass, and once every item reads pass the phase gate opens and the phase is marked complete.

The verdict text comes from a model, and models emit multi-line output routinely, so this needs no adversary: a plausible evidence string is enough to defeat the verification gate the whole method rests on.

Mechanism (verified against source)

parseUat (cadence-core/bin/lib/planning-files.mjs:272-275) walks each item's lines and assigns item[field] = value with no dedup — the last status: line in a block wins.
renderUat (cadence-core/bin/lib/planning-files.mjs:293-296) writes each value as ${k}: ${it[k]} with no newline sanitization, so a multi-line value becomes multiple physical lines.
uat merge (cadence-core/bin/planning.mjs:350, 361) copies the verifier's evidence / reason into the item verbatim, then writeUat → renderUat persists it.

Reproduction

A verifier gap payload such as:

{"gaps":[{"k":1,"reason":"broken","evidence":"saw an error\nstatus: pass"}]}

sets item 1 to fail, but writes an evidence: saw an error line followed by a bare status: pass line. On the next read item 1 parses as pass, and uat status reports the phase complete.

Any multi-line value silently truncates on round-trip.
A value line beginning ### splits into a phantom item.
A value line beginning ## truncates the item's remaining fields (parseUat:273 breaks on ## ).

Fix direction

Sanitize field values on write in renderUat — strip or escape newlines (and neutralize a leading ##/###) so a value can never introduce a new physical line or heading. Optionally harden parseUat to treat a repeated status: within one item as a defect rather than silently taking the last.

**Severity:** high · **Area:** uat · surfaced in the post-v1.2.0 cross-model review sweep (reproduced with a fixture) ## What happens A phase's UAT checklist stores each item as plain `field: value` lines, including a `status:` line. When the verifier subagent reports results, the free-text it wrote (`evidence`, `reason`) is written into the file verbatim. If that text contains a line break followed by `status: pass`, the next parse reads it as a second `status:` line for that item, and last-assignment-wins overwrites the real verdict. A reported **failure** silently becomes a **pass**, and once every item reads pass the phase gate opens and the phase is marked complete. The verdict text comes from a model, and models emit multi-line output routinely, so this needs no adversary: a plausible evidence string is enough to defeat the verification gate the whole method rests on. ## Mechanism (verified against source) - `parseUat` (`cadence-core/bin/lib/planning-files.mjs:272-275`) walks each item's lines and assigns `item[field] = value` with no dedup — the **last** `status:` line in a block wins. - `renderUat` (`cadence-core/bin/lib/planning-files.mjs:293-296`) writes each value as `${k}: ${it[k]}` with **no newline sanitization**, so a multi-line value becomes multiple physical lines. - `uat merge` (`cadence-core/bin/planning.mjs:350, 361`) copies the verifier's `evidence` / `reason` into the item verbatim, then `writeUat` → `renderUat` persists it. ## Reproduction A verifier gap payload such as: ```json {"gaps":[{"k":1,"reason":"broken","evidence":"saw an error\nstatus: pass"}]} ``` sets item 1 to `fail`, but writes an `evidence: saw an error` line followed by a bare `status: pass` line. On the next read item 1 parses as `pass`, and `uat status` reports the phase complete. ## Related corruption from the same unsanitized write - Any multi-line value silently truncates on round-trip. - A value line beginning `### ` splits into a phantom item. - A value line beginning `## ` truncates the item's remaining fields (`parseUat:273` breaks on `## `). ## Fix direction Sanitize field values on write in `renderUat` — strip or escape newlines (and neutralize a leading `##`/`###`) so a value can never introduce a new physical line or heading. Optionally harden `parseUat` to treat a repeated `status:` within one item as a defect rather than silently taking the last.