review: drop the anti-padding clauses that make reviewers self-suppress ahead of the adjudication filter #66

New issue

Open

opened 2026-07-28 17:41:23 +00:00 by crenshawdev · 0 comments

crenshawdev commented

2026-07-28 17:41:23 +00:00

(Migrated from github.com)

What you want Cadence to do

Delete two clauses.

agents/cad-reviewer.md:70 currently reads:

- No severity inflation; no padding with style nits that do not change behavior.

Keep the first half. Drop the second. agents/cad-plan-checker.md:87 carries the matching shape and gets the same treatment:

- No severity inflation: a finding that would not stop the goal stays a WARNING.

That one is already clean, since it defines a severity floor rather than instructing suppression. It is listed here only so the pass covers both files and the divergence is deliberate rather than accidental.

Net change: one clause removed from one file, and a check that the sibling contract does not grow the same clause back.

Why the default no is wrong here

This is a deletion, so the usual objection does not apply. The question is whether the clause earns its place, and there are three reasons it does not.

It filters ahead of a filter that already exists. The default plan and pre_ship gates are adjudicated. On that path the main model opens the cited code and kills false positives and overstatements itself. A reviewer that pre-suppresses is double filtering, and only one of the two filters can actually read the surrounding code.

Opus 5 follows conservatism instructions literally. Anthropic's Opus 5 prompting guide is explicit: "If your review prompt says 'only report high-severity issues' or 'be conservative,' the model may follow that instruction literally and report less; ask it to report everything and filter in a separate pass instead." Cadence's review subsystem is built on exactly the separate-pass architecture that guidance recommends, and then instructs the first pass to self-filter anyway.

The asymmetry runs the wrong way. For a subsystem whose stated job is to refute rather than bless, a suppressed finding is the expensive failure and a padded one is cheap. The adjudicator drops a nit in a sentence. Nothing recovers a finding the reviewer decided not to mention.

What the current surface cannot earn: a reviewer whose recall is actually the reviewer's recall. Today it is the reviewer's recall minus whatever it judged too small to say, and that subtraction is invisible to every downstream consumer.

The clause is partially self-cancelling already, since both files are balanced by "One pass. Report everything you find now; there is no second look." That makes this a nudge rather than a live defect, which is why it is a proposal and not a bug. But two instructions pointing opposite directions in the same guardrails block is the exact overlapping-guidance cost Anthropic describes in the context engineering post, and the resolution should be stated rather than left to the model.

What it costs

Noisier reviews. Real, and the honest cost. Expect more low-severity findings per pass, and the adjudicated gates absorb them.

Nothing new can fail. No command, no config key, no agent, no track. The change removes text from an existing prompt.

What must not be lost. Severity accuracy is load-bearing: the blocking gate keys off the blocker/high threshold, so a reviewer that inflates severity can hard-stop a phase on a nit. "No severity inflation" stays for that reason, in both files, and it should be called out in the diff review that this half was kept deliberately.

The measurement that would validate or kill it. Findings-per-pass and adjudicator rejection rate, before and after, on the same diffs. If the extra findings are all rejected by the adjudicator, the clause was doing useful work and this proposal is wrong. No such measurement exists yet, and the sample size available from one developer's phases is thin enough that it may stay a judgment call rather than a measured one. Say so rather than pretending otherwise.

Why this belongs in `main` and not a fork

The reviewer contract is the product. Cadence's pitch is that plans and diffs get refuted rather than approved, and this clause quietly reduces how much refuting happens, on the default path, for every user.

There is nothing user-specific about it. A fork would not be adapting the review posture to a different workflow; it would be fixing a prompt that instructs against the subsystem's own purpose. If the clause is wrong, it is wrong in main.

The tuning that legitimately belongs to a user already exists and is untouched: review.mode, review.triggers.*.gate, and the reviewer set are all config. Someone who wants a quieter review has four gates and three modes to reach for, none of which requires a reviewer that keeps findings to itself.

## What you want Cadence to do Delete two clauses. `agents/cad-reviewer.md:70` currently reads: ``` - No severity inflation; no padding with style nits that do not change behavior. ``` Keep the first half. Drop the second. `agents/cad-plan-checker.md:87` carries the matching shape and gets the same treatment: ``` - No severity inflation: a finding that would not stop the goal stays a WARNING. ``` That one is already clean, since it defines a severity floor rather than instructing suppression. It is listed here only so the pass covers both files and the divergence is deliberate rather than accidental. Net change: one clause removed from one file, and a check that the sibling contract does not grow the same clause back. ## Why the default no is wrong here This is a deletion, so the usual objection does not apply. The question is whether the clause earns its place, and there are three reasons it does not. **It filters ahead of a filter that already exists.** The default `plan` and `pre_ship` gates are `adjudicated`. On that path the main model opens the cited code and kills false positives and overstatements itself. A reviewer that pre-suppresses is double filtering, and only one of the two filters can actually read the surrounding code. **Opus 5 follows conservatism instructions literally.** Anthropic's [Opus 5 prompting guide](https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/prompting-claude-opus-5) is explicit: *"If your review prompt says 'only report high-severity issues' or 'be conservative,' the model may follow that instruction literally and report less; ask it to report everything and filter in a separate pass instead."* Cadence's review subsystem is built on exactly the separate-pass architecture that guidance recommends, and then instructs the first pass to self-filter anyway. **The asymmetry runs the wrong way.** For a subsystem whose stated job is to refute rather than bless, a suppressed finding is the expensive failure and a padded one is cheap. The adjudicator drops a nit in a sentence. Nothing recovers a finding the reviewer decided not to mention. What the current surface cannot earn: a reviewer whose recall is actually the reviewer's recall. Today it is the reviewer's recall minus whatever it judged too small to say, and that subtraction is invisible to every downstream consumer. The clause is partially self-cancelling already, since both files are balanced by "One pass. Report everything you find now; there is no second look." That makes this a nudge rather than a live defect, which is why it is a proposal and not a bug. But two instructions pointing opposite directions in the same guardrails block is the exact overlapping-guidance cost Anthropic describes in [the context engineering post](https://claude.com/blog/the-new-rules-of-context-engineering-for-claude-5-generation-models), and the resolution should be stated rather than left to the model. ## What it costs **Noisier reviews.** Real, and the honest cost. Expect more low-severity findings per pass, and the adjudicated gates absorb them. **Nothing new can fail.** No command, no config key, no agent, no track. The change removes text from an existing prompt. **What must not be lost.** Severity accuracy is load-bearing: the blocking gate keys off the blocker/high threshold, so a reviewer that inflates severity can hard-stop a phase on a nit. "No severity inflation" stays for that reason, in both files, and it should be called out in the diff review that this half was kept deliberately. **The measurement that would validate or kill it.** Findings-per-pass and adjudicator rejection rate, before and after, on the same diffs. If the extra findings are all rejected by the adjudicator, the clause was doing useful work and this proposal is wrong. No such measurement exists yet, and the sample size available from one developer's phases is thin enough that it may stay a judgment call rather than a measured one. Say so rather than pretending otherwise. ## Why this belongs in `main` and not a fork The reviewer contract is the product. Cadence's pitch is that plans and diffs get refuted rather than approved, and this clause quietly reduces how much refuting happens, on the default path, for every user. There is nothing user-specific about it. A fork would not be adapting the review posture to a different workflow; it would be fixing a prompt that instructs against the subsystem's own purpose. If the clause is wrong, it is wrong in `main`. The tuning that legitimately belongs to a user already exists and is untouched: `review.mode`, `review.triggers.*.gate`, and the reviewer set are all config. Someone who wants a quieter review has four gates and three modes to reach for, none of which requires a reviewer that keeps findings to itself.