Raising stakes upgrades the subagent reviewer but not the cross-model one #207

New issue

Closed

opened 2026-08-17 18:51:34 +00:00 by john · 1 comment

john commented

2026-08-17 18:51:34 +00:00

Owner

Raising stakes upgrades one half of a cross-model review panel and leaves the other half where it was.

The asymmetry

route-table.json carries three separate vocabularies for what a user reads as one axis:

stakes_order: solo / shipped / critical — the project's risk posture
rung_order: low / medium / high / xhigh / max — the claude-subagent reviewer's effort, which the routing cells select by stakes
tier_names: flagship / balanced / cheap — the model id a cross-model provider resolves, via review.providers.<name>.tiers[<tier>]

The tiers map is keyed by trigger alone:

"tiers": { "plan": "flagship", "diff": "balanced", "risk_surface": "flagship", "phase_diff": "flagship" }

There is no stakes dimension in it. So moving stakes from shipped to critical:

moves the claude-subagent reviewer from cad-reviewer (high) to cad-reviewer-xhigh, and
leaves every cross-model reviewer on exactly the same model id it used at solo.

Under a panel, the two backends are supposed to be two independent voices on the same artifact, merged blind by the adjudicator. Scaling one and not the other means the panel's composition silently changes character with stakes, in a direction nobody chose: at critical the Claude voice is stronger relative to the provider voice than it was at solo.

Why it reads as a wording problem first

Three vocabularies is the surface symptom. flagship/balanced/cheap and solo/shipped/critical are both three-valued and both ordered least-to-most-serious, so a user setting critical reasonably expects flagship to follow. Renaming alone would not fix it — the tier genuinely does not move — but the naming is what makes the gap invisible.

Options, not a decision

Key tiers by stakes as well as trigger, the way the routing cells already key rungs. Most consistent with what exists; largest table.
Derive the tier from the resolved rung, so one ladder drives both backends.
Leave the mechanism and fix only the vocabulary plus the docs, stating explicitly that tier is per-trigger and stakes-independent by design. Cheapest, and honest, if the current behaviour is actually wanted.

Whichever is chosen, references/config-catalog.md:59 glosses the tiers as flagship→strongest · balanced→mid · cheap→cheapest, which is a per-provider claim the config cannot guarantee: DeepSeek publishes two models, so in this repo flagship and balanced both resolve deepseek-v4-pro and the gloss is wrong for that provider.

Acceptance

Either the cross-model tier moves with stakes, or the documentation states in one place that it deliberately does not and why.
The three vocabularies are reconciled or their independence is stated where a user configuring them will read it.
A test pins whichever answer is chosen, so the two backends cannot drift apart again unnoticed.

Raising `stakes` upgrades one half of a cross-model review panel and leaves the other half where it was. ## The asymmetry `route-table.json` carries three separate vocabularies for what a user reads as one axis: - `stakes_order`: `solo` / `shipped` / `critical` — the project's risk posture - `rung_order`: `low` / `medium` / `high` / `xhigh` / `max` — the claude-subagent reviewer's effort, which the routing cells select **by stakes** - `tier_names`: `flagship` / `balanced` / `cheap` — the model id a cross-model provider resolves, via `review.providers.<name>.tiers[<tier>]` The `tiers` map is keyed by **trigger alone**: ```json "tiers": { "plan": "flagship", "diff": "balanced", "risk_surface": "flagship", "phase_diff": "flagship" } ``` There is no stakes dimension in it. So moving `stakes` from `shipped` to `critical`: - moves the claude-subagent reviewer from `cad-reviewer` (high) to `cad-reviewer-xhigh`, and - leaves every cross-model reviewer on exactly the same model id it used at `solo`. Under a panel, the two backends are supposed to be two independent voices on the same artifact, merged blind by the adjudicator. Scaling one and not the other means the panel's composition silently changes character with stakes, in a direction nobody chose: at `critical` the Claude voice is stronger relative to the provider voice than it was at `solo`. ## Why it reads as a wording problem first Three vocabularies is the surface symptom. `flagship`/`balanced`/`cheap` and `solo`/`shipped`/`critical` are both three-valued and both ordered least-to-most-serious, so a user setting `critical` reasonably expects `flagship` to follow. Renaming alone would not fix it — the tier genuinely does not move — but the naming is what makes the gap invisible. ## Options, not a decision 1. Key `tiers` by stakes as well as trigger, the way the routing cells already key rungs. Most consistent with what exists; largest table. 2. Derive the tier from the resolved rung, so one ladder drives both backends. 3. Leave the mechanism and fix only the vocabulary plus the docs, stating explicitly that tier is per-trigger and stakes-independent by design. Cheapest, and honest, if the current behaviour is actually wanted. Whichever is chosen, `references/config-catalog.md:59` glosses the tiers as `flagship→strongest · balanced→mid · cheap→cheapest`, which is a per-provider claim the config cannot guarantee: DeepSeek publishes two models, so in this repo `flagship` and `balanced` both resolve `deepseek-v4-pro` and the gloss is wrong for that provider. ## Acceptance - Either the cross-model tier moves with `stakes`, or the documentation states in one place that it deliberately does not and why. - The three vocabularies are reconciled or their independence is stated where a user configuring them will read it. - A test pins whichever answer is chosen, so the two backends cannot drift apart again unnoticed.