routing: make the axis stakes, not spend - one question (what happens if this is wrong) driving model, effort, review, and verification #81

New issue

Open

opened 2026-07-28 19:49:18 +00:00 by crenshawdev · 0 comments

crenshawdev commented

2026-07-28 19:49:18 +00:00

(Migrated from github.com)

Supersedes the routing half of #63. #63 diagnosed the problem correctly and got the replacement wrong; this is the replacement. #63 stays open as the trace of how the argument got here.

What you want Cadence to do

Change the question the routing axis asks.

Today model.profile asks how much are you willing to spend: fast, balanced, quality. That is a budget question wearing a capability costume, and on a Max subscription it is not even a question the user has.

Replace it with the question they actually have, which is what happens if this is wrong:

rung	the question it answers
`personal`	Nobody else runs this. If it breaks, I notice and I fix it.
`production`	This ships. If it breaks, it breaks for people who did not write it.
`critical`	This touches secrets, money, auth, data integrity, or destructive operations. If it breaks, it is not a bug report.

That is answerable in one second by someone who has never opened route-table.json. "How much do you want to spend" is answerable but useless. "Is this hardened work" (the earlier proposal in #63) is neither, because it asks the user to grade their own rigor rather than describe their situation.

Stakes are a property of the artifact, not of the author. Cadence itself is a personal project that ships to strangers, so personal has to mean "nobody else runs this," not "I am the one typing."

The cell value gets richer, and that is the point

The existing matrix shape is (profile × tier) → model. The shape is right; two dimensions are correct, because roles carry different failure consequences at the same stakes level. A planner's error costs a whole phase; the plan checker is one gate.

What is wrong, besides the axis label, is that a cell yields only a model. Quality is not one knob:

(stakes × role) → { model, effort, review, verify }

model — which model runs the role
effort — the rung it runs at, which is a coverage dial as much as a thinking dial
review — advisory or blocking; one reviewer or a panel; same-model or cross-model
verify — how deep the goal-backward pass goes, and whether a second verifier fires

One question in, a bundle out. This is also why collapsing everything to a single effort dial (as #63 proposed) is under-powered: effort cannot express "fire a blocking cross-model review," and that is exactly the knob that matters most at the top rung.

The top rung is computed, not claimed

This is the strongest argument here and it is the one that only exists after the reframe.

Cadence already detects auth/authz, DB schema, money, concurrency, destructive operations, secrets/crypto, public API contracts, and untrusted-input parsing. It computes this today, for every user, in order to fire the blocking review trigger.

That list is the critical rung. Not "similar to" it, not "a signal that could inform" it. It is the same question, already answered by the codebase.

So the top rung asserts itself from the code rather than from the user's self-assessment. Direction of control matters: detection sets a floor, the user may raise it, and lowering below a detected floor requires an explicit override that names what is being overridden. A rung that is free to claim gets claimed for everything and then means nothing.

That also settles the scope question. A project has a baseline; a phase has a surface. The user sets the baseline once, and detection raises individual phases above it. A personal project with one phase that writes credentials gets a critical phase without the user having to notice.

Why the default no is wrong here

Because the current question cannot express the thing users need to say. You cannot auto-set a spend level from a risk signal without it being absurd: touching auth is no reason to become willing to pay more. You can absolutely auto-set a stakes level from one. "This phase touches money, so verify harder" is a sentence the current config cannot represent at all.

Because the signal already exists and is half-wired. The risk surface is computed on every phase and used for exactly one thing. Everything else it should be driving reads from somewhere else instead. That is the same defect shape as #64 and #78 (a value resolved and then not delivered), one level up the stack.

Because the knobs are scattered across four places with no question behind them. Model comes from the profile matrix. Effort comes from agent frontmatter. Review gating comes from the triggers config. Verification depth comes from the workflow. A user who wants "be careful with this phase" has to know all four exist and edit each one. That is not a configuration surface, it is a scavenger hunt.

Because it rescues the matrix instead of deleting it. #63 proposed deleting the profile matrix, the tier ladder, and escalate_effort_variant. Under a stakes axis the two-dimensional lookup is the correct shape and should stay. What changes is the axis label and the cell value. That is a smaller, more reversible change than deleting three mechanisms, and it keeps a structure that already works.

The MANIFESTO objection, met

Tokens are a budget, every dispatch is a bet on whether the answer is worth what it burns.

The stated principle is value per dispatch, not minimize spend. Stakes is a strictly better proxy for value than price is. "How much does it cost if this dispatch is wrong" is the numerator the principle was always reaching for; fast/balanced/quality only ever measured the denominator. The principle is unchanged; the instrument improved.

The observation that started this

Conventional wisdom is heavy model plans, cheap model executes. Flipping that specifically at the executor produced fewer iterative cycles of bug-finding in adversarial review in observed use. A looping executor burns more than the model it saved, so the cheap placement was never cheap.

Anthropic's Opus 5 guidance gives a mechanism rather than just a correlation: it is strongest on difficult multi-file work and "completes tasks rather than leaving stubs or placeholders," while the gap over prior models is smaller on easy single-turn edits. Fewer defects introduced means fewer review cycles to find them. It also predicts the effect concentrates on heavy phases and nearly vanishes on trivial ones, which is a falsifiable claim worth checking.

Under this proposal the executor's model stops being a pinned value and becomes a derived one, which is why the standalone "route the executor to opus" issue (#75) does not survive as its own change.

What it costs

A breaking config change. model.profile enum values change with no back-compat alias. This is the v2.0.0 break and the reason this cannot ship in a minor release.

Naming is load-bearing and it is the part I hold least firmly. personal/production/critical are a proposal, not a conclusion. Whatever they end up being, they have to describe a situation rather than grade an effort, or the ladder degrades back into self-assessment.

It depends on #74. Effort is frozen per agent file on the dispatch path Cadence uses (established under #64), so varying effort per role needs per-rung agent files plus the single-sourced contract mechanism. This proposal cannot land before that one.

A richer cell value is more surface that can drift. Three stakes levels times six roles times four knobs is a lot of cells. They must be computed from a small table rather than enumerated by hand, and self-verify needs to assert every cell resolves to a valid model, a valid effort, and a real trigger.

Auto-detection that fires on the wrong surface makes a cheap phase expensive. A false positive on the risk detector now costs more than it did when it only fired a review.

What does not cost what it looks like. This is not a cost optimization and should not be argued as one. Cadence targets Max subscribers, where the allowance is a ceiling rather than a budget. It should be argued on quality or not at all.

Measurements that would validate or kill this

Executor loop and retry counts by model, on heavy phases versus trivial ones. This is the empirical claim underneath everything and it currently has only an informal result behind it. If raising the executor's model does not reduce review cycles, the seed observation was noise.
Risk-detector precision. How often does the surface detector fire on a phase that turns out not to warrant it? If that rate is high, floor-raising is a tax rather than a safeguard.
Verifier abstention rate by rung. If UNCERTAIN findings do not fall at higher rungs, the verify knob buys nothing and should be dropped from the bundle.

Why this belongs in `main` and not a fork

The wrong question is in main and is wrong for everyone. Every user is asked how much they want to spend; no user is asked what happens if the work is wrong.

The risk detection is also in main, computed for every user on every phase, and then used for one thing and discarded. Every fork would rediscover the same mapping from the same signals, and no fork can fix the shipped default.

The escape hatches stay: model.overrides.<role> keeps pinning a model, fable stays reachable by explicit pin, and the personal rung exists precisely so throwaway work does not pay for any of this.

Files this touches

cadence-core/route-table.json, cadence-core/bin/route.mjs and route.test.mjs, cadence-core/config.schema.json, cadence-core/templates/config.json, cadence-core/references/review-triggers.md, cadence-core/references/seams.md, agents/ (per-rung files), skills/ (role contracts, via #74), cadence-core/workflows/config.md, cadence-core/bin/self-verify.mjs, INTERNALS.md "Model routing" (a rewrite, not an edit), DESIGN.md.

#63 — the earlier form of this argument. Correct that the axis is wrong; wrong that the replacement is rigor-as-adjective, wrong that one effort dial is enough, wrong that the matrix should be deleted. Kept open as the trace.
#74 — hard dependency. Per-role effort is not expressible without it.
#75 — superseded. The executor's model becomes derived, not pinned.
#78 — the triggers.<t>.tier defect may resolve here rather than separately, since the tier vocabulary is one of the things being replaced.
#54 — wanted per-role tier and effort user-configurable. This changes what is configurable and by which question.

Supersedes the routing half of #63. #63 diagnosed the problem correctly and got the replacement wrong; this is the replacement. #63 stays open as the trace of how the argument got here. ## What you want Cadence to do Change the question the routing axis asks. Today `model.profile` asks **how much are you willing to spend**: `fast`, `balanced`, `quality`. That is a budget question wearing a capability costume, and on a Max subscription it is not even a question the user has. Replace it with the question they actually have, which is **what happens if this is wrong**: | rung | the question it answers | |---|---| | `personal` | Nobody else runs this. If it breaks, I notice and I fix it. | | `production` | This ships. If it breaks, it breaks for people who did not write it. | | `critical` | This touches secrets, money, auth, data integrity, or destructive operations. If it breaks, it is not a bug report. | That is answerable in one second by someone who has never opened `route-table.json`. "How much do you want to spend" is answerable but useless. "Is this hardened work" (the earlier proposal in #63) is neither, because it asks the user to grade their own rigor rather than describe their situation. **Stakes are a property of the artifact, not of the author.** Cadence itself is a personal project that ships to strangers, so `personal` has to mean "nobody else runs this," not "I am the one typing." ### The cell value gets richer, and that is the point The existing matrix shape is `(profile × tier) → model`. The shape is right; two dimensions are correct, because roles carry different failure consequences at the same stakes level. A planner's error costs a whole phase; the plan checker is one gate. What is wrong, besides the axis label, is that a cell yields **only a model**. Quality is not one knob: ``` (stakes × role) → { model, effort, review, verify } ``` - **model** — which model runs the role - **effort** — the rung it runs at, which is a coverage dial as much as a thinking dial - **review** — advisory or blocking; one reviewer or a panel; same-model or cross-model - **verify** — how deep the goal-backward pass goes, and whether a second verifier fires One question in, a bundle out. This is also why collapsing everything to a single `effort` dial (as #63 proposed) is under-powered: effort cannot express "fire a blocking cross-model review," and that is exactly the knob that matters most at the top rung. ### The top rung is computed, not claimed This is the strongest argument here and it is the one that only exists after the reframe. Cadence **already detects** auth/authz, DB schema, money, concurrency, destructive operations, secrets/crypto, public API contracts, and untrusted-input parsing. It computes this today, for every user, in order to fire the blocking review trigger. That list is the `critical` rung. Not "similar to" it, not "a signal that could inform" it. It is the same question, already answered by the codebase. So the top rung asserts itself from the code rather than from the user's self-assessment. Direction of control matters: **detection sets a floor, the user may raise it, and lowering below a detected floor requires an explicit override that names what is being overridden.** A rung that is free to claim gets claimed for everything and then means nothing. That also settles the scope question. A project has a baseline; a phase has a surface. The user sets the baseline once, and detection raises individual phases above it. A `personal` project with one phase that writes credentials gets a `critical` phase without the user having to notice. ## Why the default no is wrong here **Because the current question cannot express the thing users need to say.** You cannot auto-set a *spend* level from a risk signal without it being absurd: touching auth is no reason to become willing to pay more. You can absolutely auto-set a *stakes* level from one. "This phase touches money, so verify harder" is a sentence the current config cannot represent at all. **Because the signal already exists and is half-wired.** The risk surface is computed on every phase and used for exactly one thing. Everything else it should be driving reads from somewhere else instead. That is the same defect shape as #64 and #78 (a value resolved and then not delivered), one level up the stack. **Because the knobs are scattered across four places with no question behind them.** Model comes from the profile matrix. Effort comes from agent frontmatter. Review gating comes from the triggers config. Verification depth comes from the workflow. A user who wants "be careful with this phase" has to know all four exist and edit each one. That is not a configuration surface, it is a scavenger hunt. **Because it rescues the matrix instead of deleting it.** #63 proposed deleting the profile matrix, the tier ladder, and `escalate_effort_variant`. Under a stakes axis the two-dimensional lookup is the correct shape and should stay. What changes is the axis label and the cell value. That is a smaller, more reversible change than deleting three mechanisms, and it keeps a structure that already works. ### The MANIFESTO objection, met > Tokens are a budget, every dispatch is a bet on whether the answer is worth what it burns. The stated principle is **value per dispatch**, not minimize spend. Stakes is a strictly better proxy for value than price is. "How much does it cost if this dispatch is wrong" is the numerator the principle was always reaching for; `fast`/`balanced`/`quality` only ever measured the denominator. The principle is unchanged; the instrument improved. ### The observation that started this Conventional wisdom is heavy model plans, cheap model executes. Flipping that specifically at the executor produced **fewer iterative cycles of bug-finding in adversarial review** in observed use. A looping executor burns more than the model it saved, so the cheap placement was never cheap. Anthropic's Opus 5 guidance gives a mechanism rather than just a correlation: it is strongest on difficult multi-file work and "completes tasks rather than leaving stubs or placeholders," while the gap over prior models is smaller on easy single-turn edits. Fewer defects introduced means fewer review cycles to find them. It also predicts the effect concentrates on heavy phases and nearly vanishes on trivial ones, which is a falsifiable claim worth checking. Under this proposal the executor's model stops being a pinned value and becomes a derived one, which is why the standalone "route the executor to opus" issue (#75) does not survive as its own change. ## What it costs **A breaking config change.** `model.profile` enum values change with no back-compat alias. This is the v2.0.0 break and the reason this cannot ship in a minor release. **Naming is load-bearing and it is the part I hold least firmly.** `personal`/`production`/`critical` are a proposal, not a conclusion. Whatever they end up being, they have to describe a situation rather than grade an effort, or the ladder degrades back into self-assessment. **It depends on #74.** Effort is frozen per agent file on the dispatch path Cadence uses (established under #64), so varying effort per role needs per-rung agent files plus the single-sourced contract mechanism. This proposal cannot land before that one. **A richer cell value is more surface that can drift.** Three stakes levels times six roles times four knobs is a lot of cells. They must be computed from a small table rather than enumerated by hand, and self-verify needs to assert every cell resolves to a valid model, a valid effort, and a real trigger. **Auto-detection that fires on the wrong surface makes a cheap phase expensive.** A false positive on the risk detector now costs more than it did when it only fired a review. **What does not cost what it looks like.** This is not a cost optimization and should not be argued as one. Cadence targets Max subscribers, where the allowance is a ceiling rather than a budget. It should be argued on quality or not at all. ### Measurements that would validate or kill this - **Executor loop and retry counts by model**, on heavy phases versus trivial ones. This is the empirical claim underneath everything and it currently has only an informal result behind it. If raising the executor's model does not reduce review cycles, the seed observation was noise. - **Risk-detector precision.** How often does the surface detector fire on a phase that turns out not to warrant it? If that rate is high, floor-raising is a tax rather than a safeguard. - **Verifier abstention rate by rung.** If UNCERTAIN findings do not fall at higher rungs, the verify knob buys nothing and should be dropped from the bundle. ## Why this belongs in `main` and not a fork The wrong question is in `main` and is wrong for everyone. Every user is asked how much they want to spend; no user is asked what happens if the work is wrong. The risk detection is also in `main`, computed for every user on every phase, and then used for one thing and discarded. Every fork would rediscover the same mapping from the same signals, and no fork can fix the shipped default. The escape hatches stay: `model.overrides.<role>` keeps pinning a model, `fable` stays reachable by explicit pin, and the `personal` rung exists precisely so throwaway work does not pay for any of this. ## Files this touches `cadence-core/route-table.json`, `cadence-core/bin/route.mjs` and `route.test.mjs`, `cadence-core/config.schema.json`, `cadence-core/templates/config.json`, `cadence-core/references/review-triggers.md`, `cadence-core/references/seams.md`, `agents/` (per-rung files), `skills/` (role contracts, via #74), `cadence-core/workflows/config.md`, `cadence-core/bin/self-verify.mjs`, `INTERNALS.md` "Model routing" (a rewrite, not an edit), `DESIGN.md`. ## Related - **#63** — the earlier form of this argument. Correct that the axis is wrong; wrong that the replacement is rigor-as-adjective, wrong that one effort dial is enough, wrong that the matrix should be deleted. Kept open as the trace. - **#74** — hard dependency. Per-role effort is not expressible without it. - **#75** — superseded. The executor's model becomes derived, not pinned. - **#78** — the `triggers.<t>.tier` defect may resolve here rather than separately, since the tier vocabulary is one of the things being replaced. - **#54** — wanted per-role tier and effort user-configurable. This changes what is configurable and by which question.