Prose surfaces are budgeted to the byte; the live context window is budgeted by nothing #202

New issue

Closed

opened 2026-08-16 17:16:08 +00:00 by john · 0 comments

john commented

2026-08-16 17:16:08 +00:00

Owner

The asymmetry

weight-budgets.json budgets Cadence's PROSE surfaces byte-exactly - 23 entries
across references/** and templates/** - and self-verify fails on an overrun
AND on a surface that shrinks below its recorded count. That check is why the
plan-frontmatter.md budget had to be re-pinned in the same commit that grew the
file, this milestone.

Nothing budgets the RUNTIME window a dispatch actually grows to. Cadence weighs
what it ships and never what it runs.

Why the runtime window is the one that costs

Over the last 7 days on this repo, cache-read is 62.5% of the bill and the average
context window is 121,250 tokens across 15,579 messages. Cost is close to
turns x window x 0.10, so the window is one of the two terms that multiply out to
the bill. The shipped prose surfaces are a small constant inside it.

burnrate records peak_ctx per session. The worst on this project is 436,169 -
more than three and a half times the average window, in a single session.

The proposal

A window ceiling per dispatch, in the same spirit as the byte budgets:

A recorded ceiling, and a dispatch whose peak context crosses it is reported.
Advisory first. A crossed ceiling is a signal that the PLAN was too big, not
that the executor misbehaved, so the remedy is upstream and a hard failure would
punish the wrong actor.
The natural home is beside the existing budget check, so the same
self-verify posture (a recorded number, an explicit re-pin when it legitimately
moves) applies to both what Cadence ships and what Cadence runs.

What it depends on

Peak context per dispatch is not in trace.jsonl today - no window field exists
there. Either the close seam gains one, or this reads burnrate's per-session
peak_ctx. Both are filed separately; this issue is the consumer, not the
instrumentation.

Why it is worth having as its own idea

The byte budgets work. They caught a real drift this milestone and they are
cheap. The argument here is only that they measure the wrong thing for cost: a
16KB reference file is budgeted to the byte while a 436,169-token live window is
governed by nothing at all.

## The asymmetry `weight-budgets.json` budgets Cadence's PROSE surfaces byte-exactly - 23 entries across `references/**` and `templates/**` - and `self-verify` fails on an overrun AND on a surface that shrinks below its recorded count. That check is why the `plan-frontmatter.md` budget had to be re-pinned in the same commit that grew the file, this milestone. Nothing budgets the RUNTIME window a dispatch actually grows to. Cadence weighs what it ships and never what it runs. ## Why the runtime window is the one that costs Over the last 7 days on this repo, cache-read is 62.5% of the bill and the average context window is 121,250 tokens across 15,579 messages. Cost is close to `turns x window x 0.10`, so the window is one of the two terms that multiply out to the bill. The shipped prose surfaces are a small constant inside it. burnrate records `peak_ctx` per session. The worst on this project is 436,169 - more than three and a half times the average window, in a single session. ## The proposal A window ceiling per dispatch, in the same spirit as the byte budgets: - A recorded ceiling, and a dispatch whose peak context crosses it is reported. - Advisory first. A crossed ceiling is a signal that the PLAN was too big, not that the executor misbehaved, so the remedy is upstream and a hard failure would punish the wrong actor. - The natural home is beside the existing budget check, so the same `self-verify` posture (a recorded number, an explicit re-pin when it legitimately moves) applies to both what Cadence ships and what Cadence runs. ## What it depends on Peak context per dispatch is not in `trace.jsonl` today - no window field exists there. Either the close seam gains one, or this reads burnrate's per-session `peak_ctx`. Both are filed separately; this issue is the consumer, not the instrumentation. ## Why it is worth having as its own idea The byte budgets work. They caught a real drift this milestone and they are cheap. The argument here is only that they measure the wrong thing for cost: a 16KB reference file is budgeted to the byte while a 436,169-token live window is governed by nothing at all.