Refuse a Coder tier this deployment cannot answer

4773472fe2f0 · AtlantisPleb · · parent fb705a506a2d

Refuse a Coder tier this deployment cannot answer

The tier rename (#40) gave every lane a friendlier name, which also made a
dead lane a better hiding place: a reader who watched `gemini-3.7-flash`
fail could at least search for it, while a reader who watches `Coder Flash`
fail has been told nothing.

The model a session opens with was already checked — `chooseBackend` reads
`availability` from `/api/v1/models` and picks a lane that can answer. The
tier a reader reaches with shift+tab was not: `buildTier` pinned
`TIER_MODELS[tier]` and opened a thread on it whether or not the deployment
served it, so the switch was spent arriving at a lane that fails on its
first turn.

`tierUnavailable` closes that seam against the same catalog and the same
`available` reading, and the refusal names tiers only — the invariant does
not lapse when the news is bad. A refused tier keeps the session on the one
that was answering and says which tiers can, because `applyPendingTier`
already reports a failed build that way. An unreadable catalog allows the
tier: "could not ask" is not "serves nothing". `Coder Local` is never
refused here, since it answers from the reader's machine.

Availability is the only claim added. No tier is presented as cheaper than
another: the catalog quotes a rate for some lanes and none for others, and
a price the server never declared would be a worse lie than the vendor name
the tier replaced.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KnhfrafYx5ZGaMbzZEJQ2d
Co-Authored-By
Claude Opus 5 (1M context) <noreply@anthropic.com>

Deploy story

What this commit did to the running system — joined from the forge receipt chain, the part a commit page elsewhere cannot show.

Not deployed through the forge lane

No push, promotion, build, or deploy receipt references this commit (receipts are scanned over a bounded recent window). Changes shipped by full node replacement carry their proof in the release gate receipt instead.

Changed files

  • modified INVARIANTS.md
  • modified packages/openagents-cli/src/cli.ts
  • modified packages/openagents-cli/src/coder-tiers.ts
  • modified packages/openagents-cli/test/coder-tiers.test.ts

Diff

4 files changed, +202 -20

INVARIANTS.md modified +15

@@ -2521,5 +2521,20 @@ renders as the bare product name `Coder`, never as its id.

2521 2521
  the reasoning level. A fresh `openagents coder` session with no `--model`
2522 2522
  runs as `Coder Auto`: its thread is opened unpinned and the inference proxy
2523 2523
  request names no model, so the server selects the lane per call.
2524
- A tier is checked against what the server says it can answer before a thread
2525
  opens on it. `tierUnavailable` reads the same `/api/v1/models` catalog and
2526
  the same `availability` field that `chooseBackend` applies to the model a
2527
  session opens with, so shift+tab cannot strand a reader on a lane the
2528
  deployment cannot serve. A refused tier keeps the session on the tier that
2529
  was answering and names the tiers that can answer. The rename is why this
2530
  matters: a reader who saw a vendor id fail could search for it, and a reader
2531
  who sees `Coder Flash` fail has been told nothing. Where the catalog could
2532
  not be read the tier is allowed, because "could not ask" is not "serves
2533
  nothing"; `Coder Local` is never refused here, since it answers from the
2534
  reader's machine rather than the deployment.
2535
- Availability is the only claim a tier makes. No tier is presented as cheaper
2536
  or dearer than another: the catalog declares a rate for some lanes and none
2537
  for others, and a tier label implying a price the server never quoted would
2538
  be a worse lie than the vendor name it replaced.
2524 2539
- Held by `packages/openagents-cli/test/coder-tiers.test.ts`. Issue
2525 2540
  OpenAgentsInc/openagents#40.
packages/openagents-cli/src/cli.ts modified +14 -20

@@ -55,12 +55,7 @@ import type { ReplySource } from "./coder-session.js";

55 55
import { CoderSession, DummyReplySource } from "./coder-session.js";
56 56
import { CoderTaskRegistry } from "./coder-tasks.js";
57 57
import { runCoderUi } from "./coder-ui.js";
58
import {
59
  backendIds,
60
  chooseBackend,
61
  fetchServedCatalog,
62
  refuseBackend,
63
} from "./coder-backends.js";
58
import { backendIds, chooseBackend, fetchServedCatalog, refuseBackend } from "./coder-backends.js";
64 59
import {
65 60
  discoverOllamaModel,
66 61
  isOllamaModelFlag,

@@ -115,7 +110,7 @@ import { fileURLToPath } from "node:url";

115 110
import { rebuild, RELOAD_EXIT_CODE, sourceCheckout } from "./coder-reload.js";
116 111
import { loadSkillSelection, standingContext } from "./coder-skills.js";
117 112
import { startDevServer } from "./coder-dev-server.js";
118
import { TIER_MODELS, tierForModel, type CoderTierId } from "./coder-tiers.js";
113
import { TIER_MODELS, tierForModel, tierUnavailable, type CoderTierId } from "./coder-tiers.js";
119 114
import { ZenReplySource, zenCredential } from "./coder-zen.js";
120 115
import { describeWorkspace } from "./coder-workspace.js";
121 116
import { ComputerClient } from "./computer-client.js";

@@ -1727,15 +1722,11 @@ async function buildDelegation(options: {

1727 1722
        // different agent under the name the caller asked for.
1728 1723
        new SelfHarness({ grant: nonNullGrant(options.grant) })
1729 1724
      : /^claude(:.+)?$/.test(choice)
1730
        ? new ClaudeCodeHarness(
1731
            choice.startsWith("claude:") ? { model: choice.slice(7) } : {},
1732
          )
1725
        ? new ClaudeCodeHarness(choice.startsWith("claude:") ? { model: choice.slice(7) } : {})
1733 1726
        : /^devin(:.+)?$/.test(choice)
1734 1727
          ? new DevinHarness(choice.startsWith("devin:") ? { permissionMode: choice.slice(6) } : {})
1735 1728
          : /^codex(:.+)?$/.test(choice)
1736
            ? new CodexHarness(
1737
                choice.startsWith("codex:") ? { model: choice.slice(6) } : {},
1738
              )
1729
            ? new CodexHarness(choice.startsWith("codex:") ? { model: choice.slice(6) } : {})
1739 1730
            : new OpencodeHarness({
1740 1731
                model: choice,
1741 1732
                ...(command === undefined ? {} : { command }),

@@ -2341,6 +2332,14 @@ const coderCommand = Command.make(

2341 2332
        : async (tier: CoderTierId, history: ReadonlyArray<unknown>): Promise<ReplySource> => {
2342 2333
            const accountToken = Redacted.value(stored.value.token);
2343 2334
            const carried = history as ReadonlyArray<WireMessage>;
2335
            // A tier this deployment cannot answer is refused before a thread
2336
            // is opened on it. Throwing here is what keeps the session on the
2337
            // tier that was answering: `applyPendingTier` catches it, says
2338
            // why, and does not switch. Opening the thread anyway would spend
2339
            // the switch to reach a lane that fails at its first turn, under a
2340
            // friendly name that tells the reader nothing.
2341
            const unavailable = tierUnavailable(served, tier);
2342
            if (unavailable !== undefined) throw new Error(unavailable);
2344 2343
            if (tier === "local") {
2345 2344
              const name = await discoverOllamaModel(process.env["OLLAMA_HOST"] ?? undefined);
2346 2345
              if (name === undefined) {

@@ -2563,9 +2562,7 @@ const coderCommand = Command.make(

2563 2562
2564 2563
      const foreignSessionsManifest = locateForeignSessionsManifest();
2565 2564
2566
      const runForeignSessionResume = async (
2567
        selection: number | undefined,
2568
      ): Promise<string> => {
2565
      const runForeignSessionResume = async (selection: number | undefined): Promise<string> => {
2569 2566
        if (foreignSessionsManifest === undefined) {
2570 2567
          return (
2571 2568
            "The foreign session scanner is not available from this installation. " +

@@ -2592,10 +2589,7 @@ const coderCommand = Command.make(

2592 2589
            );
2593 2590
          }
2594 2591
        };
2595
        return runForeignResume(
2596
          { now_ms: Date.now(), cwd: process.cwd(), selection },
2597
          invoke,
2598
        );
2592
        return runForeignResume({ now_ms: Date.now(), cwd: process.cwd(), selection }, invoke);
2599 2593
      };
2600 2594
2601 2595
      // Only when there is really no way to run a child. A refused child
packages/openagents-cli/src/coder-tiers.ts modified +62

@@ -57,6 +57,68 @@ export const coderTierLabel = (modelId: string | undefined): string => {

57 57
  return tier === undefined ? "Coder" : LABELS[tier];
58 58
};
59 59
60
/** One lane of the served catalog, as much of it as a tier decision needs. */
61
interface ServedLane {
62
  readonly id: string;
63
  readonly available: boolean;
64
}
65
66
/**
67
 * Why this tier cannot open a thread on this deployment, or `undefined` if it
68
 * can.
69
 *
70
 * A tier is a friendlier name than a vendor id, which makes it a better place
71
 * to hide a dead lane — the reader who used to see `gemini-3.7-flash` fail
72
 * could at least search for it, while a reader who sees `Coder Flash` fail has
73
 * been told nothing. So the tier a reader picks is checked against what the
74
 * server says it can answer, the same catalog and the same `available` reading
75
 * that `chooseBackend` already applies to the model a session opens with. The
76
 * refusal names tiers only, because the invariant does not lapse when the news
77
 * is bad.
78
 *
79
 * `undefined` for `served` means the catalog could not be read — an older
80
 * server, an unreachable one, a resumed session that never asked. That is not
81
 * the same as "serves nothing", so the tier is allowed and the turn reports
82
 * the truth. `Coder Local` is never refused here: it answers from this machine
83
 * rather than this deployment, and its own discovery says so when no local
84
 * server is running.
85
 *
86
 * Availability is the only claim made. No tier is described as cheaper or
87
 * dearer than another: the catalog declares a rate for some lanes and none for
88
 * others, and a tier label that implied a price the server never quoted would
89
 * be a worse lie than the vendor name it replaced.
90
 */
91
export const tierUnavailable = (
92
  served: ReadonlyArray<ServedLane> | undefined,
93
  tier: CoderTierId,
94
): string | undefined => {
95
  if (served === undefined || tier === "local") return undefined;
96
97
  const answering = (id: string): boolean =>
98
    served.some((lane) => lane.id === id && lane.available);
99
100
  // Auto is openable whenever anything is, because it names no model and lets
101
  // the server pick the lane per call.
102
  const openable: CoderTierId[] = [];
103
  if (served.some((lane) => lane.available)) openable.push("auto");
104
  if (answering(TIER_MODELS.flash)) openable.push("flash");
105
  if (answering(TIER_MODELS.pro)) openable.push("pro");
106
107
  if (openable.includes(tier)) return undefined;
108
109
  // A reason clause, not a sentence: the caller that reports this already
110
  // names the tier that could not be had, and saying it twice reads as a
111
  // stutter. It still names the tiers that *can* answer, because a refusal
112
  // that leaves the reader guessing which way to press shift+tab has only
113
  // half done the job.
114
  return openable.length === 0
115
    ? `no lane on this deployment has a configured provider credential. ` +
116
        `${tierLabel("local")} still runs on this machine.`
117
    : `this deployment is not serving that lane right now. ` +
118
        `Available: ${openable.map(tierLabel).join(", ")}, and ` +
119
        `${tierLabel("local")} on this machine.`;
120
};
121
60 122
/** Shift+Tab's orbit: Auto → Flash → Pro → Local → Auto. */
61 123
export const nextTier = (tier: CoderTierId): CoderTierId => {
62 124
  switch (tier) {
packages/openagents-cli/test/coder-tiers.test.ts modified +111

@@ -8,6 +8,7 @@ import {

8 8
  TIER_MODELS,
9 9
  tierForModel,
10 10
  tierLabel,
11
  tierUnavailable,
11 12
} from "../src/coder-tiers.js";
12 13
13 14
// The invariant (INVARIANTS.md "Coder Model Naming"): a vendor model name

@@ -51,6 +52,78 @@ describe("coder tiers", () => {

51 52
    }
52 53
  });
53 54
55
  it("opens a tier whose lane the server says is available", () => {
56
    const served = [
57
      { id: TIER_MODELS.flash, available: true },
58
      { id: TIER_MODELS.pro, available: true },
59
    ];
60
    expect(tierUnavailable(served, "auto")).toBeUndefined();
61
    expect(tierUnavailable(served, "flash")).toBeUndefined();
62
    expect(tierUnavailable(served, "pro")).toBeUndefined();
63
  });
64
65
  it("refuses a tier whose lane the server serves but cannot answer on", () => {
66
    const served = [
67
      { id: TIER_MODELS.flash, available: false },
68
      { id: TIER_MODELS.pro, available: true },
69
    ];
70
    // The dead lane is refused; the live one and Auto still open. This is the
71
    // case the tier rename could have buried: `Coder Flash` is a friendlier
72
    // name to fail under than the vendor id a reader could have searched for.
73
    // The refusal points at the tiers that can answer.
74
    expect(tierUnavailable(served, "flash")).toContain("Coder Pro");
75
    expect(tierUnavailable(served, "flash")).not.toContain("Coder Flash");
76
    expect(tierUnavailable(served, "pro")).toBeUndefined();
77
    expect(tierUnavailable(served, "auto")).toBeUndefined();
78
  });
79
80
  it("refuses a tier this deployment does not serve at all", () => {
81
    const served = [{ id: TIER_MODELS.pro, available: true }];
82
    expect(tierUnavailable(served, "flash")).toContain("not serving that lane");
83
  });
84
85
  it("says Auto cannot select when no lane on the deployment can answer", () => {
86
    const served = [
87
      { id: TIER_MODELS.flash, available: false },
88
      { id: TIER_MODELS.pro, available: false },
89
    ];
90
    for (const tier of ["auto", "flash", "pro"] as const) {
91
      expect(tierUnavailable(served, tier)).toContain("no lane on this deployment");
92
    }
93
  });
94
95
  it("allows every tier when the catalog could not be read", () => {
96
    // Not the same as "serves nothing": an unreachable or older server is a
97
    // reason to let the turn report the truth, not to refuse up front.
98
    for (const tier of ["auto", "flash", "pro", "local"] as const) {
99
      expect(tierUnavailable(undefined, tier)).toBeUndefined();
100
    }
101
  });
102
103
  it("never refuses Coder Local, which answers from this machine", () => {
104
    expect(tierUnavailable([], "local")).toBeUndefined();
105
    expect(tierUnavailable([{ id: TIER_MODELS.pro, available: false }], "local")).toBeUndefined();
106
  });
107
108
  it("names no vendor model and quotes no price when it refuses", () => {
109
    const served = [
110
      { id: TIER_MODELS.flash, available: false },
111
      { id: TIER_MODELS.pro, available: false },
112
    ];
113
    for (const tier of ["auto", "flash", "pro"] as const) {
114
      const refusal = tierUnavailable(served, tier);
115
      expect(refusal).toBeDefined();
116
      // The invariant does not lapse when the news is bad.
117
      for (const id of [TIER_MODELS.flash, TIER_MODELS.pro, "ox-alpha", "gemini", "gpt", "ollama"])
118
        expect(refusal).not.toContain(id);
119
      // Availability is the only claim. A tier carries no price comparison,
120
      // because the catalog declares a rate for some lanes and none for
121
      // others.
122
      for (const priced of ["cheap", "cheaper", "free", "$", "per million", "cost"])
123
        expect(refusal?.toLowerCase()).not.toContain(priced);
124
    }
125
  });
126
54 127
  it("never leaks a vendor id through any label", () => {
55 128
    for (const id of [
56 129
      TIER_MODELS.flash,

@@ -91,4 +164,42 @@ describe("session tier cycling", () => {

91 164
    expect(session.cycleTier().label).toBe("Coder Local");
92 165
    expect(session.cycleTier().label).toBe("Coder Auto");
93 166
  });
167
168
  // What `tierUnavailable` is for, end to end: the build throws for a lane the
169
  // deployment cannot answer, and the session stays on the tier that was
170
  // answering rather than stranding the reader on a dead one.
171
  it("stays on the answering tier when the next tier cannot be built", async () => {
172
    const refusal = tierUnavailable(
173
      [
174
        { id: TIER_MODELS.flash, available: false },
175
        { id: TIER_MODELS.pro, available: true },
176
      ],
177
      "flash",
178
    );
179
    expect(refusal).toBeDefined();
180
181
    const session = new CoderSession(silent, "repo", "main", undefined, undefined, undefined, {
182
      initial: "auto",
183
      build: (tier) =>
184
        tier === "flash" ? Promise.reject(new Error(refusal)) : Promise.resolve(silent),
185
    });
186
187
    session.cycleTier();
188
    // The label flips optimistically; the build decides whether it holds.
189
    expect(session.snapshot().model).toBe("Coder Flash");
190
191
    await session.submit("hello");
192
193
    // The refused tier did not take, and the reader was told which tier it
194
    // was and where to go instead — without a vendor name.
195
    expect(session.snapshot().model).toBe("Coder Auto");
196
    const notices = session
197
      .snapshot()
198
      .entries.filter((entry) => entry.role === "notice")
199
      .map((entry) => entry.text)
200
      .join("\n");
201
    expect(notices).toContain("Coder Flash is not available");
202
    expect(notices).toContain("Coder Pro");
203
    expect(notices).not.toContain(TIER_MODELS.flash);
204
  });
94 205
});

This page updates live while a promote is in flight · changelog