Put the local model behind --local

89ec3c1125ce · AtlantisPleb · · parent f74d10130b5a

Put the local model behind --local

Local was the default whenever a machine happened to be running Ollama. That
made a session's model a property of the laptop rather than a choice: a reader
who had started the server for something else got a different coder without
asking for one. It is `--local` now, and a session opens on the coder backend
otherwise.

`--local` with no Ollama running says so and stops, rather than falling through
to something else the reader did not ask for.

Also, a missing route is no longer reported as a refusal. The transcript writer
posts to `POST /api/v3/threads/{id}/events` and treated a 404 as the server
calling the event invalid, so a server older than the client produced that
sentence on every turn. It says once that the server does not serve a transcript
yet and that the work is unaffected, which is what is true.

This commit also named `gemini-3.7-flash` as the backend a coder session leads
with. That part was wrong and the next commit corrects it: no deployment has
ever served a model by that id, so every session that took the name opened its
thread against a catalog that refused it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012TRDRrfL1khQhQtNr3SRrA
Co-Authored-By
Claude Opus 5 (1M context) <noreply@anthropic.com>

Deploy story

What this commit did to the running system — joined from the forge receipt chain, the part a commit page elsewhere cannot show.

Not deployed through the forge lane

No push, promotion, build, or deploy receipt references this commit (receipts are scanned over a bounded recent window). Changes shipped by full node replacement carry their proof in the release gate receipt instead.

Changed files

  • modified packages/openagents-cli/src/cli.ts
  • modified packages/openagents-cli/src/coder-backends.ts
  • modified packages/openagents-cli/src/coder-transcript.ts
  • modified packages/openagents-cli/test/coder-ollama.test.ts

Diff

4 files changed, +77 -3

packages/openagents-cli/src/cli.ts modified +25 -2

@@ -37,7 +37,7 @@ import type { ReplySource } from "./coder-session.js";

37 37
import { CoderSession, DummyReplySource } from "./coder-session.js";
38 38
import { CoderTaskRegistry } from "./coder-tasks.js";
39 39
import { runCoderUi } from "./coder-ui.js";
40
import { backendIds } from "./coder-backends.js";
40
import { backendIds, defaultBackendId } from "./coder-backends.js";
41 41
import {
42 42
  discoverOllamaModel,
43 43
  isOllamaModelFlag,

@@ -1475,6 +1475,11 @@ const coderPlainFlag = Flag.boolean("plain").pipe(

1475 1475
const coderOfflineFlag = Flag.boolean("offline").pipe(
1476 1476
  Flag.withDescription("Answer from the built-in stand-in instead of the chat API"),
1477 1477
);
1478
const coderLocalFlag = Flag.boolean("local").pipe(
1479
  Flag.withDescription(
1480
    "Answer from a model running on this machine through Ollama, instead of the coder backend",
1481
  ),
1482
);
1478 1483
const coderReasoningFlag = Flag.choice("reasoning", [
1479 1484
  "minimal",
1480 1485
  "low",

@@ -1772,6 +1777,7 @@ const coderCommand = Command.make(

1772 1777
    prompt: coderPrompt,
1773 1778
    plain: coderPlainFlag,
1774 1779
    offline: coderOfflineFlag,
1780
    local: coderLocalFlag,
1775 1781
    resume: coderResumeFlag,
1776 1782
    last: coderLastFlag,
1777 1783
    all: coderAllFlag,

@@ -1787,6 +1793,7 @@ const coderCommand = Command.make(

1787 1793
    prompt,
1788 1794
    plain,
1789 1795
    offline,
1796
    local,
1790 1797
    resume,
1791 1798
    last,
1792 1799
    all,

@@ -1835,11 +1842,25 @@ const coderCommand = Command.make(

1835 1842
      }
1836 1843
1837 1844
      const named = Option.getOrUndefined(model);
1845
1846
      // Local is asked for, not fallen into. It used to be the default whenever
1847
      // a machine happened to be running Ollama, which made the session's model
1848
      // a property of what was running on the laptop rather than a choice, and
1849
      // a reader who started the server for something else got a different
1850
      // coder without asking for one.
1838 1851
      const localModel =
1839
        named === undefined && !offline && !resume
1852
        local && named === undefined && !offline && !resume
1840 1853
          ? yield* Effect.promise(() => discoverOllamaModel())
1841 1854
          : undefined;
1842 1855
1856
      if (local && named === undefined && localModel === undefined && !offline && !resume) {
1857
        return yield* new InputError({
1858
          message:
1859
            "--local answers from a model on this machine, and no Ollama server is running " +
1860
            "with one. Start it, pull a model, or drop --local to use the coder backend.",
1861
        });
1862
      }
1863
1843 1864
      const wantsOllama = named === undefined ? localModel !== undefined : isOllamaModelFlag(named);
1844 1865
      const askedFor =
1845 1866
        named === undefined

@@ -1977,6 +1998,8 @@ const coderCommand = Command.make(

1977 1998
                    // records it on the thread, and `--resume` filters on it
1978 1999
                    // rather than parsing the objective back.
1979 2000
                    repository: workspace.repository,
2001
                    // The named backend, or the one this build leads with.
2002
                    model: named ?? defaultBackendId(),
1980 2003
                    reasoning: Option.getOrUndefined(reasoning),
1981 2004
                  }),
1982 2005
                // The server's own code and sentence, which is what turns a ninth
packages/openagents-cli/src/coder-backends.ts modified +20

@@ -32,5 +32,25 @@ export const CODER_BACKENDS: readonly CoderBackend[] = [

32 32
  { id: "gemini-3.7-flash", label: "Gemini 3.7 Flash" },
33 33
];
34 34
35
/**
36
 * What a coder session opens on when nobody names a backend.
37
 *
38
 * Deliberately not the server's own default, which is the catalog's first entry
39
 * and serves every caller of the chat API. A coder turn is a long one with tools
40
 * in it, and this build leads with the fast model for that; a reader who wants
41
 * the other says so with `--model`.
42
 *
43
 * Named rather than taken from the list's order, because the order here mirrors
44
 * the server's published enum and a test holds the two together. Expressing a
45
 * preference by reordering would have broken that agreement to say something the
46
 * list was never saying.
47
 */
48
export const DEFAULT_CODER_BACKEND = "gemini-3.7-flash";
49
50
export const defaultBackendId = (): string =>
51
  CODER_BACKENDS.some((backend) => backend.id === DEFAULT_CODER_BACKEND)
52
    ? DEFAULT_CODER_BACKEND
53
    : (CODER_BACKENDS[0]?.id ?? "");
54
35 55
/** Every id, for a flag's error message and its accepted values. */
36 56
export const backendIds = (): readonly string[] => CODER_BACKENDS.map((backend) => backend.id);
packages/openagents-cli/src/coder-transcript.ts modified +18 -1

@@ -184,6 +184,16 @@ export class ThreadTranscriptWriter implements TranscriptSink {

184 184
          continue;
185 185
        }
186 186
187
        if (outcome === "unsupported") {
188
          this.threadClosed = true;
189
          this.queue.length = 0;
190
          this.trouble(
191
            "This server does not serve a thread transcript yet, so this session is not " +
192
              "recorded on it. The work is unaffected.",
193
          );
194
          break;
195
        }
196
187 197
        if (outcome === "thread_closed") {
188 198
          this.threadClosed = true;
189 199
          this.queue.length = 0;

@@ -227,7 +237,7 @@ export class ThreadTranscriptWriter implements TranscriptSink {

227 237
  /** One POST, translated to what the pump can act on. Never throws. */
228 238
  private async send(
229 239
    event: QueuedEvent,
230
  ): Promise<"posted" | "retry" | "refused" | "thread_closed"> {
240
  ): Promise<"posted" | "retry" | "refused" | "thread_closed" | "unsupported"> {
231 241
    let response: Response;
232 242
    try {
233 243
      response = await this.post(

@@ -250,6 +260,13 @@ export class ThreadTranscriptWriter implements TranscriptSink {

250 260
    // A server that is down answers 5xx; the event is still good.
251 261
    if (response.status >= 500) return "retry";
252 262
263
    // A server without the route is a server older than this client, not a
264
    // server calling the event invalid. Retrying cannot help and neither can
265
    // editing the payload, so the record stops for the session and says why
266
    // once — rather than reporting a refusal on every turn for a route that was
267
    // never reached.
268
    if (response.status === 404 || response.status === 405) return "unsupported";
269
253 270
    const body = (await response.json().catch(() => ({}))) as Record<string, unknown>;
254 271
    if (body["code"] === "thread_terminal") return "thread_closed";
255 272
    return "refused";
packages/openagents-cli/test/coder-ollama.test.ts modified +14

@@ -557,3 +557,17 @@ describe("what goes back to the model each round", () => {

557 557
    expect(chunks.at(-1)).toMatchObject({ type: "usage" });
558 558
  });
559 559
});
560
561
describe("which lane a session opens on", () => {
562
  it("leads with Gemini 3.7 Flash when nobody names a backend", async () => {
563
    const { defaultBackendId, CODER_BACKENDS } = await import("../src/coder-backends.js");
564
565
    // Local used to be the default whenever a machine happened to be running
566
    // Ollama, which made the session's model a property of the laptop rather
567
    // than a choice.
568
    expect(defaultBackendId()).toBe("gemini-3.7-flash");
569
    // And the list still mirrors the server's enum in its own order: a
570
    // preference is named, not expressed by reordering an agreement.
571
    expect(CODER_BACKENDS.map((backend) => backend.id)).toEqual(["ox-alpha", "gemini-3.7-flash"]);
572
  });
573
});

This page updates live while a promote is in flight · changelog