Change how hard the model thinks, and say what it is set to

c0e5957660db · AtlantisPleb · · parent 7be4b94e7fac

Change how hard the model thinks, and say what it is set to

There was no answer to "what reasoning level are we on". `--reasoning` existed
but only reached the thread lane, where it is recorded at thread creation; the
local lane had no level at all and nothing showed one.

The status line names it — `thinking medium` — and shift+tab cycles it, in both
spellings: the classic back-tab and the one the keyboard protocol reports, which
this interface now turns on for shift+enter anyway.

A source declares its own levels rather than sharing one ladder, because they do
not have the same rungs. Ollama takes `think` as a boolean or one of low,
medium, high — four rungs, confirmed against the running server and the client's
own types — so the local lane offers `off low medium high`. `off` is the boolean
because a model asked to think at no level still thinks; the way to stop it is
to say not to.

The flag's five names map onto those four: `minimal` is off, and `max` is high
because there is nothing above it. Mapped by name rather than clamped silently,
so a reader who asked for `max` and sees `high` can find out why.

A source with one level and no other reports the level and offers no key, since
a key that does nothing is worse than no key. Cycling is refused while a turn
runs, for the reason switching a backend is: a turn already accepted keeps the
shape it was accepted with.

409 tests pass, including that the level reaches the request on every round and
that both spellings of shift+tab move it.

Deploy story

What this commit did to the running system — joined from the forge receipt chain, the part a commit page elsewhere cannot show.

Not deployed through the forge lane

No push, promotion, build, or deploy receipt references this commit (receipts are scanned over a bounded recent window). Changes shipped by full node replacement carry their proof in the release gate receipt instead.

Changed files

  • modified packages/openagents-cli/src/cli.ts
  • modified packages/openagents-cli/src/coder-ollama.ts
  • modified packages/openagents-cli/src/coder-session.ts
  • modified packages/openagents-cli/src/coder-ui.ts
  • modified packages/openagents-cli/test/coder-ollama.test.ts
  • modified packages/openagents-cli/test/coder-ui.test.ts

Diff

6 files changed, +224 -1

packages/openagents-cli/src/cli.ts modified +4 -1

@@ -1763,7 +1763,10 @@ const coderCommand = Command.make(

1763 1763
      // so it takes neither a thread nor the stand-in.
1764 1764
      const source: ReplySource =
1765 1765
        wantsOllama && ollamaName !== undefined
1766
          ? new OllamaReplySource({ model: ollamaName })
1766
          ? new OllamaReplySource({
1767
              model: ollamaName,
1768
              ...(Option.isSome(reasoning) ? { reasoning: reasoning.value } : {}),
1769
            })
1767 1770
          : (thread ?? new DummyReplySource());
1768 1771
1769 1772
      // Children get their own thread on their own model. The conversation
packages/openagents-cli/src/coder-ollama.ts modified +52

@@ -31,11 +31,48 @@ const DEFAULT_HOST = "http://127.0.0.1:11434";

31 31
 */
32 32
const MAX_TOOL_STEPS = 100;
33 33
34
/**
35
 * What a reasoning level means to Ollama.
36
 *
37
 * `think` takes a boolean or, on a model that advertises thinking, one of these
38
 * names. `off` is the boolean, because a model asked to think at no level still
39
 * thinks; the way to stop it is to say not to.
40
 */
41
const THINK: Record<string, boolean | "low" | "medium" | "high"> = {
42
  off: false,
43
  low: "low",
44
  medium: "medium",
45
  high: "high",
46
};
47
48
/**
49
 * What `--reasoning` means here.
50
 *
51
 * The flag's ladder is the thread lane's — minimal through max — and Ollama has
52
 * four rungs, not five. `minimal` is off because a minimal amount of thinking
53
 * from a model that cannot be asked for a little is none, and `max` is `high`
54
 * because there is nothing above it. Named rather than silently clamped, so a
55
 * reader who asked for `max` and sees `high` can see why.
56
 */
57
const FROM_FLAG: Record<string, string> = {
58
  minimal: "off",
59
  off: "off",
60
  low: "low",
61
  medium: "medium",
62
  high: "high",
63
  max: "high",
64
};
65
66
/** The levels, in the order the interface cycles them. */
67
export const OLLAMA_REASONING_LEVELS = Object.keys(THINK);
68
34 69
export interface OllamaOptions {
35 70
  /** The Ollama model name, without the `ollama:` prefix. */
36 71
  readonly model: string;
37 72
  /** The Ollama server endpoint. Defaults to `http://127.0.0.1:11434`. */
38 73
  readonly host?: string | undefined;
74
  /** Where the reasoning level starts. Defaults to the model's own default. */
75
  readonly reasoning?: string | undefined;
39 76
}
40 77
41 78
/**

@@ -195,6 +232,7 @@ export class OllamaReplySource implements ReplySource {

195 232
   * the difference between steering a model and waiting one out.
196 233
   */
197 234
  private steered: string[] = [];
235
  private reasoningLevel: string;
198 236
  private callCount = 0;
199 237
200 238
  get model(): string {

@@ -212,6 +250,8 @@ export class OllamaReplySource implements ReplySource {

212 250
  }
213 251
214 252
  constructor(options: OllamaOptions) {
253
    this.reasoningLevel =
254
      options.reasoning === undefined ? "medium" : (FROM_FLAG[options.reasoning] ?? "medium");
215 255
    this.host = options.host ?? DEFAULT_HOST;
216 256
    this.client = new Ollama({ host: this.host });
217 257
    this.modelName = options.model;

@@ -265,6 +305,17 @@ export class OllamaReplySource implements ReplySource {

265 305
   * which model is the part a reader needs, and so is knowing the transcript
266 306
   * survives.
267 307
   */
308
  get reasoning(): { readonly level: string; readonly levels: ReadonlyArray<string> } {
309
    return { level: this.reasoningLevel, levels: OLLAMA_REASONING_LEVELS };
310
  }
311
312
  cycleReasoning(): string {
313
    const at = OLLAMA_REASONING_LEVELS.indexOf(this.reasoningLevel);
314
    this.reasoningLevel =
315
      OLLAMA_REASONING_LEVELS[(at + 1) % OLLAMA_REASONING_LEVELS.length] ?? "medium";
316
    return this.reasoningLevel;
317
  }
318
268 319
  /** Take a message for the next step of the running turn. */
269 320
  steer(text: string): boolean {
270 321
    this.steered.push(text);

@@ -337,6 +388,7 @@ export class OllamaReplySource implements ReplySource {

337 388
        // request nobody can reason about.
338 389
        messages: [...this.transcript],
339 390
        stream: true,
391
        think: THINK[this.reasoningLevel] ?? "medium",
340 392
        ...(this.tools.length === 0 || finalRound
341 393
          ? {}
342 394
          : {
packages/openagents-cli/src/coder-session.ts modified +38

@@ -127,6 +127,8 @@ export interface CoderSnapshot {

127 127
  readonly repository: string;
128 128
  readonly branch: string;
129 129
  readonly model: string;
130
  /** How hard the model is asked to think, when the source says. */
131
  readonly reasoning: string | undefined;
130 132
  /**
131 133
   * Turns this process has submitted, counted from the moment one starts.
132 134
   *

@@ -187,6 +189,16 @@ export interface ReplySource {

187 189
   * only where pressing it would do something.
188 190
   */
189 191
  cycleBackend?(): string;
192
  /**
193
   * How hard the model is asked to think, and what else it could be asked.
194
   *
195
   * A source that cannot vary it reports the level it is fixed at and offers no
196
   * others, which is how the interface knows to show the level without offering
197
   * a key that would do nothing.
198
   */
199
  readonly reasoning?: { readonly level: string; readonly levels: ReadonlyArray<string> };
200
  /** Move to the next reasoning level and return it. */
201
  cycleReasoning?(): string;
190 202
  /**
191 203
   * Declare the tools the model may call.
192 204
   *

@@ -377,6 +389,7 @@ export class CoderSession {

377 389
      repository: this.repository,
378 390
      branch: this.branch,
379 391
      model: this.source.model,
392
      reasoning: this.source.reasoning?.level,
380 393
      turns: this.turnCount,
381 394
      budget: this.source.budget,
382 395
      tasks: this.delegation?.registry.list() ?? [],

@@ -428,6 +441,31 @@ export class CoderSession {

428 441
   * — the status line would name a model that did not produce the text on
429 442
   * screen. The caller shows the refusal rather than switching silently.
430 443
   */
444
  /** Whether another reasoning level exists to move to. */
445
  get canCycleReasoning(): boolean {
446
    return (this.source.reasoning?.levels.length ?? 0) > 1;
447
  }
448
449
  /**
450
   * Move to the next reasoning level.
451
   *
452
   * Refused while a turn runs, for the reason switching a backend is: a turn
453
   * already accepted keeps the shape it was accepted with, and a level that
454
   * changed halfway through would describe neither half.
455
   */
456
  cycleReasoning(): { readonly changed: boolean; readonly level: string | undefined } {
457
    if (!this.canCycleReasoning || this.source.cycleReasoning === undefined) {
458
      return { changed: false, level: this.source.reasoning?.level };
459
    }
460
    if (this.running) {
461
      this.notice("A turn is running. The reasoning level changes on the next turn.");
462
      return { changed: false, level: this.source.reasoning?.level };
463
    }
464
    const level = this.source.cycleReasoning();
465
    this.notice(`Reasoning set to ${level}.`);
466
    return { changed: true, level };
467
  }
468
431 469
  cycleBackend(): { readonly switched: boolean; readonly label: string | undefined } {
432 470
    if (this.source.cycleBackend === undefined) return { switched: false, label: undefined };
433 471
    if (this.controller !== undefined) {
packages/openagents-cli/src/coder-ui.ts modified +13

@@ -574,6 +574,7 @@ export function runCoderUi(session: CoderSession, options: CoderUiOptions): Prom

574 574
      // checkout they are in, they can ask git for the branch, and nothing on
575 575
      // screen but this says which model answers or what the thread has left.
576 576
      const facts = [snapshot.repository, snapshot.branch, snapshot.model];
577
      if (snapshot.reasoning !== undefined) facts.push(`thinking ${snapshot.reasoning}`);
577 578
      if (snapshot.budget !== undefined) facts.push(snapshot.budget);
578 579
      let where = "";
579 580
      for (let from = 0; from < facts.length; from += 1) {

@@ -630,6 +631,9 @@ export function runCoderUi(session: CoderSession, options: CoderUiOptions): Prom

630 631
      // Only when there is another model to switch to, and only while nothing
631 632
      // is running: a turn already accepted keeps the backend it named.
632 633
      if (session.canCycleBackend && !snapshot.running) keys.push({ text: "tab to switch model" });
634
      if (session.canCycleReasoning && !snapshot.running) {
635
        keys.push({ text: "shift+tab to change thinking" });
636
      }
633 637
      if (lines.length > transcriptRows) keys.push({ text: "pgup/pgdn to scroll" });
634 638
      if (focusedTool(snapshot) !== undefined) keys.push({ text: "ctrl+o to expand" });
635 639

@@ -877,6 +881,15 @@ export function runCoderUi(session: CoderSession, options: CoderUiOptions): Prom

877 881
            dirty = false;
878 882
            continue;
879 883
          }
884
          // Shift+tab, in both spellings: the classic back-tab and the one the
885
          // keyboard protocol reports. Tab moves the model, shift+tab moves how
886
          // hard it is asked to think.
887
          if (sequence === "\x1b[Z" || sequence === "\x1b[9;2u") {
888
            session.cycleReasoning();
889
            dirty = true;
890
            continue;
891
          }
892
880 893
          const page = Math.max(1, viewport - 1);
881 894
          if (sequence === "\x1b[5~") scrollBy(-page);
882 895
          else if (sequence === "\x1b[6~") scrollBy(page);
packages/openagents-cli/test/coder-ollama.test.ts modified +53

@@ -389,3 +389,56 @@ describe("naming a model by the part a reader remembers", () => {

389 389
    await expect(resolveOllamaModel("qwen3.8")).resolves.toEqual({ installed: ["llama3:8b"] });
390 390
  });
391 391
});
392
393
describe("how hard the model is asked to think", () => {
394
  const seen: unknown[] = [];
395
  const sourceWithThink = () => {
396
    const source = new OllamaReplySource({ model: "m" });
397
    (source as unknown as { client: unknown }).client = {
398
      chat: async (request: Record<string, unknown>) => {
399
        seen.push(request["think"]);
400
        return Object.assign(
401
          (async function* () {
402
            yield { message: { content: "x" }, done: true };
403
          })(),
404
          { abort: () => {} },
405
        );
406
      },
407
    };
408
    return source;
409
  };
410
411
  it("sends the level on every request", async () => {
412
    seen.length = 0;
413
    const source = sourceWithThink();
414
415
    for await (const _ of source.reply("x", new AbortController().signal)) void _;
416
417
    expect(seen.at(-1)).toBe("medium");
418
  });
419
420
  it("cycles through the levels Ollama has, and off is not a level", async () => {
421
    seen.length = 0;
422
    const source = sourceWithThink();
423
424
    expect(source.reasoning.levels).toEqual(["off", "low", "medium", "high"]);
425
426
    const sent: unknown[] = [];
427
    for (let round = 0; round < 4; round += 1) {
428
      for await (const _ of source.reply("x", new AbortController().signal)) void _;
429
      sent.push(seen.at(-1));
430
      source.cycleReasoning();
431
    }
432
433
    // A model asked to think at no level still thinks; the way to stop it is to
434
    // say not to, which is the boolean.
435
    expect(sent).toEqual(["medium", "high", false, "low"]);
436
  });
437
438
  it("maps the flag's ladder onto the four rungs Ollama has", () => {
439
    // The flag is the thread lane's, and has five names.
440
    expect(new OllamaReplySource({ model: "m", reasoning: "minimal" }).reasoning.level).toBe("off");
441
    expect(new OllamaReplySource({ model: "m", reasoning: "max" }).reasoning.level).toBe("high");
442
    expect(new OllamaReplySource({ model: "m", reasoning: "low" }).reasoning.level).toBe("low");
443
  });
444
});
packages/openagents-cli/test/coder-ui.test.ts modified +64

@@ -775,3 +775,67 @@ describe("reasoning in the transcript", () => {

775 775
    await running;
776 776
  });
777 777
});
778
779
describe("changing how hard the model thinks", () => {
780
  const thinking = () => {
781
    let at = 0;
782
    const levels = ["off", "low", "medium", "high"] as const;
783
    return {
784
      model: "scripted",
785
      get reasoning() {
786
        return { level: levels[at] ?? "medium", levels };
787
      },
788
      cycleReasoning() {
789
        at = (at + 1) % levels.length;
790
        return levels[at] ?? "medium";
791
      },
792
      async *reply() {
793
        yield { type: "text", value: "ok" } as const;
794
      },
795
    };
796
  };
797
798
  it("shows the level in the status line", async () => {
799
    const stdin = new FakeIn();
800
    const stdout = new FakeOut();
801
    const session = new CoderSession(thinking(), "repo", "main");
802
    const running = runCoderUi(session, {
803
      stdin: stdin as unknown as NodeJS.ReadStream,
804
      stdout: stdout as unknown as NodeJS.WriteStream,
805
    });
806
807
    expect(screen(stdout.written).join("\n")).toContain("thinking off");
808
809
    stdin.emit("data", "\x04");
810
    await running;
811
  });
812
813
  it("cycles it on shift+tab, in both spellings", async () => {
814
    const stdin = new FakeIn();
815
    const stdout = new FakeOut();
816
    const session = new CoderSession(thinking(), "repo", "main");
817
    const running = runCoderUi(session, {
818
      stdin: stdin as unknown as NodeJS.ReadStream,
819
      stdout: stdout as unknown as NodeJS.WriteStream,
820
    });
821
822
    // The classic back-tab.
823
    stdin.emit("data", "\x1b[Z");
824
    expect(session.snapshot().reasoning).toBe("low");
825
826
    // And the one the keyboard protocol reports.
827
    stdin.emit("data", "\x1b[9;2u");
828
    expect(session.snapshot().reasoning).toBe("medium");
829
830
    stdin.emit("data", "\x04");
831
    await running;
832
  });
833
834
  it("offers no key when a source has one level and no other", async () => {
835
    const session = new CoderSession(source([]), "repo", "main");
836
837
    // Showing a key that does nothing is worse than showing none.
838
    expect(session.canCycleReasoning).toBe(false);
839
    expect(session.snapshot().reasoning).toBeUndefined();
840
  });
841
});

This page updates live while a promote is in flight · changelog