Show what the model was told, with /system

14608415c5f5 · AtlantisPleb · · parent fde22295af0b

Show what the model was told, with /system

Nothing in the session showed the standing context a turn carries. Working out
why a model answered as it did meant reading the source that composed the
prompt, and the case that started this -- a local model listing tools it did not
have -- is exactly the case where reading the source is what you cannot afford
to rely on.

`/system` prints it. It is a notice, not a turn: the model is not asked, the
turn count does not move, and the interface already dims a notice, so what the
model was told is visibly not something the model said. Reading what the model
was told does not change what the model was told.

A source reports what it actually sends rather than a description of it, through
an optional `describeContext` on `ReplySource`. The local lane renders its
system message from the same function the request uses and lists every tool with
its description and parameter schema, so the two cannot drift.

The thread lane says where the boundary is. The proxy is a completions surface
and the server composes what precedes the turn, so the client never sees that
message and does not print one. It reports the tool declarations it does send
and states plainly that the system message is the server's. A `/system` that
guessed would be worse than one that admits it cannot see.

The command is the whole line, so "what is in your /system prompt" is still an
ordinary question for the model.

295 tests pass, three on this: that it notices rather than answers, that it
never reaches the model, and that a source composing no context says so.

Deploy story

What this commit did to the running system — joined from the forge receipt chain, the part a commit page elsewhere cannot show.

Not deployed through the forge lane

No push, promotion, build, or deploy receipt references this commit (receipts are scanned over a bounded recent window). Changes shipped by full node replacement carry their proof in the release gate receipt instead.

Changed files

  • modified packages/openagents-cli/src/coder-ollama.ts
  • modified packages/openagents-cli/src/coder-session.ts
  • modified packages/openagents-cli/src/coder-thread.ts
  • modified packages/openagents-cli/src/coder-ui.ts
  • modified packages/openagents-cli/test/coder-session.test.ts

Diff

5 files changed, +126 -1

packages/openagents-cli/src/coder-ollama.ts modified +21

@@ -121,6 +121,27 @@ export class OllamaReplySource implements ReplySource {

121 121
    this.tools = tools;
122 122
  }
123 123
124
  /**
125
   * Everything standing that goes to the model: the system message and the tool
126
   * declarations, rendered from the same values the request carries.
127
   */
128
  describeContext(): string {
129
    const parts = [`System message sent with every turn:\n\n${systemPrompt(this.tools)}`];
130
131
    parts.push(
132
      this.tools.length === 0
133
        ? "\nNo tools are declared to the model."
134
        : `\n${String(this.tools.length)} tool${this.tools.length === 1 ? "" : "s"} declared to the model:\n\n${this.tools
135
            .map(
136
              (tool) =>
137
                `- \`${tool.name}\`\n  ${tool.description}\n  parameters: ${JSON.stringify(tool.parameters)}`,
138
            )
139
            .join("\n\n")}`,
140
    );
141
142
    return parts.join("\n");
143
  }
144
124 145
  async *reply(prompt: string, signal: AbortSignal): AsyncIterable<ReplyChunk> {
125 146
    // Built on the first turn rather than in the constructor: the tools are
126 147
    // declared after construction, and the prompt is derived from them.
packages/openagents-cli/src/coder-session.ts modified +24

@@ -132,6 +132,15 @@ export interface ReplySource {

132 132
   * session then declares no tools rather than declaring tools nothing runs.
133 133
   */
134 134
  useTools?(tools: ReadonlyArray<CoderTool>): void;
135
  /**
136
   * The standing context this source sends with every turn, as text.
137
   *
138
   * What `/system` shows. A source reports what it actually sends rather than a
139
   * description of it, so the two cannot drift: a reader checking why a model
140
   * behaved a certain way is reading the thing the model read. A source that
141
   * does not compose its own context leaves this undefined.
142
   */
143
  describeContext?(): string;
135 144
  /**
136 145
   * Yield the reply to `prompt` in chunks. Rendering appends each chunk as it
137 146
   * arrives, so a slow source shows partial text rather than nothing.

@@ -358,6 +367,21 @@ export class CoderSession {

358 367
    // Delegation is not a turn: it does not go to the model, it does not block
359 368
    // the next prompt, and it is allowed while a reply is streaming. That is
360 369
    // the point of a fleet — the console keeps working while children run.
370
    // `/system` is not a turn either: it reads what the session already holds,
371
    // shows it as a notice, and sends nothing. A reader checking what the model
372
    // was told should not have to change what the model was told to find out.
373
    if (/^\/system\s*$/.test(prompt.trim())) {
374
      this.entries.push({ role: "you", text: prompt, settled: true });
375
      const context = this.source.describeContext?.();
376
      this.notice(
377
        context === undefined
378
          ? "This reply source composes no context of its own, so there is nothing to show."
379
          : context,
380
      );
381
      this.emit();
382
      return;
383
    }
384
361 385
    const delegate = parseDelegateCommand(prompt);
362 386
    if (delegate !== undefined) {
363 387
      this.entries.push({ role: "you", text: prompt, settled: true });
packages/openagents-cli/src/coder-thread.ts modified +29

@@ -257,6 +257,35 @@ export class ThreadReplySource implements ReplySource {

257 257
    this.tools = tools;
258 258
  }
259 259
260
  /**
261
   * What this lane sends as standing context.
262
   *
263
   * The tool declarations are the client's and are reported in full. The system
264
   * message is not: the proxy is a completions surface and the server composes
265
   * what precedes the turn, so this says so rather than printing a prompt this
266
   * process never saw. A `/system` that guessed would be worse than one that
267
   * admits the boundary.
268
   */
269
  describeContext(): string {
270
    const declarations =
271
      this.tools.length === 0
272
        ? "No tools are declared to the model."
273
        : `${String(this.tools.length)} tool${this.tools.length === 1 ? "" : "s"} declared to the model:\n\n${this.tools
274
            .map(
275
              (tool) =>
276
                `- \`${tool.name}\`\n  ${tool.description}\n  parameters: ${JSON.stringify(tool.parameters)}`,
277
            )
278
            .join("\n\n")}`;
279
280
    return [
281
      "This session runs on a thread. The system message is composed by the server for the",
282
      "thread's grant and is not sent from this machine, so it cannot be shown here. What follows",
283
      "is what this process does send with every turn.",
284
      "",
285
      declarations,
286
    ].join("\n");
287
  }
288
260 289
  /**
261 290
   * The grant, for lending to child agents.
262 291
   *
packages/openagents-cli/src/coder-ui.ts modified +1 -1

@@ -757,7 +757,7 @@ export function runCoderUi(session: CoderSession, options: CoderUiOptions): Prom

757 757
758 758
    session.notice(
759 759
      "openagents coder — development build. Type a message and press enter. " +
760
        "Ctrl+D quits, Esc interrupts a reply.",
760
        "Ctrl+D quits, Esc interrupts a reply. `/system` shows what the model is told.",
761 761
    );
762 762
    render();
763 763
  });
packages/openagents-cli/test/coder-session.test.ts modified +51

@@ -313,3 +313,54 @@ describe("CoderSession", () => {

313 313
    });
314 314
  });
315 315
});
316
317
describe("the /system command", () => {
318
  /** A source that records whether the model was reached. */
319
  const watched = (context?: string): ReplySource & { prompts: string[] } => {
320
    const prompts: string[] = [];
321
    return {
322
      model: "scripted",
323
      prompts,
324
      ...(context === undefined ? {} : { describeContext: () => context }),
325
      // eslint-disable-next-line require-yield -- a turn that must not happen
326
      async *reply(prompt) {
327
        prompts.push(prompt);
328
      },
329
    };
330
  };
331
332
  it("shows the source's context as a notice and never reaches the model", async () => {
333
    const reply = watched("System message sent with every turn:\n\nYou are `openagents coder`.");
334
    const session = new CoderSession(reply, "repo", "main");
335
336
    await session.submit("/system");
337
338
    const { entries, turns } = session.snapshot();
339
    // A notice, so the interface dims it: what the model was told is not
340
    // something the model said.
341
    expect(entries.map((entry) => entry.role)).toEqual(["you", "notice"]);
342
    expect(entries[1]?.text).toContain("You are `openagents coder`.");
343
    // Reading what the model was told must not change what the model was told.
344
    expect(reply.prompts).toEqual([]);
345
    expect(turns).toBe(0);
346
  });
347
348
  it("says so when the source composes no context of its own", async () => {
349
    const reply = watched();
350
    const session = new CoderSession(reply, "repo", "main");
351
352
    await session.submit("/system");
353
354
    expect(session.snapshot().entries[1]?.text).toContain("composes no context of its own");
355
    expect(reply.prompts).toEqual([]);
356
  });
357
358
  it("is the whole line, so a question about the system prompt is still a turn", async () => {
359
    const reply = watched("context");
360
    const session = new CoderSession(reply, "repo", "main");
361
362
    await session.submit("what is in your /system prompt");
363
364
    expect(reply.prompts).toEqual(["what is in your /system prompt"]);
365
  });
366
});

This page updates live while a promote is in flight · changelog