OpenAgents

Commit

Tell a local model what it is, and number children once

be7971fd0443 · AtlantisPleb · · parent fc1b2d94063b 

Tell a local model what it is, and number children once

Two things the read-tool transcript exposed.

A local session sent no system prompt. The thread lane sends the server an
objective at thread creation; the local lane sent a bare user message and the
tool schemas. Asked what tools it had, the model answered with what a coding
agent usually has -- read/write, search, bash, web fetch -- none of which this
session declares. That answer then sat in the transcript, and the next turn read
it back as instruction: its own reasoning says the tools are "not in the function
list" and then defers to "the instructions state that Read/Write etc. exist".
There were no such instructions. The model file carries no system directive and
its template is `{{ .Prompt }}`, so the vacuum was ours to fill.

So the source now opens with one. It is derived from the tools actually declared
rather than written out, which is the point: it cannot name a tool the session
does not pass, cannot miss one it does, and needs no edit when the tool list
changes. It says the list is complete, that the session has no file, shell,
search, or web tools of its own, and that a capability named in a tool's
description belongs to the child that tool starts, not to the model reading it.
Three runs that had been inventing tools now answer with `delegate` alone and
say what they cannot do.

The delegate description invited the second bug. It said each child is told its
number "so a prompt may say work on your own numbered file", and the model wrote
`You are child #1 of 3` into the prompt every child receives. The harness
already numbers each child, so children two and three were told they were both
themselves and child one, and did identical work: three agents, one file, one
result, reported as "same result -- a good sign". It was the opposite. The
description now says every child runs the same prompt and is told its own number
separately, and to write for whichever child reads it rather than naming one.
The same fan-out now reads three different files and reports three line counts,
all three correct.

280 tests pass, including four on the system prompt: that it names the declared
tools, that it says there are none when there are none, that it is derived
rather than fixed, and that it is sent once rather than per turn.

Deploy story

What this commit did to the running system — joined from the forge receipt chain, the part a commit page elsewhere cannot show.

Not deployed through the forge lane

No push, promotion, build, or deploy receipt references this commit (receipts are scanned over a bounded recent window). Changes shipped by full node replacement carry their proof in the release gate receipt instead.

Changed files

  • modified packages/openagents-cli/src/coder-ollama.ts
  • modified packages/openagents-cli/src/coder-tools.ts
  • modified packages/openagents-cli/test/coder-ollama.test.ts

Diff

3 files changed, +126 -4

packages/openagents-cli/src/coder-ollama.ts modified +47

@@ -53,6 +53,47 @@ export const parseOllamaModelFlag = (value: string): string | undefined => {

53 53 
 */
54 54 
type WireMessage = OllamaMessage;
55 55
56 
/**
57 
 * What the session tells a local model about itself.
58 
 *
59 
 * Derived from the tools actually declared rather than written out, so it
60 
 * cannot claim a tool the session does not pass or miss one it does.
61 
 *
62 
 * The thread lane sends the server an objective at thread creation. The local
63 
 * lane sent nothing, and a model with no system prompt has nothing anchoring
64 
 * what it is: asked what tools it has, it answered from what a coding agent
65 
 * usually has -- files, shell, search, web -- and none of that is declared
66 
 * here. The invented answer then sat in the transcript, and the next turn read
67 
 * it back as instruction. So the anchor is the tool list itself.
68 
 */
69 
const systemPrompt = (tools: ReadonlyArray<CoderTool>): string => {
70 
  const lines = [
71 
    "You are `openagents coder`, a coding assistant in a terminal. You answer from a model " +
72 
      "running locally on this machine.",
73 
    "",
74 
  ];
75
76 
  if (tools.length === 0) {
77 
    lines.push(
78 
      "You have no tools in this session. You cannot read or write files, run commands, search " +
79 
        "the repository, or fetch a URL. Answer from what the reader tells you, and say plainly " +
80 
        "when something would need a tool you do not have.",
81 
    );
82 
  } else {
83 
    lines.push(
84 
      `You have ${String(tools.length)} tool${tools.length === 1 ? "" : "s"}, and no others:`,
85 
      ...tools.map((tool) => `- \`${tool.name}\``),
86 
      "",
87 
      "That list is complete. You have no file, shell, search, or web tools of your own: any " +
88 
        "capability not on that list is one you do not have. Where a tool's description says what " +
89 
        "a child agent can do, that is the child's capability and not yours. Never say you ran " +
90 
        "something you did not run.",
91 
    );
92 
  }
93
94 
  return lines.join("\n");
95 
};
96
56 97 
export class OllamaReplySource implements ReplySource {
57 98 
  private readonly client: Ollama;
58 99 
  private readonly modelName: string;

@@ -81,6 +122,12 @@ export class OllamaReplySource implements ReplySource {

81 122 
  }
82 123
83 124 
  async *reply(prompt: string, signal: AbortSignal): AsyncIterable<ReplyChunk> {
125 
    // Built on the first turn rather than in the constructor: the tools are
126 
    // declared after construction, and the prompt is derived from them.
127 
    if (this.transcript.length === 0) {
128 
      this.transcript.push({ role: "system", content: systemPrompt(this.tools) });
129 
    }
130
84 131 
    this.transcript.push({ role: "user", content: prompt });
85 132
86 133 
    // A turn is a loop, not a single call: the model may answer, or it may ask
packages/openagents-cli/src/coder-tools.ts modified +5 -3

@@ -56,9 +56,11 @@ export function delegateTool(delegation: CoderDelegation): CoderTool {

56 56 
      "depend on each other: several files to change the same way, several hypotheses to check, " +
57 57 
      "several tests to run down. Each child is a full coding agent with its own file and shell " +
58 58 
      "tools, it starts with no context from this conversation, and it cannot ask questions, so " +
59 
      "the prompt has to be self-contained. Children run on this session's budget. Each child is " +
60 
      'told which number it is out of the count, so a prompt may say "work on your own numbered ' +
61 
      'file". Prefer one call with a count over several calls. At most ' +
59 
      "the prompt has to be self-contained. Children run on this session's budget. Every child " +
60 
      "runs the same prompt, and each is told separately which number it is, so write the prompt " +
61 
      'for whichever child reads it: say "read the file at your own number" rather than naming ' +
62 
      'one child ("you are child 1"), which gives every child the same work and wastes the ' +
63 
      "fan-out. Prefer one call with a count over several calls. At most " +
62 64 
      `${String(MAX_DELEGATE_COUNT)} children.`,
63 65 
    parameters: {
64 66 
      type: "object",
packages/openagents-cli/test/coder-ollama.test.ts modified +74 -1

@@ -119,7 +119,12 @@ describe("an ollama turn that calls a tool", () => {

119 119 
    // The tool was declared, and the second round carried the exchange back.
120 120 
    expect(stub.requests[0]).toHaveProperty("tools");
121 121 
    const messages = stub.requests[1]?.["messages"] as ReadonlyArray<Record<string, unknown>>;
122 
    expect(messages.map((message) => message["role"])).toEqual(["user", "assistant", "tool"]);
122 
    expect(messages.map((message) => message["role"])).toEqual([
123 
      "system",
124 
      "user",
125 
      "assistant",
126 
      "tool",
127 
    ]);
123 128 
    expect(messages.at(-1)).toMatchObject({ content: "child 1 said PONG", tool_name: "delegate" });
124 129 
  });
125 130

@@ -173,3 +178,71 @@ describe("an ollama turn that calls a tool", () => {

173 178 
    expect((chunks.at(-1) as { value: string }).value).toContain("Stopped after 6 rounds");
174 179 
  });
175 180 
});
181
182 
describe("what a local session tells the model about itself", () => {
183 
  const systemOf = (stub: { requests: Record<string, unknown>[] }, round = 0) => {
184 
    const messages = stub.requests[round]?.["messages"] as ReadonlyArray<Record<string, unknown>>;
185 
    return messages[0] as { role: string; content: string };
186 
  };
187
188 
  it("opens with a system message naming every declared tool and no others", async () => {
189 
    const { source, stub } = sourceWith([[chunk({ content: "ok" }, true)]]);
190 
    source.useTools([delegate([])]);
191
192 
    await collect(source, "hi");
193
194 
    const system = systemOf(stub);
195 
    expect(system.role).toBe("system");
196 
    expect(system.content).toContain("1 tool");
197 
    expect(system.content).toContain("`delegate`");
198 
    // The failure this exists to stop: the model answering with the tools a
199 
    // coding agent usually has rather than the ones it was given.
200 
    expect(system.content).toContain("no file, shell, search, or web tools of your own");
201 
    // A tool description says what a child can do. That is not the model's.
202 
    expect(system.content).toContain("that is the child's capability and not yours");
203 
  });
204
205 
  it("says it has none when the session declares no tools", async () => {
206 
    const { source, stub } = sourceWith([[chunk({ content: "ok" }, true)]]);
207
208 
    await collect(source, "hi");
209
210 
    expect(systemOf(stub).content).toContain("You have no tools in this session");
211 
  });
212
213 
  it("is derived from the tools, so it cannot name one the session does not pass", async () => {
214 
    const { source, stub } = sourceWith([[chunk({ content: "ok" }, true)]]);
215 
    source.useTools([
216 
      { name: "alpha", description: "a", parameters: {}, run: () => Promise.resolve("") },
217 
      { name: "beta", description: "b", parameters: {}, run: () => Promise.resolve("") },
218 
    ]);
219
220 
    await collect(source, "hi");
221
222 
    const system = systemOf(stub);
223 
    expect(system.content).toContain("2 tools");
224 
    expect(system.content).toContain("`alpha`");
225 
    expect(system.content).toContain("`beta`");
226 
    expect(system.content).not.toContain("`delegate`");
227 
  });
228
229 
  it("says it once, not on every turn", async () => {
230 
    const { source, stub } = sourceWith([
231 
      [chunk({ content: "one" }, true)],
232 
      [chunk({ content: "two" }, true)],
233 
    ]);
234 
    source.useTools([delegate([])]);
235
236 
    await collect(source, "first");
237 
    await collect(source, "second");
238
239 
    const messages = stub.requests[1]?.["messages"] as ReadonlyArray<Record<string, unknown>>;
240 
    expect(messages.filter((message) => message["role"] === "system")).toHaveLength(1);
241 
    expect(messages.map((message) => message["role"])).toEqual([
242 
      "system",
243 
      "user",
244 
      "assistant",
245 
      "user",
246 
    ]);
247 
  });
248 
});

This page updates live while a promote is in flight · changelog