Adopt the proxy's reasoning and tool-call fidelity in the coder

Diff

2 files changed, +340 -48

 *   against the stream anyway rather than against `await response.text()`, so
 *   chunked delivery becomes visible here the day the server sends it, with no
 *   change on this side.
 * - **It carries no reasoning.** `OpenAgents.Providers.ProviderEvent` has no
 *   reasoning member at all — the union is `response_started`, `text_delta`,
 *   `tool_call`, `usage`, `response_completed`, `failed`, `cancelled` — and the
 *   proxy drops everything it cannot name. The chat event log had
 *   `reasoning_delta`; nothing on this path does. So no `reasoning` chunk is
 *   ever produced here, and the interface's dim-italic reasoning entry, which
 *   the stand-in behind `--offline` still exercises, never appears against a
 *   live model.
 * - **It carries reasoning as `delta.reasoning`.** Since openagents.com
 *   `c26c188` the proxy forwards a model's thinking as string chunks on
 *   `choices[0].delta.reasoning`, interleaved with `delta.content` in stream
 *   order. The parser below turns each into a `reasoning` chunk, which the
 *   interface renders as its dim-italic reasoning entry and the transcript
 *   writer records whole as `turn.reasoning`.
 * - **Tools run here.** The chat lane ran tools on the server and reported each
 *   one. The proxy is a bare completions surface: it forwards the `tools` a
 *   caller declares and returns the calls the model asks for, and the caller

 *   the tools a session declares, and a turn continues until the model stops
 *   asking for one.
 *
 *   A tool result is fed back as plain turns rather than as a `tool` message.
 *   The proxy maps a `tool` message to a `function_call_output` item and sends
 *   it on its own, which the provider refuses without the `function_call` that
 *   preceded it, and the response id that would link the two is never given to
 *   a client. Until the proxy carries a tool exchange, the honest thing is to
 *   say what was called and what came back in turns it does accept.
 *   A tool exchange is fed back in the standard chat shape: the assistant
 *   message carries its `tool_calls` array (content may be empty), and each
 *   result follows as a `role: "tool"` message named by `tool_call_id`. The
 *   same `c26c188` made the proxy replay both faithfully to the provider, so
 *   the plain-turn paraphrase this file used to send — `[tool call]` and
 *   `[tool result]` written into user turns — is gone.
 *
 * Nothing here announces any of that on screen. `2c15c6ed20` removed the
 * `scopeNotice` seam with the reasoning that a session private to its own

  readonly budget: ThreadBudget;
}

/** One chat-completions message, which is what the proxy takes as its input. */
interface WireMessage {
  readonly role: "user" | "assistant";
  readonly content: string;
/** One call in an assistant message, in the chat-completions wire shape. */
interface WireToolCall {
  readonly id: string;
  readonly type: "function";
  readonly function: { readonly name: string; readonly arguments: string };
}

/**
 * One chat-completions message, which is what the proxy takes as its input.
 *
 * The `arguments` a model produced stay the raw JSON string on the way back:
 * the proxy replays them to the provider without interpreting them, and a
 * parse-and-reserialize here could reorder keys or normalize whitespace in a
 * string the provider expects byte for byte.
 */
type WireMessage =
  | { readonly role: "user"; readonly content: string }
  | {
      readonly role: "assistant";
      readonly content: string;
      readonly tool_calls?: ReadonlyArray<WireToolCall>;
    }
  | { readonly role: "tool"; readonly tool_call_id: string; readonly content: string };

/** A call the model asked for, assembled from its fragments. */
interface WireCall {
  readonly id: string;

          yield chunk;
        }

        // One event per block, whole, never deltas. The proxy carries no
        // reasoning today, so this records nothing against a live model; the
        // day a `reasoning` chunk exists here, its record does too.
        // One event per block, whole, never deltas: the record is what was
        // thought, not the pieces it arrived in.
        if (reasoning.length > 0) this.sink?.record("turn.reasoning", { text: reasoning });

        // Whatever the model said belongs to the thread even when the turn was
        // interrupted, or the next turn answers a question it cannot see it
        // half-answered.
        if (assistant.length > 0) {
          this.transcript.push({ role: "assistant", content: assistant });
          turnText = turnText.length === 0 ? assistant : `${turnText}\n\n${assistant}`;
        }
        if (signal.aborted || calls.length === 0) {
          if (assistant.length > 0) this.transcript.push({ role: "assistant", content: assistant });
          this.recordAnswer(turnText, turnToolCalls, signal.aborted);
          return;
        }

        if (step >= MAX_TOOL_STEPS) {
          // Take the tools away for one more round rather than stopping on a
          // tool result. The work already done is the reason the turn is long,
          // and ending on "stopped" throws all of it away.
          // and ending on "stopped" throws all of it away. The calls are
          // dropped rather than written: an assistant `tool_calls` message
          // whose results never follow is a transcript the provider refuses.
          if (assistant.length > 0) this.transcript.push({ role: "assistant", content: assistant });
          this.mustAnswer = true;
          this.transcript.push({
            role: "user",

          continue;
        }

        // The exchange in the standard chat shape: one assistant message
        // carrying every call of the round — content may be empty, and the
        // arguments stay the raw JSON string the model produced — then, after
        // the tools have run, one `tool` message per result in the order the
        // calls were made, whatever order they finished in.
        this.transcript.push({
          role: "assistant",
          content: assistant,
          tool_calls: calls.map((call) => ({
            id: call.id,
            type: "function" as const,
            function: { name: call.name, arguments: call.args },
          })),
        });
        turnToolCalls += calls.length;
        // Concurrently. A model asking for two tools in one turn is saying they do
        // not depend on each other, and running them in order anyway makes a fan-out
        // to two models cost the sum of both.
        if (signal.aborted) {
          this.recordAnswer(turnText, turnToolCalls, true);
          return;
        const results = new Map<string, string>();
        yield* merge(calls.map((call) => this.invoke(call, signal, results)));
        for (const call of calls) {
          this.transcript.push({
            role: "tool",
            tool_call_id: call.id,
            content: results.get(call.id) ?? "",
          });
        }
        turnToolCalls += calls.length;
        yield* merge(calls.map((call) => this.invoke(call, signal)));
      }
    } finally {
      // Read the budget on the way out of every turn, including an interrupted

  }

  /**
   * Run one call, report it, and put the exchange on the thread.
   * Run one call, report it, and leave its result for the caller to file.
   *
   * The transcript keeps the call and its result as an assistant turn and a
   * user turn for the reason given at the top of this file: a `tool` message is
   * not carried by the proxy today, and a model that cannot see what its own
   * call returned calls it again.
   * The result goes into `results` under the call's id rather than onto the
   * transcript here: calls of one round run concurrently and finish in any
   * order, and the turn loop writes the `tool` messages afterward in the
   * order the calls were made, so the wire history is deterministic.
   */
  private async *invoke(call: WireCall, signal: AbortSignal): AsyncIterable<ReplyChunk> {
  private async *invoke(
    call: WireCall,
    signal: AbortSignal,
    results: Map<string, string>,
  ): AsyncIterable<ReplyChunk> {
    yield { type: "tool_call", callId: call.id, name: call.name, arguments: call.args };

    const tool = this.tools.find((candidate) => candidate.name === call.name);

        : { error: bounded(failure, EVENT_RESULT_KEPT) }),
    });

    this.transcript.push({
      role: "assistant",
      content: `[tool call]\n${call.name}(${call.args})`,
    });
    // Bounded on the way onto the transcript, not on the way to the reader:
    // this is what goes back to the model on every round after, and a session
    // that re-sends everything it has already read spends its wall clock on
    // reading it again.
    this.transcript.push({
      role: "user",
      content: `[tool result ${call.name}]\n${boundedResult(output)}`,
    });
    // Bounded on the way toward the model, not on the way to the reader or the
    // record: this is what goes back on every round after, and a session that
    // re-sends everything it has already read spends its wall clock on reading
    // it again. The `tool.ran` event above kept the fuller copy.
    results.set(call.id, boundedResult(output));
  }

  /**

      for (const choice of choices) {
        const delta = record(record(choice)["delta"]);

        // Reasoning precedes the words it produced, and the proxy interleaves
        // the two in stream order, so within one delta it is yielded first.
        const thought = delta["reasoning"];
        if (typeof thought === "string" && thought.length > 0) {
          yield { type: "reasoning", value: thought };
        }

        const content = delta["content"];
        if (typeof content === "string" && content.length > 0) {
          yield { type: "text", value: content };

    const second = proxied[1];
    expect(second?.body["messages"]).toEqual([
      { role: "user", content: "hello" },
      { role: "assistant", content: `[tool call]\ndelegate({"prompt":"add tests"})` },
      { role: "user", content: "[tool result delegate]\n2 of 2 children completed." },
      {
        role: "assistant",
        content: "",
        tool_calls: [
          {
            id: "call-1",
            type: "function",
            function: { name: "delegate", arguments: `{"prompt":"add tests"}` },
          },
        ],
      },
      { role: "tool", tool_call_id: "call-1", content: "2 of 2 children completed." },
    ]);
  });

  it("translates reasoning deltas into reasoning chunks, in stream order", async () => {
    stub({
      proxy: [
        sse([
          [
            `data: {"choices":[{"delta":{"reasoning":"Let me think."},"index":0}]}`,
            `data: {"choices":[{"delta":{"reasoning":" Two files."},"index":0}]}`,
            `data: {"choices":[{"delta":{"content":"Answer."},"index":0}]}`,
            `data: [DONE]`,
            "",
          ].join("\n\n"),
        ]),
      ],
    });

    expect(await chunks(await open())).toEqual([
      { type: "reasoning", value: "Let me think." },
      { type: "reasoning", value: " Two files." },
      { type: "text", value: "Answer." },
    ]);
  });

  it("keeps reasoning off the wire transcript: the next turn replays only what was said", async () => {
    const calls = stub({
      proxy: [
        sse([
          [
            `data: {"choices":[{"delta":{"reasoning":"Thinking."},"index":0}]}`,
            `data: {"choices":[{"delta":{"content":"Answer."},"index":0}]}`,
            `data: [DONE]`,
            "",
          ].join("\n\n"),
        ]),
        sse([LIVE_SSE]),
      ],
    });

    const source = await open();
    await chunks(source, "first");
    await chunks(source, "second");

    const spends = calls.filter((call) => call.url.endsWith("/api/inference/proxy"));
    expect(spends[1]?.body["messages"]).toEqual([
      { role: "user", content: "first" },
      { role: "assistant", content: "Answer." },
      { role: "user", content: "second" },
    ]);
  });

  it("replays two calls of one round in call order with their raw arguments", async () => {
    // The second call's arguments carry spacing the model chose. A parse and
    // re-serialize would normalize it; the wire must not.
    const rawFirst = `{"command":  "ls -la"}`;
    const rawSecond = `{"pattern": "thread",  "limit": 2}`;
    const round = [
      `data: ${JSON.stringify({
        choices: [
          {
            index: 0,
            delta: {
              tool_calls: [
                {
                  index: 0,
                  id: "call-a",
                  type: "function",
                  function: { name: "shell", arguments: rawFirst },
                },
                {
                  index: 1,
                  id: "call-b",
                  type: "function",
                  function: { name: "grep", arguments: rawSecond },
                },
              ],
            },
          },
        ],
      })}`,
      `data: {"choices":[{"index":0,"delta":{},"finish_reason":"tool_calls"}]}`,
      `data: [DONE]`,
      "",
    ].join("\n\n");

    const calls = stub({ proxy: [sse([round]), sse([LIVE_SSE])] });
    const source = await open();

    // The first tool finishes last, so completion order is the reverse of
    // call order, and the transcript must not care.
    let releaseFirst: () => void = () => undefined;
    const gate = new Promise<void>((resolve) => {
      releaseFirst = resolve;
    });
    source.useTools([
      {
        name: "shell",
        description: "run a command",
        parameters: { type: "object" },
        run: async () => {
          await gate;
          return "shell output";
        },
      },
      {
        name: "grep",
        description: "search",
        parameters: { type: "object" },
        run: async () => {
          releaseFirst();
          return "grep output";
        },
      },
    ]);

    await chunks(source, "run both");

    const spends = calls.filter((call) => call.url.endsWith("/api/inference/proxy"));
    expect(spends[1]?.body["messages"]).toEqual([
      { role: "user", content: "run both" },
      {
        role: "assistant",
        content: "",
        tool_calls: [
          { id: "call-a", type: "function", function: { name: "shell", arguments: rawFirst } },
          { id: "call-b", type: "function", function: { name: "grep", arguments: rawSecond } },
        ],
      },
      { role: "tool", tool_call_id: "call-a", content: "shell output" },
      { role: "tool", tool_call_id: "call-b", content: "grep output" },
    ]);
  });

  it("carries the words an assistant said alongside the calls it made", async () => {
    const round = [
      `data: {"choices":[{"delta":{"content":"Looking now."},"index":0}]}`,
      `data: {"choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"id":"call-1","type":"function","function":{"name":"shell","arguments":"{}"}}]}}]}`,
      `data: {"choices":[{"index":0,"delta":{},"finish_reason":"tool_calls"}]}`,
      `data: [DONE]`,
      "",
    ].join("\n\n");
    const calls = stub({ proxy: [sse([round]), sse([LIVE_SSE])] });
    const source = await open();
    source.useTools([withTool(async () => "done")]);

    await chunks(source, "look");

    const spends = calls.filter((call) => call.url.endsWith("/api/inference/proxy"));
    const messages = spends[1]?.body["messages"] as Array<Record<string, unknown>>;
    expect(messages[1]).toMatchObject({ role: "assistant", content: "Looking now." });
    expect(messages[1]?.["tool_calls"]).toHaveLength(1);
  });

  it("bounds a tool result on the wire while the record keeps the fuller copy", async () => {
    const round = [
      `data: {"choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"id":"call-1","type":"function","function":{"name":"shell","arguments":"{}"}}]}}]}`,
      `data: {"choices":[{"index":0,"delta":{},"finish_reason":"tool_calls"}]}`,
      `data: [DONE]`,
      "",
    ].join("\n\n");
    const calls = stub({ proxy: [sse([round]), sse([LIVE_SSE])] });
    const source = await open();
    const sink = recorder();
    source.useTranscript(sink);
    source.useTools([withTool(async () => "x".repeat(10_000))]);

    await chunks(source, "dump it");

    const spends = calls.filter((call) => call.url.endsWith("/api/inference/proxy"));
    const messages = spends[1]?.body["messages"] as Array<Record<string, unknown>>;
    const result = messages.find((message) => message["role"] === "tool");
    const wire = result?.["content"] as string;
    // The 4,000-char context budget still holds on the way to the model...
    expect(wire.length).toBeLessThan(4_200);
    expect(wire).toContain("characters omitted");
    // ...while the durable event keeps the result whole, as before.
    const ran = sink.events.find((event) => event.eventType === "tool.ran");
    expect(ran?.payload["output"]).toBe("x".repeat(10_000));
  });

  it("drops the calls past the step limit and asks for an answer without tools", async () => {
    const round = [
      `data: {"choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"id":"call-1","type":"function","function":{"name":"shell","arguments":"{}"}}]}}]}`,
      `data: {"choices":[{"index":0,"delta":{},"finish_reason":"tool_calls"}]}`,
      `data: [DONE]`,
      "",
    ].join("\n\n");
    // Steps 0..100 each ask for a tool; step 100 is past the limit, so its
    // call is dropped and step 101 must answer in words.
    const calls = stub({
      proxy: [...Array.from({ length: 101 }, () => sse([round])), sse([LIVE_SSE])],
    });
    const source = await open();
    source.useTools([withTool(async () => "ok")]);

    expect(textOf(await chunks(source, "loop"))).toBe("Hello! Nice");

    const spends = calls.filter((call) => call.url.endsWith("/api/inference/proxy"));
    expect(spends).toHaveLength(102);
    const last = spends[101];
    // The tools are withheld for the answering round.
    expect(last?.body["tools"]).toBeUndefined();
    const messages = last?.body["messages"] as Array<Record<string, unknown>>;
    // The dropped round left no orphan: every assistant `tool_calls` message
    // is answered by a `tool` message, and the nudge closes the transcript.
    const asked = messages.filter((message) => Array.isArray(message["tool_calls"])).length;
    const answered = messages.filter((message) => message["role"] === "tool").length;
    expect(asked).toBe(100);
    expect(answered).toBe(100);
    expect(messages[messages.length - 1]?.["content"]).toContain(
      "You have reached this turn's limit on tool calls.",
    );
  });

  it("lends the grant to children without handing over the token", async () => {

543	765		`expect(sink.events[2]?.payload).toEqual({ text: "only the top level", steered: true });`
544	766		`});`
545	767
	768	+	`it("records the turn's reasoning whole, one event per block", async () => {`
	769	+	`stub({`
	770	+	`proxy: [`
	771	+	`sse([`
	772	+	`[`
	773	+	`data: {"choices":[{"delta":{"reasoning":"Let me think."},"index":0}]}`,
	774	+	`data: {"choices":[{"delta":{"reasoning":" Two files."},"index":0}]}`,
	775	+	`data: {"choices":[{"delta":{"content":"Two files."},"index":0}]}`,
	776	+	`data: [DONE]`,
	777	+	`"",`
	778	+	`].join("\n\n"),`
	779	+	`]),`
	780	+	`],`
	781	+	`});`
	782	+	`const source = await open();`
	783	+	`const sink = recorder();`
	784	+	`source.useTranscript(sink);`
	785	+
	786	+	`await chunks(source, "what is in this repo?");`
	787	+
	788	+	`expect(sink.events.map((event) => event.eventType)).toEqual([`
	789	+	`"turn.user",`
	790	+	`"turn.reasoning",`
	791	+	`"turn.assistant",`
	792	+	`]);`
	793	+	`// The deltas are how the thinking arrived, not what it is: one event`
	794	+	`// carries the block whole.`
	795	+	`expect(sink.events[1]?.payload).toEqual({ text: "Let me think. Two files." });`
	796	+	`});`
	797	+
546	798		`it("records nothing extra for a turn without tools", async () => {`
547	799		`stub({});`
548	800		`const source = await open();`

26	26		* against the stream anyway rather than against `await response.text()`, so
27	27		`* chunked delivery becomes visible here the day the server sends it, with no`
28	28		`* change on this side.`
29		-	* - It carries no reasoning. `OpenAgents.Providers.ProviderEvent` has no
30		-	* reasoning member at all — the union is `response_started`, `text_delta`,
31		-	* `tool_call`, `usage`, `response_completed`, `failed`, `cancelled` — and the
32		-	`* proxy drops everything it cannot name. The chat event log had`
33		-	* `reasoning_delta`; nothing on this path does. So no `reasoning` chunk is
34		-	`* ever produced here, and the interface's dim-italic reasoning entry, which`
35		-	* the stand-in behind `--offline` still exercises, never appears against a
36		-	`* live model.`
	29	+	* - It carries reasoning as `delta.reasoning`. Since openagents.com
	30	+	* `c26c188` the proxy forwards a model's thinking as string chunks on
	31	+	* `choices[0].delta.reasoning`, interleaved with `delta.content` in stream
	32	+	* order. The parser below turns each into a `reasoning` chunk, which the
	33	+	`* interface renders as its dim-italic reasoning entry and the transcript`
	34	+	* writer records whole as `turn.reasoning`.
37	35		`* - Tools run here. The chat lane ran tools on the server and reported each`
38	36		* one. The proxy is a bare completions surface: it forwards the `tools` a
39	37		`* caller declares and returns the calls the model asks for, and the caller`

229	227		`readonly budget: ThreadBudget;`
230	228		`}`
231	229
232		-	`/** One chat-completions message, which is what the proxy takes as its input. */`
233		-	`interface WireMessage {`
234		-	`readonly role: "user" \| "assistant";`
235		-	`readonly content: string;`
	230	+	`/** One call in an assistant message, in the chat-completions wire shape. */`
	231	+	`interface WireToolCall {`
	232	+	`readonly id: string;`
	233	+	`readonly type: "function";`
	234	+	`readonly function: { readonly name: string; readonly arguments: string };`
236	235		`}`
237	236
	237	+	`/**`
	238	+	`* One chat-completions message, which is what the proxy takes as its input.`
	239	+	`*`
	240	+	* The `arguments` a model produced stay the raw JSON string on the way back:
	241	+	`* the proxy replays them to the provider without interpreting them, and a`
	242	+	`* parse-and-reserialize here could reorder keys or normalize whitespace in a`
	243	+	`* string the provider expects byte for byte.`
	244	+	`*/`
	245	+	`type WireMessage =`
	246	+	`\| { readonly role: "user"; readonly content: string }`
	247	+	`\| {`
	248	+	`readonly role: "assistant";`
	249	+	`readonly content: string;`
	250	+	`readonly tool_calls?: ReadonlyArray<WireToolCall>;`
	251	+	`}`
	252	+	`\| { readonly role: "tool"; readonly tool_call_id: string; readonly content: string };`
	253	+
238	254		`/** A call the model asked for, assembled from its fragments. */`
239	255		`interface WireCall {`
240	256		`readonly id: string;`

408	424		`yield chunk;`
409	425		`}`
410	426
411		-	`// One event per block, whole, never deltas. The proxy carries no`
412		-	`// reasoning today, so this records nothing against a live model; the`
413		-	// day a `reasoning` chunk exists here, its record does too.
	427	+	`// One event per block, whole, never deltas: the record is what was`
	428	+	`// thought, not the pieces it arrived in.`
414	429		`if (reasoning.length > 0) this.sink?.record("turn.reasoning", { text: reasoning });`
415	430
416	431		`// Whatever the model said belongs to the thread even when the turn was`
417	432		`// interrupted, or the next turn answers a question it cannot see it`
418	433		`// half-answered.`
419	434		`if (assistant.length > 0) {`
420		-	`this.transcript.push({ role: "assistant", content: assistant });`
421	435		turnText = turnText.length === 0 ? assistant : `${turnText}\n\n${assistant}`;
422	436		`}`
423	437		`if (signal.aborted \|\| calls.length === 0) {`
	438	+	`if (assistant.length > 0) this.transcript.push({ role: "assistant", content: assistant });`
424	439		`this.recordAnswer(turnText, turnToolCalls, signal.aborted);`
425	440		`return;`
426	441		`}`

428	443		`if (step >= MAX_TOOL_STEPS) {`
429	444		`// Take the tools away for one more round rather than stopping on a`
430	445		`// tool result. The work already done is the reason the turn is long,`
431		-	`// and ending on "stopped" throws all of it away.`
	446	+	`// and ending on "stopped" throws all of it away. The calls are`
	447	+	// dropped rather than written: an assistant `tool_calls` message
	448	+	`// whose results never follow is a transcript the provider refuses.`
	449	+	`if (assistant.length > 0) this.transcript.push({ role: "assistant", content: assistant });`
432	450		`this.mustAnswer = true;`
433	451		`this.transcript.push({`
434	452		`role: "user",`

439	457		`continue;`
440	458		`}`
441	459
	460	+	`// The exchange in the standard chat shape: one assistant message`
	461	+	`// carrying every call of the round — content may be empty, and the`
	462	+	`// arguments stay the raw JSON string the model produced — then, after`
	463	+	// the tools have run, one `tool` message per result in the order the
	464	+	`// calls were made, whatever order they finished in.`
	465	+	`this.transcript.push({`
	466	+	`role: "assistant",`
	467	+	`content: assistant,`
	468	+	`tool_calls: calls.map((call) => ({`
	469	+	`id: call.id,`
	470	+	`type: "function" as const,`
	471	+	`function: { name: call.name, arguments: call.args },`
	472	+	`})),`
	473	+	`});`
	474	+	`turnToolCalls += calls.length;`
442	475		`// Concurrently. A model asking for two tools in one turn is saying they do`
443	476		`// not depend on each other, and running them in order anyway makes a fan-out`
444	477		`// to two models cost the sum of both.`
445		-	`if (signal.aborted) {`
446		-	`this.recordAnswer(turnText, turnToolCalls, true);`
447		-	`return;`
	478	+	`const results = new Map<string, string>();`
	479	+	`yield* merge(calls.map((call) => this.invoke(call, signal, results)));`
	480	+	`for (const call of calls) {`
	481	+	`this.transcript.push({`
	482	+	`role: "tool",`
	483	+	`tool_call_id: call.id,`
	484	+	`content: results.get(call.id) ?? "",`
	485	+	`});`
448	486		`}`
449		-	`turnToolCalls += calls.length;`
450		-	`yield* merge(calls.map((call) => this.invoke(call, signal)));`
451	487		`}`
452	488		`} finally {`
453	489		`// Read the budget on the way out of every turn, including an interrupted`

Commit

Adopt the proxy's reasoning and tool-call fidelity in the coder

Deploy story

Not deployed through the forge lane

Changed files

Diff

41	39		`* the tools a session declares, and a turn continues until the model stops`
42	40		`* asking for one.`
43	41		`*`
44		-	* A tool result is fed back as plain turns rather than as a `tool` message.
45		-	* The proxy maps a `tool` message to a `function_call_output` item and sends
46		-	* it on its own, which the provider refuses without the `function_call` that
47		-	`* preceded it, and the response id that would link the two is never given to`
48		-	`* a client. Until the proxy carries a tool exchange, the honest thing is to`
49		-	`* say what was called and what came back in turns it does accept.`
	42	+	`* A tool exchange is fed back in the standard chat shape: the assistant`
	43	+	* message carries its `tool_calls` array (content may be empty), and each
	44	+	* result follows as a `role: "tool"` message named by `tool_call_id`. The
	45	+	* same `c26c188` made the proxy replay both faithfully to the provider, so
	46	+	* the plain-turn paraphrase this file used to send — `[tool call]` and
	47	+	* `[tool result]` written into user turns — is gone.
50	48		`*`
51	49		* Nothing here announces any of that on screen. `2c15c6ed20` removed the
52	50		* `scopeNotice` seam with the reasoning that a session private to its own

484	520		`}`
485	521
486	522		`/**`
487		-	`* Run one call, report it, and put the exchange on the thread.`
	523	+	`* Run one call, report it, and leave its result for the caller to file.`
488	524		`*`
489		-	`* The transcript keeps the call and its result as an assistant turn and a`
490		-	* user turn for the reason given at the top of this file: a `tool` message is
491		-	`* not carried by the proxy today, and a model that cannot see what its own`
492		-	`* call returned calls it again.`
	525	+	* The result goes into `results` under the call's id rather than onto the
	526	+	`* transcript here: calls of one round run concurrently and finish in any`
	527	+	* order, and the turn loop writes the `tool` messages afterward in the
	528	+	`* order the calls were made, so the wire history is deterministic.`
493	529		`*/`
494		-	`private async *invoke(call: WireCall, signal: AbortSignal): AsyncIterable<ReplyChunk> {`
	530	+	`private async *invoke(`
	531	+	`call: WireCall,`
	532	+	`signal: AbortSignal,`
	533	+	`results: Map<string, string>,`
	534	+	`): AsyncIterable<ReplyChunk> {`
495	535		`yield { type: "tool_call", callId: call.id, name: call.name, arguments: call.args };`
496	536
497	537		`const tool = this.tools.find((candidate) => candidate.name === call.name);`

531	571		`: { error: bounded(failure, EVENT_RESULT_KEPT) }),`
532	572		`});`
533	573
534		-	`this.transcript.push({`
535		-	`role: "assistant",`
536		-	content: `[tool call]\n${call.name}(${call.args})`,
537		-	`});`
538		-	`// Bounded on the way onto the transcript, not on the way to the reader:`
539		-	`// this is what goes back to the model on every round after, and a session`
540		-	`// that re-sends everything it has already read spends its wall clock on`
541		-	`// reading it again.`
542		-	`this.transcript.push({`
543		-	`role: "user",`
544		-	content: `[tool result ${call.name}]\n${boundedResult(output)}`,
545		-	`});`
	574	+	`// Bounded on the way toward the model, not on the way to the reader or the`
	575	+	`// record: this is what goes back on every round after, and a session that`
	576	+	`// re-sends everything it has already read spends its wall clock on reading`
	577	+	// it again. The `tool.ran` event above kept the fuller copy.
	578	+	`results.set(call.id, boundedResult(output));`
546	579		`}`
547	580
548	581		`/**`

633	666		`for (const choice of choices) {`
634	667		`const delta = record(record(choice)["delta"]);`
635	668
	669	+	`// Reasoning precedes the words it produced, and the proxy interleaves`
	670	+	`// the two in stream order, so within one delta it is yielded first.`
	671	+	`const thought = delta["reasoning"];`
	672	+	`if (typeof thought === "string" && thought.length > 0) {`
	673	+	`yield { type: "reasoning", value: thought };`
	674	+	`}`
	675	+
636	676		`const content = delta["content"];`
637	677		`if (typeof content === "string" && content.length > 0) {`
638	678		`yield { type: "text", value: content };`

344	344		`const second = proxied[1];`
345	345		`expect(second?.body["messages"]).toEqual([`
346	346		`{ role: "user", content: "hello" },`
347		-	{ role: "assistant", content: `[tool call]\ndelegate({"prompt":"add tests"})` },
348		-	`{ role: "user", content: "[tool result delegate]\n2 of 2 children completed." },`
	347	+	`{`
	348	+	`role: "assistant",`
	349	+	`content: "",`
	350	+	`tool_calls: [`
	351	+	`{`
	352	+	`id: "call-1",`
	353	+	`type: "function",`
	354	+	function: { name: "delegate", arguments: `{"prompt":"add tests"}` },
	355	+	`},`
	356	+	`],`
	357	+	`},`
	358	+	`{ role: "tool", tool_call_id: "call-1", content: "2 of 2 children completed." },`
	359	+	`]);`
	360	+	`});`
	361	+
	362	+	`it("translates reasoning deltas into reasoning chunks, in stream order", async () => {`
	363	+	`stub({`
	364	+	`proxy: [`
	365	+	`sse([`
	366	+	`[`
	367	+	`data: {"choices":[{"delta":{"reasoning":"Let me think."},"index":0}]}`,
	368	+	`data: {"choices":[{"delta":{"reasoning":" Two files."},"index":0}]}`,
	369	+	`data: {"choices":[{"delta":{"content":"Answer."},"index":0}]}`,
	370	+	`data: [DONE]`,
	371	+	`"",`
	372	+	`].join("\n\n"),`
	373	+	`]),`
	374	+	`],`
	375	+	`});`
	376	+
	377	+	`expect(await chunks(await open())).toEqual([`
	378	+	`{ type: "reasoning", value: "Let me think." },`
	379	+	`{ type: "reasoning", value: " Two files." },`
	380	+	`{ type: "text", value: "Answer." },`
	381	+	`]);`
	382	+	`});`
	383	+
	384	+	`it("keeps reasoning off the wire transcript: the next turn replays only what was said", async () => {`
	385	+	`const calls = stub({`
	386	+	`proxy: [`
	387	+	`sse([`
	388	+	`[`
	389	+	`data: {"choices":[{"delta":{"reasoning":"Thinking."},"index":0}]}`,
	390	+	`data: {"choices":[{"delta":{"content":"Answer."},"index":0}]}`,
	391	+	`data: [DONE]`,
	392	+	`"",`
	393	+	`].join("\n\n"),`
	394	+	`]),`
	395	+	`sse([LIVE_SSE]),`
	396	+	`],`
	397	+	`});`
	398	+
	399	+	`const source = await open();`
	400	+	`await chunks(source, "first");`
	401	+	`await chunks(source, "second");`
	402	+
	403	+	`const spends = calls.filter((call) => call.url.endsWith("/api/inference/proxy"));`
	404	+	`expect(spends[1]?.body["messages"]).toEqual([`
	405	+	`{ role: "user", content: "first" },`
	406	+	`{ role: "assistant", content: "Answer." },`
	407	+	`{ role: "user", content: "second" },`
	408	+	`]);`
	409	+	`});`
	410	+
	411	+	`it("replays two calls of one round in call order with their raw arguments", async () => {`
	412	+	`// The second call's arguments carry spacing the model chose. A parse and`
	413	+	`// re-serialize would normalize it; the wire must not.`
	414	+	const rawFirst = `{"command": "ls -la"}`;
	415	+	const rawSecond = `{"pattern": "thread", "limit": 2}`;
	416	+	`const round = [`
	417	+	`data: ${JSON.stringify({
	418	+	`choices: [`
	419	+	`{`
	420	+	`index: 0,`
	421	+	`delta: {`
	422	+	`tool_calls: [`
	423	+	`{`
	424	+	`index: 0,`
	425	+	`id: "call-a",`
	426	+	`type: "function",`
	427	+	`function: { name: "shell", arguments: rawFirst },`
	428	+	`},`
	429	+	`{`
	430	+	`index: 1,`
	431	+	`id: "call-b",`
	432	+	`type: "function",`
	433	+	`function: { name: "grep", arguments: rawSecond },`
	434	+	`},`
	435	+	`],`
	436	+	`},`
	437	+	`},`
	438	+	`],`
	439	+	})}`,
	440	+	`data: {"choices":[{"index":0,"delta":{},"finish_reason":"tool_calls"}]}`,
	441	+	`data: [DONE]`,
	442	+	`"",`
	443	+	`].join("\n\n");`
	444	+
	445	+	`const calls = stub({ proxy: [sse([round]), sse([LIVE_SSE])] });`
	446	+	`const source = await open();`
	447	+
	448	+	`// The first tool finishes last, so completion order is the reverse of`
	449	+	`// call order, and the transcript must not care.`
	450	+	`let releaseFirst: () => void = () => undefined;`
	451	+	`const gate = new Promise<void>((resolve) => {`
	452	+	`releaseFirst = resolve;`
	453	+	`});`
	454	+	`source.useTools([`
	455	+	`{`
	456	+	`name: "shell",`
	457	+	`description: "run a command",`
	458	+	`parameters: { type: "object" },`
	459	+	`run: async () => {`
	460	+	`await gate;`
	461	+	`return "shell output";`
	462	+	`},`
	463	+	`},`
	464	+	`{`
	465	+	`name: "grep",`
	466	+	`description: "search",`
	467	+	`parameters: { type: "object" },`
	468	+	`run: async () => {`
	469	+	`releaseFirst();`
	470	+	`return "grep output";`
	471	+	`},`
	472	+	`},`
349	473		`]);`
	474	+
	475	+	`await chunks(source, "run both");`
	476	+
	477	+	`const spends = calls.filter((call) => call.url.endsWith("/api/inference/proxy"));`
	478	+	`expect(spends[1]?.body["messages"]).toEqual([`
	479	+	`{ role: "user", content: "run both" },`
	480	+	`{`
	481	+	`role: "assistant",`
	482	+	`content: "",`
	483	+	`tool_calls: [`
	484	+	`{ id: "call-a", type: "function", function: { name: "shell", arguments: rawFirst } },`
	485	+	`{ id: "call-b", type: "function", function: { name: "grep", arguments: rawSecond } },`
	486	+	`],`
	487	+	`},`
	488	+	`{ role: "tool", tool_call_id: "call-a", content: "shell output" },`
	489	+	`{ role: "tool", tool_call_id: "call-b", content: "grep output" },`
	490	+	`]);`
	491	+	`});`
	492	+
	493	+	`it("carries the words an assistant said alongside the calls it made", async () => {`
	494	+	`const round = [`
	495	+	`data: {"choices":[{"delta":{"content":"Looking now."},"index":0}]}`,
	496	+	`data: {"choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"id":"call-1","type":"function","function":{"name":"shell","arguments":"{}"}}]}}]}`,
	497	+	`data: {"choices":[{"index":0,"delta":{},"finish_reason":"tool_calls"}]}`,
	498	+	`data: [DONE]`,
	499	+	`"",`
	500	+	`].join("\n\n");`
	501	+	`const calls = stub({ proxy: [sse([round]), sse([LIVE_SSE])] });`
	502	+	`const source = await open();`
	503	+	`source.useTools([withTool(async () => "done")]);`
	504	+
	505	+	`await chunks(source, "look");`
	506	+
	507	+	`const spends = calls.filter((call) => call.url.endsWith("/api/inference/proxy"));`
	508	+	`const messages = spends[1]?.body["messages"] as Array<Record<string, unknown>>;`
	509	+	`expect(messages[1]).toMatchObject({ role: "assistant", content: "Looking now." });`
	510	+	`expect(messages[1]?.["tool_calls"]).toHaveLength(1);`
	511	+	`});`
	512	+
	513	+	`it("bounds a tool result on the wire while the record keeps the fuller copy", async () => {`
	514	+	`const round = [`
	515	+	`data: {"choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"id":"call-1","type":"function","function":{"name":"shell","arguments":"{}"}}]}}]}`,
	516	+	`data: {"choices":[{"index":0,"delta":{},"finish_reason":"tool_calls"}]}`,
	517	+	`data: [DONE]`,
	518	+	`"",`
	519	+	`].join("\n\n");`
	520	+	`const calls = stub({ proxy: [sse([round]), sse([LIVE_SSE])] });`
	521	+	`const source = await open();`
	522	+	`const sink = recorder();`
	523	+	`source.useTranscript(sink);`
	524	+	`source.useTools([withTool(async () => "x".repeat(10_000))]);`
	525	+
	526	+	`await chunks(source, "dump it");`
	527	+
	528	+	`const spends = calls.filter((call) => call.url.endsWith("/api/inference/proxy"));`
	529	+	`const messages = spends[1]?.body["messages"] as Array<Record<string, unknown>>;`
	530	+	`const result = messages.find((message) => message["role"] === "tool");`
	531	+	`const wire = result?.["content"] as string;`
	532	+	`// The 4,000-char context budget still holds on the way to the model...`
	533	+	`expect(wire.length).toBeLessThan(4_200);`
	534	+	`expect(wire).toContain("characters omitted");`
	535	+	`// ...while the durable event keeps the result whole, as before.`
	536	+	`const ran = sink.events.find((event) => event.eventType === "tool.ran");`
	537	+	`expect(ran?.payload["output"]).toBe("x".repeat(10_000));`
	538	+	`});`
	539	+
	540	+	`it("drops the calls past the step limit and asks for an answer without tools", async () => {`
	541	+	`const round = [`
	542	+	`data: {"choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"id":"call-1","type":"function","function":{"name":"shell","arguments":"{}"}}]}}]}`,
	543	+	`data: {"choices":[{"index":0,"delta":{},"finish_reason":"tool_calls"}]}`,
	544	+	`data: [DONE]`,
	545	+	`"",`
	546	+	`].join("\n\n");`
	547	+	`// Steps 0..100 each ask for a tool; step 100 is past the limit, so its`
	548	+	`// call is dropped and step 101 must answer in words.`
	549	+	`const calls = stub({`
	550	+	`proxy: [...Array.from({ length: 101 }, () => sse([round])), sse([LIVE_SSE])],`
	551	+	`});`
	552	+	`const source = await open();`
	553	+	`source.useTools([withTool(async () => "ok")]);`
	554	+
	555	+	`expect(textOf(await chunks(source, "loop"))).toBe("Hello! Nice");`
	556	+
	557	+	`const spends = calls.filter((call) => call.url.endsWith("/api/inference/proxy"));`
	558	+	`expect(spends).toHaveLength(102);`
	559	+	`const last = spends[101];`
	560	+	`// The tools are withheld for the answering round.`
	561	+	`expect(last?.body["tools"]).toBeUndefined();`
	562	+	`const messages = last?.body["messages"] as Array<Record<string, unknown>>;`
	563	+	// The dropped round left no orphan: every assistant `tool_calls` message
	564	+	// is answered by a `tool` message, and the nudge closes the transcript.
	565	+	`const asked = messages.filter((message) => Array.isArray(message["tool_calls"])).length;`
	566	+	`const answered = messages.filter((message) => message["role"] === "tool").length;`
	567	+	`expect(asked).toBe(100);`
	568	+	`expect(answered).toBe(100);`
	569	+	`expect(messages[messages.length - 1]?.["content"]).toContain(`
	570	+	`"You have reached this turn's limit on tool calls.",`
	571	+	`);`
350	572		`});`
351	573
352	574		`it("lends the grant to children without handing over the token", async () => {`