Carry the turn's cost into the trajectory on the thread lane

fe816df06eb0 · AtlantisPleb · 4d ago · parent 544bb6e8b9d8

Carry the turn's cost into the trajectory on the thread lane

The local lane yields a usage report at the end of a turn and the
thread lane did not, so a thread-backed session's ATIF export carried
no step metrics and its final totals were empty — the first real Gym
trial showed tokens as unknown for a run whose cost the proxy had
metered all along. The thread lane now yields the same end-of-turn
report the local lane does, the session attaches it to the closing
entry, and the exporter's existing aggregation fills
total_prompt_tokens and total_completion_tokens with no further change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GoYpb8FEmdxVErsv7ABCYi

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GoYpb8FEmdxVErsv7ABCYi

Deploy story

What this commit did to the running system — joined from the forge receipt chain, the part a commit page elsewhere cannot show.

Not deployed through the forge lane

No push, promotion, build, or deploy receipt references this commit (receipts are scanned over a bounded recent window). Changes shipped by full node replacement carry their proof in the release gate receipt instead.

Changed files

modified packages/openagents-cli/src/coder-thread.ts
modified packages/openagents-cli/test/coder-thread.test.ts

Diff

2 files changed, +28 -0

        }
        if (signal.aborted || calls.length === 0) {
          if (assistant.length > 0) this.transcript.push({ role: "assistant", content: assistant });
          // The turn's cost reaches the session too, not only the durable
          // transcript: the closing entry carries it into the ATIF export,
          // where a benchmark computes cost per outcome from the trajectory
          // alone. Same shape and place as the local lane's report.
          if (this.turnUsage.calls > 0) {
            yield {
              type: "usage",
              promptTokens: this.turnUsage.promptTokens,
              completionTokens: this.turnUsage.completionTokens,
              calls: this.turnUsage.calls,
            };
          }
          this.recordAnswer(turnText, turnToolCalls, signal.aborted);
          return;
        }

    });
  });

  it("yields the turn's usage so the session's export can carry it", async () => {
    stub({ proxy: [sse([TOOL_ROUND]), sse([ANSWER_ROUND])] });
    const source = await open();
    source.useTools([withTool(async () => "README.md\nsrc")]);

    const received = await chunks(source, "what is in this repo?");

    // One usage chunk, at the end of the turn, summing both model calls —
    // the same report the local lane yields, so the ATIF export's step
    // metrics and final totals hold on every lane.
    const usage = received.filter((chunk) => chunk.type === "usage");
    expect(usage).toEqual([
      { type: "usage", promptTokens: 100, completionTokens: 16, calls: 2 },
    ]);
  });

  it("records a failed tool as one event carrying its error", async () => {
    stub({ proxy: [sse([TOOL_ROUND]), sse([ANSWER_ROUND])] });
    const source = await open();

585	585		`}`
586	586		`if (signal.aborted \|\| calls.length === 0) {`
587	587		`if (assistant.length > 0) this.transcript.push({ role: "assistant", content: assistant });`
	588	+	`// The turn's cost reaches the session too, not only the durable`
	589	+	`// transcript: the closing entry carries it into the ATIF export,`
	590	+	`// where a benchmark computes cost per outcome from the trajectory`
	591	+	`// alone. Same shape and place as the local lane's report.`
	592	+	`if (this.turnUsage.calls > 0) {`
	593	+	`yield {`
	594	+	`type: "usage",`
	595	+	`promptTokens: this.turnUsage.promptTokens,`
	596	+	`completionTokens: this.turnUsage.completionTokens,`
	597	+	`calls: this.turnUsage.calls,`
	598	+	`};`
	599	+	`}`
588	600		`this.recordAnswer(turnText, turnToolCalls, signal.aborted);`
589	601		`return;`
590	602		`}`

738	738		`});`
739	739		`});`
740	740
	741	+	`it("yields the turn's usage so the session's export can carry it", async () => {`
	742	+	`stub({ proxy: [sse([TOOL_ROUND]), sse([ANSWER_ROUND])] });`
	743	+	`const source = await open();`
	744	+	`source.useTools([withTool(async () => "README.md\nsrc")]);`
	745	+
	746	+	`const received = await chunks(source, "what is in this repo?");`
	747	+
	748	+	`// One usage chunk, at the end of the turn, summing both model calls —`
	749	+	`// the same report the local lane yields, so the ATIF export's step`
	750	+	`// metrics and final totals hold on every lane.`
	751	+	`const usage = received.filter((chunk) => chunk.type === "usage");`
	752	+	`expect(usage).toEqual([`
	753	+	`{ type: "usage", promptTokens: 100, completionTokens: 16, calls: 2 },`
	754	+	`]);`
	755	+	`});`
	756	+
741	757		`it("records a failed tool as one event carrying its error", async () => {`
742	758		`stub({ proxy: [sse([TOOL_ROUND]), sse([ANSWER_ROUND])] });`
743	759		`const source = await open();`