Write the coder transcript to thread_events as the turn loop runs

f69fc6d5d602 · AtlantisPleb · · parent 079a4c710a18

Write the coder transcript to thread_events as the turn loop runs

The thread lane now posts its session to POST /api/v3/threads/{id}/events
in the decided vocabulary: turn.user for what the reader asked (steered
messages marked as such), turn.reasoning whole per block, tool.ran as one
event per call carrying the tool's identity, its arguments, and a bounded
result, and turn.assistant with the turn's summed usage and call count.
Deltas and interface notices are not recorded. The server copy is the
only durable copy; the offline, Ollama, and stand-in lanes attach nothing
and are unchanged.

The writer (coder-transcript.ts) never costs the turn loop anything:
record is synchronous enqueue, one pump posts in order in the background,
transient failures retry up a backoff ladder without reordering, three in
a row surface one status-line notice and keep queueing, a 4xx drops only
the refused event, and thread_terminal stops the writer for good. Exit
flushes with a deadline before the revoke that would close the
transcript, so nothing queued is refused as terminal.

Closes OpenAgentsInc/openagents#23.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GoYpb8FEmdxVErsv7ABCYi
Co-Authored-By
Claude Fable 5 <noreply@anthropic.com>
Closes
OpenAgentsInc/openagents#23 (another repository — recorded, not closed)

Deploy story

What this commit did to the running system — joined from the forge receipt chain, the part a commit page elsewhere cannot show.

Not deployed through the forge lane

No push, promotion, build, or deploy receipt references this commit (receipts are scanned over a bounded recent window). Changes shipped by full node replacement carry their proof in the release gate receipt instead.

Changed files

  • modified packages/openagents-cli/src/cli.ts
  • modified packages/openagents-cli/src/coder-thread.ts
  • added packages/openagents-cli/src/coder-transcript.ts
  • modified packages/openagents-cli/test/coder-thread.test.ts
  • added packages/openagents-cli/test/coder-transcript.test.ts

Diff

5 files changed, +828 -8

packages/openagents-cli/src/cli.ts modified +23

@@ -45,6 +45,7 @@ import {

45 45
  resolveOllamaModel,
46 46
} from "./coder-ollama.js";
47 47
import { openThread, ThreadUnavailable, type ThreadReplySource } from "./coder-thread.js";
48
import { ThreadTranscriptWriter } from "./coder-transcript.js";
48 49
import { delegateTool, openagentsTool, shellTool, skillTool } from "./coder-tools.js";
49 50
import {
50 51
  describeLoad,

@@ -1903,6 +1904,25 @@ const coderCommand = Command.make(

1903 1904
        standingContext(skills.active(), process.cwd()),
1904 1905
      );
1905 1906
1907
      // The thread lane writes its transcript to the server as the turn loop
1908
      // runs — `POST /api/v3/threads/{id}/events`, on the account token that
1909
      // opened the thread. The server copy is the only durable copy; the
1910
      // offline, Ollama, and stand-in lanes keep no record and attach nothing.
1911
      // A failed post never reaches the turn loop: the writer queues, retries,
1912
      // and says so once on the status line.
1913
      const transcript =
1914
        thread !== undefined && Option.isSome(stored)
1915
          ? new ThreadTranscriptWriter({
1916
              origin: endpoint.origin,
1917
              threadId: thread.threadId,
1918
              token: Redacted.value(stored.value.token),
1919
              onTrouble: (message) => {
1920
                session.notice(message);
1921
              },
1922
            })
1923
          : undefined;
1924
      if (transcript !== undefined) thread?.useTranscript(transcript);
1925
1906 1926
      // The model is told what it can do rather than the reader being asked to
1907 1927
      // remember a slash command. A turn that needs three agents asks for them
1908 1928
      // mid-sentence, and `/delegate` stays as the way to launch a fan-out

@@ -2002,6 +2022,9 @@ const coderCommand = Command.make(

2002 2022
          // expired an hour later, which is the ninth session refused for a
2003 2023
          // session nobody is in. A process killed outright still leaves it to
2004 2024
          // the server's expiry reap.
2025
          // Flush before revoking: revoking makes the thread terminal, and a
2026
          // terminal thread refuses the events that are still queued.
2027
          if (transcript !== undefined) await transcript.close();
2005 2028
          if (thread !== undefined) await thread.revoke();
2006 2029
          if (childThread?.kind === "opened") await childThread.thread.revoke();
2007 2030
          if (setup !== undefined) await setup.close();
packages/openagents-cli/src/coder-thread.ts modified +117 -8

@@ -66,6 +66,7 @@ import type { ChildGrant } from "./coder-child-gateway.js";

66 66
import { merge } from "./coder-merge.js";
67 67
import type { ReplyChunk, ReplySource } from "./coder-session.js";
68 68
import type { CoderTool } from "./coder-tools.js";
69
import type { TranscriptSink } from "./coder-transcript.js";
69 70
70 71
const THREADS_PATH = "/api/v3/threads";
71 72

@@ -83,14 +84,28 @@ const MAX_TOOL_STEPS = 100;

83 84
/** How much of one tool's output is kept on the transcript. */
84 85
const TOOL_RESULT_KEPT = 4_000;
85 86
86
/** A long tool result, kept at both ends, which is what it is read for. */
87
const boundedResult = (output: string): string => {
88
  if (output.length <= TOOL_RESULT_KEPT) return output;
89
  const half = Math.floor(TOOL_RESULT_KEPT / 2);
90
  const cut = output.length - TOOL_RESULT_KEPT;
87
/**
88
 * How much of one tool's output reaches the durable `tool.ran` event.
89
 *
90
 * A separate figure from `TOOL_RESULT_KEPT`, because they answer different
91
 * questions. The 4,000 above is a context-budget decision made against a
92
 * model's window: it is re-sent on every round of the turn. This one bounds a
93
 * record written once, so it is set where every result a real session has
94
 * produced fits whole — the largest measured was 8.4 KB — and only a
95
 * pathological dump is cut, kept at both ends the same way.
96
 */
97
const EVENT_RESULT_KEPT = 64_000;
98
99
/** A long tool output, kept at both ends, which is what it is read for. */
100
const bounded = (output: string, keep: number): string => {
101
  if (output.length <= keep) return output;
102
  const half = Math.floor(keep / 2);
103
  const cut = output.length - keep;
91 104
  return `${output.slice(0, half)}\n\n[${String(cut)} characters omitted from the middle; run it again more narrowly if you need them]\n\n${output.slice(-half)}`;
92 105
};
93 106
107
const boundedResult = (output: string): string => bounded(output, TOOL_RESULT_KEPT);
108
94 109
/** What the thread may still spend, as the server last reported it. */
95 110
export interface ThreadBudget {
96 111
  readonly calls: number;

@@ -238,6 +253,14 @@ export class ThreadReplySource implements ReplySource {

238 253
  private readonly transcript: WireMessage[] = [];
239 254
  private remaining: ThreadBudget;
240 255
  private tools: ReadonlyArray<CoderTool> = [];
256
  /**
257
   * Where the turn loop writes the durable transcript, when the session has
258
   * one. Absent, nothing is recorded — the offline and local lanes never
259
   * attach one — and every call below is a no-op through optional chaining.
260
   */
261
  private sink: TranscriptSink | undefined;
262
  /** The running turn's token usage, accumulated across its model calls. */
263
  private turnUsage = { promptTokens: 0, completionTokens: 0, totalTokens: 0, calls: 0 };
241 264
  /** Set for the one round that must answer rather than call another tool. */
242 265
  private mustAnswer = false;
243 266
  /**

@@ -281,6 +304,19 @@ export class ThreadReplySource implements ReplySource {

281 304
    this.tools = tools;
282 305
  }
283 306
307
  /**
308
   * Attach the writer that puts this session's turns on the server.
309
   *
310
   * Set after construction for the same reason `useTools` is: the writer needs
311
   * the thread's id, so it cannot exist until the thread does, and the failure
312
   * notice it surfaces needs the session, which is built later still. The
313
   * server copy is the only durable copy — this process keeps no transcript
314
   * file of its own.
315
   */
316
  useTranscript(sink: TranscriptSink): void {
317
    this.sink = sink;
318
  }
319
284 320
  /**
285 321
   * What this lane sends as standing context.
286 322
   *

@@ -342,7 +378,13 @@ export class ThreadReplySource implements ReplySource {

342 378
    // Per turn, not per session: a turn that had to answer without tools must
343 379
    // not leave the next one without them.
344 380
    this.mustAnswer = false;
381
    this.turnUsage = { promptTokens: 0, completionTokens: 0, totalTokens: 0, calls: 0 };
382
    /** The answer so far, across steps, for the one `turn.assistant` event. */
383
    let turnText = "";
384
    /** How many tools this turn ran, reported on `turn.assistant`. */
385
    let turnToolCalls = 0;
345 386
    this.transcript.push({ role: "user", content: prompt });
387
    this.sink?.record("turn.user", { text: prompt });
346 388
347 389
    try {
348 390
      for (let step = 0; ; step += 1) {

@@ -350,22 +392,38 @@ export class ThreadReplySource implements ReplySource {

350 392
        // model is asked again.
351 393
        for (const said of this.steered.splice(0)) {
352 394
          this.transcript.push({ role: "user", content: said });
395
          // Steered mid-turn rather than asked between turns, and the record
396
          // says so, or a replay would show a question the answer ignores.
397
          this.sink?.record("turn.user", { text: said, steered: true });
353 398
        }
354 399
355 400
        const calls: WireCall[] = [];
356 401
        let assistant = "";
402
        let reasoning = "";
357 403
358 404
        for await (const chunk of this.stream(signal, calls)) {
359 405
          if (signal.aborted) break;
360 406
          if (chunk.type === "text") assistant += chunk.value;
407
          if (chunk.type === "reasoning") reasoning += chunk.value;
361 408
          yield chunk;
362 409
        }
363 410
411
        // One event per block, whole, never deltas. The proxy carries no
412
        // reasoning today, so this records nothing against a live model; the
413
        // day a `reasoning` chunk exists here, its record does too.
414
        if (reasoning.length > 0) this.sink?.record("turn.reasoning", { text: reasoning });
415
364 416
        // Whatever the model said belongs to the thread even when the turn was
365 417
        // interrupted, or the next turn answers a question it cannot see it
366 418
        // half-answered.
367
        if (assistant.length > 0) this.transcript.push({ role: "assistant", content: assistant });
368
        if (signal.aborted || calls.length === 0) return;
419
        if (assistant.length > 0) {
420
          this.transcript.push({ role: "assistant", content: assistant });
421
          turnText = turnText.length === 0 ? assistant : `${turnText}\n\n${assistant}`;
422
        }
423
        if (signal.aborted || calls.length === 0) {
424
          this.recordAnswer(turnText, turnToolCalls, signal.aborted);
425
          return;
426
        }
369 427
370 428
        if (step >= MAX_TOOL_STEPS) {
371 429
          // Take the tools away for one more round rather than stopping on a

@@ -384,7 +442,11 @@ export class ThreadReplySource implements ReplySource {

384 442
        // Concurrently. A model asking for two tools in one turn is saying they do
385 443
        // not depend on each other, and running them in order anyway makes a fan-out
386 444
        // to two models cost the sum of both.
387
        if (signal.aborted) return;
445
        if (signal.aborted) {
446
          this.recordAnswer(turnText, turnToolCalls, true);
447
          return;
448
        }
449
        turnToolCalls += calls.length;
388 450
        yield* merge(calls.map((call) => this.invoke(call, signal)));
389 451
      }
390 452
    } finally {

@@ -397,6 +459,30 @@ export class ThreadReplySource implements ReplySource {

397 459
    }
398 460
  }
399 461
462
  /**
463
   * Record the turn's answer, with what it cost.
464
   *
465
   * One event per turn, whatever the turn took to get there: several model
466
   * calls, several tool rounds, one figure each. An interrupted turn is
467
   * recorded too, marked as such, because whatever streamed before Escape was
468
   * said and the next reader of this thread will be answered against it.
469
   */
470
  private recordAnswer(text: string, toolCalls: number, interrupted: boolean): void {
471
    if (this.sink === undefined) return;
472
    if (text.length === 0 && this.turnUsage.calls === 0) return;
473
    this.sink.record("turn.assistant", {
474
      text,
475
      usage: {
476
        prompt_tokens: this.turnUsage.promptTokens,
477
        completion_tokens: this.turnUsage.completionTokens,
478
        total_tokens: this.turnUsage.totalTokens,
479
        calls: this.turnUsage.calls,
480
      },
481
      tool_calls: toolCalls,
482
      ...(interrupted ? { interrupted: true } : {}),
483
    });
484
  }
485
400 486
  /**
401 487
   * Run one call, report it, and put the exchange on the thread.
402 488
   *

@@ -431,6 +517,20 @@ export class ThreadReplySource implements ReplySource {

431 517
      error: failure,
432 518
    };
433 519
520
    // Call and result are one fact, so they are one event. The tool's name is
521
    // its identity for later attribution — a plugin's tool carries the name it
522
    // was declared under — and the result is bounded far above the model-wire
523
    // bound, where every result a real session has produced is stored whole.
524
    this.sink?.record("tool.ran", {
525
      call_id: call.id,
526
      tool: call.name,
527
      arguments: bounded(call.args, EVENT_RESULT_KEPT),
528
      status: failure === undefined ? "succeeded" : "failed",
529
      ...(failure === undefined
530
        ? { output: bounded(output, EVENT_RESULT_KEPT) }
531
        : { error: bounded(failure, EVENT_RESULT_KEPT) }),
532
    });
533
434 534
    this.transcript.push({
435 535
      role: "assistant",
436 536
      content: `[tool call]\n${call.name}(${call.args})`,

@@ -562,6 +662,15 @@ export class ThreadReplySource implements ReplySource {

562 662
      totalTokens: Math.max(0, this.remaining.totalTokens - total),
563 663
      costMicrousd: this.remaining.costMicrousd,
564 664
    };
665
    // The same report feeds the turn's own tally, which `turn.assistant`
666
    // carries: a turn is several calls, and the record holds their sum with
667
    // the count rather than the last call's figures presented as the turn's.
668
    this.turnUsage = {
669
      promptTokens: this.turnUsage.promptTokens + number(usage["prompt_tokens"]),
670
      completionTokens: this.turnUsage.completionTokens + number(usage["completion_tokens"]),
671
      totalTokens: this.turnUsage.totalTokens + total,
672
      calls: this.turnUsage.calls + 1,
673
    };
565 674
  }
566 675
567 676
  /** Read what the server says the thread has left. Failure keeps the estimate. */
packages/openagents-cli/src/coder-transcript.ts added +274

@@ -0,0 +1,274 @@

1
/**
2
 * The thread's durable transcript, written as the turn loop runs.
3
 *
4
 * `POST /api/v3/threads/{id}/events` is append-only and the server's copy is
5
 * the only copy: this process keeps no file of its own, so what lands here is
6
 * what `--resume`, the export, and every other machine reading the thread will
7
 * ever see. The vocabulary is the one decided in the openagents.com audit of
8
 * 2026-08-24 — `turn.user`, `turn.reasoning`, `tool.ran`, `turn.assistant` —
9
 * and deltas and interface notices are deliberately not recorded: a delta is
10
 * how a reply arrived rather than what it is, and a notice never reached a
11
 * model.
12
 *
13
 * The writer must never cost the session anything. A turn loop that blocked on
14
 * a slow POST would make every tool call wait on the network twice, and a turn
15
 * that died because the transcript endpoint was down would have traded the work
16
 * for the record of it. So `record` is synchronous enqueue, one pump posts the
17
 * queue in order in the background, a failed post is retried with backoff, and
18
 * a persistent failure surfaces one notice and keeps queueing rather than ever
19
 * throwing into the loop that called it.
20
 */
21
22
const THREADS_PATH = "/api/v3/threads";
23
24
/**
25
 * How many consecutive failed posts before the reader is told once.
26
 *
27
 * One flaky request is the network being the network. Three in a row is an
28
 * outage the reader should know about, because from that point the durable
29
 * record is running behind the session it records.
30
 */
31
const TROUBLE_AFTER = 3;
32
33
/** Backoff between retries of one event, capped where waiting longer buys nothing. */
34
const RETRY_DELAYS_MS: ReadonlyArray<number> = [500, 1_000, 2_000, 5_000, 10_000, 30_000];
35
36
/**
37
 * How long `close` waits for the queue to drain before giving up.
38
 *
39
 * Flushing runs between the last turn and the revoke that closes the thread,
40
 * and a server that is down at exit must not hold the terminal open forever: a
41
 * reader who typed `exit` has left. What cannot be posted by this deadline is
42
 * lost, and that is the one place loss is accepted.
43
 */
44
const CLOSE_DEADLINE_MS = 5_000;
45
46
/** What one event needs from the caller. The writer supplies the rest. */
47
interface QueuedEvent {
48
  readonly eventType: string;
49
  readonly payload: Record<string, unknown>;
50
}
51
52
/**
53
 * The one call shape the writer makes of its transport. Named so a test can
54
 * hand in a plain function without matching `fetch`'s full overload set.
55
 */
56
export type TranscriptTransport = (input: URL, init?: RequestInit) => Promise<Response>;
57
58
/**
59
 * The slice of the writer a reply source calls. Narrow on purpose so a test
60
 * can stand a recorder in for the whole machinery.
61
 */
62
export interface TranscriptSink {
63
  record(eventType: string, payload: Record<string, unknown>): void;
64
}
65
66
export interface TranscriptWriterOptions {
67
  readonly origin: string;
68
  readonly threadId: string;
69
  /** The account token, the same authority that opened the thread. */
70
  readonly token: string;
71
  /** Where one sentence about persistent failure goes. Usually the status line. */
72
  readonly onTrouble?: ((message: string) => void) | undefined;
73
  /** Injection seam for tests. Defaults to the global `fetch`. */
74
  readonly fetch?: TranscriptTransport | undefined;
75
  /** Injection seam for tests. Defaults to the production backoff ladder. */
76
  readonly retryDelaysMs?: ReadonlyArray<number> | undefined;
77
}
78
79
export class ThreadTranscriptWriter implements TranscriptSink {
80
  private readonly queue: QueuedEvent[] = [];
81
  private readonly delays: ReadonlyArray<number>;
82
  private readonly post: TranscriptTransport;
83
  private pumping = false;
84
  private consecutiveFailures = 0;
85
  private toldOfTrouble = false;
86
  /**
87
   * Set when the server said the thread is terminal. Nothing can ever land on
88
   * a closed transcript, so from here events are dropped rather than queued
89
   * against a refusal that cannot change.
90
   */
91
  private threadClosed = false;
92
  /**
93
   * Set when `close` has run. The session is leaving: whatever could not be
94
   * posted by the flush deadline is not retried into a process that no longer
95
   * exists, and that is the one place loss is accepted.
96
   */
97
  private stopped = false;
98
  /** Resolvers waiting in `close` for the queue to drain. */
99
  private drainWaiters: Array<() => void> = [];
100
101
  constructor(private readonly options: TranscriptWriterOptions) {
102
    this.delays = options.retryDelaysMs ?? RETRY_DELAYS_MS;
103
    this.post = options.fetch ?? globalThis.fetch.bind(globalThis);
104
  }
105
106
  /** How many events are queued and not yet on the server. For tests and `close`. */
107
  get pending(): number {
108
    return this.queue.length;
109
  }
110
111
  /**
112
   * Queue one event for the thread's transcript.
113
   *
114
   * Returns immediately. Events post in the order they were recorded, whatever
115
   * the network does in between, because the transcript is a sequence and a
116
   * reordered one describes a session that never happened.
117
   */
118
  record(eventType: string, payload: Record<string, unknown>): void {
119
    if (this.threadClosed || this.stopped) return;
120
    this.queue.push({ eventType, payload });
121
    // The pump is started here rather than awaited: the caller is the turn
122
    // loop, and the whole contract is that it never waits on this.
123
    void this.pump().catch(() => undefined);
124
  }
125
126
  /**
127
   * Wait for the queue to drain, up to a deadline.
128
   *
129
   * Called before the thread is revoked, because revoking closes the
130
   * transcript and anything still queued would then be refused
131
   * `thread_terminal`. Failure to drain by the deadline resolves rather than
132
   * rejects: a session on its way out has nobody left to throw to.
133
   */
134
  async close(deadlineMs = CLOSE_DEADLINE_MS): Promise<void> {
135
    try {
136
      if (this.queue.length === 0 && !this.pumping) return;
137
138
      await new Promise<void>((resolve) => {
139
        const timer = setTimeout(() => {
140
          this.drainWaiters = this.drainWaiters.filter((waiter) => waiter !== settle);
141
          resolve();
142
        }, deadlineMs);
143
        const settle = () => {
144
          clearTimeout(timer);
145
          resolve();
146
        };
147
        this.drainWaiters.push(settle);
148
      });
149
    } finally {
150
      // Whatever the flush achieved, the writer stops here: a retry loop that
151
      // outlived the session it was recording would hold the process open for
152
      // a transcript nobody is in.
153
      this.stopped = true;
154
    }
155
  }
156
157
  /**
158
   * Post the queue, in order, one event at a time.
159
   *
160
   * One pump runs at a time. A transient failure — the network refusing, a
161
   * 5xx — retries the same event up the backoff ladder and never skips it,
162
   * because posting the next event first would reorder the transcript. A
163
   * refusal that cannot change is treated by kind: `thread_terminal` closes
164
   * the writer, and any other 4xx drops that one event and moves on, since a
165
   * payload the server has already called invalid will be invalid tomorrow.
166
   */
167
  private async pump(): Promise<void> {
168
    if (this.pumping) return;
169
    this.pumping = true;
170
171
    try {
172
      while (this.queue.length > 0 && !this.threadClosed && !this.stopped) {
173
        const event = this.queue[0];
174
        if (event === undefined) break;
175
176
        // Events post strictly in order; each depends on the one before it
177
        // being on the server, so there is nothing here to run concurrently.
178
        // eslint-disable-next-line no-await-in-loop
179
        const outcome = await this.send(event);
180
181
        if (outcome === "posted") {
182
          this.queue.shift();
183
          this.consecutiveFailures = 0;
184
          continue;
185
        }
186
187
        if (outcome === "thread_closed") {
188
          this.threadClosed = true;
189
          this.queue.length = 0;
190
          this.trouble(
191
            "This thread is closed, so the rest of the session will not reach its transcript.",
192
          );
193
          break;
194
        }
195
196
        if (outcome === "refused") {
197
          // The server named the event invalid. Retrying cannot change that,
198
          // and holding the queue on it would silently stop the record.
199
          this.queue.shift();
200
          this.trouble(
201
            "The server refused a transcript event, which was dropped. The session continues.",
202
          );
203
          continue;
204
        }
205
206
        // Transient. Same event, next rung of the ladder.
207
        this.consecutiveFailures += 1;
208
        if (this.consecutiveFailures >= TROUBLE_AFTER) {
209
          this.trouble(
210
            "The thread transcript is not reaching the server. " +
211
              "Events are queued and will keep retrying in the background.",
212
          );
213
        }
214
215
        const rung = Math.min(this.consecutiveFailures - 1, this.delays.length - 1);
216
        // eslint-disable-next-line no-await-in-loop
217
        await sleep(this.delays[rung] ?? 0);
218
      }
219
    } finally {
220
      this.pumping = false;
221
      if (this.queue.length === 0 || this.threadClosed) {
222
        for (const waiter of this.drainWaiters.splice(0)) waiter();
223
      }
224
    }
225
  }
226
227
  /** One POST, translated to what the pump can act on. Never throws. */
228
  private async send(
229
    event: QueuedEvent,
230
  ): Promise<"posted" | "retry" | "refused" | "thread_closed"> {
231
    let response: Response;
232
    try {
233
      response = await this.post(
234
        new URL(`${THREADS_PATH}/${this.options.threadId}/events`, this.options.origin),
235
        {
236
          method: "POST",
237
          headers: {
238
            authorization: `Bearer ${this.options.token}`,
239
            "content-type": "application/json",
240
            accept: "application/json",
241
          },
242
          body: JSON.stringify({ event_type: event.eventType, payload: event.payload }),
243
        },
244
      );
245
    } catch {
246
      return "retry";
247
    }
248
249
    if (response.status >= 200 && response.status < 300) return "posted";
250
    // A server that is down answers 5xx; the event is still good.
251
    if (response.status >= 500) return "retry";
252
253
    const body = (await response.json().catch(() => ({}))) as Record<string, unknown>;
254
    if (body["code"] === "thread_terminal") return "thread_closed";
255
    return "refused";
256
  }
257
258
  /** One sentence, once. A status line repeating itself is a status line ignored. */
259
  private trouble(message: string): void {
260
    if (this.toldOfTrouble) return;
261
    this.toldOfTrouble = true;
262
    this.options.onTrouble?.(message);
263
  }
264
}
265
266
const sleep = (ms: number) =>
267
  new Promise<void>((resolve) => {
268
    // Always a real timer, even at zero: a retry loop that resumed on the
269
    // microtask queue would starve the event loop the session runs on. The
270
    // timer is unreferenced so a backoff mid-wait cannot hold the process
271
    // open after the session that was being recorded has ended.
272
    const timer = setTimeout(resolve, Math.max(0, ms));
273
    if (typeof timer === "object" && "unref" in timer) timer.unref();
274
  });
packages/openagents-cli/test/coder-thread.test.ts modified +190

@@ -2,6 +2,7 @@ import { afterEach, describe, expect, it, vi } from "vitest";

2 2
3 3
import type { ReplyChunk } from "../src/coder-session.js";
4 4
import { openThread, ThreadUnavailable, type ThreadReplySource } from "../src/coder-thread.js";
5
import { ThreadTranscriptWriter } from "../src/coder-transcript.js";
5 6
6 7
const ORIGIN = "https://openagents.test";
7 8
const ACCOUNT_TOKEN = "oa_pat_account";

@@ -401,3 +402,192 @@ describe("ThreadReplySource", () => {

401 402
    await expect((await open()).revoke()).resolves.toBeUndefined();
402 403
  });
403 404
});
405
406
/** A sink that just remembers, standing in for the writer. */
407
const recorder = () => {
408
  const events: Array<{ eventType: string; payload: Record<string, unknown> }> = [];
409
  return {
410
    events,
411
    record(eventType: string, payload: Record<string, unknown>) {
412
      events.push({ eventType, payload });
413
    },
414
  };
415
};
416
417
const withTool = (run: (args: Record<string, unknown>) => Promise<string>) => ({
418
  name: "shell",
419
  description: "run a command",
420
  parameters: { type: "object" },
421
  run,
422
});
423
424
describe("the thread's durable transcript", () => {
425
  const TOOL_ROUND = [
426
    `data: {"choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"id":"call-1","type":"function","function":{"name":"shell","arguments":"{\\"command\\":\\"ls\\"}"}}]}}]}`,
427
    `data: {"choices":[{"index":0,"delta":{},"finish_reason":"tool_calls"}]}`,
428
    `data: {"choices":[],"usage":{"completion_tokens":5,"prompt_tokens":40,"total_tokens":45}}`,
429
    `data: [DONE]`,
430
    "",
431
  ].join("\n\n");
432
433
  const ANSWER_ROUND = [
434
    `data: {"choices":[{"delta":{"content":"Two files."},"index":0}]}`,
435
    `data: {"choices":[],"usage":{"completion_tokens":11,"prompt_tokens":60,"total_tokens":71}}`,
436
    `data: [DONE]`,
437
    "",
438
  ].join("\n\n");
439
440
  it("records the turn in order: what was asked, each tool run, the answer", async () => {
441
    stub({ proxy: [sse([TOOL_ROUND]), sse([ANSWER_ROUND])] });
442
    const source = await open();
443
    const sink = recorder();
444
    source.useTranscript(sink);
445
    source.useTools([withTool(async () => "README.md\nsrc")]);
446
447
    await chunks(source, "what is in this repo?");
448
449
    expect(sink.events.map((event) => event.eventType)).toEqual([
450
      "turn.user",
451
      "tool.ran",
452
      "turn.assistant",
453
    ]);
454
    expect(sink.events[0]?.payload).toEqual({ text: "what is in this repo?" });
455
    expect(sink.events[1]?.payload).toEqual({
456
      call_id: "call-1",
457
      tool: "shell",
458
      arguments: `{"command":"ls"}`,
459
      status: "succeeded",
460
      output: "README.md\nsrc",
461
    });
462
  });
463
464
  it("records the answer with the turn's summed usage and its call count", async () => {
465
    stub({ proxy: [sse([TOOL_ROUND]), sse([ANSWER_ROUND])] });
466
    const source = await open();
467
    const sink = recorder();
468
    source.useTranscript(sink);
469
    source.useTools([withTool(async () => "README.md\nsrc")]);
470
471
    await chunks(source, "what is in this repo?");
472
473
    const answer = sink.events.find((event) => event.eventType === "turn.assistant");
474
    // The turn took two model calls; the record holds their sum with the
475
    // count, not the last call's figures presented as the turn's.
476
    expect(answer?.payload).toEqual({
477
      text: "Two files.",
478
      usage: { prompt_tokens: 100, completion_tokens: 16, total_tokens: 116, calls: 2 },
479
      tool_calls: 1,
480
    });
481
  });
482
483
  it("records a failed tool as one event carrying its error", async () => {
484
    stub({ proxy: [sse([TOOL_ROUND]), sse([ANSWER_ROUND])] });
485
    const source = await open();
486
    const sink = recorder();
487
    source.useTranscript(sink);
488
    source.useTools([
489
      withTool(async () => {
490
        throw new Error("permission denied");
491
      }),
492
    ]);
493
494
    await chunks(source, "try it");
495
496
    const ran = sink.events.find((event) => event.eventType === "tool.ran");
497
    expect(ran?.payload).toEqual({
498
      call_id: "call-1",
499
      tool: "shell",
500
      arguments: `{"command":"ls"}`,
501
      status: "failed",
502
      error: "permission denied",
503
    });
504
  });
505
506
  it("bounds a tool result on its way into the record", async () => {
507
    stub({ proxy: [sse([TOOL_ROUND]), sse([ANSWER_ROUND])] });
508
    const source = await open();
509
    const sink = recorder();
510
    source.useTranscript(sink);
511
    source.useTools([withTool(async () => "x".repeat(200_000))]);
512
513
    await chunks(source, "dump it");
514
515
    const ran = sink.events.find((event) => event.eventType === "tool.ran");
516
    const output = ran?.payload["output"] as string;
517
    // Kept at both ends around a marker, and far above the model-wire bound,
518
    // so every result a real session has produced is stored whole.
519
    expect(output.length).toBeLessThan(70_000);
520
    expect(output).toContain("characters omitted");
521
  });
522
523
  it("records a message steered into the running turn as the reader's", async () => {
524
    stub({ proxy: [sse([TOOL_ROUND]), sse([ANSWER_ROUND])] });
525
    const source = await open();
526
    const sink = recorder();
527
    source.useTranscript(sink);
528
    source.useTools([
529
      withTool(async () => {
530
        source.steer("only the top level");
531
        return "README.md\nsrc";
532
      }),
533
    ]);
534
535
    await chunks(source, "list the files");
536
537
    expect(sink.events.map((event) => event.eventType)).toEqual([
538
      "turn.user",
539
      "tool.ran",
540
      "turn.user",
541
      "turn.assistant",
542
    ]);
543
    expect(sink.events[2]?.payload).toEqual({ text: "only the top level", steered: true });
544
  });
545
546
  it("records nothing extra for a turn without tools", async () => {
547
    stub({});
548
    const source = await open();
549
    const sink = recorder();
550
    source.useTranscript(sink);
551
552
    await chunks(source, "hello");
553
554
    expect(sink.events.map((event) => event.eventType)).toEqual(["turn.user", "turn.assistant"]);
555
    expect(sink.events[1]?.payload).toMatchObject({ text: "Hello! Nice", tool_calls: 0 });
556
  });
557
558
  it("still answers when every transcript post fails", async () => {
559
    // The real writer against a server that refuses the events route: the
560
    // turn loop must neither throw nor wait on it.
561
    vi.stubGlobal(
562
      "fetch",
563
      vi.fn(async (target: URL | string, init?: RequestInit) => {
564
        const url = typeof target === "string" ? target : target.toString();
565
        if (url.endsWith("/events")) throw new Error("socket closed");
566
        if (url.endsWith("/api/inference/proxy")) return sse([LIVE_SSE]);
567
        if ((init?.method ?? "GET") === "POST") return json(201, CREATED);
568
        return json(200, { thread: {}, grant: {} });
569
      }),
570
    );
571
572
    const source = await openThread({
573
      origin: ORIGIN,
574
      token: ACCOUNT_TOKEN,
575
      objective: "coder in repo on main",
576
    });
577
    const notices: string[] = [];
578
    const writer = new ThreadTranscriptWriter({
579
      origin: ORIGIN,
580
      threadId: source.threadId,
581
      token: ACCOUNT_TOKEN,
582
      retryDelaysMs: [0, 0, 10],
583
      onTrouble: (message) => {
584
        notices.push(message);
585
      },
586
    });
587
    source.useTranscript(writer);
588
589
    expect(textOf(await chunks(source))).toBe("Hello! Nice");
590
    await writer.close(100);
591
    expect(notices).toHaveLength(1);
592
  });
593
});
packages/openagents-cli/test/coder-transcript.test.ts added +224

@@ -0,0 +1,224 @@

1
import { describe, expect, it, vi } from "vitest";
2
3
import { ThreadTranscriptWriter, type TranscriptTransport } from "../src/coder-transcript.js";
4
5
const ORIGIN = "https://openagents.test";
6
const TOKEN = "oa_pat_account";
7
const THREAD_ID = "9bb19447-ecf4-4f1b-b44e-6b128664da9c";
8
9
const json = (status: number, body: unknown) =>
10
  new Response(JSON.stringify(body), { status, headers: { "content-type": "application/json" } });
11
12
const posted = () => json(201, { thread: { id: THREAD_ID, event_count: 1 } });
13
14
interface Recorded {
15
  readonly url: string;
16
  readonly method: string;
17
  readonly authorization: string;
18
  readonly body: Record<string, unknown>;
19
}
20
21
/**
22
 * A transport that answers from a script, one response or failure per call,
23
 * and records what was sent. The last entry repeats, so a drain after the
24
 * interesting part needs no padding.
25
 */
26
const transport = (script: ReadonlyArray<Response | Error>) => {
27
  const calls: Recorded[] = [];
28
  let at = 0;
29
30
  const fetch = vi.fn(async (target: URL | string, init?: RequestInit) => {
31
    const headers = (init?.headers ?? {}) as Record<string, string>;
32
    calls.push({
33
      url: typeof target === "string" ? target : target.toString(),
34
      method: init?.method ?? "GET",
35
      authorization: headers["authorization"] ?? "",
36
      body: JSON.parse(typeof init?.body === "string" ? init.body : "{}") as Record<
37
        string,
38
        unknown
39
      >,
40
    });
41
    const answer = script[Math.min(at, script.length - 1)];
42
    at += 1;
43
    if (answer instanceof Error) throw answer;
44
    return (answer ?? posted()).clone();
45
  });
46
47
  return { calls, fetch };
48
};
49
50
const writer = (
51
  wire: { fetch: TranscriptTransport },
52
  extra?: Partial<ConstructorParameters<typeof ThreadTranscriptWriter>[0]>,
53
) =>
54
  new ThreadTranscriptWriter({
55
    origin: ORIGIN,
56
    threadId: THREAD_ID,
57
    token: TOKEN,
58
    fetch: wire.fetch,
59
    retryDelaysMs: [0, 0, 0],
60
    ...extra,
61
  });
62
63
describe("ThreadTranscriptWriter", () => {
64
  it("posts each event to the thread's transcript on the account token", async () => {
65
    const wire = transport([posted()]);
66
    const sink = writer(wire);
67
68
    sink.record("turn.user", { text: "list the open issues" });
69
    await sink.close();
70
71
    expect(wire.calls).toHaveLength(1);
72
    const call = wire.calls[0];
73
    expect(call?.method).toBe("POST");
74
    expect(call?.url).toBe(`${ORIGIN}/api/v3/threads/${THREAD_ID}/events`);
75
    expect(call?.authorization).toBe(`Bearer ${TOKEN}`);
76
    expect(call?.body).toEqual({
77
      event_type: "turn.user",
78
      payload: { text: "list the open issues" },
79
    });
80
  });
81
82
  it("posts events in the order they were recorded", async () => {
83
    const wire = transport([posted()]);
84
    const sink = writer(wire);
85
86
    sink.record("turn.user", { text: "hello" });
87
    sink.record("tool.ran", { tool: "shell", status: "succeeded" });
88
    sink.record("turn.assistant", { text: "done" });
89
    await sink.close();
90
91
    expect(wire.calls.map((call) => call.body["event_type"])).toEqual([
92
      "turn.user",
93
      "tool.ran",
94
      "turn.assistant",
95
    ]);
96
  });
97
98
  it("retries a refused connection and keeps the order across the retry", async () => {
99
    const wire = transport([new Error("socket closed"), posted()]);
100
    const sink = writer(wire);
101
102
    sink.record("turn.user", { text: "first" });
103
    sink.record("turn.assistant", { text: "second" });
104
    await sink.close();
105
106
    // The failed event is retried, not skipped: posting the next one first
107
    // would reorder the transcript.
108
    expect(wire.calls.map((call) => call.body["event_type"])).toEqual([
109
      "turn.user",
110
      "turn.user",
111
      "turn.assistant",
112
    ]);
113
    expect(sink.pending).toBe(0);
114
  });
115
116
  it("retries a server failure the same way", async () => {
117
    const wire = transport([json(502, {}), posted()]);
118
    const sink = writer(wire);
119
120
    sink.record("turn.user", { text: "hello" });
121
    await sink.close();
122
123
    expect(wire.calls).toHaveLength(2);
124
    expect(sink.pending).toBe(0);
125
  });
126
127
  it("never throws into the caller, whatever the transport does", async () => {
128
    const sink = writer({
129
      fetch: (() => {
130
        throw new Error("broken before the promise");
131
      }) as unknown as TranscriptTransport,
132
    });
133
134
    expect(() => {
135
      sink.record("turn.user", { text: "hello" });
136
    }).not.toThrow();
137
138
    // Stop the retry loop rather than leaving it running under later tests.
139
    await sink.close(20);
140
  });
141
142
  it("says so once when posting keeps failing, and keeps queueing", async () => {
143
    const notices: string[] = [];
144
    const wire = transport([new Error("down")]);
145
    const sink = writer(wire, {
146
      // A real rung after the notice, so the retry loop idles rather than
147
      // spins while the close deadline runs down.
148
      retryDelaysMs: [0, 0, 10],
149
      onTrouble: (message) => {
150
        notices.push(message);
151
      },
152
    });
153
154
    sink.record("turn.user", { text: "one" });
155
    sink.record("turn.user", { text: "two" });
156
    await sink.close(200);
157
158
    // Three consecutive failures are an outage worth one sentence; the fourth
159
    // and fifth are the same outage and get no second sentence.
160
    expect(notices).toHaveLength(1);
161
    expect(notices[0]).toContain("not reaching the server");
162
    // Nothing was dropped: both events are still queued for retry.
163
    expect(sink.pending).toBe(2);
164
  });
165
166
  it("drops an event the server called invalid and continues with the next", async () => {
167
    const notices: string[] = [];
168
    const wire = transport([json(422, { errors: { event_type: ["cannot be blank"] } }), posted()]);
169
    const sink = writer(wire, {
170
      onTrouble: (message) => {
171
        notices.push(message);
172
      },
173
    });
174
175
    sink.record("turn.user", { text: "refused" });
176
    sink.record("turn.assistant", { text: "still recorded" });
177
    await sink.close();
178
179
    // A payload the server has already called invalid will be invalid
180
    // tomorrow; retrying it would silently stop the whole record.
181
    expect(wire.calls.map((call) => call.body["event_type"])).toEqual([
182
      "turn.user",
183
      "turn.assistant",
184
    ]);
185
    expect(notices).toHaveLength(1);
186
    expect(sink.pending).toBe(0);
187
  });
188
189
  it("stops for good when the thread is terminal", async () => {
190
    const notices: string[] = [];
191
    const wire = transport([json(422, { code: "thread_terminal" })]);
192
    const sink = writer(wire, {
193
      onTrouble: (message) => {
194
        notices.push(message);
195
      },
196
    });
197
198
    sink.record("turn.user", { text: "one" });
199
    sink.record("turn.user", { text: "two" });
200
    await sink.close();
201
    sink.record("turn.user", { text: "three" });
202
    await sink.close();
203
204
    // A closed transcript can never take another event, so nothing is queued
205
    // against a refusal that cannot change.
206
    expect(wire.calls).toHaveLength(1);
207
    expect(notices).toHaveLength(1);
208
    expect(sink.pending).toBe(0);
209
  });
210
211
  it("gives up the flush at the deadline rather than holding the exit", async () => {
212
    const sink = writer({
213
      // A transport that never answers, which is what a dead server looks
214
      // like from a process that is trying to leave.
215
      fetch: (() => new Promise<Response>(() => undefined)) as TranscriptTransport,
216
    });
217
218
    sink.record("turn.user", { text: "unsendable" });
219
220
    const started = Date.now();
221
    await sink.close(50);
222
    expect(Date.now() - started).toBeLessThan(1_000);
223
  });
224
});

This page updates live while a promote is in flight · changelog