`openagents-cli-identity`, keyed by the identity directory; the key never

  enters the identity directory, so a copy of that directory is not an identity.

  The envelope (`openagents.cli_identity_seed.v1`), the AEAD, the keychain

  service, and the account key are one contract across

  `crates/openagents-cli/src/identity.rs` and

  `packages/openagents-cli/src/seed-identity.ts`: a format only one CLI

  understands makes the other a downgrade attack on it, because both read one

  file at one path.

  service, and the account key are one contract, owned by

  `packages/openagents-cli/src/seed-identity.ts`. It was written as a contract

  across two CLIs because both read one file at one path, so a format only one

  of them understood would be a downgrade attack on the other. The Rust CLI's

  identity surface was removed on 2026-08-26 in `02cdaea275` — nothing consumed

  the identity it derived — which leaves one reader. Any second reader of that

  path re-enters the contract exactly as written above.

- Where no keychain exists — CI, a container, an unattended agent host — the

  phrase is stored as plaintext `0600` and every surface that shows an identity

  must say so. `identity show`, `create`, `import`, and `backup` carry the

- A plaintext seed written before this was migrated on the next `identity`

  command by renaming the sealed envelope over the same path, so the phrase is

  never in two places at once. Coverage is

  `crates/openagents-cli/tests/identity_test.rs` and

  `packages/openagents-cli/test/seed-identity.test.ts`, which assert the bytes on

  disk carry no word of the phrase, and that migration preserves the `npub`

  while removing the plaintext.

  `packages/openagents-cli/test/seed-identity.test.ts`, which asserts the bytes

  on disk carry no word of the phrase, and that migration preserves the `npub`

  while removing the plaintext. Its Rust counterpart went with the surface it

  covered in `02cdaea275`.

- The wallet receives; it does not spend. The spending rail is an owner decision

  that is not recorded, so no CLI surface may imply a spend path exists until it

  is.

  "note": "Heuristic false-green LEADS, not findings. A finding requires a demonstrated reproduction (surviving mutation via mutation-runner). Do not treat a candidate as a confirmed false green. Coverage-theater leads may include tests that delegate their assertion to a custom helper the classifier does not recognise; verify before acting.",

  "sourceDigest": "sha256:dd810dd48c5bdbc9becd7fcc01dd41a4ca2abf0b2d6f6a545907247f6e3e8361",

  "summary": {

    "filesScanned": 2505,

    "filesScanned": 2507,

    "candidateCount": 16,

    "byMode": {

      "false_green_coverage_theater": 15,

{

  "schemaVersion": "1",

  "repository": "OpenAgentsInc/openagents",

  "sourceDigest": "sha256:931a55aba01fd586c019996b1f02bf4d383179c895bc0053a5ed770bac5deb7b",

  "sourceDigest": "sha256:70b6d652076b37c10109b54de527c61c1e549afad5b1e2664207f8432292ebed",

  "surfaces": [

    {

      "id": "app:@openagentsinc/acceptance-runner",

      "oracles": [

        {

          "type": "test",

          "ref": "packages/openagents-cli (91 tracked test files)"

          "ref": "packages/openagents-cli (93 tracked test files)"

        },

        {

          "type": "behavior-contract",

  lineage: {

    origin:      "review" | "optimizer" | "human"

    parent:      string | null            // the candidateId this came from

    producedBy:  string                   // e.g. "coder review-run:<reviewId>"

    producedBy:  string                   // e.g. "coder-review:<jobDir>:<reviewer ref>"

  }

  transferLabel: { modelFamily: string, lane: string }

  evidence:      [{ ref: string, note: string }]

`surfaces[].surface` names a staged artifact from `surfaces/coder/index.json`.

The whole current vocabulary:

| `surface` | Artifact | What it holds |

| --- | --- | --- |

| `system-prompt` | `surfaces/coder/system-prompt.v1.json` | The instructions, the concision sentence, the no-tools and tool-list sentences, and the lane notices, for all three harnesses |

| `tool-descriptions` | `surfaces/coder/tool-descriptions.v1.json` | The description of each declared tool, plus the per-model-family emphasis overrides |

| `catalog-lines` | `surfaces/coder/catalog-lines.v1.json` | Each installed plugin's catalog line, keyed by plugin id |

| `surface`           | Artifact                                   | What it holds                                                                                                                 |

| ------------------- | ------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------- |

| `system-prompt`     | `surfaces/coder/system-prompt.v1.json`     | The instructions, the concision sentence, the no-tools and tool-list sentences, and the lane notices, for all three harnesses |

| `tool-descriptions` | `surfaces/coder/tool-descriptions.v1.json` | The description of each declared tool, plus the per-model-family emphasis overrides                                           |

| `catalog-lines`     | `surfaces/coder/catalog-lines.v1.json`     | Each installed plugin's catalog line, keyed by plugin id                                                                      |

`diff` is a unified diff over that artifact file where one exists, and the

proposed text otherwise. The artifact's `text` map is flat and keyed, so a

# Cycle reviews

One file pair per reviewed autoimprovement cycle, written by

`pnpm run coder:review` (runbook §6):

- `YYYY-MM-DD-<lever-slug>.md` — the review a human reads and adopts from.

- `YYYY-MM-DD-<lever-slug>.json` — the same review as data

  (`openagents.coder_review_document.v1`), so the adopt step can become a

  diff rather than a reading exercise. Its proposals are

  `openagents.coder_candidate.v1` candidates: the same type an optimizer

  mutation will carry, so a reflection and a mutation are one object.

Over time this directory is a dataset of which process changes actually

helped, which is the reason each file keeps its evidence refs rather than

only its conclusion.

**A refused review leaves nothing here.** Every claim in a review cites the

artifacts the reviewer was given — `trial:<task>#step-<id>`,

`trial:<task>#outcome`, `row:<suite>#<recordedAt>`, `ledger:<id>`,

`diff:<path>` — and a citation that does not resolve refuses the whole

review by name, before any file is written. A file in this directory is a

record of a judgment about a run that happened; a document that reads like

one and is not is the failure the loop exists to catch.

A review whose reviewer ref begins `replay:` was replayed from a recorded

response rather than produced fresh. Read it as a record of the replay.

  ```

  `pnpm pack`, never `npm pack` (best practice R1).

- **Ollama** with the local-lane model pulled, for local-lane runs. Local

  runs need no token; keep `--n-concurrent 1` because the model owns the

  cores.

   axis: rounds, prompt tokens, and wall clock per accepted outcome are

   where process levers show first (the fix-git analysis is the worked

   example).

5. **Decide.** Improved or neutral-but-simpler: keep. Worse: revert the

   lever, keep the recorded row (best practice M3), and write the refutation

   into the review. Timeout-shaped failures are not efficiency signal —

- the two store rows (before and after),

- the current `docs/coder/best-practices.md`.

Review prompt skeleton:

Assembling those by hand is what `coder:review` does mechanically

(OpenAgentsInc/openagents#121):

```

You are reviewing one autoimprovement cycle of `openagents coder`.

<lever>{what changed, and the predicted delta}</lever>

<baseline-row>{jsonl row}</baseline-row>

<result-row>{jsonl row}</result-row>

<trials>{per-trial: instruction, verifier decision, ATIF metrics,

notable transcript spans}</trials>

<diff>{the lever's diff}</diff>

<practices>{docs/coder/best-practices.md}</practices>

Score the cycle 0–10 with specific evidence for each point gained or

lost. Answer: did the lever cause the delta, or does a confounder

explain it? Were any ledger practices violated (cite the entry and the

transcript step)? Propose one to three changes, each typed as

{lever, evidence: trajectory steps, risk, verification: suite and

expected delta direction}. Finally, list ledger entries to add, promote,

demote, or refute, with provenance.

```sh

pnpm run coder:review -- /tmp/gym-jobs/<job>/<run> \

  --suite tb2-quick --lane proxy --lever HEAD~1 \

  --slug <lever-slug> \

  --reviewer-model <a model from a different family than the cycle ran on>

```

The review must cite trajectory steps for every claim; a proposal without

an evidence pointer is rejected at the adopt step. Save the output to

`docs/coder/reviews/YYYY-MM-DD-<lever-slug>.md`.

It reads the job directory, redacts everything through the one ATIF rule

list before it leaves the process, renders the prompt with the citable

evidence refs printed in it, asks the reviewer, checks every citation, and

writes `docs/coder/reviews/YYYY-MM-DD-<lever-slug>.md` plus the

machine-readable review beside it. Name the lever with `--lever <ref>`,

`--diff <file>`, or `--no-diff` for a baseline: a review that cannot see the

change cannot attribute the delta to it. The reviewer is reached over

`/api/v1/responses` at `--api-url` (default `$OPENAGENTS_CODER_API_URL`,

else `http://localhost:4000`) with `$OPENAGENTS_TOKEN`.

**Every claim carries refs, and the refs are checked.** The grammar is

`trial:<task>#step-<id>`, `trial:<task>#outcome`, `row:<suite>#<recordedAt>`,

`ledger:<id>`, and `diff:<path>`, resolved against the artifacts the

reviewer actually read — a truncated trajectory's dropped steps are not

citable, and the prompt says so. Every proposal must cite at least one

trajectory step: a store row or a ledger entry says what changed, never

what the coder did. A review with an unresolvable ref is refused whole,

by name, and nothing is written.

Exit codes: `0` accepted and written, `1` the reviewer answered and the

answer was refused (the named reasons are on stderr), `2` the review could

not be run at all — no job directory, no lever, no reviewer.

Two flags matter for the lanes below the live one:

- `--print-prompt` renders the prompt and exits without asking anyone. This

  is the manual §6 lane: paste it into a fresh agent context and save the

  answer.

- `--offline <file>` **replays** a recorded reviewer response instead of

  asking a model. It replays; it never generates. Its ref says `replay:` in

  the review file, so a replayed review cannot be read as a fresh judgment.

  There is no third behavior — a canned score would be the exact failure

  this command exists to prevent.

## 7. Record, adopt, land

   `docs/coder/best-practices.md` (checking new entries against existing

   ones for contradiction), carry rejected proposals into the review file

   with a one-line reason.

2. **Commit** the cycle as one unit: the lever, the review file, the ledger

   change, and the appended store rows. The commit message states the lever

   and the measured delta with its suite — a number, not an adjective

   (best practice V3).

2. **Commit** the cycle as one unit: the lever, the review file and the JSON

   written beside it, the ledger change, and the appended store rows. The

   commit message states the lever and the measured delta with its suite — a

   number, not an adjective (best practice V3).

3. **Push to the forge** (`git push openagents HEAD:main`), reconcile the

   canonical checkout, remove the worktree.

  (OpenAgentsInc/openagents.com#220); until then, metered-lane dollar

  figures are ceilings, and lane comparisons lean on success rate and

  rounds.

- The PTY-driven interactive harness (autoimprove §7.4) does not exist

  yet; best practice V2 is enforced by rule, not by gate.

- The optimizer lane (autoimprove §2.4) does not exist yet. The text

  surfaces it would mutate are still string literals in the two CLIs

  rather than staged artifacts; until that lands, every cycle is a

  hand-written lever and the ledger's O-series applies to nothing running.

- The PTY-driven interactive harness (autoimprove §7.4) exists for

  coder-lite as `cargo test -p coder-lite --test interactive_pty` (§0). The

  completion gate does not run it (#124), so best practice V2 is enforced by

  rule for every surface the gate does reach, and by that harness for the

  one it does not.

- The automated review (autoimprove §7.5) exists as `pnpm run coder:review`

  (#121). Its candidate schema, `openagents.coder_candidate.v1` in

  `packages/openagents-cli/src/coder-review-candidate.ts`, is the object

  #122 staged surfaces for and #123's optimizer will mutate; a review

  proposal and an optimizer mutation are the same type, documented in

  `docs/coder/candidate-format.md`. No review has been recorded through it

  yet, so `docs/coder/reviews/` holds only its README.

- The optimizer lane (autoimprove §2.4) does not exist yet, but the text it

  would mutate is now staged (#122): `surfaces/coder/`, pinned by digest and

  guarded by `check:coder-surfaces`. Until #123 lands, every cycle is still a

  hand-written lever, and the ledger's O-series applies to nothing running.

    "check:assure-repo-audit": "node --import tsx packages/assure-repo/src/cli.ts audit-check",

    "effectiveness:compare": "node --import tsx packages/coder-effectiveness/src/compare-cli.ts",

    "effectiveness:report": "node --import tsx packages/coder-effectiveness/src/cli.ts",

    "coder:review": "node --import tsx packages/openagents-cli/src/coder-review-cli.ts",

    "effectiveness:suites": "node bench/build-suites.mjs",

    "test:coder-effectiveness": "vp test --run packages/coder-effectiveness/src",

    "test:product-spec": "vp test --run packages/product-spec",

  }

  const categories = Object.keys(counts);

  const hard = new Set(ENGRAM_HARD_UNSAFE_CATEGORIES as unknown as ReadonlyArray<string>);

  // `new Set(...)` on an `as const` tuple infers a set of the literal union,

  // whose `has` then refuses the `string` keys it is being asked about. Widening

  // the element type at construction says the same thing without a cast, which

  // AFS authority code may not use to recover type safety.

  const hard = new Set<string>(ENGRAM_HARD_UNSAFE_CATEGORIES);

  const storable = categories.every((category) => !hard.has(category));

  const total = Object.values(counts).reduce((a, b) => a + b, 0);

 */

export const verifySupersessionChain = (events: ReadonlyArray<EngramEvent>): boolean => {

  for (let i = 0; i < events.length; i += 1) {

    const event = events[i];

    const event = events[i]!;

    if (!verifyEngramEventId(event)) {

      return false;

    }

    assertDigest(input.capabilityDigest, "capability digest");

    assertDigest(input.budgetSnapshotDigest, "budget snapshot digest");

    input.capabilityRefs.forEach((ref) => assertRef(ref, "capability ref"));

    // `budgetLimits` is an open record, so its `outputBytes` may be absent.

    // An absent output bound is already rejected by the safe-integer test; it is

    // named here so the type says what the test was always enforcing.

    const outputBytes: number | undefined = input.budgetLimits["outputBytes"];

    if (

      !input.workingDirectory.startsWith("/") ||

      input.workingDirectory.includes("\0") ||

      !Number.isSafeInteger(input.budgetLimits.outputBytes) ||

      input.budgetLimits.outputBytes < 0 ||

      outputBytes === undefined ||

      !Number.isSafeInteger(outputBytes) ||

      outputBytes < 0 ||

      Object.values(input.budgetLimits).some((value) => !Number.isSafeInteger(value) || value < 0)

    )

      throw new CloudComputerCommandStoreError("invalid", "request authority is invalid");

/**

 * Assemble one cycle review's input from the artifacts a Harbor run left.

 *

 * The runbook's §6 prompt takes five things: the trials, the lever's diff, the

 * store rows before and after, and the current ledger. An agent assembling

 * those by hand is the step OpenAgentsInc/openagents#121 replaces, and doing it

 * mechanically buys three properties a copy-and-paste cannot have.

 *

 * BOUNDED, AND NAMED WHERE IT IS BOUND. A trajectory is as long as the task

 * was hard, and the transcript of a fifteen-round trial does not fit in a

 * review prompt. Every cut this file makes is reported in the request itself,

 * in the `read-conversation` plugin's vocabulary: `tail_only`,

 * `dropped_leading_steps`, `kept_steps`, `total_steps`. A reviewer that cannot

 * see the opening of a trial is told so, and a proposal that cites a dropped

 * step is refused by name rather than resolving against a step the reviewer

 * never read.

 *

 * REDACTED WITH THE ONE RULE LIST. Everything assembled here is on its way out

 * of the working session and into another conversation, so it goes through

 * `redactForExternalInference` from the vendored ATIF rules — the same list

 * `openagents trace redact` folds in. #97 is the reason that is stated rather

 * than assumed: a second hand-written rule list forgot `oa_pat_` and `smct_`,

 * and the command reported success over a file of live tokens. There is no

 * second list here.

 *

 * A RESOLVABLE EVIDENCE INDEX. The request knows exactly which steps, rows,

 * ledger entries, and diff paths it contains, and hands that set to the parser

 * as {@link EvidenceIndex}. That is what makes "this proposal cites nothing"

 * a mechanical finding instead of a reviewer's opinion.

 */

import { existsSync, readdirSync, readFileSync, statSync } from "node:fs";

import { basename, join } from "node:path";

import type { EvidenceIndex } from "./coder-review-candidate.js";

import { redactForExternalInference } from "./memory/redaction.js";

export const CODER_REVIEW_REQUEST_SCHEMA = "openagents.coder_review_request.v1";

/** The coder's `--plain` thread announcement, the contract `bench` parses. */

const THREAD_LINE = /\[oa:thread ([0-9a-fA-F-]{36})\]/u;

/** Matches `### T1. Batch independent commands ...` in the ledger. */

const LEDGER_HEADING = /^###\s+([A-Z]+\d+)\.\s/gmu;

/** Matches the paths in a unified diff's `diff --git a/x b/x` lines. */

const DIFF_PATH = /^diff --git a\/(\S+) b\/(\S+)$/gmu;

/** What a verifier decided. Same three buckets `harbor-job.ts` grades into. */

export type ReviewTrialOutcome = "accepted" | "rejected" | "ungraded";

export interface ReviewToolCall {

  readonly name: string;

  readonly arguments: string;

  readonly argumentsTruncated: boolean;

  /**

   * What the tool returned, bounded.

   *

   * A trajectory step carries the call and its observation together, and a

   * review that can see the command but not its output cannot tell a wasted

   * round from a productive one — which is most of what runbook §6 asks it to

   * judge.

   */

  readonly observation: string;

  readonly observationTruncated: boolean;

}

export interface ReviewStep {

  /** The step id as the trajectory spells it, so a ref can name it back. */

  readonly stepId: string;

  readonly source: string;

  readonly text: string;

  readonly textTruncated: boolean;

  readonly toolCalls: ReadonlyArray<ReviewToolCall>;

  readonly promptTokens: number | null;

  readonly completionTokens: number | null;

  readonly cachedInputTokens: number | null;

}

/**

 * How much of a trial's trajectory reached the reviewer.

 *

 * Named in the `read-conversation` plugin's spelling on purpose: a truncated

 * read that says what it left out is a different artifact from one that does

 * not, and having two vocabularies for it would make a reader check which is

 * which.

 */

export interface ReviewTruncation {

  readonly tail_only: boolean;

  readonly dropped_leading_steps: number;

  readonly kept_steps: number;

  readonly total_steps: number;

  readonly text_truncated_steps: number;

  readonly max_step_chars: number;

}

export interface ReviewTrial {

  readonly task: string;

  /** The trial directory's own name. Never its path. */

  readonly trialDir: string;

  readonly instruction: string | null;

  /** Where the instruction was read from, or `absent` when nothing carried it. */

  readonly instructionSource:

    | "trial_result"

    | "trial_config"

    | "trajectory_first_user_step"

    | "absent";

  readonly outcome: ReviewTrialOutcome;

  readonly modelId: string | null;

  readonly agentVersion: string | null;

  readonly promptTokens: number | null;

  readonly completionTokens: number | null;

  readonly cachedInputTokens: number;

  readonly toolCalls: number | null;

  readonly wallClockSeconds: number | null;

  readonly threadId: string | null;

  readonly exception: string | null;

  readonly steps: ReadonlyArray<ReviewStep>;

  readonly truncation: ReviewTruncation;

}

export interface ReviewLever {

  /** What the diff was taken against, e.g. `HEAD` or a commit ref. */

  readonly ref: string;

  readonly diff: string;

  readonly paths: ReadonlyArray<string>;

  readonly truncation: {

    readonly tail_only: false;

    readonly kept_lines: number;

    readonly total_lines: number;

    readonly dropped_trailing_lines: number;

  };

}

export interface ReviewBenchRow {

  readonly suite: string;

  readonly recordedAt: string;

  /** The row's own fields, as the store wrote them. */

  readonly row: Record<string, unknown>;

}

export interface ReviewPractices {

  readonly path: string;

  readonly text: string;

  readonly entryIds: ReadonlyArray<string>;

}

export interface ReviewRedactionStamp {

  readonly serviceRef: string;

  readonly surface: "trace_capture";

  readonly appliedBeforeExternalInference: true;

  readonly counts: Readonly<Record<string, number>>;

  readonly total: number;

}

/** Everything the reviewer is given, and nothing the working session holds. */

export interface ReviewRequest {

  readonly schema: typeof CODER_REVIEW_REQUEST_SCHEMA;

  /** The job directory's own name. Never its path: a path is a local fact. */

  readonly jobDir: string;

  readonly jobId: string | null;

  readonly suite: string;

  readonly lane: string;

  readonly trials: ReadonlyArray<ReviewTrial>;

  readonly lever: ReviewLever;

  readonly rows: ReadonlyArray<ReviewBenchRow>;

  readonly practices: ReviewPractices;

  readonly redaction: ReviewRedactionStamp;

}

export interface AssembleOptions {

  readonly suite: string;

  readonly lane: string;

  readonly lever: { readonly ref: string; readonly diff: string };

  readonly rows: ReadonlyArray<ReviewBenchRow>;

  readonly practices: { readonly path: string; readonly text: string };

  /** Steps kept per trial. The tail is kept: a trial ends where it failed. */

  readonly maxSteps: number;

  readonly maxStepChars: number;

  readonly maxDiffLines: number;

}

export const DEFAULT_MAX_STEPS = 40;

export const DEFAULT_MAX_STEP_CHARS = 1200;

export const DEFAULT_MAX_DIFF_LINES = 800;

export class NotAHarborJob extends Error {

  constructor(jobDir: string, reason: string) {

    super(

      `not a Harbor job directory (${reason}): ${jobDir}. Point at the directory harbor run created under its --jobs-dir.`,

    );

    this.name = "NotAHarborJob";

  }

}

/**

 * Read a completed Harbor job into a bounded, redacted review request.

 *

 * Throws {@link NotAHarborJob} rather than reviewing an empty directory: a

 * review of nothing would still produce a score, and a score with no run

 * behind it is the shape of claim this loop exists to stop.

 */

export const assembleReviewRequest = (

  jobDir: string,

  options: AssembleOptions,

): { readonly request: ReviewRequest; readonly index: EvidenceIndex } => {

  const jobResult = readJson(join(jobDir, "result.json"));

  if (jobResult === undefined) throw new NotAHarborJob(jobDir, "no result.json");

  const trials: Array<ReviewTrial> = [];

  for (const entry of readdirSync(jobDir).sort()) {

    const trialDir = join(jobDir, entry);

    if (!statSync(trialDir).isDirectory()) continue;

    const trialResult = readJson(join(trialDir, "result.json"));

    if (trialResult === undefined) continue;

    trials.push(readTrial(entry, trialDir, trialResult, options));

  }

  // A trial directory has a `result.json` too, so pointing at one would

  // otherwise assemble a job of zero trials — and a review of zero trials still

  // produces a score. Refusing here is the same rule the store applies to a run

  // that skipped its pinned tasks: it is not a smaller measurement, it is none.

  if (trials.length === 0) {

    throw new NotAHarborJob(jobDir, "it holds no trial directories");

  }

  const diffLines = options.lever.diff === "" ? [] : options.lever.diff.split("\n");

  const keptDiffLines = diffLines.slice(0, options.maxDiffLines);

  const lever: ReviewLever = {

    ref: options.lever.ref,

    diff: keptDiffLines.join("\n"),

    paths: diffPathsOf(options.lever.diff),

    truncation: {

      tail_only: false,

      kept_lines: keptDiffLines.length,

      total_lines: diffLines.length,

      dropped_trailing_lines: diffLines.length - keptDiffLines.length,

    },

  };

  const practices: ReviewPractices = {

    path: options.practices.path,

    text: options.practices.text,

    entryIds: ledgerEntryIds(options.practices.text),

  };

  const draft = {

    schema: CODER_REVIEW_REQUEST_SCHEMA,

    jobDir: basename(jobDir),

    jobId: readString(readField(jobResult, "id")),

    suite: options.suite,

    lane: options.lane,

    trials,

    lever,

    rows: options.rows,

    practices,

  } as const;

  // Everything above is on its way into another conversation, so the whole

  // request goes through the one rule list before it leaves this process. The

  // stamp is added after, so the report counts what the reviewer will not see.

  const redacted = redactForExternalInference(draft, { surface: "trace_capture" });

  const request: ReviewRequest = {

    ...redacted.value,

    redaction: {

      serviceRef: redacted.policy.serviceRef,

      surface: "trace_capture",

      appliedBeforeExternalInference: true,

      counts: redacted.report.counts,

      total: redacted.report.total,

    },

  };

  return { request, index: indexOf(request) };

};

/** The set of refs a proposal may cite, built from the request itself. */

export const indexOf = (request: ReviewRequest): EvidenceIndex => {

  const trajectorySteps = new Set<string>();

  const trialOutcomes = new Set<string>();

  for (const trial of request.trials) {

    trialOutcomes.add(trial.task);

    for (const step of trial.steps) {

      trajectorySteps.add(`${trial.task}#step-${step.stepId}`);

    }

  }

  return {

    trajectorySteps,

    trialOutcomes,

    benchRows: new Set(request.rows.map((row) => `${row.suite}#${row.recordedAt}`)),

    ledgerEntries: new Set(request.practices.entryIds),

    diffPaths: new Set(request.lever.paths),

  };

};

const readTrial = (

  dirName: string,

  trialDir: string,

  trialResult: unknown,

  options: AssembleOptions,

): ReviewTrial => {

  const trajectory = readJson(join(trialDir, "agent", "trajectory.json"));

  const rawSteps = readArray(readField(trajectory, "steps"));

  const agent = readField(trajectory, "agent");

  const finalMetrics = readField(trajectory, "final_metrics");

  let cachedInputTokens = 0;

  let toolCalls = 0;

  for (const step of rawSteps) {

    const extra = readField(readField(step, "metrics"), "extra");

    cachedInputTokens += readNumber(readField(extra, "cache_read_input_tokens")) ?? 0;

    toolCalls += readArray(readField(step, "tool_calls")).length;

  }

  // The tail is what is kept. A trial that failed failed at its end, and the

  // opening rounds of a long transcript are the least informative part of it.

  const dropped = Math.max(0, rawSteps.length - options.maxSteps);

  const keptRaw = rawSteps.slice(dropped);

  let textTruncatedSteps = 0;

  const steps = keptRaw.map((step, position) => {

    const built = readStep(step, dropped + position, options.maxStepChars);

    if (built.textTruncated) textTruncatedSteps += 1;

    return built;

  });

  const instruction = readInstruction(trialDir, trialResult, rawSteps);

  return {

    // The trial spells its own task name. The directory name is the fallback

    // for a tree that did not, and it carries a random per-trial suffix.

    task:

      readString(readField(trialResult, "task_name")) ??

      (dirName.includes("__") ? dirName.slice(0, dirName.lastIndexOf("__")) : dirName),

    trialDir: dirName,

    instruction: instruction.text,

    instructionSource: instruction.source,

    outcome: outcomeOf(trialResult),

    modelId:

      readString(readField(agent, "model_name")) ??

      modelFromTrialConfig(readJson(join(trialDir, "config.json"))),

    agentVersion: readString(readField(agent, "version")),

    promptTokens: readNumber(readField(finalMetrics, "total_prompt_tokens")),

    completionTokens: readNumber(readField(finalMetrics, "total_completion_tokens")),

    cachedInputTokens,

    toolCalls: rawSteps.length === 0 ? null : toolCalls,

    wallClockSeconds: wallClockOf(trialResult),

    threadId: threadIdOf(trialDir),

    exception: readString(readField(readField(trialResult, "exception_info"), "exception_type")),

    steps,

    truncation: {

      tail_only: dropped > 0,

      dropped_leading_steps: dropped,

      kept_steps: steps.length,

      total_steps: rawSteps.length,

      text_truncated_steps: textTruncatedSteps,

      max_step_chars: options.maxStepChars,

    },

  };

};

const readStep = (step: unknown, position: number, maxChars: number): ReviewStep => {

  const rawId = readField(step, "step_id");

  const stepId =

    typeof rawId === "number" || typeof rawId === "string" ? String(rawId) : String(position + 1);

  const message = readString(readField(step, "message")) ?? "";

  const clipped = clip(message, maxChars);

  const metrics = readField(step, "metrics");

  return {

    stepId,

    source: readString(readField(step, "source")) ?? "unknown",

    text: clipped.text,

    textTruncated: clipped.truncated,

    toolCalls: readArray(readField(step, "tool_calls")).map((call) => {

      // The coder's ATIF exporter writes `function_name` and an `arguments`

      // object; the OpenAI-shaped spellings are read too, so a trajectory from

      // another adapter is not silently reported as a step of `unknown` calls.

      const args =

        stringify(readField(call, "arguments")) ?? stringify(readField(call, "input")) ?? "";

      const clippedArgs = clip(args, maxChars);

      const callId =

        readString(readField(call, "tool_call_id")) ?? readString(readField(call, "call_id"));

      const clippedObservation = clip(observationFor(step, callId), maxChars);

      return {

        name:

          readString(readField(call, "function_name")) ??

          readString(readField(call, "name")) ??

          readString(readField(readField(call, "function"), "name")) ??

          "unknown",

        arguments: clippedArgs.text,

        argumentsTruncated: clippedArgs.truncated,

        observation: clippedObservation.text,

        observationTruncated: clippedObservation.truncated,

      };

    }),

    promptTokens: readNumber(readField(metrics, "prompt_tokens")),

    completionTokens: readNumber(readField(metrics, "completion_tokens")),

    cachedInputTokens: readNumber(

      readField(readField(metrics, "extra"), "cache_read_input_tokens"),

    ),

  };

};

/**

 * The output of one call, out of the step's observation block.

 *

 * ATIF puts every result of a step in one `observation.results` array keyed by

 * `source_call_id`, so a step with two calls carries two results and matching

 * them by id is what keeps a command next to its own output. A step with one

 * result and no id still matches its single call: dropping the output because

 * the exporter omitted an id would lose the more useful half of the step.

 */

const observationFor = (step: unknown, callId: string | null): string => {

  const results = readArray(readField(readField(step, "observation"), "results"));

  if (results.length === 0) return "";

  const matched =

    callId === null

      ? undefined

      : results.find((result) => readString(readField(result, "source_call_id")) === callId);

  const chosen = matched ?? (results.length === 1 ? results[0] : undefined);

  if (chosen === undefined) return "";

  return stringify(readField(chosen, "content")) ?? "";

};

/**

 * The task instruction, from whichever artifact carried it.

 *

 * Harbor spells it differently across its own files, and a trial that timed

 * out before the coder started may carry it in only one of them. The source is

 * recorded next to the text so a reviewer reading a suspiciously short

 * instruction can tell a truncated task statement from a recovered one.

 */

const readInstruction = (

  trialDir: string,

  trialResult: unknown,

  steps: ReadonlyArray<unknown>,

): { readonly text: string | null; readonly source: ReviewTrial["instructionSource"] } => {

  const fromResult =

    readString(readField(trialResult, "instruction")) ??

    readString(readField(readField(trialResult, "task"), "instruction"));

  if (fromResult !== null) return { text: fromResult, source: "trial_result" };

  const config = readJson(join(trialDir, "config.json"));

  const fromConfig =

    readString(readField(config, "instruction")) ??

    readString(readField(readField(config, "task"), "instruction"));

  if (fromConfig !== null) return { text: fromConfig, source: "trial_config" };

  for (const step of steps) {

    if (readString(readField(step, "source")) !== "user") continue;

    const message = readString(readField(step, "message"));

    if (message !== null) return { text: message, source: "trajectory_first_user_step" };

  }

  return { text: null, source: "absent" };

};

/**

 * The grading rule, held identical to `harbor-job.ts`.

 *

 * A verifier that never ran leaves no `verifier_result`, and that is

 * `ungraded` — not a failure and not a pass. Two readers of the same run

 * disagreeing about what a pass is would make the review and the score

 * describe different runs.

 */

const outcomeOf = (trialResult: unknown): ReviewTrialOutcome => {

  const verifier = readField(trialResult, "verifier_result");

  if (verifier === undefined || verifier === null) return "ungraded";

  const rewards = readField(verifier, "rewards") ?? readField(trialResult, "rewards");

  const reward = rewardValue(rewards);

  return reward !== null && reward > 0 ? "accepted" : "rejected";

};

const rewardValue = (rewards: unknown): number | null => {

  const direct = readNumber(readField(rewards, "reward"));

  if (direct !== null) return direct;

  if (typeof rewards !== "object" || rewards === null) return null;

  const values = Object.values(rewards as Record<string, unknown>);

  return values.length === 1 ? readNumber(values[0]) : null;

};

const wallClockOf = (trialResult: unknown): number | null => {

  const execution = readField(trialResult, "agent_execution");

  const started = readString(readField(execution, "started_at"));

  const finished = readString(readField(execution, "finished_at"));

  if (started === null || finished === null) return null;

  const span = Date.parse(finished) - Date.parse(started);

  return Number.isFinite(span) ? span / 1000 : null;

};

const modelFromTrialConfig = (trialConfig: unknown): string | null => {

  const spelled =

    readString(readField(readField(trialConfig, "agent"), "model_name")) ??

    readString(readField(readField(readField(trialConfig, "config"), "agent"), "model_name"));

  if (spelled === null) return null;

  const separator = spelled.indexOf("/");

  if (separator === -1) return spelled;

  const name = spelled.slice(separator + 1);

  return name === "" ? spelled.slice(0, separator) : name;

};

const threadIdOf = (trialDir: string): string | null => {

  const path = join(trialDir, "agent", "coder.txt");

  if (!existsSync(path)) return null;

  return THREAD_LINE.exec(readFileSync(path, "utf8"))?.[1] ?? null;

};

/** Ledger entry ids, so `ledger:T1` can be resolved instead of trusted. */

export const ledgerEntryIds = (practices: string): ReadonlyArray<string> => {

  const ids: Array<string> = [];

  LEDGER_HEADING.lastIndex = 0;

  let match = LEDGER_HEADING.exec(practices);

  while (match !== null) {

    if (match[1] !== undefined) ids.push(match[1]);

    match = LEDGER_HEADING.exec(practices);

  }

  return ids;

};

/** Paths a unified diff touches, so `diff:<path>` can be resolved. */

export const diffPathsOf = (diff: string): ReadonlyArray<string> => {

  const paths = new Set<string>();

  DIFF_PATH.lastIndex = 0;

  let match = DIFF_PATH.exec(diff);

  while (match !== null) {

    if (match[1] !== undefined) paths.add(match[1]);

    if (match[2] !== undefined) paths.add(match[2]);

    match = DIFF_PATH.exec(diff);

  }

  return [...paths].sort();

};

/** Read the rows of one `bench-results` store file, newest last. */

export const readBenchRows = (

  storePath: string,

  suite: string,

  limit: number,

): ReadonlyArray<ReviewBenchRow> => {

  if (!existsSync(storePath)) return [];

  const rows: Array<ReviewBenchRow> = [];

  for (const line of readFileSync(storePath, "utf8").split("\n")) {

    if (line.trim() === "") continue;

    let parsed: unknown;

    try {

      parsed = JSON.parse(line) as unknown;

    } catch {

      continue;

    }

    const row = readRecord(parsed);

    if (row === undefined) continue;

    rows.push({

      suite: readString(row["suite"]) ?? suite,

      recordedAt: readString(row["recordedAt"]) ?? "",

      row,

    });

  }

  return rows.slice(Math.max(0, rows.length - limit));

};

const clip = (text: string, maxChars: number): { text: string; truncated: boolean } =>

  text.length <= maxChars

    ? { text, truncated: false }

    : {

        text: `${text.slice(0, maxChars)}\n[${String(text.length - maxChars)} of ${String(text.length)} characters dropped from the end of this step]`,

        truncated: true,

      };

const stringify = (value: unknown): string | null => {

  if (value === undefined || value === null) return null;

  if (typeof value === "string") return value === "" ? null : value;

  try {

    return JSON.stringify(value) ?? null;

  } catch {

    return null;

  }

};

const readJson = (path: string): unknown => {

  if (!existsSync(path)) return undefined;

  try {

    return JSON.parse(readFileSync(path, "utf8")) as unknown;

  } catch {

    return undefined;

  }

};

const readField = (value: unknown, key: string): unknown =>

  typeof value === "object" && value !== null ? (value as Record<string, unknown>)[key] : undefined;

const readRecord = (value: unknown): Record<string, unknown> | undefined =>

  typeof value === "object" && value !== null && !Array.isArray(value)

    ? (value as Record<string, unknown>)

    : undefined;

const readArray = (value: unknown): ReadonlyArray<unknown> => (Array.isArray(value) ? value : []);

const readString = (value: unknown): string | null =>

  typeof value === "string" && value !== "" ? value : null;

const readNumber = (value: unknown): number | null =>

  typeof value === "number" && Number.isFinite(value) ? value : null;

/**

 * The candidate: the one object a cycle review proposes and an optimizer

 * mutates.

 *

 * `docs/coder/autoimprove.md` §3 types a review proposal as

 * `{lever, evidence, risk, verification}`, and §7.6 says the optimizer lane

 * shares that schema "so a reflection and a mutation are the same object".

 * OpenAgentsInc/openagents#122 names the other half of the same object: a

 * candidate is a diff over the staged text surfaces plus the lineage it came

 * from, the model family it was written or evolved against (ledger O5), and

 * the evidence rows behind it. This file is that single definition. A review

 * proposal is a candidate whose `lineage.origin` is `review`; an optimizer

 * mutation will be one whose origin is `optimizer`, and nothing else about it

 * changes.

 *

 * THE EVIDENCE RULE IS ENFORCED HERE, AT PARSE TIME. `autoimprove.md` §6

 * lists "confident review without understanding" as a failure mode and its

 * control as "a proposal with no evidence pointer is rejected in the adopt

 * step". Rejecting it in the adopt step means a human notices, or does not.

 * {@link parseCycleReview} refuses the whole review instead: every evidence

 * ref must resolve against the artifacts the review was actually given, and

 * every proposal must cite at least one trajectory step. A reviewer that

 * invents a step number produces a named rejection, not a plausible document.

 *

 * The refusals are named because an unnamed refusal is the failure this loop

 * exists to stop: `openagents trace redact` reported "Nothing matched the

 * redaction rules" over a file full of live tokens (#97), and the sentence was

 * true. A reviewer whose citations do not resolve is the same shape of lie one

 * level up, so its rejection says which ref, in which proposal, and why.

 */

/** The schema id a candidate carries on the wire. */

export const CODER_CANDIDATE_SCHEMA = "openagents.coder_candidate.v1";

/** The schema id a parsed cycle review carries. */

export const CODER_REVIEW_SCHEMA = "openagents.coder_review.v1";

/** The improvement axes of `docs/coder/autoimprove.md` §2, plus the ledger. */

export type LeverAxis = "process" | "plugin" | "harness" | "optimizer" | "routing" | "ledger";

export const LEVER_AXES: ReadonlyArray<LeverAxis> = [

  "process",

  "plugin",

  "harness",

  "optimizer",

  "routing",

  "ledger",

];

/** Which way a suite figure has to move for the candidate to be confirmed. */

export type DeltaDirection = "up" | "down" | "unchanged";

export const DELTA_DIRECTIONS: ReadonlyArray<DeltaDirection> = ["up", "down", "unchanged"];

/**

 * One pointer into the artifacts the review was given.

 *

 * The grammar is deliberately small, because every scheme in it has to be

 * resolvable against something the assembler put in the request:

 *

 * - `trial:<task>#step-<id>` — one step of that trial's ATIF trajectory

 * - `trial:<task>#outcome` — that trial's verifier decision

 * - `row:<suite>#<recordedAt>` — one `bench-results` row

 * - `ledger:<id>` — one `docs/coder/best-practices.md` entry, e.g. `ledger:T1`

 * - `diff:<path>` — one file the lever's diff touches

 */

export interface EvidenceRef {

  readonly ref: string;

  /** What this ref is being cited for. Prose, and it may be empty. */

  readonly note: string;

}

/** What kind of artifact a ref resolved to. */

export type EvidenceKind =

  | "trajectory_step"

  | "trial_outcome"

  | "bench_row"

  | "ledger_entry"

  | "diff_path";

/** One staged text surface a candidate changes, and the change to it. */

export interface CandidateSurfaceDiff {

  /**

   * The staged surface's id, from `surfaces/coder/index.json` — `system-prompt`,

   * `tool-descriptions`, or `catalog-lines` as #122 staged them, and whatever a