OpenAgents

Commit

Print the rows a trend is over, not only the movement between them

f86647b4d376 · Claude Fable 5 · · parent 046939295007 

Print the rows a trend is over, not only the movement between them

#34's acceptance clause asks for two consecutive runs producing comparable
rows, and the trend view printed only the deltas — so it could say cost per
accepted outcome rose 40% without ever saying what it rose from, which lane
model produced either figure, or which Harbor jobs to go and read.

Each row now prints its success rate, cost per accepted outcome, accepted over
graded, gate verdict, model, and job id, above the steps between them.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SoZMfWRSGnf6FZX2Ar9rQ2
Co-Authored-By
Claude Fable 5 <noreply@anthropic.com>

Deploy story

What this commit did to the running system — joined from the forge receipt chain, the part a commit page elsewhere cannot show.

Not deployed through the forge lane

No push, promotion, build, or deploy receipt references this commit (receipts are scanned over a bounded recent window). Changes shipped by full node replacement carry their proof in the release gate receipt instead.

Changed files

  • modified packages/coder-effectiveness/src/compare.test.ts
  • modified packages/coder-effectiveness/src/render-compare.ts

Diff

2 files changed, +26 -0

packages/coder-effectiveness/src/compare.test.ts modified +14

@@ -221,4 +221,18 @@ describe("rendering", () => {

221 221 
    expect(text).toContain("Trend — tb2-cross-section on the proxy lane");
222 222 
    expect(text).toContain("worse");
223 223 
  });
224
225 
  test("prints each row's own figures, not only the movement between them", () => {
226 
    // #34's acceptance clause asks to see two comparable rows. A trend printed
227 
    // as deltas alone says something moved without saying what it moved from.
228 
    const first = row("priced-lane", "proxy", "2026-08-25T10:00:00.000Z");
229 
    const second = row("regressed-lane", "proxy", "2026-08-25T11:00:00.000Z");
230
231 
    const text = renderComparison(compareRuns([first, second]));
232
233 
    expect(text).toContain("2026-08-25T10:00:00.000Z");
234 
    expect(text).toContain("2026-08-25T11:00:00.000Z");
235 
    expect(text).toContain(`job ${String(first.jobId)}`);
236 
    expect(text).toContain(`job ${String(second.jobId)}`);
237 
  });
224 238 
});
packages/coder-effectiveness/src/render-compare.ts modified +12

@@ -62,6 +62,18 @@ const renderLaneComparison = (comparison: LaneComparison): ReadonlyArray<string>

62 62 
const renderTrend = (trend: LaneTrend): ReadonlyArray<string> => {
63 63 
  const lines: Array<string> = [];
64 64 
  lines.push(`Trend — ${trend.suite} on the ${trend.lane} lane, ${String(trend.rows.length)} runs`);
65
66 
  // The rows themselves before the deltas between them. A trend printed as
67 
  // deltas alone reads as movement without saying what moved, and the thing
68 
  // #34's acceptance clause asks to see is the rows.
69 
  for (const row of trend.rows) {
70 
    lines.push(
71 
      `  ${row.recordedAt}  ${pct(row.successRate).padStart(7)} success  ${usd(row.costPerAcceptedOutcomeUsd).padStart(9)}/accepted  ${String(row.accepted)}/${String(row.graded)} graded  ${row.gateStatus ?? "ungated"}`,
72 
    );
73 
    lines.push(`    ${row.models.join(", ") || "model unknown"} · job ${row.jobId ?? "unknown"}`);
74 
  }
75 
  lines.push("");
76
65 77 
  for (const step of trend.steps) {
66 78 
    lines.push(`  ${step.from.recordedAt} → ${step.to.recordedAt}`);
67 79 
    lines.push(deltaLine("Δ cost per accepted", step.costDelta, 4, true));

This page updates live while a promote is in flight · changelog