Stream Gym runs live at /gym with the chat transcript components #242

Closed AtlantisPleb opened this 2d ago 1 comment

The ask

Watching a suite run from /gym should feel like watching /chat: a running suite appears the moment it starts, its trials show up as Harbor launches them, and opening a trial streams the coder's transcript live — rendered with the same components the chat page uses, not a raw event dump. Today GymLive is a static post-hoc table; the run only exists after it is over, and the transcript (which already streams through this server on the thread lane) is reachable only by knowing the thread id and visiting /threads/:id, where it renders as typed raw rows.

Design

Index goes live. GymLive subscribes to the gym PubSub topic (from #241). Running runs render at the top with their live trial tally and a working indicator (the shimmer treatment from the AI components), and flip to graded rows in place when finalized. The existing table stays the record below.

Run detail page. A run route (for example /gym/runs/:id) shows the run header (suite, agent, model, lane, recipe digest, status), its trials with per-trial state, and the selected trial's transcript streaming as it happens.

The transcript renders with the chat components. The components are already shared — OpenAgentsWeb.AI.Conversation (conversation/1, conversation_content/1, message/1, message_content/1, shimmer/1), AI.Reasoning, and the tool-activity presentation ChatLive uses. What is missing is the mapping: thread events (turn.user, turn.reasoning, tool.ran, turn.assistant, payloads client-written and read defensively) rendered into those components. Build that mapping as a shared transcript renderer, not a private helper of the gym page, so ThreadShowLive can adopt the same rendering later instead of keeping a second vocabulary. Follow ThreadShowLive's projection protocol exactly: subscribe first, then snapshot, then drain buffered broadcasts with a monotonic last_event_id dedup.

Gating. Operator-only on every path, the way GymLive already is: route behind the :operator pipeline, mount rechecks, every event rechecks. The gym read path reaches a trial's thread through the run linkage (verified at ingest by #241), and the budget/grant card stays owner-only exactly as /threads/:id holds it (THREAD-002).

Out of scope. Local-lane (ollama/...) trials have no thread and render as state-only rows. No public leaderboard; widening past operators stays a deliberate later decision, per docs/2026-08-24-harbor-terminal-bench-plan.md.

Acceptance

  • Start bench/run-suite.sh against the dev forge: the run appears on /gym while running, trials appear as they start, and the run flips to graded in place — no reload anywhere.
  • Open the run: the active trial's transcript streams turn by turn with the same conversation components /chat renders, including reasoning and tool activity, and holds up under the defensive-payload rule (a malformed payload degrades to a neutral row, never a crash).
  • A non-operator account cannot reach the index, the run page, or any linked transcript.
  • mix precommit green.

Blocked by

  • #241 — run lifecycle, trial records, thread linkage, PubSub.
  • OpenAgentsInc/openagents#38 — the bench lane registering runs and reporting trial threads.

Part of the Gym lane (project 14).

  1. AtlantisPleb opened this issue 2d ago
  2. A AtlantisPleb Author 2d ago

    Shipped in c60a364. /gym subscribes to the gym topic: running runs render above the table with live tallies and flip to graded in place; /gym/runs/:id shows the run, its trials, and the selected trial's transcript streamed through the same components /chat uses (OpenAgentsWeb.AI.ThreadTranscript maps turn.user / turn.reasoning / tool.ran / turn.assistant into Conversation, Reasoning, and tool presentation; malformed payloads degrade to a neutral row). Read path is Gym.fetch_trial_thread/1 — only through an ingest-verified linkage, operator-gated, stated in THREAD-001/ADMIN-001 with enumeration proofs updated. Verified with a real Harbor terminal-bench run: the run appeared live while executing and its fix-git trial's transcript streamed turn by turn. Local-lane trials stream too now that OpenAgentsInc/openagents#39 reports them.

  3. closed this as completed 2d ago
Sign in with GitHub to comment on this issue.