ATIF exporter: automatic wasted-reexecution section in extra #157

Closed AtlantisPleb opened this 5d ago 2 comments

Problem

Nothing in the current export distinguishes useful executions from repeat executions of the same work. In trajectory 2026-08-27T12:29:02: npx vp test --run executed twice for ~150s each (second was pure failure-name recovery), and pnpm run test:rust executed three times in one invocation — roughly 5–7 minutes of the 22-minute session spent redundantly, invisible to anyone reading the final metrics.

Recommendation

Add automatic waste detection to the ATIF exporter (extra.waste or similar):

Normalize every executed command to its head (executable + first meaningful args, i.e. everything before the first pipeline operator).
Cluster steps by equal normalized head; flag any family with execution count > 1.
Emit structured records:
```
{ "head": "npx vp test --run", "executions": 2, "approx_wasted_seconds": 150 }
```
(wasted_seconds = sum of durations beyond the first in the family, using duration_ms from the per-step-timing change.)
Same for the pre-normalized line level so step 55's three-in-one-invocation triple-run gets caught even though it's one shell call.

This surfaces repeat-execution cost at export time without anyone hand-parsing the trajectory, which is exactly what had to happen manually today.

Acceptance criteria

[ ] Exporter emits per-head execution counts + wasted-seconds on every session.
[ ] Against a replayed/fresh run reproducing this trajectory's shape, it reports npx vp test --run: 2× and pnpm run test:rust: 3× automatically.
[ ] Depends on / pairs with the duration_ms issue; without timing, wasted_seconds falls back to wall-clock deltas between duplicated heads.

AtlantisPleb opened this issue 5d ago
A AtlantisPleb Author 5d ago
Placement (from codebase survey): Rust, same exporter as #156.
- Head-normalization + clustering runs over the already-recorded ToolCall records inside crates/openagents-cli/src/coder/export.rs at export time — pure function, no runtime cost during the session.
- Consumes duration_ms from #156; wall-clock at_ms deltas are the fallback when a record predates it.
- The pipeline-head normalizer should be the same code #153 uses for its pre-spawn lint — one normalizer, two consumers. Both live in crates/openagents-cli/src/tools.rs or a small shared module.
  Rust only.
A AtlantisPleb Author 5d ago
Done on branch waste-series.

Shipped:
- extra.waste.repeated_command_heads in the ATIF export (coder/export.rs): per-family {head, executions, approx_wasted_seconds}, clustered with command_heads — the same normalizer #153 refuses on, per the issue's one-normalizer-two-consumers requirement.
- approx_wasted_seconds = equal share of total family wall time for executions beyond the first, from duration_ms (#156), so the whole chain runs off measured per-call timing.
- Consumes shell/bash/run tool calls only; heads run once are omitted.
- Normalizer refinement the test forced: npx/pnpx carry their package as word two (npx vp test ≠ npx vp lint), matching the three-word rule pnpm run X already had.
- New integration test coder_export_waste.rs (own process — the exporter resolves output via HOME): two 150s suite runs export {head: "npx vp test", executions: 2, approx_wasted_seconds: 150}.
- Full crate: 1,498 integration + 954 lib tests green.
Against a replay of this morning's trajectory this section reports npx vp test --run: 2× and pnpm run test:rust: 3× automatically — the acceptance criterion.
closed this as completed 5d ago