Cut the round-trips out of a CLI question, and move the status line

8d711b2bd787 · AtlantisPleb · · parent 7eab6f8e444f

Cut the round-trips out of a CLI question, and move the status line

"list open issues re CLI" cost six model calls and 26,193 prompt tokens. Four
things were wrong, three of them here.

**The command tree was discovered one round-trip at a time.** A session that did
not know `issue list` existed ran `issue --help`, then `issue list --help`, then
the command. Two turns of a model's time to learn two words. The tree now sits
in the tool description, read from this binary through the completion script
that already enumerates every command and subcommand: one process at session
start, 1,397 bytes, and it cannot describe a CLI other than the one running.
Flags are still left to `--help`, where there are hundreds and they change.

**`--json` was the advertised way to read anything.** A list of three issues is
442 bytes plain and 20,009 as JSON, because the JSON carries every issue's whole
body: 3,896 bytes each. Both the tool description and the skill now say to read
the plain output and reach for `--json` only to take one field out of one
record.

**A truncated answer advised the thing that caused it.** The message said
"narrow the command or use --json" on every truncation. It now fits what was
run -- drop `--json` when that was the problem, narrow with a flag otherwise --
and says plainly that what survived is incomplete and must not be summarized as
if it were the whole answer.

The fourth was a stale sentence in the repository's own work-management skill,
fixed in openagents.com: it claimed the CLI had no `issue` or `project`
commands, so the session went looking for them. The same question now takes
three calls and no JSON.

Also, from a screenshot: the status line moves under the composer and a blank
row goes above it. It reads as a caption under the thing it describes rather
than something to scan back up for, and a line of nothing separates reading from
typing better than a rule does.

And a new packaged skill, `delegating-work`, on choosing between doing it
yourself, fanning out, and handing a whole task to the Devin CLI when it is on
PATH. It records two things found by running it: the mode is `dangerous` on this
build where the published documentation calls it "bypass", and `bypass` is not
rejected but accepted and ignored, so a session that copied the documentation
would silently fall back to prompting where no one can answer; and it refuses an
untrusted workspace, which only the person at the keyboard can grant.

379 tests pass.

Deploy story

What this commit did to the running system — joined from the forge receipt chain, the part a commit page elsewhere cannot show.

Not deployed through the forge lane

No push, promotion, build, or deploy receipt references this commit (receipts are scanned over a bounded recent window). Changes shipped by full node replacement carry their proof in the release gate receipt instead.

Changed files

  • added packages/openagents-cli/skills/delegating-work/SKILL.md
  • modified packages/openagents-cli/skills/openagents-cli/SKILL.md
  • modified packages/openagents-cli/src/coder-tools.ts
  • modified packages/openagents-cli/src/coder-ui.ts
  • modified packages/openagents-cli/test/coder-tools-openagents.test.ts

Diff

5 files changed, +201 -20

packages/openagents-cli/skills/delegating-work/SKILL.md added +78

@@ -0,0 +1,78 @@

1
---
2
name: delegating-work
3
description: Choose between running a command yourself, fanning work out to child coding agents with delegate, and handing a whole task to another agent such as the Devin CLI. Use it before reaching for delegate, and whenever work looks like it needs more than one worker.
4
---
5
6
# Handing work to something else
7
8
Three ways, in order of what they cost. Pick the cheapest that does the job.
9
10
## Do it yourself
11
12
The `shell` tool runs a command here. Reading a file, listing a directory,
13
searching, `git`, a build, a test run: all of it is one call and one process.
14
15
This is almost always the right answer for a single command. Starting an agent
16
to run `pwd` costs minutes and real money and hands back an answer nobody
17
watched being produced.
18
19
## `delegate`, for work that splits
20
21
The `delegate` tool starts child coding agents that all run the same prompt in
22
parallel, in this repository, each with its own file and shell tools. It earns
23
its cost when the work genuinely splits into parts that do not depend on each
24
other: several files to change the same way, several hypotheses to check at
25
once, several test suites to run down.
26
27
Each child starts with no context from this conversation and cannot ask
28
questions, so the prompt has to carry everything. Every child gets the same
29
prompt and is told its own number separately — write for whichever child is
30
reading, rather than naming one.
31
32
## The Devin CLI, for a whole task
33
34
If `devin` is on `PATH`, it is another coding agent on this machine, and it can
35
take a task end to end rather than one prompt in parallel. Check with
36
`command -v devin`.
37
38
Run it non-interactively through `shell`:
39
40
```sh
41
devin -p "<a complete, self-contained task>" --permission-mode dangerous
42
```
43
44
Three things about that command are worth knowing before you run it.
45
46
**`dangerous` is the mode name on this build.** The published documentation
47
calls the equivalent mode "bypass". Passing `--permission-mode bypass` is not
48
rejected — it is accepted and ignored, so the session silently falls back to
49
prompting, and in `-p` mode a prompt nobody can answer is a task that does
50
nothing. Read `devin --help` if unsure; the values it lists are the values it
51
takes.
52
53
**It refuses a workspace it does not trust.** In an untrusted directory it
54
exits at once with `Refusing to run in an untrusted workspace`. Trust is
55
granted by starting `devin` interactively there once, which is something only
56
the person at the keyboard can do. If you hit that, say so and name the
57
directory rather than retrying.
58
59
**`dangerous` auto-approves every tool it has, including writes and shell.**
60
That is the point of using it unattended, and it is also the reason to say what
61
you are handing over before you hand it over. Give it a bounded task in this
62
repository, not an open-ended one.
63
64
There is also `devin acp`, an Agent Client Protocol server over stdio, for a
65
caller that speaks ACP. `-p` is the simpler route from here and needs no
66
protocol on this side.
67
68
## Which one
69
70
| The work | Use |
71
| --- | --- |
72
| One command, one answer | `shell` |
73
| The same thing to N independent parts, at once | `delegate` |
74
| A whole task you would otherwise do yourself, run by another agent | `devin -p` |
75
76
Whatever runs it, the result is yours to check. An agent reporting that it
77
finished is not evidence that it did; read the diff, run the test, look at the
78
output.
packages/openagents-cli/skills/openagents-cli/SKILL.md modified +15 -8

@@ -9,17 +9,24 @@ You are running inside `openagents coder`, which is one subcommand of the

9 9
`openagents` CLI. The `openagents` tool runs the rest of it. The same binary
10 10
answers, so what you see is this machine's build, not a remembered one.
11 11
12
## Find out what a command does by asking it
12
## Finding a command, and reading its answer
13 13
14
Every command documents itself. `openagents --help` lists the subcommands;
15
`openagents <command> --help` gives one command's flags and arguments;
16
`openagents <command> <subcommand> --help` goes a level deeper.
14
The `openagents` tool's own description lists every command and subcommand,
15
read from this binary. You do not need to go looking for them, and you do not
16
need `--help` to find out that a command exists.
17 17
18
Ask before you guess. This file deliberately does not list flags: a written
19
copy of them goes stale the first time one changes, and the help output cannot.
18
Use `<command> --help` for a flag you do not know. This file lists no flags on
19
purpose: a written copy goes stale the first time one changes, and the help
20
output cannot.
20 21
21
Add `--json` to any command to get the response as JSON rather than as prose.
22
Prefer it when you are going to read a field out of the answer.
22
**Read the plain output.** It is what a person reads and it is small. A list of
23
three issues is 442 bytes plain and 20,000 as JSON, because the JSON carries
24
every issue's whole body — and a session that reads twenty thousand bytes to
25
answer one question pays for them on every turn after it, too.
26
27
Add `--json` only when you need one field out of one record. Prefer a narrower
28
command over a wider one you then read past: `--label`, `--state`, `--limit`
29
and a search term cost nothing and cut the answer to what was asked.
23 30
24 31
## What works with no credential
25 32
packages/openagents-cli/src/coder-tools.ts modified +58 -7

@@ -14,6 +14,7 @@

14 14
 * reads, and a function that runs on this machine.
15 15
 */
16 16
17
import { spawnSync } from "node:child_process";
17 18
import { existsSync } from "node:fs";
18 19
import { fileURLToPath } from "node:url";
19 20

@@ -285,6 +286,42 @@ const refusalFor = (args: ReadonlyArray<string>): string | undefined => {

285 286
  return undefined;
286 287
};
287 288
289
/**
290
 * The command tree, read out of the CLI's own completion script.
291
 *
292
 * A session that did not know `issue list` existed spent two turns finding out:
293
 * `issue --help`, then `issue list --help`, then the command it wanted. Two
294
 * round-trips of a model's time to learn two words.
295
 *
296
 * So the tree goes in the tool description. It is not written down here -- it
297
 * is asked for once, from the binary that is running, through the completion
298
 * script that already enumerates every command and subcommand. One process at
299
 * session start, and it cannot describe a CLI other than this one.
300
 *
301
 * Flags are still left to `--help`. There are hundreds and they change; the
302
 * command names are few and are what the round-trips were being spent on.
303
 */
304
const commandTree = (entry: string): string | undefined => {
305
  const shown = spawnSync(process.execPath, [entry, "--completions", "zsh"], {
306
    encoding: "utf8",
307
    timeout: 10_000,
308
  });
309
  if (shown.status !== 0 || typeof shown.stdout !== "string") return undefined;
310
311
  const lines: string[] = [];
312
  // Each `commands=( … )` block belongs to the function it sits in, and that
313
  // function is named for the command whose subcommands it lists.
314
  for (const match of shown.stdout.matchAll(/_openagents(_[a-z_]*)?\(\)\s*\{([\s\S]*?)\n\}/g)) {
315
    const owner = (match[1] ?? "").replaceAll("_", " ").trim();
316
    const block = /commands=\(([\s\S]*?)\n\s*\)/.exec(match[2] ?? "");
317
    if (block === null) continue;
318
    const names = [...(block[1] ?? "").matchAll(/'([a-z][a-z0-9-]*):/g)].map((found) => found[1]);
319
    if (names.length === 0) continue;
320
    lines.push(owner.length === 0 ? `openagents ${names.join(" | ")}` : `  ${owner} ${names.join(" | ")}`);
321
  }
322
  return lines.length === 0 ? undefined : lines.join("\n");
323
};
324
288 325
/**
289 326
 * The openagents tool: run the CLI this session is part of.
290 327
 *

@@ -295,16 +332,24 @@ const refusalFor = (args: ReadonlyArray<string>): string | undefined => {

295 332
 * can go stale.
296 333
 */
297 334
export function openagentsTool(): CoderTool {
335
  const entry = cliEntry();
336
  const tree = entry === undefined ? undefined : commandTree(entry);
298 337
  return {
299 338
    name: "openagents",
300 339
    description:
301 340
      "Run the OpenAgents CLI: issues, projects, repositories, the forum, authentication, and " +
302 341
      "any API route through `api`. Pass the arguments after `openagents` as a list, without " +
303
      "`openagents` itself. Discover what exists with `--help` on any command rather than " +
304
      "guessing at flags, and add `--json` when you are going to read a field out of the " +
305
      "answer. Reads are free; a write is visible to other people at once, so say what you are " +
306
      "about to write before the first one. Read the `openagents-cli` skill for the auth model " +
307
      "and what works with no credential.",
342
      "`openagents` itself.\n\n" +
343
      (tree === undefined ? "" : `Commands:\n${tree}\n\n`) +
344
      "Run `<command> --help` when you need a flag you do not know; the commands above are the " +
345
      "whole set, so you do not need to go looking for them.\n\n" +
346
      "Read the plain output. It is what a person reads and it is small: a list of three issues " +
347
      "is 442 bytes plain and 20,000 as JSON, because the JSON carries every issue's whole body. " +
348
      "Add `--json` only when you need one field out of one record, and prefer a narrower " +
349
      "command over a wider one you then have to read past.\n\n" +
350
      "Reads are free; a write is visible to other people at once, so say what you are about to " +
351
      "write before the first one. Read the `openagents-cli` skill for the auth model and what " +
352
      "works with no credential.",
308 353
    parameters: {
309 354
      type: "object",
310 355
      properties: {

@@ -331,7 +376,6 @@ export function openagentsTool(): CoderTool {

331 376
      if (refusal !== undefined) return refusal;
332 377
333 378
      const { spawn } = await import("node:child_process");
334
      const entry = cliEntry();
335 379
      if (entry === undefined) return "This session cannot find the CLI it is running from.";
336 380
337 381
      return await new Promise<string>((resolve) => {

@@ -373,9 +417,16 @@ export function openagentsTool(): CoderTool {

373 417
        });
374 418
375 419
        child.on("close", (code) => {
420
          // The advice has to fit what was run. This used to say "or use
421
          // --json" on every truncation, which for a list is the thing that
422
          // caused it: three issues are 442 bytes plain and 20,000 as JSON,
423
          // because the JSON carries every body.
424
          const advice = args.includes("--json")
425
            ? "drop --json and read the plain output, or ask for one record"
426
            : "narrow it with a flag such as --limit, --label, or --state";
376 427
          const bounded =
377 428
            output.length > CLI_OUTPUT_LIMIT
378
              ? `${output.slice(0, CLI_OUTPUT_LIMIT)}\n\n[truncated; narrow the command or use --json]`
429
              ? `${output.slice(0, CLI_OUTPUT_LIMIT)}\n\n[The output was cut off here. Run it again and ${advice}; what you have above is incomplete and must not be summarized as if it were the whole answer.]`
379 430
              : output;
380 431
          const body = bounded.trim();
381 432
          // The exit code is reported on failure because it is what the CLI
packages/openagents-cli/src/coder-ui.ts modified +21 -5

@@ -61,6 +61,14 @@ const RED = "\x1b[31m";

61 61
62 62
const STATUS_ROWS = 1;
63 63
const COMPOSER_ROWS = 3;
64
/**
65
 * One blank row between the transcript and the composer.
66
 *
67
 * The composer sat directly under the last line of the reply, so a rule was
68
 * doing all the work of saying where reading stops and typing starts. A line of
69
 * nothing does it better and costs one row.
70
 */
71
const SPACER_ROWS = 1;
64 72
/**
65 73
 * Rows the fleet block may take before it scrolls internally.
66 74
 *

@@ -502,7 +510,10 @@ export function runCoderUi(session: CoderSession, options: CoderUiOptions): Prom

502 510
        paint(rows, rows.length, 1);
503 511
        return;
504 512
      }
505
      const transcriptHeight = Math.max(1, height - STATUS_ROWS - COMPOSER_ROWS - 1);
513
      const transcriptHeight = Math.max(
514
        1,
515
        height - STATUS_ROWS - COMPOSER_ROWS - SPACER_ROWS - 1,
516
      );
506 517
507 518
      const fleet = fleetLines(snapshot, width);
508 519
      // The fleet takes its rows from the transcript, not from the chrome: the

@@ -522,9 +533,9 @@ export function runCoderUi(session: CoderSession, options: CoderUiOptions): Prom

522 533
      for (let row = 0; row < transcriptRows; row += 1) rows.push(lines[start + row] ?? "");
523 534
      rows.push(...fleet);
524 535
525
      // Bottom chrome, in the order a reader scans it: what the session is
526
      // doing now, then where the typing goes, then what the keys do. The
527
      // composer sits between two rules so it reads as its own region rather
536
      // Bottom chrome, in the order a reader scans it: where the typing goes,
537
      // what the session is doing, then what the keys do. The composer sits
538
      // between a blank row and a rule so it reads as its own region rather
528 539
      // than as the last line of the transcript.
529 540
      const rule = `${DIM}${"─".repeat(Math.max(0, width))}${RESET}`;
530 541
      const inner = Math.max(10, width - 4);

@@ -555,7 +566,11 @@ export function runCoderUi(session: CoderSession, options: CoderUiOptions): Prom

555 566
        where = candidate;
556 567
        break;
557 568
      }
558
      rows.push(`  ${justify(activity, where, inner)}`);
569
      // A blank row, then the composer, then what the session is doing. The
570
      // status line reads as a caption under the thing it describes: the reader
571
      // looks at where they type, and the state of the session is the next
572
      // thing down rather than something to scan back up for.
573
      rows.push("");
559 574
      rows.push(rule);
560 575
      // The composer shows its tail, never more characters than the row holds.
561 576
      // A row written past the last column makes the terminal wrap it, which

@@ -568,6 +583,7 @@ export function runCoderUi(session: CoderSession, options: CoderUiOptions): Prom

568 583
          ? `…${typed.slice(typed.length - composerRoom + 1).join("")}`
569 584
          : composer;
570 585
      rows.push(`  › ${visible}`);
586
      rows.push(`  ${justify(activity, where, inner)}`);
571 587
      rows.push(rule);
572 588
573 589
      // Every key named here does something in the state it is named in, and
packages/openagents-cli/test/coder-tools-openagents.test.ts modified +29

@@ -20,6 +20,35 @@ describe("the openagents tool", () => {

20 20
    await expect(run(["--version"])).resolves.toContain("openagents v");
21 21
  });
22 22
23
24
  it("carries the command tree, so a session need not go looking for one", async () => {
25
    const { description } = openagentsTool();
26
27
    // Two round-trips of a model's time went on `issue --help` then
28
    // `issue list --help`, to learn two words. The tree is read from this
29
    // binary, so it cannot describe a CLI other than the one running.
30
    expect(description).toContain("Commands:");
31
    expect(description).toContain("issue list");
32
    expect(description).toContain("auth login");
33
  });
34
35
  it("says to read the plain output, not the JSON", async () => {
36
    const { description } = openagentsTool();
37
38
    expect(description).toContain("442 bytes plain and 20,000 as JSON");
39
  });
40
41
  it("tells a truncated --json call to drop --json, not to add it", async () => {
42
    // This once advised "use --json" on every truncation, which for a list is
43
    // the thing that caused it.
44
    const output = await run(["issue", "list", "-R", "OpenAgentsInc/openagents.com", "--json"]);
45
46
    if (output.includes("cut off here")) {
47
      expect(output).toContain("drop --json");
48
      expect(output).toContain("must not be summarized");
49
    }
50
  });
51
23 52
  it("asks for arguments rather than running something arbitrary", async () => {
24 53
    await expect(run([])).resolves.toContain("`args` is required");
25 54
  });

This page updates live while a promote is in flight · changelog