Refuse a missing headless prompt, and a plugin named after a builtin

4f3a557fd818 · AtlantisPleb · · parent 343870bfdb29

Refuse a missing headless prompt, and a plugin named after a builtin

Two places where the CLI did something the caller did not ask for and did
not say so.

`oa coder --headless` with no prompt substituted the literal `Analyze
workspace and run tests`, opened a thread, and spent the grant on an
instruction nobody gave. One screen above it, `--offline` refuses the
identical omission by name, so the same missing input was handled two
opposite ways in one function. It now refuses, names the form that works,
exits 2, and opens no thread.

A plugin whose manifest name is `shell`, `skill`, `openagents`,
`capability`, or `delegate` loaded, was declared to the model, and was
permanently unreachable: those five names are answered by match arms above
the plugin lookup, so every call landed on the builtin. `validate_manifest`
now refuses such a manifest under a `name_reserved` code that names the
collision, which keeps it out of the catalog as well as out of a load — a
capability that failed to install is better than one silently shadowed.
The reserved list lives in `tools::BUILTIN_TOOL_NAMES` beside the arms it
mirrors, and `every_declared_tool_has_an_arm_that_answers_it` now asserts
the two agree.

The prompt test asserts the server was never asked to open a thread, and
asserts it first: an invented prompt that opens a thread and then fails
would satisfy an exit code alone. The collision test plants two copies of
one valid, digest-pinned, pure-compute guest -- one under `shell`, one
under a free name -- and asserts the declared list exactly, which is what
catches the duplicate, then walks every declared name through
`execute_tool`. Both were verified by reverting each fix: the reverted
prompt path fails on the thread assertion, the reverted collision check
fails on the catalog and, with that muted, on the declared list.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SoZMfWRSGnf6FZX2Ar9rQ2
Co-Authored-By
Claude Opus 5 (1M context) <noreply@anthropic.com>

Deploy story

What this commit did to the running system — joined from the forge receipt chain, the part a commit page elsewhere cannot show.

Not deployed through the forge lane

No push, promotion, build, or deploy receipt references this commit (receipts are scanned over a bounded recent window). Changes shipped by full node replacement carry their proof in the release gate receipt instead.

Changed files

  • modified crates/openagents-cli/src/cli.rs
  • modified crates/openagents-cli/src/plugins.rs
  • modified crates/openagents-cli/src/tools.rs
  • modified crates/openagents-cli/tests/flags.rs
  • modified crates/openagents-cli/tests/plugin_host_test.rs

Diff

5 files changed, +312 -3

crates/openagents-cli/src/cli.rs modified +17 -3

@@ -4292,6 +4292,12 @@ fn run_offline_coder(coder: CoderArgs) {

4292 4292
/// Lifted out of the dispatch so `--export` is written here too. It used to be
4293 4293
/// read only by the full-screen session, which meant a headless run that asked
4294 4294
/// for a transcript got none and was told nothing.
4295
///
4296
/// A missing prompt is a missing input, not a licence to invent one. This
4297
/// substituted the literal `Analyze workspace and run tests`, opened a thread,
4298
/// and spent the grant on an instruction nobody gave — one screen above
4299
/// [`run_offline_coder`], which refuses the identical omission by name. The
4300
/// same input is now handled the same way on both paths.
4295 4301
async fn run_headless_coder(
4296 4302
    coder: CoderArgs,
4297 4303
    api_base: &str,

@@ -4299,10 +4305,18 @@ async fn run_headless_coder(

4299 4305
    repository: Option<String>,
4300 4306
    resumed: Option<crate::resume::Resumption>,
4301 4307
) -> Result<(), Box<dyn std::error::Error>> {
4302
    let prompt = coder
4308
    let Some(prompt) = coder
4303 4309
        .prompt
4304
        .clone()
4305
        .unwrap_or_else(|| "Analyze workspace and run tests".to_string());
4310
        .as_deref()
4311
        .map(str::trim)
4312
        .filter(|p| !p.is_empty())
4313
        .map(str::to_string)
4314
    else {
4315
        fail(
4316
            "--headless runs one prompt and exits. Give it one: \
4317
             `oa coder --headless \"<prompt>\"`",
4318
        );
4319
    };
4306 4320
    println!("Executing coder prompt headlessly: {}", prompt);
4307 4321
    let lane_name = coder.lane_name().unwrap_or_else(|reason| fail(&reason));
4308 4322
    // A headless session may start children. They run on the same lane and the
crates/openagents-cli/src/plugins.rs modified +74

@@ -184,6 +184,23 @@ pub fn validate_manifest(value: &serde_json::Value) -> Result<Manifest, Refusal>

184 184
            "`name` (lowercase identifier, it becomes the tool name)",
185 185
        ));
186 186
    }
187
    // The manifest name *is* the tool name, and the session's own tools are
188
    // dispatched before any plugin. A plugin under one of those names would be
189
    // declared to the model and then permanently unreachable, which is worse
190
    // than one that failed to install: the model is told a capability exists
191
    // and every call to it lands on the builtin. Refused here, so it reaches
192
    // neither the catalog nor a load.
193
    if crate::tools::BUILTIN_TOOL_NAMES.contains(&name) {
194
        return Err(refuse(
195
            "name_reserved",
196
            format!(
197
                "the manifest is named `{name}`, which is a built-in tool of this session. \
198
                 The builtin answers first, so the plugin could never run. Rename it; \
199
                 the reserved names are {}",
200
                crate::tools::BUILTIN_TOOL_NAMES.join(", ")
201
            ),
202
        ));
203
    }
187 204
    let version = record.get("version").and_then(|v| v.as_str()).unwrap_or("");
188 205
    if version.is_empty() {
189 206
        return Err(bad("`version`"));

@@ -1586,6 +1603,63 @@ mod tests {

1586 1603
        );
1587 1604
    }
1588 1605
1606
    /// Every name the session dispatches itself is refused, and the refusal
1607
    /// says which one it collided with.
1608
    ///
1609
    /// The manifest is otherwise valid, so a host without this check installs
1610
    /// it, declares its tool, and routes every call to the builtin instead.
1611
    #[test]
1612
    fn a_plugin_named_after_a_builtin_tool_is_refused_by_name() {
1613
        for reserved in crate::tools::BUILTIN_TOOL_NAMES {
1614
            let mut value = manifest_json(serde_json::json!([]), serde_json::json!([]));
1615
            value["name"] = serde_json::json!(reserved);
1616
            let refusal = validate_manifest(&value).unwrap_err();
1617
            assert_eq!(refusal.code, "name_reserved", "`{reserved}`: {refusal}");
1618
            assert!(
1619
                refusal.reason.contains(reserved),
1620
                "the refusal did not name the collision: {}",
1621
                refusal.reason
1622
            );
1623
        }
1624
        // A name that collides with nothing still loads, so the check is a
1625
        // reservation and not a ban on plugins.
1626
        assert_eq!(
1627
            validate_manifest(&manifest_json(serde_json::json!([]), serde_json::json!([])))
1628
                .unwrap()
1629
                .name,
1630
            "probe_plugin"
1631
        );
1632
    }
1633
1634
    /// A colliding manifest never reaches the catalog either, so nothing can
1635
    /// ask for it by name and no tool is declared for it.
1636
    #[test]
1637
    fn a_colliding_manifest_is_left_out_of_the_catalog() {
1638
        let root = tempfile::tempdir().unwrap();
1639
        let plugins = root.path().join("plugins");
1640
        for name in ["shell", "word_count"] {
1641
            let dir = plugins.join(name);
1642
            std::fs::create_dir_all(&dir).unwrap();
1643
            let mut value = manifest_json(serde_json::json!([]), serde_json::json!([]));
1644
            value["name"] = serde_json::json!(name);
1645
            std::fs::write(
1646
                dir.join("manifest.json"),
1647
                serde_json::to_vec_pretty(&value).unwrap(),
1648
            )
1649
            .unwrap();
1650
        }
1651
1652
        let names: Vec<String> = discover_catalog(root.path())
1653
            .into_iter()
1654
            .map(|entry| entry.name)
1655
            .collect();
1656
        assert_eq!(
1657
            names,
1658
            vec!["word_count"],
1659
            "a plugin named after a builtin reached the catalog"
1660
        );
1661
    }
1662
1589 1663
    #[test]
1590 1664
    fn an_unknown_abi_is_refused_by_name() {
1591 1665
        let mut value = manifest_json(serde_json::json!([]), serde_json::json!([]));
crates/openagents-cli/src/tools.rs modified +24

@@ -37,6 +37,22 @@ use crate::plugins::{

37 37
38 38
pub const OUTPUT_LIMIT: usize = 30_000;
39 39
40
/// The names [`HarnessToolRegistry::execute_tool`] answers itself.
41
///
42
/// A loaded plugin is dispatched from the fallthrough arm, *below* all five of
43
/// these, so a plugin carrying one of these names would be declared to the
44
/// model and never reached: the builtin answers first, every time. That is why
45
/// [`crate::plugins::validate_manifest`] refuses such a manifest by name
46
/// rather than installing a capability nothing can call.
47
///
48
/// `delegate` is on the list even though it is declared only where a
49
/// delegation gate exists, because its match arm is unconditional — a gateless
50
/// session answers it with a refusal, which shadows a plugin just as
51
/// completely. `every_declared_tool_has_an_arm_that_answers_it` keeps this
52
/// list and the arms in step.
53
pub const BUILTIN_TOOL_NAMES: [&str; 5] =
54
    ["shell", "skill", "openagents", "capability", "delegate"];
55
40 56
/// The largest index at or below `max` that is a character boundary in `text`.
41 57
///
42 58
/// Slicing a `String` by a byte index panics when the index lands inside a

@@ -1288,6 +1304,14 @@ mod tests {

1288 1304
            names,
1289 1305
            vec!["shell", "skill", "openagents", "capability", "delegate"]
1290 1306
        );
1307
        // The same five names `plugins::validate_manifest` refuses a plugin
1308
        // for taking. An arm added here and not there would leave a name a
1309
        // plugin can claim and never be called under.
1310
        assert_eq!(
1311
            names,
1312
            BUILTIN_TOOL_NAMES.to_vec(),
1313
            "the reserved list and the arms disagree"
1314
        );
1291 1315
1292 1316
        let runtime = tokio::runtime::Runtime::new().unwrap();
1293 1317
        for name in &names {
crates/openagents-cli/tests/flags.rs modified +93

@@ -989,6 +989,99 @@ fn offline_reaches_no_server_and_the_live_path_reaches_one() {

989 989
    );
990 990
}
991 991
992
// -------------------------------------------- `--headless` with no prompt
993
994
/// A headless run with no prompt refuses, and opens no thread.
995
///
996
/// It used to substitute the literal `Analyze workspace and run tests`, open a
997
/// thread, and spend the grant on an instruction nobody gave — one screen
998
/// below where `--offline` refuses the identical omission by name. The exit
999
/// code alone does not catch that: an invented turn can fail afterwards and
1000
/// exit non-zero too. What catches it is the server, which is never asked to
1001
/// open anything.
1002
#[test]
1003
fn headless_without_a_prompt_refuses_and_opens_no_thread() {
1004
    let server = RouteServer::start(coder_routes);
1005
    let origin = server.origin();
1006
1007
    let base = origin.as_str();
1008
    for bare in [
1009
        vec!["--api-url", base, "coder", "--headless"],
1010
        // Whitespace is not a prompt either; it is the same omission with a
1011
        // space in it.
1012
        vec!["--api-url", base, "coder", "--headless", "   "],
1013
    ] {
1014
        let run = oa_env(&bare, &[("OPENAGENTS_TOKEN", "t")]);
1015
        // Asserted first, because it is the assertion that carries the test:
1016
        // an invented prompt that opens a thread and then fails would satisfy
1017
        // an exit code and nothing else.
1018
        let paths: Vec<String> = server.hits().into_iter().map(|hit| hit.path).collect();
1019
        assert!(
1020
            !paths.iter().any(|p| p == "/api/v1/threads"),
1021
            "{bare:?} still opened a thread: {paths:?}"
1022
        );
1023
        assert!(
1024
            !run.stdout.contains("Analyze workspace"),
1025
            "a prompt nobody gave was run anyway: {}",
1026
            run.stdout
1027
        );
1028
        assert_eq!(run.status, Some(2), "{bare:?} stdout: {}", run.stdout);
1029
        assert!(
1030
            run.stderr.contains("--headless") && run.stderr.contains("<prompt>"),
1031
            "the refusal did not say what is missing or how to give it: {}",
1032
            run.stderr
1033
        );
1034
    }
1035
1036
    // The control, on the same fixture: with a prompt it does open one. Without
1037
    // this the assertion above would also pass against a binary that could not
1038
    // reach the server at all.
1039
    let given = oa_env(
1040
        &["--api-url", &origin, "coder", "--headless", "hello"],
1041
        &[("OPENAGENTS_TOKEN", "t")],
1042
    );
1043
    assert_eq!(given.status, Some(0), "stderr: {}", given.stderr);
1044
    let paths: Vec<String> = server.hits().into_iter().map(|hit| hit.path).collect();
1045
    assert!(
1046
        paths.iter().any(|p| p == "/api/v1/threads"),
1047
        "the fixture never opens a thread, so the test proves nothing: {paths:?}"
1048
    );
1049
}
1050
1051
/// The same omission, refused the same way on both coder paths.
1052
///
1053
/// This is the defect stated as a property: `--offline` and `--headless` were
1054
/// two arms of one function handling one missing input two opposite ways, and
1055
/// the reader who forgot the prompt got a refusal or an invented instruction
1056
/// depending on which arm they were in.
1057
#[test]
1058
fn a_missing_prompt_is_refused_the_same_way_offline_and_headless() {
1059
    let dead = "http://127.0.0.1:1";
1060
    let offline = oa_env(
1061
        &["--api-url", dead, "coder", "--offline"],
1062
        &[("OPENAGENTS_TOKEN", "t")],
1063
    );
1064
    let headless = oa_env(
1065
        &["--api-url", dead, "coder", "--headless"],
1066
        &[("OPENAGENTS_TOKEN", "t")],
1067
    );
1068
    assert_eq!(offline.status, Some(2), "stdout: {}", offline.stdout);
1069
    assert_eq!(
1070
        headless.status,
1071
        Some(2),
1072
        "the headless path accepted a missing prompt: {} {}",
1073
        headless.stdout,
1074
        headless.stderr
1075
    );
1076
    for run in [&offline, &headless] {
1077
        assert!(
1078
            run.stderr.contains("<prompt>"),
1079
            "the refusal did not show the form that works: {}",
1080
            run.stderr
1081
        );
1082
    }
1083
}
1084
992 1085
// ----------------------------------------------------------------- `--model`
993 1086
994 1087
/// `--model` decides the id sent at thread open; without it the default lane's
crates/openagents-cli/tests/plugin_host_test.rs modified +104

@@ -18,6 +18,7 @@ use std::time::Instant;

18 18
use openagents_cli::plugins::{
19 19
    invoke, load_plugin, Approval, CatalogEntry, Mount, MOUNT_FILE_LIMIT,
20 20
};
21
use openagents_cli::tools::{HarnessToolRegistry, ToolCall};
21 22
use sha2::{Digest, Sha256};
22 23
23 24
/// Bump-allocating `packet-v0` scaffolding every fixture shares.

@@ -446,6 +447,109 @@ fn a_mounted_capability_needs_an_operator_before_it_can_be_loaded() {

446 447
    .is_ok());
447 448
}
448 449
450
// ──────────────────────────────────────── a name the session already answers
451
452
/// A valid, digest-pinned, pure-compute guest that loads and does nothing.
453
///
454
/// The collision test needs a plugin that would otherwise install cleanly, so
455
/// the only thing between it and the model's tool list is the reserved-name
456
/// check.
457
fn inert_guest() -> Vec<u8> {
458
    wasm(&format!(
459
        r#"(module (memory (export "memory") 1) {PREAMBLE}
460
             (func (export "handle_packet") (param i32 i32) (result i64) (i64.const 0)))"#
461
    ))
462
}
463
464
fn call(name: &str, arguments: serde_json::Value) -> ToolCall {
465
    ToolCall {
466
        id: "1".to_string(),
467
        name: name.to_string(),
468
        arguments,
469
    }
470
}
471
472
/// A plugin named after a builtin is refused at install, so the declared tool
473
/// list never carries a name the session would answer for itself.
474
///
475
/// The two plugins below are the same fixture under two names. `word_count`
476
/// is the control: it installs, it is declared, and it dispatches to the
477
/// plugin. `shell` is the collision — a builtin match arm wins before the
478
/// plugin lookup, so a host that installed it would declare a tool the model
479
/// could never reach. The list assertion is exact, which is what catches the
480
/// duplicate: a host without the check declares `shell` twice.
481
#[tokio::test]
482
async fn a_plugin_named_after_a_builtin_is_never_declared_and_every_declared_tool_answers() {
483
    let dir = tempfile::tempdir().unwrap();
484
    let artifact = inert_guest();
485
    for name in ["shell", "word_count"] {
486
        let home = dir.path().join("plugins").join(name);
487
        std::fs::create_dir_all(&home).unwrap();
488
        plant(&home, name, &artifact, serde_json::json!([]), 2000, 16);
489
    }
490
491
    let registry = HarnessToolRegistry::new(Some(dir.path().to_path_buf()));
492
    let installed: Vec<&str> = registry
493
        .catalog
494
        .iter()
495
        .map(|entry| entry.name.as_str())
496
        .collect();
497
    assert_eq!(
498
        installed,
499
        vec!["word_count"],
500
        "a plugin named after a builtin reached the catalog"
501
    );
502
503
    // Asked for by exact name: the free name loads, the taken one is not
504
    // there to load, and the model is told so rather than left with a tool
505
    // that answers from somewhere else.
506
    let taken = registry
507
        .execute_tool(&call("capability", serde_json::json!({"name": "shell"})))
508
        .await;
509
    assert!(
510
        taken.is_error,
511
        "`shell` was loaded as a plugin: {}",
512
        taken.output
513
    );
514
    let free = registry
515
        .execute_tool(&call(
516
            "capability",
517
            serde_json::json!({"name": "word_count"}),
518
        ))
519
        .await;
520
    assert!(
521
        !free.is_error,
522
        "the control plugin did not load: {}",
523
        free.output
524
    );
525
526
    // This registry has no delegation gate, so `delegate` is not declared.
527
    let names: Vec<String> = registry.list_tools().into_iter().map(|t| t.name).collect();
528
    assert_eq!(
529
        names,
530
        vec!["shell", "skill", "openagents", "capability", "word_count"],
531
        "the declared list is not the set of tools that can be called"
532
    );
533
534
    // And every name on it reaches an implementation. `word_count` is the one
535
    // that matters here: it is the plugin, dispatched from the arm below all
536
    // the builtins, which is the arm a collision would have hidden it behind.
537
    for name in &names {
538
        if name == "openagents" {
539
            // Shelling out to another CLI is not this test's business.
540
            continue;
541
        }
542
        let out = registry
543
            .execute_tool(&call(name, serde_json::json!({})))
544
            .await;
545
        assert!(
546
            !out.output.starts_with("Unknown tool:"),
547
            "`{name}` is declared and unanswered: {}",
548
            out.output
549
        );
550
    }
551
}
552
449 553
// ─────────────────────────────────────────────────── what actually ships
450 554
451 555
#[test]

This page updates live while a promote is in flight · changelog