Name the next capabilities to harvest from the Grok workspace

c812b8e016b1 · AtlantisPleb · · parent d474c0045c44

Name the next capabilities to harvest from the Grok workspace

The first two coder plugins came from Grok's session lineage and the
loop they close is proven, so the question becomes what to harvest
next. This survey reads the reconstructed Grok CLI workspace against
the Terminal-Bench task classes the Gym grades and names seven
targets in delta-per-effort order — repo_map from the tree-sitter
codebase graph, gitignore-aware repo_tree, session_search,
git_facts on gitoxide, a pure test_report parser, patch_check, and
token_count — plus the one host change most of them share (a
parameterized workspace mount), and the refusals: lifecycle features,
network reach, LSP, and media stay out of the sandbox on purpose.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01E1mRkPGYmTVvMKqAzmQvy5
Co-Authored-By
Claude Fable 5 <noreply@anthropic.com>

Deploy story

What this commit did to the running system — joined from the forge receipt chain, the part a commit page elsewhere cannot show.

pushed
by user · WAL seq 410 · 2026-08-25T15:45:59.903450Z

Changed files

  • added docs/2026-08-25-plugin-harvest-targets.md

Diff

1 file changed, +186 -0

docs/2026-08-25-plugin-harvest-targets.md added +186

@@ -0,0 +1,186 @@

1
# Plugin harvest targets for the coder
2
3
**Date:** 2026-08-25
4
**Status:** Survey and backlog. Nothing here is committed work until it has an
5
issue and a Gym delta.
6
**Method:** direct reading of the reconstructed Grok CLI Rust workspace — the
7
`grok-build` clone in the workspace's external reference area,
8
`projects/repos/grok-build`, about 80 crates under `crates/codegen/` —,
9
the teardown of that codebase in the `openagents` monorepo
10
(`docs/teardowns/2026-07-15-grok-build-teardown.md`), the plugin model
11
assessment (`docs/2026-08-24-triage-and-plugin-model-assessment.md`), the
12
tool zero-base audit (`docs/2026-08-23-agent-tools-zero-base.md`), and the
13
Harbor Terminal-Bench plan (`docs/2026-08-24-harbor-terminal-bench-plan.md`).
14
15
## Why this document exists
16
17
The first two working coder plugins — `foreign_sessions` and
18
`read_conversation`, both live in the capability catalog today — were
19
harvested from Grok's session lineage: Grok ships `xai-grok-active-sessions`
20
and `xai-grok-session-search` crates for exactly the discover-and-read work
21
we rebuilt as sandboxed WASM guests. The harvest worked: one spoken request
22
in `openagents coder` now discovers, digest-verifies, loads, and runs both
23
plugins with no improvised shell in the loop.
24
25
Grok's workspace holds more of the same shape. This document names the next
26
targets, ties each to the Terminal-Bench task classes the Gym grades the
27
coder on, and says what each needs from the plugin host. The Gym rule from
28
the Harbor plan applies to every entry: a plugin lands with a before-and-after
29
suite score on the same recipe, and its ATIF-stamped `tool.ran` steps make
30
the delta attributable to the exact digest. A plugin that moves no score is
31
questioned.
32
33
## What makes a good harvest
34
35
The plugin model admits a narrow shape, on purpose:
36
37
- **Read-only or pure.** The sandbox grants read-only mounts and nothing
38
  else. Edits, writes, and process execution stay in the coder's core tools,
39
  where the permission profile governs them.
40
- **One-shot and bounded.** One packet in, one packet out, under a time and
41
  memory bound. Daemons, watchers, and anything that must outlive a call
42
  (`xai-fsnotify`, background tasks) do not fit until the host grows a
43
  lifecycle for them.
44
- **Typed and honest.** Input and output schemas in the manifest, and output
45
  that names its own truncation the way `read_conversation` reports
46
  `tail_only` and `dropped_leading_turns`.
47
- **Worth a catalog line.** The capability tool carries each installed
48
  plugin's name and first sentence in its standing description, capped at
49
  twelve. A plugin competes for that space with everything else installed.
50
51
Grok links these capabilities into one native process. We deliberately do
52
not: the WASM boundary is what makes a harvested capability reviewable,
53
priceable, and safe to auto-approve from a digest-pinned catalog.
54
55
## Targets from the Grok workspace
56
57
Ordered by expected Gym delta per unit of work.
58
59
### 1. `repo_map` — the codebase graph
60
61
**Source:** `xai-codebase-graph` (tree-sitter queries, go-to-definition,
62
go-to-references, full and incremental indexing).
63
64
The single largest gap between the coder and the agents it benchmarks
65
against is orientation speed on an unfamiliar repository. Terminal-Bench
66
fix-the-bug and implement-the-feature tasks begin with minutes of `ls`,
67
`grep`, and dead-end file reads. A `repo_map` plugin takes a workspace
68
mount and returns a bounded structural map: directories that matter, public
69
symbols per file, definition sites for a named symbol, reference counts.
70
Tree-sitter compiles to WASM cleanly, which is why this crate is the
71
highest-confidence port in the list. Start with the two or three grammars
72
the Gym suites actually hit (Python, TypeScript, Rust) rather than Grok's
73
full language set.
74
75
**Needs from the host:** a workspace mount. Today's manifests declare fixed
76
home-relative paths (`~/.claude`); this plugin needs the manifest to declare
77
a mount the host resolves to the session's working directory. That
78
parameterized mount is the one host change this whole document asks for, and
79
targets 2, 4, 5, and 6 reuse it.
80
81
### 2. `repo_tree` — gitignore-aware listing
82
83
**Source:** `xai-grok-tools/src/gitignore.rs`, `xai-file-utils`,
84
`xai-fuzzy-file-search`.
85
86
The first thing every benchmark run does is look around, and the coder does
87
it today with raw `ls` and `find` — unfiltered, unbounded, and re-sent to
88
the model every round it stays in context. A `repo_tree` plugin returns a
89
gitignore-filtered tree with per-entry size and kind, bounded by depth and
90
entry ceilings, plus a fuzzy name lookup (`"authcontroller"` finds
91
`auth_controller.ex`) so the model stops spending rounds guessing paths.
92
Pure listing over the workspace mount; small; probably the best
93
delta-per-effort in the list.
94
95
### 3. `session_search` — search what any agent already learned
96
97
**Source:** `xai-grok-session-search` (the other half of the lineage
98
`foreign_sessions` came from), `xai-sqlite-journal` for Grok's own store.
99
100
`read_conversation` reads one session; this searches all of them.
101
Terminal-Bench aside, this is the memory primitive the fleet keeps
102
reinventing: "have I — or any agent on this machine — hit this error
103
before?" Input is a query string and the same source/cwd filters the
104
scanner takes; output is matching sessions with bounded surrounding
105
context per hit. Same two mounts the existing pair already declares, so it
106
ships with no host changes at all. Plain substring and identifier matching
107
first; ranking can come later without a schema change.
108
109
### 4. `git_facts` — structured status, log, and blame
110
111
**Source:** `xai-gix-status` (gitoxide status), `xai-hunk-tracker`
112
(hunk-level change tracking), `xai-fast-worktree` for the read side.
113
114
The coder answers "what changed?" by shelling `git` and re-reading prose.
115
Benchmark tasks that hand the agent a dirty repository — and the coder's own
116
verify-before-claim discipline — want typed answers: current branch, staged
117
and unstaged paths with hunk counts, the last N commits touching a path,
118
who last changed a line. Gitoxide is pure Rust and compiles to WASM;
119
`.git` is just files under the workspace mount. Read-only by construction —
120
commits and pushes stay in core, where the push guards live.
121
122
### 5. `test_report` — parse the test run the shell just did
123
124
**Source:** none — this one is ours, but it earns its place here because
125
Grok's `tool_taxonomy` treats "run" and "understand the run" as one native
126
step, and a sandboxed coder cannot. The shell tool runs `pytest`, `cargo
127
test`, `mix test`, or `jest`; this pure plugin parses the captured output
128
into typed failures: file, test name, assertion, the relevant traceback
129
lines. Terminal-Bench grades on making tests pass, and today the model
130
re-reads the same 300-line dump every round it iterates. A pure text-in,
131
JSON-out plugin, no mounts at all — the cheapest kind the ABI supports —
132
and likely the largest single Gym delta on the fix-the-tests task class.
133
134
### 6. `patch_check` — validate a diff before claiming it
135
136
**Source:** `xai-hunk-tracker`, and Grok's edit-tool normalization
137
(`xai-grok-tools/src/normalization.rs`).
138
139
A pure plugin that takes a unified diff plus the target file's current
140
content and answers: does this hunk apply, where does it drift, what would
141
the file look like after. The coder's edit loop occasionally emits patches
142
that no longer apply after an earlier edit; today that surfaces as a shell
143
round-trip and a confused retry. Validation as a capability keeps the write
144
itself in core while making the check free.
145
146
### 7. `token_count` — budget arithmetic for context planning
147
148
**Source:** `xai-token-estimation`.
149
150
Pure computation: text or file-listing in, token estimates per item out.
151
Long Gym runs die of context exhaustion mid-task; a model that can ask
152
"which of these five files fits my remaining budget" plans instead of
153
truncating. Pairs with `xai-compaction-transcript`'s idea — a
154
`compact_notes` plugin that reduces a long tool transcript to a bounded
155
structured summary — but token counting alone is an afternoon of work and
156
immediately measurable.
157
158
## Explicitly not harvested
159
160
Named so the next reader does not re-derive the refusals:
161
162
- **Subagents, background tasks, worktree checkpoints, rewind**
163
  (`xai-workflow`, `xai-fast-worktree`'s write side, section 9 of the
164
  teardown): lifecycle features, not one-shot capabilities. The coder's
165
  delegation and thread lanes own this ground.
166
- **Web search and fetch:** need host allowlists, which the approval
167
  posture currently auto-approves nothing for. A capability with network
168
  reach is a different trust conversation, deliberately deferred.
169
- **LSP integration** (`ToolKind::Lsp`): a long-lived server per language,
170
  wrong shape for the sandbox. `repo_map` covers the read-only half of what
171
  the benchmarks need from it.
172
- **Media generation, deploy, voice** (`ImageGen`, `DeployApp`,
173
  `xai-grok-voice`): not coding-agent work; no Gym suite grades them.
174
- **Grok's plugin marketplace** (`xai-grok-plugin-marketplace`): the
175
  registry lane already exists here (forge-hosted, issue #206 closed);
176
  what to harvest is their manifest fields, not their store.
177
178
## Sequencing
179
180
`repo_tree` and `test_report` first — no host changes, small crates, both
181
sit directly under the Terminal-Bench task classes the Gym already grades.
182
The parameterized workspace mount lands with `repo_tree`'s successor
183
`repo_map`, and `git_facts` follows it on the same mount. `session_search`
184
can land any time; it needs nothing new. Every landing follows the Gym
185
rule: same recipe, score before, score after, delta attributed to the
186
digest.

This page updates live while a promote is in flight · changelog