Stream repository imports and publish CLI docs

575b99f108e8 · AtlantisPleb · · parent 521c208d8ee8

Stream repository imports and publish CLI docs

Publish the complete CLI and GitHub import guide on the docs site, support bounded large-repository transfers, and use GitHub-shaped clone URLs.

Changelog: Repository imports now stream large Git bundles with bounded limits, the CLI supports headless device login, and clone URLs use /owner/repository.git.

Changelog-Category: feature

Changelog-Visibility: public
Changelog
Repository imports now stream large Git bundles with bounded limits, the CLI supports headless device login, and clone URLs use /owner/repository.git.
Changelog-Category
feature
Changelog-Visibility
public

Deploy story

What this commit did to the running system — joined from the forge receipt chain, the part a commit page elsewhere cannot show.

Not deployed through the forge lane

No push, promotion, build, or deploy receipt references this commit (receipts are scanned over a bounded recent window). Changes shipped by full node replacement carry their proof in the release gate receipt instead.

Changed files

  • modified docs/openagents-cli/command-reference.md
  • modified docs/openagents-cli/create-repository.md
  • modified docs/openagents-cli/git.md
  • modified docs/openagents-cli/import-github.md
  • modified docs/openagents-cli/index.md
  • modified docs/openagents-cli/install.md
  • modified docs/repository-creation-and-openagents-cli-spec.md
  • modified docs/repository-creation-cli-implementation-roadmap.md
  • modified lib/openagents/forge.ex
  • modified lib/openagents/forge/git_http.ex
  • modified lib/openagents/forge/sync.ex
  • modified lib/openagents/forge/wal.ex
  • modified lib/openagents/forge/wal/gcs.ex
  • modified lib/openagents/forge/wal/local.ex
  • modified lib/openagents/repositories/importer.ex
  • modified lib/openagents_web/controllers/repository_json.ex
  • modified lib/openagents_web/docs_catalog.ex
  • modified lib/openagents_web/live/repository_index_live.ex
  • modified lib/openagents_web/repository_access.ex
  • modified lib/openagents_web/route_authority.ex
  • modified lib/openagents_web/router.ex
  • modified priv/docs/api-tokens.md
  • modified priv/docs/browsing-code.md
  • added priv/docs/cli-command-reference.md
  • added priv/docs/clone-push-pull.md
  • added priv/docs/create-repository.md
  • added priv/docs/import-github.md
  • added priv/docs/install-cli.md
  • added priv/docs/openagents-cli.md
  • modified priv/docs/rest-api.md
  • modified priv/docs/signing-in.md
  • modified priv/docs/welcome.md
  • modified test/openagents/forge/sync_test.exs
  • modified test/openagents/forge/wal_test.exs
  • modified test/openagents/repositories/cli_local_e2e_test.exs
  • modified test/openagents/repositories/provisioner_test.exs
  • modified test/openagents_web/controllers/repository_controller_test.exs
  • modified test/openagents_web/docs_catalog_test.exs
  • modified test/openagents_web/live/repository_live_test.exs
  • modified test/openagents_web/route_authority_test.exs

Diff

40 files changed, +1511 -73

docs/openagents-cli/command-reference.md modified +4 -1

@@ -9,6 +9,10 @@ openagents <subcommand> [flags]

9 9
Run `openagents <command> --help` for the reference installed with your CLI
10 10
version.
11 11
12
For one command without a global installation, replace the `openagents` prefix
13
with `npx --yes @openagentsinc/cli@latest`. Do not run `auth setup-git` through
14
`npx`; install the CLI globally before saving a persistent Git helper.
15
12 16
## Global flags
13 17
14 18
| Flag | Description |

@@ -142,4 +146,3 @@ In a noninteractive process:

142 146
This release does not provide `repo delete`, `repo mirror`, pull-request,
143 147
ruleset, SSH-key, generic API, or self-update commands. Use only commands shown
144 148
by the installed version's `--help` output.
145
docs/openagents-cli/create-repository.md modified +9 -3

@@ -44,6 +44,12 @@ The CLI waits up to 300 seconds for provisioning by default. Pass

44 44
`--wait-timeout 0` to return after the server accepts the durable request. The
45 45
repository continues provisioning on the server.
46 46
47
Run one create command without a global installation:
48
49
```sh
50
npx --yes @openagentsinc/cli@latest repo create my-project
51
```
52
47 53
## Attach an existing local project
48 54
49 55
Create the remote repository and attach it to an existing Git worktree:

@@ -68,7 +74,8 @@ URL. If remote attachment fails, the remote repository still exists.

68 74
69 75
## Push the first commit
70 76
71
For an existing local project:
77
Install the CLI globally before configuring persistent Git authentication. For
78
an existing local project:
72 79
73 80
```sh
74 81
openagents auth setup-git --local

@@ -81,7 +88,7 @@ To start from an empty directory:

81 88
git init my-project
82 89
cd my-project
83 90
git branch -M main
84
git remote add origin https://openagents.com/git/OWNER/my-project.git
91
git remote add origin https://openagents.com/OWNER/my-project.git
85 92
# Add files, then commit them.
86 93
openagents auth setup-git --local
87 94
git push -u origin main

@@ -119,4 +126,3 @@ are not available yet.

119 126
- [Clone, push, and pull](git.md)
120 127
- [Import a GitHub repository](import-github.md)
121 128
- [CLI command reference](command-reference.md)
122
docs/openagents-cli/git.md modified +13 -4

@@ -15,6 +15,12 @@ Choose a destination directory:

15 15
openagents repo clone OWNER/REPOSITORY ./local-directory
16 16
```
17 17
18
Run one clone without a global installation:
19
20
```sh
21
npx --yes @openagentsinc/cli@latest repo clone OWNER/REPOSITORY
22
```
23
18 24
The command uses the server-provided clone URL and invokes Git without putting
19 25
the token in the URL or process arguments. For the clone operation, it scopes
20 26
the OpenAgents credential helper to the selected API origin.

@@ -24,19 +30,23 @@ the OpenAgents credential helper to the selected API origin.

24 30
Copy the HTTPS URL from the repository page, then run:
25 31
26 32
```sh
27
git clone https://openagents.com/git/OWNER/REPOSITORY.git
33
git clone https://openagents.com/OWNER/REPOSITORY.git
28 34
```
29 35
30 36
Public repositories support anonymous clone and fetch. Private repositories
31 37
require an authorized credential.
32 38
33
Before you use standard Git with a private repository, configure the helper:
39
Before you use standard Git with a private repository, install the CLI globally
40
and configure the helper:
34 41
35 42
```sh
36 43
cd existing-worktree
37 44
openagents auth setup-git --local
38 45
```
39 46
47
Do not run `auth setup-git` through `npx`. The saved helper calls
48
`openagents`, and the temporary executable is unavailable after `npx` exits.
49
40 50
Use global setup only when you want every local repository to use the helper
41 51
for the selected OpenAgents origin:
42 52

@@ -67,7 +77,7 @@ From a worktree whose `origin` is an exact OpenAgents clone URL:

67 77
openagents repo view
68 78
```
69 79
70
The CLI accepts only `/git/OWNER/REPOSITORY.git` on the selected API origin. It
80
The CLI accepts only `/OWNER/REPOSITORY.git` on the selected API origin. It
71 81
does not infer authority from an arbitrary URL that resembles a repository
72 82
path. Override inference explicitly when needed:
73 83

@@ -93,4 +103,3 @@ not available yet.

93 103
94 104
- [Install and authenticate the CLI](install.md)
95 105
- [CLI command reference](command-reference.md)
96
docs/openagents-cli/import-github.md modified +49

@@ -66,6 +66,55 @@ The CLI waits up to 300 seconds by default. Pass `--wait-timeout 0` to return

66 66
after the server accepts the durable import. A client timeout does not cancel
67 67
the server-side import.
68 68
69
Run one import without a global installation:
70
71
```sh
72
npx --yes @openagentsinc/cli@latest repo import OWNER/REPOSITORY
73
```
74
75
Pin the package version for a reproducible qualification run:
76
77
```sh
78
npx --yes @openagentsinc/cli@0.1.3 \
79
  --profile staging \
80
  repo import OWNER/REPOSITORY \
81
  --private \
82
  --wait-timeout 300
83
```
84
85
In a headless agent process, `auth login` prints the complete approval URL and
86
user code, then waits. The agent can surface both values while you approve the
87
request in any browser. Install the CLI globally before you run
88
`auth setup-git`; a saved Git helper cannot call the temporary executable after
89
`npx` exits.
90
91
## Import a large repository
92
93
OpenAgents keeps large Git bundles on disk and streams them to and from the
94
durable forge WAL in 1 MiB chunks. The application does not read the complete
95
bundle into the BEAM heap. The default server limits allow a bundle up to 20
96
GiB and an import to run for up to six hours.
97
98
The CLI's `--wait-timeout` controls only how long that client waits. It does
99
not change or cancel the server import. For a large repository, accept the
100
operation immediately and check it separately:
101
102
```sh
103
openagents repo import OWNER/REPOSITORY --wait-timeout 0
104
openagents repo view OWNER/REPOSITORY
105
```
106
107
The repository page reports queued, copying, storing, ready, and failed states.
108
Server logs record every stage and the bundle byte count. A bundle over the
109
server limit fails with `import_too_large`; an operation over the server time
110
limit fails with `import_timeout`.
111
112
Large imports still need enough temporary disk for the fetched Git objects and
113
the bundle. Git LFS objects remain outside the import. Start an environment's
114
first import with a small repository so you can qualify authentication,
115
lifecycle reporting, storage, and clone behavior before spending the time and
116
bandwidth of a large transfer.
117
69 118
## Verify the first production import
70 119
71 120
Use a small private GitHub repository for the first production import. Include
docs/openagents-cli/index.md modified +14

@@ -7,6 +7,20 @@ interface and leaves Git data transfer to standard Git.

7 7
The CLI is available from npm. OpenAgents qualified repository imports on
8 8
staging before deploying the same server revision to the production fleet.
9 9
10
The same guides are published on the [OpenAgents documentation
11
site](https://openagents.com/docs/openagents-cli).
12
13
Install the CLI globally for regular use, or run a one-time command through
14
`npx`:
15
16
```sh
17
npm install --global @openagentsinc/cli
18
npx --yes @openagentsinc/cli@latest --version
19
```
20
21
Do not configure a persistent Git credential helper through `npx`. Install the
22
CLI globally before you run `openagents auth setup-git`.
23
10 24
## What you can do
11 25
12 26
The first release lets you:
docs/openagents-cli/install.md modified +35 -2

@@ -1,7 +1,7 @@

1 1
# Install the OpenAgents CLI
2 2
3 3
The npm package is `@openagentsinc/cli`, and it installs the `openagents`
4
command. The package requires Node.js 24.15 or later.
4
command. The package requires Node.js 20 or later.
5 5
6 6
## Install with npm
7 7

@@ -23,6 +23,33 @@ npm install --global @openagentsinc/cli@latest

23 23
24 24
The CLI does not include an `openagents update` command.
25 25
26
## Run one command with npx
27
28
Use `npx` when you want to run one command without a global installation:
29
30
```sh
31
npx --yes @openagentsinc/cli@latest --version
32
npx --yes @openagentsinc/cli@latest repo list
33
```
34
35
Pin a version for a reproducible run:
36
37
```sh
38
npx --yes @openagentsinc/cli@0.1.3 --version
39
```
40
41
Place all CLI arguments after the package name:
42
43
```sh
44
npx --yes @openagentsinc/cli@latest --profile staging auth status
45
npx --yes @openagentsinc/cli@latest repo import OWNER/REPOSITORY
46
```
47
48
Do not run `auth setup-git` through `npx`. The command writes a persistent Git
49
helper configuration that calls `openagents`, but the temporary executable is
50
unavailable after `npx` exits. Install the CLI globally before you configure a
51
local or global Git helper.
52
26 53
## Run the CLI from source
27 54
28 55
From the root of the `openagents` monorepo:

@@ -44,6 +71,12 @@ The CLI prints a verification URL and user code, opens the URL when your

44 71
operating system supports it, and waits for approval. Complete the flow with
45 72
the GitHub account connected to OpenAgents.
46 73
74
In a headless or noninteractive agent process, the command prints the complete
75
authorization URL and user code to standard error and waits. Have the agent
76
surface both values to you. Open the URL in any browser, review the request,
77
and approve it. The waiting command then stores the OpenAgents token and
78
continues; the agent never receives your GitHub token.
79
47 80
The CLI stores the resulting `oa_pat_` token for the selected API origin:
48 81
49 82
- On macOS, it uses Keychain through the `security` command.

@@ -127,7 +160,7 @@ The file accepts `profile` or `api_url` and never stores credentials.

127 160
128 161
## Configure Git authentication
129 162
130
Configure only the current Git repository:
163
After you install the CLI globally, configure only the current Git repository:
131 164
132 165
```sh
133 166
openagents auth setup-git --local
docs/repository-creation-and-openagents-cli-spec.md modified +3 -4

@@ -390,7 +390,7 @@ The new surface extends the bounded GitHub-shaped API under `/api/v3`.

390 390
| `GET /api/v3/user/repos` | Authenticated API | List repositories visible to the caller, including private repositories |
391 391
| `GET /api/v3/repos/{owner}/{repo}` | Optional API principal | Return a public repository or a repository visible to the principal |
392 392
| `GET /api/v3/repository-imports/{id}` | Authenticated API | Return bounded status for an import owned by the caller |
393
| Git smart HTTP under `/git/{owner}/{repo}.git` | Public read or authenticated Git principal | Clone, fetch, push, and pull |
393
| Git smart HTTP under `/{owner}/{repo}.git` | Public read or authenticated Git principal | Clone, fetch, push, and pull |
394 394
395 395
Do not add an endpoint that accepts an arbitrary owner string. The user route
396 396
derives the GitHub user namespace from the authenticated principal. The

@@ -499,7 +499,7 @@ repository is already `ready`.

499 499
  "description": "An optional description",
500 500
  "default_branch": "main",
501 501
  "lifecycle_state": "ready",
502
  "clone_url": "https://openagents.com/git/octavia/my-project.git",
502
  "clone_url": "https://openagents.com/octavia/my-project.git",
503 503
  "html_url": "https://openagents.com/octavia/my-project",
504 504
  "permissions": {
505 505
    "admin": true,

@@ -662,8 +662,7 @@ Resolve Git paths through the repository database:

662 662
- A private, missing, suspended, failed, or unauthorized repository returns an
663 663
  indistinguishable refusal where the transport permits it.
664 664
665
Change the canonical Git path from a name-only route to
666
`/git/{owner}/{repo}.git`. Keep a tested compatibility alias for the existing
665
Use `/{owner}/{repo}.git` as the canonical Git path. Keep a tested compatibility alias for the existing
667 666
`/git/openagents.com.git` remote until the canonical repository cutover plan
668 667
retires it.
669 668
docs/repository-creation-cli-implementation-roadmap.md modified +1 -1

@@ -275,7 +275,7 @@ and assertions that neither side receives a synchronization call.

275 275
276 276
**Owner:** `openagents.com`.
277 277
278
1. Change the Git route to `/git/{owner}/{repo}.git` while keeping the temporary
278
1. Use the canonical `/{owner}/{repo}.git` route while keeping the temporary
279 279
   `/git/openagents.com.git` compatibility route required by the forge cutover
280 280
   contract.
281 281
2. Allow anonymous upload-pack only for a ready public repository.
lib/openagents/forge.ex modified +2 -2

@@ -2,7 +2,7 @@ defmodule OpenAgents.Forge do

2 2
  @moduledoc """
3 3
  Sarah's own git forge — the public API of the bounded context (audit A6).
4 4
5
  This app serves the forge at `openagents.com/git/<repo>.git` via stock git
5
  This app serves the forge at `openagents.com/<owner>/<repo>.git` via stock Git
6 6
  smart-HTTP (`OpenAgents.Forge.GitHTTP`), with the WAL in object storage as ref
7 7
  truth (`OpenAgents.Forge.WAL`, Continuity-shaped per audit A7), bare repos on
8 8
  the stateful partition as per-node cache (`OpenAgents.Forge.Repos` /

@@ -86,6 +86,6 @@ defmodule OpenAgents.Forge do

86 86
87 87
  @doc "The clone URL for a repo on this deployment."
88 88
  def clone_url(repo) do
89
    OpenAgentsWeb.Endpoint.url() <> "/git/#{repo}.git"
89
    OpenAgentsWeb.Endpoint.url() <> "/OpenAgentsInc/#{repo}.git"
90 90
  end
91 91
end
lib/openagents/forge/git_http.ex modified +16 -4

@@ -3,7 +3,7 @@ defmodule OpenAgents.Forge.GitHTTP do

3 3
  Git smart-HTTP v0, wrapping the stock git binary ("Spokes got that exactly
4 4
  right" — standard packfiles, upstream clients, no custom object format).
5 5
6
  Serves, under the mount point (`/git` in the router):
6
  Serves under the canonical `/:owner/:repo.git` mount point:
7 7
8 8
      GET  /:repo.git/info/refs?service=git-upload-pack|git-receive-pack
9 9
      POST /:repo.git/git-upload-pack

@@ -35,7 +35,7 @@ defmodule OpenAgents.Forge.GitHTTP do

35 35
36 36
  @impl true
37 37
  def call(conn, _opts) do
38
    case {conn.method, split_repo(conn.path_info)} do
38
    case {conn.method, split_repo(conn)} do
39 39
      {"GET", {:ok, owner, name, ["info", "refs"]}} ->
40 40
        advertise(conn, owner, name, first_query(conn, "service"))
41 41

@@ -137,7 +137,19 @@ defmodule OpenAgents.Forge.GitHTTP do

137 137
138 138
  # ── helpers ─────────────────────────────────────────────────────────────
139 139
140
  defp split_repo([segment | rest]) do
140
  defp split_repo(%Plug.Conn{
141
         path_params: %{"owner" => owner, "repo" => segment},
142
         path_info: [_owner, _repo | rest]
143
       }) do
144
    case strip_git_suffix(segment) do
145
      {:ok, name} -> {:ok, owner, name, rest}
146
      :error -> :error
147
    end
148
  end
149
150
  defp split_repo(%Plug.Conn{path_info: path_info}), do: split_legacy_repo(path_info)
151
152
  defp split_legacy_repo([segment | rest]) do
141 153
    case strip_git_suffix(segment) do
142 154
      {:ok, "openagents.com"} -> {:ok, "OpenAgentsInc", "openagents.com", rest}
143 155
      {:ok, name} -> {:ok, nil, name, rest}

@@ -145,7 +157,7 @@ defmodule OpenAgents.Forge.GitHTTP do

145 157
    end
146 158
  end
147 159
148
  defp split_repo(_), do: :error
160
  defp split_legacy_repo(_), do: :error
149 161
150 162
  defp split_namespaced_repo(owner, [segment | rest]) do
151 163
    case strip_git_suffix(segment) do
lib/openagents/forge/sync.ex modified +10 -10

@@ -49,15 +49,15 @@ defmodule OpenAgents.Forge.Sync do

49 49
  end
50 50
51 51
  defp apply_entry!(repo, %{"seq" => seq, "object" => object} = entry) do
52
    {:ok, payload} = WAL.get_entry(repo, object)
53 52
    path = Repos.bare_path(repo)
54 53
55 54
    case entry["format"] || "receive_pack" do
56 55
      "receive_pack" ->
56
        {:ok, payload} = WAL.get_entry(repo, object)
57 57
        {_output, 0} = run_receive_pack(path, payload)
58 58
59 59
      "git_bundle" ->
60
        :ok = unbundle(path, payload)
60
        :ok = unbundle_entry(repo, path, object)
61 61
62 62
      "empty_import" ->
63 63
        :ok

@@ -77,20 +77,20 @@ defmodule OpenAgents.Forge.Sync do

77 77
    end
78 78
  end
79 79
80
  defp unbundle(path, payload) do
80
  defp unbundle_entry(repo, path, object) do
81 81
    temporary_path =
82 82
      Path.join(
83
        System.tmp_dir!(),
83
        Application.get_env(:openagents, :repository_import_temp_dir, System.tmp_dir!()),
84 84
        "openagents-import-#{System.unique_integer([:positive, :monotonic])}.bundle"
85 85
      )
86 86
87 87
    try do
88
      File.write!(temporary_path, payload, [:binary, :exclusive])
89
      File.chmod!(temporary_path, 0o600)
90
91
      case Repos.git(path, ["bundle", "unbundle", temporary_path]) do
92
        {_output, 0} -> :ok
93
        {_output, _status} -> raise "repository bundle could not be materialized"
88
      with :ok <- WAL.get_entry_file(repo, object, temporary_path),
89
           :ok <- File.chmod(temporary_path, 0o600) do
90
        case Repos.git(path, ["bundle", "unbundle", temporary_path]) do
91
          {_output, 0} -> :ok
92
          {_output, _status} -> raise "repository bundle could not be materialized"
93
        end
94 94
      end
95 95
    after
96 96
      File.rm(temporary_path)
lib/openagents/forge/wal.ex modified +64

@@ -44,7 +44,11 @@ defmodule OpenAgents.Forge.WAL do

44 44
              {:ok, generation} | {:error, :cas_conflict} | {:error, term}
45 45
  @callback put_entry(repo, seq :: non_neg_integer(), payload :: binary()) ::
46 46
              {:ok, object_key :: String.t()} | {:error, term}
47
  @callback put_entry_file(repo, seq :: non_neg_integer(), path :: String.t()) ::
48
              {:ok, object_key :: String.t()} | {:error, term}
47 49
  @callback get_entry(repo, object_key :: String.t()) :: {:ok, binary()} | {:error, term}
50
  @callback get_entry_file(repo, object_key :: String.t(), path :: String.t()) ::
51
              :ok | {:error, term}
48 52
  @callback put_object(repo, object_key :: String.t(), payload :: binary()) ::
49 53
              {:ok, String.t()} | {:error, term}
50 54

@@ -91,6 +95,25 @@ defmodule OpenAgents.Forge.WAL do

91 95
    with :ok <- validate_repo(repo), do: adapter().put_entry(repo, seq, payload)
92 96
  end
93 97
98
  @doc """
99
  Stream one immutable WAL entry from `path` through the configured adapter.
100
101
  The adapter derives the same content-addressed key as `put_entry/3` without
102
  loading the complete file into the BEAM heap.
103
  """
104
  @spec put_entry_file(repo, non_neg_integer(), String.t()) ::
105
          {:ok, String.t()} | {:error, term}
106
  def put_entry_file(repo, seq, path)
107
      when is_integer(seq) and seq >= 0 and is_binary(path) do
108
    with :ok <- validate_repo(repo),
109
         {:ok, %File.Stat{type: :regular}} <- File.stat(path) do
110
      adapter().put_entry_file(repo, seq, path)
111
    else
112
      {:ok, _not_regular} -> {:error, :invalid_entry_file}
113
      {:error, reason} -> {:error, reason}
114
    end
115
  end
116
94 117
  @doc """
95 118
  Fetch a previously stored WAL entry payload for `repo` by its object key.
96 119
  """

@@ -102,6 +125,16 @@ defmodule OpenAgents.Forge.WAL do

102 125
    end
103 126
  end
104 127
128
  @doc "Stream a WAL entry into `path` without retaining its complete body in memory."
129
  @spec get_entry_file(repo, String.t(), String.t()) :: :ok | {:error, term}
130
  def get_entry_file(repo, object_key, path)
131
      when is_binary(object_key) and is_binary(path) do
132
    with :ok <- validate_repo(repo),
133
         :ok <- validate_entry_key(object_key) do
134
      adapter().get_entry_file(repo, object_key, path)
135
    end
136
  end
137
105 138
  @doc """
106 139
  Store a named artifact blob alongside the WAL (P6, #123): built beam tars
107 140
  land here so a replaced node — whose local artifact cache is empty — can

@@ -203,6 +236,37 @@ defmodule OpenAgents.Forge.WAL do

203 236
    "entries/" <> String.pad_leading(Integer.to_string(seq), 8, "0") <> "-" <> digest
204 237
  end
205 238
239
  @doc "Derive the immutable entry key for a file without loading the file into memory."
240
  @spec entry_key_file(non_neg_integer(), String.t()) :: {:ok, String.t()} | {:error, term}
241
  def entry_key_file(seq, path) when is_integer(seq) and seq >= 0 and is_binary(path) do
242
    with {:ok, digest} <- file_digest(path) do
243
      {:ok, entry_key_from_digest(seq, digest)}
244
    end
245
  end
246
247
  defp file_digest(path) do
248
    try do
249
      digest =
250
        path
251
        |> File.stream!(1_048_576, [])
252
        |> Enum.reduce(:crypto.hash_init(:sha256), &:crypto.hash_update(&2, &1))
253
        |> :crypto.hash_final()
254
255
      {:ok, digest}
256
    rescue
257
      File.Error -> {:error, :entry_file_unavailable}
258
    end
259
  end
260
261
  defp entry_key_from_digest(seq, digest) do
262
    prefix =
263
      digest
264
      |> Base.encode16(case: :lower)
265
      |> binary_part(0, 12)
266
267
    "entries/" <> String.pad_leading(Integer.to_string(seq), 8, "0") <> "-" <> prefix
268
  end
269
206 270
  @doc """
207 271
  Validate a repository name (`[a-z0-9][a-z0-9_-]*`).
208 272
  """
lib/openagents/forge/wal/gcs.ex modified +90

@@ -26,6 +26,8 @@ defmodule OpenAgents.Forge.WAL.Gcs do

26 26
  @metadata_token_url "http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token"
27 27
  @token_cache_key {__MODULE__, :token}
28 28
  @token_expiry_margin_seconds 60
29
  @stream_chunk_bytes 1_048_576
30
  @default_stream_timeout_ms 6 * 60 * 60 * 1_000
29 31
30 32
  @impl WAL
31 33
  def read_index(repo) do

@@ -97,6 +99,17 @@ defmodule OpenAgents.Forge.WAL.Gcs do

97 99
    end
98 100
  end
99 101
102
  @impl WAL
103
  def put_entry_file(repo, seq, path) when is_integer(seq) and seq >= 0 do
104
    with {:ok, bucket} <- bucket(),
105
         {:ok, key} <- WAL.entry_key_file(seq, path),
106
         {:ok, size} <- regular_file_size(path),
107
         {:ok, token} <- token(),
108
         :ok <- upload_file(bucket, object_name(repo, key), path, size, token) do
109
      {:ok, key}
110
    end
111
  end
112
100 113
  @impl WAL
101 114
  def put_object(repo, object_key, payload) when is_binary(payload) do
102 115
    with {:ok, bucket} <- bucket(),

@@ -124,6 +137,14 @@ defmodule OpenAgents.Forge.WAL.Gcs do

124 137
    end
125 138
  end
126 139
140
  @impl WAL
141
  def get_entry_file(repo, object_key, path) when is_binary(object_key) and is_binary(path) do
142
    with {:ok, bucket} <- bucket(),
143
         {:ok, token} <- token() do
144
      download_file(bucket, object_name(repo, object_key), path, token)
145
    end
146
  end
147
127 148
  ## Object naming (public so it is testable without a live bucket)
128 149
129 150
  @doc """

@@ -179,6 +200,75 @@ defmodule OpenAgents.Forge.WAL.Gcs do

179 200
    end
180 201
  end
181 202
203
  defp upload_file(bucket, name, path, size, token) do
204
    url =
205
      @storage_base <>
206
        "/upload/storage/v1/b/#{URI.encode_www_form(bucket)}/o?" <>
207
        URI.encode_query(uploadType: "media", name: name)
208
209
    case Req.post(url,
210
           body: File.stream!(path, @stream_chunk_bytes, []),
211
           headers:
212
             auth_headers(token) ++
213
               [
214
                 {"content-type", "application/octet-stream"},
215
                 {"content-length", Integer.to_string(size)}
216
               ],
217
           receive_timeout: stream_timeout_ms(),
218
           retry: false
219
         ) do
220
      {:ok, %Req.Response{status: 200}} -> :ok
221
      {:ok, %Req.Response{status: status, body: body}} -> {:error, {:gcs_error, status, body}}
222
      {:error, reason} -> {:error, reason}
223
    end
224
  end
225
226
  defp download_file(bucket, name, path, token) do
227
    temporary = path <> ".tmp." <> Integer.to_string(System.unique_integer([:positive]))
228
229
    result =
230
      with :ok <- File.mkdir_p(Path.dirname(path)) do
231
        case Req.get(object_url(bucket, name) <> "?alt=media",
232
               headers: auth_headers(token),
233
               into: File.stream!(temporary, @stream_chunk_bytes, [:write, :binary]),
234
               decode_body: false,
235
               receive_timeout: stream_timeout_ms(),
236
               retry: false
237
             ) do
238
          {:ok, %Req.Response{status: 200}} ->
239
            File.rename(temporary, path)
240
241
          {:ok, %Req.Response{status: 404}} ->
242
            {:error, :not_found}
243
244
          {:ok, %Req.Response{status: status, body: body}} ->
245
            {:error, {:gcs_error, status, body}}
246
247
          {:error, reason} ->
248
            {:error, reason}
249
        end
250
      end
251
252
    if result != :ok, do: File.rm(temporary)
253
    result
254
  end
255
256
  defp regular_file_size(path) do
257
    case File.stat(path) do
258
      {:ok, %File.Stat{type: :regular, size: size}} -> {:ok, size}
259
      {:ok, _not_regular} -> {:error, :invalid_entry_file}
260
      {:error, reason} -> {:error, reason}
261
    end
262
  end
263
264
  defp stream_timeout_ms do
265
    Application.get_env(
266
      :openagents,
267
      :forge_wal_stream_timeout_ms,
268
      @default_stream_timeout_ms
269
    )
270
  end
271
182 272
  defp object_url(bucket, name) do
183 273
    @storage_base <>
184 274
      "/storage/v1/b/#{URI.encode_www_form(bucket)}/o/#{URI.encode_www_form(name)}"
lib/openagents/forge/wal/local.ex modified +44

@@ -47,6 +47,14 @@ defmodule OpenAgents.Forge.WAL.Local do

47 47
    end
48 48
  end
49 49
50
  @impl WAL
51
  def put_entry_file(repo, seq, source_path) when is_integer(seq) and seq >= 0 do
52
    with {:ok, key} <- WAL.entry_key_file(seq, source_path) do
53
      destination = Path.join(repo_dir(repo), key)
54
      copy_file_atomically(source_path, destination, key)
55
    end
56
  end
57
50 58
  @impl WAL
51 59
  def put_object(repo, object_key, payload) when is_binary(payload) do
52 60
    path = Path.join(repo_dir(repo), object_key)

@@ -66,6 +74,28 @@ defmodule OpenAgents.Forge.WAL.Local do

66 74
    end
67 75
  end
68 76
77
  @impl WAL
78
  def get_entry_file(repo, object_key, destination_path) do
79
    source_path = Path.join(repo_dir(repo), object_key)
80
81
    case File.stat(source_path) do
82
      {:ok, %File.Stat{type: :regular}} ->
83
        case copy_file_atomically(source_path, destination_path, destination_path) do
84
          {:ok, _destination} -> :ok
85
          {:error, reason} -> {:error, reason}
86
        end
87
88
      {:error, :enoent} ->
89
        {:error, :not_found}
90
91
      {:ok, _not_regular} ->
92
        {:error, :not_found}
93
94
      {:error, reason} ->
95
        {:error, reason}
96
    end
97
  end
98
69 99
  ## Internal
70 100
71 101
  defp do_cas(repo, :none, index) do

@@ -99,6 +129,20 @@ defmodule OpenAgents.Forge.WAL.Local do

99 129
    end
100 130
  end
101 131
132
  defp copy_file_atomically(source, destination, result) do
133
    temporary = destination <> ".tmp." <> Integer.to_string(System.unique_integer([:positive]))
134
135
    with :ok <- File.mkdir_p(Path.dirname(destination)),
136
         {:ok, _bytes} <- File.copy(source, temporary),
137
         :ok <- File.rename(temporary, destination) do
138
      {:ok, result}
139
    else
140
      {:error, reason} ->
141
        File.rm(temporary)
142
        {:error, reason}
143
    end
144
  end
145
102 146
  defp decode_index(raw) do
103 147
    case Jason.decode(raw) do
104 148
      {:ok, %{"generation" => generation} = index} when is_integer(generation) ->
lib/openagents/repositories/importer.ex modified +53 -7

@@ -8,7 +8,8 @@ defmodule OpenAgents.Repositories.Importer do

8 8
  alias OpenAgents.Repositories.{Repository, RepositoryImport}
9 9
10 10
  @maximum_append_attempts 3
11
  @default_import_timeout_ms 10 * 60 * 1_000
11
  @default_import_timeout_ms 6 * 60 * 60 * 1_000
12
  @default_maximum_bundle_bytes 20 * 1_024 * 1_024 * 1_024
12 13
13 14
  def import(%Repository{} = repository, options \\ []) do
14 15
    repository = Repo.preload(repository, [:created_by_user, :repository_import])

@@ -145,10 +146,11 @@ defmodule OpenAgents.Repositories.Importer do

145 146
           import_stage(repository, repository_import, "verify_snapshot", fn ->
146 147
             verify_snapshot(source_repository, repository_import)
147 148
           end),
148
         {:ok, payload, format} <-
149
         {:ok, payload, format, payload_bytes} <-
149 150
           import_stage(repository, repository_import, "create_payload", fn ->
150 151
             create_payload(source_repository, refs, temporary_directory)
151 152
           end),
153
         :ok <- log_payload_ready(repository, repository_import, payload_bytes),
152 154
         :ok <-
153 155
           import_stage(repository, repository_import, "append_wal", fn ->
154 156
             append_import(repository, repository_import, payload, format, refs, 0)

@@ -284,16 +286,26 @@ defmodule OpenAgents.Repositories.Importer do

284 286
  end
285 287
286 288
  defp create_payload(_source_repository, refs, _temporary_directory) when map_size(refs) == 0,
287
    do: {:ok, "", "empty_import"}
289
    do: {:ok, "", "empty_import", 0}
288 290
289 291
  defp create_payload(source_repository, _refs, temporary_directory) do
290 292
    bundle_path = Path.join(temporary_directory, "snapshot.bundle")
291 293
292 294
    case Repos.git(source_repository, ["bundle", "create", bundle_path, "--all"]) do
293 295
      {_output, 0} ->
294
        case File.read(bundle_path) do
295
          {:ok, payload} -> {:ok, payload, "git_bundle"}
296
          {:error, _reason} -> {:error, :bundle_unavailable}
296
        case File.stat(bundle_path) do
297
          {:ok, %File.Stat{type: :regular, size: size}} ->
298
            if size <= maximum_bundle_bytes() do
299
              {:ok, {:file, bundle_path}, "git_bundle", size}
300
            else
301
              {:error, :import_too_large}
302
            end
303
304
          {:ok, _not_regular} ->
305
            {:error, :bundle_unavailable}
306
307
          {:error, _reason} ->
308
            {:error, :bundle_unavailable}
297 309
        end
298 310
299 311
      {_output, _status} ->

@@ -332,7 +344,7 @@ defmodule OpenAgents.Repositories.Importer do

332 344
         :missing <- import_entry(index, repository_import.id),
333 345
         true <- WAL.refs(index) == %{} or {:error, :destination_not_empty},
334 346
         seq = WAL.next_seq(index),
335
         {:ok, object} <- WAL.put_entry(repository.storage_key, seq, payload),
347
         {:ok, object} <- put_payload(repository.storage_key, seq, payload),
336 348
         entry = %{
337 349
           "seq" => seq,
338 350
           "object" => object,

@@ -476,6 +488,7 @@ defmodule OpenAgents.Repositories.Importer do

476 488
  defp error_code(:github_scope_required), do: "github_scope_required"
477 489
  defp error_code(:github_token_missing), do: "github_connection_required"
478 490
  defp error_code(:import_timeout), do: "import_timeout"
491
  defp error_code(:import_too_large), do: "import_too_large"
479 492
  defp error_code(_reason), do: "import_failed"
480 493
481 494
  defp import_stage(repository, repository_import, stage, operation) do

@@ -494,6 +507,10 @@ defmodule OpenAgents.Repositories.Importer do

494 507
        log_stage(repository, repository_import, stage, "completed")
495 508
        result
496 509
510
      {:ok, _value, _metadata, _measurement} = result ->
511
        log_stage(repository, repository_import, stage, "completed")
512
        result
513
497 514
      {:error, reason} = result ->
498 515
        log_stage(repository, repository_import, stage, "failed", reason)
499 516
        result

@@ -519,6 +536,35 @@ defmodule OpenAgents.Repositories.Importer do

519 536
520 537
  defp diagnostic_error(_reason), do: " error_code=unexpected_error"
521 538
539
  defp put_payload(storage_key, seq, {:file, path}),
540
    do: WAL.put_entry_file(storage_key, seq, path)
541
542
  defp put_payload(storage_key, seq, payload) when is_binary(payload),
543
    do: WAL.put_entry(storage_key, seq, payload)
544
545
  defp log_payload_ready(repository, repository_import, bytes) do
546
    Logger.info(
547
      "repository_import_payload" <>
548
        " repository_id=#{repository.id}" <>
549
        " repository_import_id=#{repository_import.id}" <>
550
        " bytes=#{bytes}" <>
551
        " storage=streamed"
552
    )
553
554
    :ok
555
  end
556
557
  defp maximum_bundle_bytes do
558
    case Application.get_env(
559
           :openagents,
560
           :repository_import_max_bundle_bytes,
561
           @default_maximum_bundle_bytes
562
         ) do
563
      value when is_integer(value) and value > 0 -> value
564
      _invalid -> @default_maximum_bundle_bytes
565
    end
566
  end
567
522 568
  defp temporary_directory(import_id) do
523 569
    root = Application.get_env(:openagents, :repository_import_temp_dir, System.tmp_dir!())
524 570
    Path.join(root, "openagents-import-#{import_id}-#{System.unique_integer([:positive])}")
lib/openagents_web/controllers/repository_json.ex modified +1 -1

@@ -21,7 +21,7 @@ defmodule OpenAgentsWeb.RepositoryJSON do

21 21
      "default_branch" => repository.default_branch,
22 22
      "lifecycle_state" => repository.lifecycle_state,
23 23
      "provision_error_code" => repository.provision_error_code,
24
      "clone_url" => base_url <> "/git/#{owner}/#{repository.name}.git",
24
      "clone_url" => base_url <> "/#{owner}/#{repository.name}.git",
25 25
      "html_url" => base_url <> "/#{owner}/#{repository.name}",
26 26
      "permissions" => permissions,
27 27
      "created_at" => DateTime.to_iso8601(repository.inserted_at),
lib/openagents_web/docs_catalog.ex modified +41

@@ -23,6 +23,47 @@ defmodule OpenAgentsWeb.DocsCatalog do

23 23
        %{slug: "api-tokens", title: "API tokens", icon: "key", route: "/settings/api-tokens"}
24 24
      ]
25 25
    },
26
    %{
27
      title: "Repositories and CLI",
28
      items: [
29
        %{
30
          slug: "openagents-cli",
31
          title: "Repositories and CLI",
32
          icon: "terminal",
33
          route: "/repositories"
34
        },
35
        %{
36
          slug: "install-cli",
37
          title: "Install the CLI",
38
          icon: "download",
39
          route: "/repositories"
40
        },
41
        %{
42
          slug: "create-repository",
43
          title: "Create a repository",
44
          icon: "square-plus",
45
          route: "/repositories/new"
46
        },
47
        %{
48
          slug: "import-github",
49
          title: "Import from GitHub",
50
          icon: "download",
51
          route: "/repositories/import/github"
52
        },
53
        %{
54
          slug: "clone-push-pull",
55
          title: "Clone, push, and pull",
56
          icon: "code",
57
          route: "/repositories"
58
        },
59
        %{
60
          slug: "cli-command-reference",
61
          title: "CLI command reference",
62
          icon: "square-code",
63
          route: "/repositories"
64
        }
65
      ]
66
    },
26 67
    %{
27 68
      title: "Issues",
28 69
      items: [
lib/openagents_web/live/repository_index_live.ex modified +5 -1

@@ -29,6 +29,7 @@ defmodule OpenAgentsWeb.RepositoryIndexLive do

29 29
  # they are interpolated rather than written into the template, where the HEEx
30 30
  # parser would read them as tags.
31 31
  @cli_steps [
32
    %{command: "npx --yes @openagentsinc/cli@latest --version", note: "try without installing"},
32 33
    %{command: "npm i -g @openagentsinc/cli", note: "install"},
33 34
    %{command: "openagents auth login", note: "sign in"},
34 35
    %{command: "openagents repo create <name>", note: "create"},

@@ -53,7 +54,7 @@ defmodule OpenAgentsWeb.RepositoryIndexLive do

53 54
     |> assign(:repositories_more?, more?)
54 55
     |> assign(:watching, MapSet.new())
55 56
     |> assign(:cli_steps, @cli_steps)
56
     |> assign(:clone_url_shape, OpenAgentsWeb.Endpoint.url() <> "/git/<owner>/<name>.git")
57
     |> assign(:clone_url_shape, OpenAgentsWeb.Endpoint.url() <> "/<owner>/<name>.git")
57 58
     |> stream(:repositories, repositories)
58 59
     |> watch(repositories)}
59 60
  end

@@ -256,6 +257,9 @@ defmodule OpenAgentsWeb.RepositoryIndexLive do

256 257
          The <code class="text-foreground">openagents</code>
257 258
          command creates, clones, and imports these repositories from a terminal.
258 259
          Clone URLs are <code class="text-foreground">{@clone_url_shape}</code>.
260
          <.link navigate={~p"/docs/openagents-cli"} class="text-foreground underline">
261
            Read the CLI guide.
262
          </.link>
259 263
        </p>
260 264
261 265
        <ul class="space-y-2">
lib/openagents_web/repository_access.ex modified +1 -1

@@ -37,7 +37,7 @@ defmodule OpenAgentsWeb.RepositoryAccess do

37 37
38 38
  def clone_url(%Repository{} = repository) do
39 39
    OpenAgentsWeb.Endpoint.url() <>
40
      "/git/#{repository.namespace.slug}/#{repository.name}.git"
40
      "/#{repository.namespace.slug}/#{repository.name}.git"
41 41
  end
42 42
43 43
  defp ordinary_public?(%Repository{visibility: "public", storage_key: storage_key}),
lib/openagents_web/route_authority.ex modified +14 -8

@@ -127,14 +127,20 @@ defmodule OpenAgentsWeb.RouteAuthority do

127 127
  defp policy(%{path: "/admin"}),
128 128
    do: declaration(:operator, "configured operator GitHub ID", "voice:metadata:read", false)
129 129
130
  defp policy(%{path: "/git"}),
131
    do:
132
      declaration(
133
        :git_transport,
134
        "operator or active paired-machine HTTP credential",
135
        "git:repository",
136
        true
137
      )
130
  defp policy(%{path: path})
131
       when path in [
132
              "/:owner/:repo/info/refs",
133
              "/:owner/:repo/git-upload-pack",
134
              "/:owner/:repo/git-receive-pack",
135
              "/git"
136
            ],
137
       do:
138
         declaration(
139
           :git_transport,
140
           "anonymous read or authorized user, operator, or paired-machine HTTP credential",
141
           "git:repository",
142
           true
143
         )
138 144
139 145
  defp policy(%{path: "/api/status", verb: verb}) when verb in [:get, :head],
140 146
    do: declaration(:public_read, "anonymous", "published:status", false)
lib/openagents_web/router.ex modified +10

@@ -171,6 +171,16 @@ defmodule OpenAgentsWeb.Router do

171 171
    get "/recordings/:id/audio", AdminRecordingController, :show
172 172
  end
173 173
174
  scope "/" do
175
    pipe_through :forge_git
176
177
    get "/:owner/:repo/info/refs", OpenAgents.Forge.GitHTTP, []
178
    post "/:owner/:repo/git-upload-pack", OpenAgents.Forge.GitHTTP, []
179
    post "/:owner/:repo/git-receive-pack", OpenAgents.Forge.GitHTTP, []
180
  end
181
182
  # Keep existing remotes operational while every newly issued clone URL uses
183
  # the canonical GitHub-shaped /:owner/:repo.git path.
174 184
  scope "/git" do
175 185
    pipe_through :forge_git
176 186
    forward "/", OpenAgents.Forge.GitHTTP
priv/docs/api-tokens.md modified +7 -2

@@ -3,6 +3,11 @@

3 3
Tokens authenticate programmatic access to the REST API. Manage them at
4 4
[API tokens](/settings/api-tokens).
5 5
6
The OpenAgents CLI normally obtains an API token through browser-assisted
7
device authorization and stores it in the operating-system credential store.
8
See [Install the CLI](/docs/install-cli) when you do not need to create a token
9
manually.
10
6 11
## Creating a token
7 12
8 13
Create a token and copy it immediately. Only a hash is stored, so the value

@@ -17,11 +22,11 @@ curl -H "Authorization: Bearer $OPENAGENTS_TOKEN" \

17 22
  https://openagents.com/api/v3/repos/OpenAgentsInc/openagents.com/issues
18 23
```
19 24
20
## Revoking
25
## Revoke a token
21 26
22 27
Revoking takes effect immediately. Any request in flight with that token fails
23 28
on its next call.
24 29
25
## Scope
30
## Understand the scope
26 31
27 32
A token acts as you. It reaches what your account reaches and nothing more.
priv/docs/browsing-code.md modified +8 -4

@@ -1,12 +1,16 @@

1 1
# Browsing code
2 2
3
Any repository the forge hosts can be read in the browser at `/:repo`.
3
Open a repository that you can access at `/:owner/:repo`. Start from
4
[Repositories](/repositories) when you do not know the exact path.
4 5
5 6
## Files
6 7
7
`/:repo/blob/:ref/*path` renders one file at one ref. The ref is part of the
8
URL, so a link to a file is a link to that file *at that revision* and does not
9
drift as the branch moves.
8
`/:owner/:repo/blob/:ref/*path` renders one file at one ref. The ref is part of
9
the URL, so a file link stays at that revision when the branch moves.
10
11
The repository page also lists branches, tags, recent commits, and the clone
12
URL. Use the [CLI and Git guide](/docs/clone-push-pull) to clone or change the
13
repository.
10 14
11 15
## What is public
12 16
priv/docs/cli-command-reference.md added +155

@@ -0,0 +1,155 @@

1
# OpenAgents CLI command reference
2
3
The `openagents` command manages authentication and hosted repositories.
4
5
```text
6
openagents <subcommand> [flags]
7
```
8
9
Run `openagents <command> --help` for the reference that matches your installed
10
version. When you use `npx`, replace the `openagents` prefix with
11
`npx --yes @openagentsinc/cli@latest`.
12
13
## Use global flags
14
15
| Flag | Description |
16
| --- | --- |
17
| `--profile production\|staging\|local` | Select a named API profile. |
18
| `--api-url ORIGIN` | Use a custom HTTPS or loopback API origin. |
19
| `--json` | Write one machine-readable JSON value. |
20
| `--no-color` | Disable ANSI output. |
21
| `--help`, `-h` | Show help. |
22
| `--version`, `-v` | Show the CLI version. |
23
| `--completions bash\|zsh\|fish\|sh` | Print a shell completion script. |
24
25
Place shared flags before the subcommand:
26
27
```sh
28
openagents --profile staging --json repo list
29
npx --yes @openagentsinc/cli@latest --profile staging --json repo list
30
```
31
32
Setting `NO_COLOR` also disables ANSI output.
33
34
## Run authentication commands
35
36
| Command | Description |
37
| --- | --- |
38
| `openagents auth login` | Start browser-assisted device authorization and store the token. |
39
| `openagents auth login --token-stdin` | Read and store a token from standard input. |
40
| `openagents auth token-stdin` | Read and store a token from standard input. |
41
| `openagents auth status` | Show the selected API, account, namespaces, expiry, and helper state. |
42
| `openagents auth logout` | Remove the stored token for the selected API origin. |
43
| `openagents auth setup-git --local` | Configure the current Git repository. |
44
| `openagents auth setup-git --global --yes` | Configure global Git settings with explicit confirmation. |
45
46
`auth git-credential` is an internal Git-helper endpoint. Do not invoke it
47
directly.
48
49
Do not run either `auth setup-git` form through `npx`. Install the CLI globally
50
before you save a persistent helper configuration.
51
52
## Create a repository
53
54
```text
55
openagents repo create [flags] <name-or-namespace/name>
56
```
57
58
| Flag | Description |
59
| --- | --- |
60
| `--description TEXT` | Set the repository description. |
61
| `--public` | Create a public repository. |
62
| `--private` | Create a private repository, which is the default. |
63
| `--default-branch NAME` | Set the initial default branch. The default is `main`. |
64
| `--wait-timeout SECONDS` | Wait for provisioning. The default is `300`; `0` does not wait. |
65
| `--source DIRECTORY` | Attach the new repository to a Git worktree. |
66
| `--remote NAME` | Set the remote name used with `--source`. The default is `origin`. |
67
68
The command creates the server repository before it configures a local remote.
69
It never pushes automatically.
70
71
## Import a repository
72
73
```text
74
openagents repo import [flags] <github-owner/repository>
75
```
76
77
| Flag | Description |
78
| --- | --- |
79
| `--name NAME` | Override the destination repository name. |
80
| `--namespace OWNER` | State the matching eligible GitHub owner. |
81
| `--public` | Create a public destination. |
82
| `--private` | Create a private destination, which is the default. |
83
| `--wait-timeout SECONDS` | Wait for import. The default is `300`; `0` does not wait. |
84
85
This command performs one import. It does not start synchronization. A client
86
timeout does not cancel the accepted server-side import.
87
88
## List repositories
89
90
```text
91
openagents repo list [--namespace OWNER] [--limit 1..100] [--after CURSOR]
92
```
93
94
The default limit is `30`. When more results exist, human output prints the
95
next opaque cursor and JSON output returns it as `next_cursor`.
96
97
## View a repository
98
99
```text
100
openagents repo view [OWNER/REPOSITORY]
101
openagents repo view --repo OWNER/REPOSITORY
102
```
103
104
When you omit the repository, the CLI infers it from an exact OpenAgents
105
`origin` remote on the selected API origin.
106
107
## Clone a repository
108
109
```text
110
openagents repo clone [OWNER/REPOSITORY] [DIRECTORY]
111
openagents repo clone --repo OWNER/REPOSITORY [DIRECTORY]
112
```
113
114
The CLI retrieves the clone URL from the API and starts standard Git.
115
116
## Use JSON in noninteractive processes
117
118
With `--json`, stdout contains machine-readable output. Human progress and
119
errors do not contaminate a successful JSON response. Responses never include
120
an API token or token digest.
121
122
In a noninteractive process:
123
124
- Set `OPENAGENTS_TOKEN` or provide an existing credential-store entry.
125
- Pass every ambiguous value as an argument or flag.
126
- Do not use browser login or global Git-helper setup.
127
- Handle `SIGINT` and `SIGTERM` as exit code `130`. The CLI cancels in-flight
128
  HTTP work and terminates its child Git process.
129
130
## Handle exit codes
131
132
| Code | Meaning |
133
| --- | --- |
134
| `0` | Success. |
135
| `1` | Git, output, or unclassified operational failure. |
136
| `2` | Usage, configuration, or validation error. |
137
| `3` | Authentication, authorization, or credential-store failure. |
138
| `4` | Repository or API resource not found. |
139
| `5` | Conflict, such as an existing repository name. |
140
| `6` | Network, server, transport, or API-contract failure. |
141
| `7` | Provisioning or import failure or timeout. |
142
| `130` | Interrupted by `SIGINT` or `SIGTERM`. |
143
144
## Know which commands are unavailable
145
146
This release does not provide `repo delete`, `repo mirror`, pull-request,
147
ruleset, SSH-key, generic API, or self-update commands. Use only commands shown
148
by the installed version's `--help` output.
149
150
## Next steps
151
152
- [Install the CLI](/docs/install-cli)
153
- [Create a repository](/docs/create-repository)
154
- [Import from GitHub](/docs/import-github)
155
- [Clone, push, and pull](/docs/clone-push-pull)
priv/docs/clone-push-pull.md added +109

@@ -0,0 +1,109 @@

1
# Clone, push, and pull
2
3
OpenAgents serves Git smart HTTP at the HTTPS clone URL returned by the
4
repository API and web interface. Standard Git performs clone, fetch, push,
5
and pull operations.
6
7
## Clone with the CLI
8
9
```sh
10
openagents repo clone OWNER/REPOSITORY
11
```
12
13
Choose a destination directory:
14
15
```sh
16
openagents repo clone OWNER/REPOSITORY ./local-directory
17
```
18
19
For a one-time clone without a global installation:
20
21
```sh
22
npx --yes @openagentsinc/cli@latest repo clone OWNER/REPOSITORY
23
```
24
25
The command retrieves the server-provided clone URL and invokes Git without
26
putting the token in the URL or process arguments. It scopes the OpenAgents
27
credential helper to the selected API origin for that clone process.
28
29
## Clone with Git
30
31
Copy the HTTPS URL from the repository page, then run:
32
33
```sh
34
git clone https://openagents.com/OWNER/REPOSITORY.git
35
```
36
37
Public repositories support anonymous clone and fetch. Private repositories
38
require an authorized credential.
39
40
Before you use standard Git with a private repository, install the CLI globally
41
and configure the helper in the worktree:
42
43
```sh
44
npm install --global @openagentsinc/cli
45
cd existing-worktree
46
openagents auth setup-git --local
47
```
48
49
Do not run `auth setup-git` through `npx`. The saved helper calls
50
`openagents`, and the temporary `npx` executable is unavailable after that
51
command exits.
52
53
Use global setup only when you want every local repository to use the helper
54
for the selected OpenAgents origin:
55
56
```sh
57
openagents auth setup-git --global --yes
58
```
59
60
## Push and pull
61
62
After you configure the helper, use standard Git commands:
63
64
```sh
65
git push -u origin main
66
git fetch origin
67
git pull --ff-only
68
```
69
70
Repository owners, maintainers, and contributors can push. Viewers can clone
71
and fetch but cannot push. OpenAgents checks token authorization for each Git
72
request, so revoking or expiring a token takes effect without changing the
73
remote URL.
74
75
## Infer a repository from origin
76
77
From a worktree whose `origin` is an exact OpenAgents clone URL:
78
79
```sh
80
openagents repo view
81
```
82
83
The CLI accepts only `/OWNER/REPOSITORY.git` on the selected API origin. It
84
does not infer authority from another URL that resembles a repository path.
85
Override inference explicitly when needed:
86
87
```sh
88
openagents repo view --repo OWNER/REPOSITORY
89
openagents repo clone -R OWNER/REPOSITORY
90
```
91
92
## Review authentication safety
93
94
The credential helper:
95
96
- Returns credentials only for the exact selected OpenAgents origin.
97
- Rejects username, port, path, or protocol mismatches.
98
- Reads `OPENAGENTS_TOKEN` or the operating-system credential store.
99
- Never writes a token into the Git remote URL.
100
- Never logs complete credential-helper input.
101
102
OpenAgents uses HTTPS in this release. SSH remotes and SSH-key management are
103
not available yet.
104
105
## Next steps
106
107
- [Install the CLI](/docs/install-cli)
108
- [Create a repository](/docs/create-repository)
109
- [CLI command reference](/docs/cli-command-reference)
priv/docs/create-repository.md added +129

@@ -0,0 +1,129 @@

1
# Create a repository
2
3
Create an empty OpenAgents repository in the browser or with the CLI. If the
4
code already exists on GitHub, use a [one-time GitHub
5
import](/docs/import-github) instead.
6
7
## Create a repository in the browser
8
9
1. Sign in to OpenAgents with GitHub.
10
2. Open [Repositories](/repositories).
11
3. Select **New repository**.
12
4. Choose your GitHub user namespace or an eligible GitHub organization.
13
5. Enter a name and optional description.
14
6. Choose **Private** or **Public**. Private is the default.
15
7. Enter the default branch. The default value is `main`.
16
8. Select **Create repository**.
17
18
The repository can briefly show a provisioning state. Push instructions appear
19
after durable provisioning finishes.
20
21
## Create a repository with the CLI
22
23
Create a private repository in your GitHub user namespace:
24
25
```sh
26
openagents repo create my-project
27
```
28
29
Create a public repository in an eligible organization namespace:
30
31
```sh
32
openagents repo create OpenAgentsInc/my-project --public
33
```
34
35
Set a description and default branch:
36
37
```sh
38
openagents repo create my-project \
39
  --description "Example repository" \
40
  --default-branch trunk
41
```
42
43
The CLI waits up to 300 seconds for provisioning by default. Pass
44
`--wait-timeout 0` to return after the server accepts the durable request. The
45
repository continues provisioning on the server.
46
47
For one command without a global install, prefix the same arguments with the
48
package name:
49
50
```sh
51
npx --yes @openagentsinc/cli@latest repo create my-project
52
```
53
54
## Attach an existing local project
55
56
Create the remote repository and attach it to an existing Git worktree:
57
58
```sh
59
openagents repo create my-project --source .
60
```
61
62
The CLI verifies the Git worktree, adds the server-provided clone URL as the
63
`origin` remote, and prints the next push command. It does not push
64
automatically.
65
66
Choose another remote name when `origin` already belongs to another host:
67
68
```sh
69
openagents repo create my-project --source . --remote openagents
70
git push -u openagents HEAD
71
```
72
73
The CLI refuses to overwrite an existing remote that points to another URL. If
74
remote attachment fails, the remote repository still exists.
75
76
## Push the first commit
77
78
Install the CLI globally before configuring persistent Git authentication:
79
80
```sh
81
npm install --global @openagentsinc/cli
82
cd existing-worktree
83
openagents auth setup-git --local
84
git push -u origin HEAD
85
```
86
87
To start from an empty directory:
88
89
```sh
90
git init my-project
91
cd my-project
92
git branch -M main
93
git remote add origin https://openagents.com/OWNER/my-project.git
94
# Add files, then commit them.
95
openagents auth setup-git --local
96
git push -u origin main
97
```
98
99
Prefer the clone URL returned by the browser or CLI instead of constructing it
100
yourself.
101
102
## Follow repository naming rules
103
104
Repository names are case-insensitive and normalize to lowercase. A name:
105
106
- Contains 1 through 64 ASCII characters.
107
- Starts with a letter or digit.
108
- Uses letters, digits, hyphens, underscores, and qualifying dots.
109
- Cannot use a reserved OpenAgents route or Git-internal name.
110
- Must be unique within the namespace.
111
112
A private repository requires an authorized OpenAgents membership. A public
113
repository permits anonymous Git reads. The creator becomes the repository
114
owner.
115
116
## Create an organization repository
117
118
OpenAgents uses GitHub to verify organization identity and membership. You
119
need an active GitHub organization administrator membership to create a
120
repository in that organization during this release.
121
122
Custom OpenAgents namespaces, repository transfer, rename, archive, and delete
123
are not available yet.
124
125
## Next steps
126
127
- [Clone, push, and pull](/docs/clone-push-pull)
128
- [Import from GitHub](/docs/import-github)
129
- [CLI command reference](/docs/cli-command-reference)
priv/docs/import-github.md added +189

@@ -0,0 +1,189 @@

1
# Import a GitHub repository
2
3
An import copies one accepted GitHub snapshot into a new OpenAgents
4
repository. It is a one-time copy, not a mirror. Later changes do not
5
synchronize in either direction.
6
7
## Check the prerequisites
8
9
You need:
10
11
- An OpenAgents account connected to GitHub.
12
- A current GitHub connection with the `repo` and `read:org` grants.
13
- Read access to the source GitHub repository.
14
- An eligible destination namespace that matches the GitHub source owner.
15
- An active GitHub organization administrator membership for an organization
16
  destination.
17
18
Reconnect your GitHub account before importing if its current grant lacks the
19
required permissions.
20
21
## Import in the browser
22
23
1. Sign in to OpenAgents with GitHub.
24
2. Open [Repositories](/repositories).
25
3. Select **Import from GitHub**.
26
4. Choose a repository from the bounded GitHub repository list.
27
5. Confirm the matching destination namespace.
28
6. Keep the source name or enter another destination name.
29
7. Choose **Private** or **Public**. Private is the default, even when the
30
   GitHub source is public.
31
8. Review the Git LFS warning.
32
9. Select **Import repository**.
33
34
The import page shows the accepted snapshot and bounded lifecycle state. It
35
does not expose raw Git output or a GitHub token.
36
37
## Import with an installed CLI
38
39
Import a repository into its matching namespace:
40
41
```sh
42
openagents repo import OpenAgentsInc/example
43
```
44
45
Choose another destination name:
46
47
```sh
48
openagents repo import OpenAgentsInc/example --name example-copy
49
```
50
51
Create a public destination explicitly:
52
53
```sh
54
openagents repo import OpenAgentsInc/example --public
55
```
56
57
State the matching organization explicitly:
58
59
```sh
60
openagents repo import OpenAgentsInc/example --namespace OpenAgentsInc
61
```
62
63
The `--namespace` value must match the GitHub source owner in this release. You
64
cannot import `SOURCE/repository` directly into an unrelated namespace.
65
66
The CLI waits up to 300 seconds by default. Pass `--wait-timeout 0` to return
67
after the server accepts the durable import. A client timeout does not cancel
68
the server-side import.
69
70
## Import once with npx
71
72
Run the same import without a global installation:
73
74
```sh
75
npx --yes @openagentsinc/cli@latest repo import OWNER/REPOSITORY
76
```
77
78
Pin the package version for a reproducible qualification run:
79
80
```sh
81
npx --yes @openagentsinc/cli@0.1.3 \
82
  --profile staging \
83
  repo import OWNER/REPOSITORY \
84
  --private \
85
  --wait-timeout 300
86
```
87
88
You can use `npx` for the import and the CLI-managed clone. Install the CLI
89
globally before you run `auth setup-git`, because a persistent Git helper
90
cannot call the temporary executable after `npx` exits.
91
92
## Import a large repository
93
94
OpenAgents keeps large Git bundles on disk and streams them to and from the
95
durable forge WAL in 1 MiB chunks. The application does not read the complete
96
bundle into the BEAM heap. The default server limits allow a bundle up to 20
97
GiB and an import to run for up to six hours.
98
99
The CLI's `--wait-timeout` controls only how long that client waits. It does
100
not change or cancel the server import. For a large repository, accept the
101
operation immediately and check it separately:
102
103
```sh
104
openagents repo import OWNER/REPOSITORY --wait-timeout 0
105
openagents repo view OWNER/REPOSITORY
106
```
107
108
The repository page reports queued, copying, storing, ready, and failed states.
109
Server logs record every stage and the bundle byte count. A bundle over the
110
server limit fails with `import_too_large`; an operation over the server time
111
limit fails with `import_timeout`.
112
113
Large imports still need enough temporary disk for the fetched Git objects and
114
the bundle. Git LFS objects remain outside the import. Start an environment's
115
first import with a small repository so you can qualify authentication,
116
lifecycle reporting, storage, and clone behavior before spending the time and
117
bandwidth of a large transfer.
118
119
## Verify an import
120
121
Use a small private repository for an initial environment check. Include a
122
second branch and an annotated tag so you can verify the accepted ref
123
snapshot. If the source uses Git LFS, expect OpenAgents to preserve pointer
124
files without copying LFS objects.
125
126
1. Sign in and confirm the selected account:
127
128
   ```sh
129
   openagents auth login
130
   openagents auth status
131
   ```
132
133
2. Start one private import and wait for its durable result:
134
135
   ```sh
136
   openagents repo import OWNER/REPOSITORY --private --wait-timeout 300
137
   ```
138
139
3. Confirm that the destination is ready, then clone it:
140
141
   ```sh
142
   openagents repo view OWNER/REPOSITORY
143
   openagents repo clone OWNER/REPOSITORY
144
   ```
145
146
4. Compare the cloned branches and tags with the accepted GitHub snapshot.
147
5. Add a later commit on GitHub and confirm that it does not appear in the
148
   OpenAgents copy.
149
150
OpenAgents does not start an import during a deployment. An authenticated user
151
must start each import explicitly.
152
153
## Understand what the import copies
154
155
| Copied | Not copied |
156
| --- | --- |
157
| Git history reachable from accepted refs | GitHub Issues |
158
| `refs/heads/*` branches | Pull requests and reviews |
159
| `refs/tags/*` tags | Actions runs and secrets |
160
| The source default branch | Releases and repository settings |
161
| Submodule pointer commits | Wikis and Git LFS objects |
162
163
OpenAgents freezes the accepted branch and tag map before copying data. It
164
verifies the same ref snapshot before marking the repository ready. A GitHub
165
commit created after acceptance is not part of the import.
166
167
Git LFS pointer files remain in Git history, but OpenAgents does not copy the
168
referenced LFS objects. Download or migrate those objects separately before
169
you rely on the imported repository.
170
171
## Work after the import
172
173
OpenAgents becomes the source of truth for the destination. Clone and work
174
with it as a normal OpenAgents repository:
175
176
```sh
177
openagents repo clone OpenAgentsInc/example
178
cd example
179
openagents auth setup-git --local
180
git push
181
```
182
183
The repository page records the GitHub source and labels the result as imported
184
once.
185
186
## Next steps
187
188
- [Clone, push, and pull](/docs/clone-push-pull)
189
- [CLI command reference](/docs/cli-command-reference)
priv/docs/install-cli.md added +178

@@ -0,0 +1,178 @@

1
# Install the OpenAgents CLI
2
3
The npm package is `@openagentsinc/cli`. It provides the `openagents` command
4
and requires Node.js 20 or later.
5
6
## Install globally with npm
7
8
Install the CLI globally when you use it regularly:
9
10
```sh
11
npm install --global @openagentsinc/cli
12
openagents --version
13
```
14
15
Install the latest release again when you want to update:
16
17
```sh
18
npm install --global @openagentsinc/cli@latest
19
```
20
21
The CLI does not include an `openagents update` command.
22
23
## Run one command with npx
24
25
Use `npx` when you want to run one CLI command without installing the package
26
globally:
27
28
```sh
29
npx --yes @openagentsinc/cli@latest --version
30
npx --yes @openagentsinc/cli@latest repo list
31
```
32
33
Pin the package version when a script or qualification run must be
34
reproducible:
35
36
```sh
37
npx --yes @openagentsinc/cli@0.1.3 --version
38
```
39
40
Place every `openagents` argument after the package name:
41
42
```sh
43
npx --yes @openagentsinc/cli@latest --profile staging auth status
44
npx --yes @openagentsinc/cli@latest repo import OWNER/REPOSITORY
45
```
46
47
`npx` works for authentication, repository creation, imports, listing,
48
inspection, and cloning. The CLI stores an approved login in the same
49
operating-system credential store that a global installation uses.
50
51
Do not run `auth setup-git` through `npx`. That command writes a persistent Git
52
helper configuration that calls `openagents`, but the temporary `npx`
53
executable disappears after the command. Install the CLI globally before you
54
configure a local or global Git helper.
55
56
## Sign in
57
58
Start the browser-assisted device authorization flow:
59
60
```sh
61
openagents auth login
62
```
63
64
The CLI prints a verification URL and user code, opens the URL when your
65
operating system supports it, and waits for approval. Complete the flow with
66
the GitHub account connected to OpenAgents.
67
68
In a headless or noninteractive agent process, the command prints the complete
69
authorization URL and user code to standard error and waits. Have the agent
70
surface both values to you. Open the URL in any browser, review the request,
71
and approve it. The waiting command then stores the OpenAgents token and
72
continues; the agent never receives your GitHub token.
73
74
The CLI stores the resulting `oa_pat_` token for the selected API origin:
75
76
- On macOS, it uses Keychain through the `security` command.
77
- On Linux, it uses Secret Service through `secret-tool`.
78
- On a system without an admitted credential store, it fails closed. Use
79
  `OPENAGENTS_TOKEN` for the current process instead.
80
81
Check the selected account, namespaces, token source, expiry, and Git-helper
82
state:
83
84
```sh
85
openagents auth status
86
```
87
88
Remove the stored credential for the selected API origin:
89
90
```sh
91
openagents auth logout
92
```
93
94
## Use a token without a browser
95
96
Read and store a token from standard input. The CLI never accepts a token as a
97
command-line argument.
98
99
```sh
100
openagents auth token-stdin
101
```
102
103
`openagents auth login --token-stdin` provides the same behavior.
104
105
For an agent or CI process, set the token for the process:
106
107
```sh
108
OPENAGENTS_TOKEN="oa_pat_..." openagents --json repo list
109
```
110
111
Do not put a token in a Git URL, configuration file, shell history, or process
112
argument.
113
114
## Select an API profile
115
116
The CLI uses the production profile by default.
117
118
| Profile | API origin |
119
| --- | --- |
120
| `production` | `https://openagents.com` |
121
| `staging` | `https://staging.openagents.com` |
122
| `local` | `http://localhost:4000` |
123
124
Place a shared profile or API flag before the subcommand:
125
126
```sh
127
openagents --profile staging auth status
128
openagents --profile local repo list
129
openagents --api-url https://forge.example.com repo list
130
```
131
132
Custom origins must use HTTPS. The CLI permits HTTP only for loopback hosts.
133
An API URL must be an origin without a path, query, fragment, username, or
134
password.
135
136
The CLI resolves endpoint settings in this order:
137
138
1. `--api-url`
139
2. `--profile`
140
3. `OPENAGENTS_API_URL`
141
4. `OPENAGENTS_PROFILE`
142
5. `api_url` in `~/.config/openagents/config.json`
143
6. `profile` in `~/.config/openagents/config.json`
144
7. The production profile
145
146
Set `OPENAGENTS_CONFIG_PATH` to read another configuration file. The file
147
accepts `profile` or `api_url` and never stores credentials.
148
149
```json
150
{
151
  "profile": "local"
152
}
153
```
154
155
## Configure Git authentication
156
157
After you install the CLI globally, configure only the current Git repository:
158
159
```sh
160
openagents auth setup-git --local
161
```
162
163
Configure every local repository only when you intend to use the same helper
164
for the selected OpenAgents origin:
165
166
```sh
167
openagents auth setup-git --global --yes
168
```
169
170
The helper is scoped to the selected OpenAgents origin. It refuses unrelated
171
hosts and never places a token in a Git URL or process argument.
172
173
## Next steps
174
175
- [Create a repository](/docs/create-repository)
176
- [Import from GitHub](/docs/import-github)
177
- [Clone, push, and pull](/docs/clone-push-pull)
178
- [CLI command reference](/docs/cli-command-reference)
priv/docs/openagents-cli.md added +120

@@ -0,0 +1,120 @@

1
# Repositories and the OpenAgents CLI
2
3
OpenAgents hosts Git repositories and serves them through Git smart HTTP. You
4
can create and import repositories in the browser or manage them from a
5
terminal with the OpenAgents CLI (`openagents`).
6
7
## What you can do
8
9
The current release lets you:
10
11
- Create a private or public repository in your GitHub user namespace.
12
- Create a repository in an eligible GitHub organization namespace.
13
- Import a GitHub repository as a one-time copy.
14
- List and inspect repositories that you can access.
15
- Clone repositories and configure Git authentication.
16
- Push, pull, and fetch with standard Git.
17
- Use JSON output and stable exit codes in scripts and agents.
18
19
The current release does not provide pull requests, repository deletion,
20
continuous GitHub mirroring, SSH transport, rulesets, or a self-update command.
21
22
## Choose how to run the CLI
23
24
Install the CLI globally when you use it regularly or when you want to
25
configure Git authentication that remains available after the current command:
26
27
```sh
28
npm install --global @openagentsinc/cli
29
openagents --version
30
```
31
32
Use `npx` for one command without a global installation:
33
34
```sh
35
npx --yes @openagentsinc/cli@latest --version
36
npx --yes @openagentsinc/cli@latest repo list
37
```
38
39
Do not configure a persistent Git credential helper through `npx`. The helper
40
configuration refers to the `openagents` executable, but the temporary `npx`
41
executable disappears when the command ends. Install the CLI globally before
42
you run `openagents auth setup-git --local` or
43
`openagents auth setup-git --global --yes`.
44
45
See [Install the CLI](/docs/install-cli) for authentication, profiles,
46
configuration, and the complete `npx` guidance.
47
48
## Understand namespaces and access
49
50
OpenAgents uses GitHub namespaces in this release. Your OpenAgents user
51
namespace matches your GitHub login. Eligible organization namespaces match
52
the GitHub organization login.
53
54
OpenAgents keys namespaces to GitHub's immutable account IDs. If a GitHub login
55
changes, OpenAgents can update the displayed login without changing the
56
repository's identity.
57
58
Creating or importing in an organization requires an active GitHub
59
organization administrator membership. OpenAgents adds the creator as the
60
repository owner. It does not automatically grant access to every member of
61
the GitHub organization.
62
63
## Understand visibility and credentials
64
65
New repositories are private unless you explicitly make them public. Anyone
66
can clone and fetch a public repository. Only authorized OpenAgents members can
67
see a private repository, and an unauthorized request does not reveal whether
68
the repository exists.
69
70
OpenAgents stores the GitHub access token used for GitHub operations on the
71
server. It never sends that token to the browser or the CLI. The CLI stores its
72
OpenAgents API token in the operating-system credential store or reads it from
73
`OPENAGENTS_TOKEN` for the current process.
74
75
Repository access does not grant deployment or operator authority.
76
77
## Manage repositories in the browser
78
79
After you sign in, open [Repositories](/repositories).
80
81
- Select **New repository** to create an empty repository.
82
- Select **Import from GitHub** to copy one GitHub repository.
83
- Open a repository to see its clone URL, lifecycle state, code, Issues,
84
  Projects, and import receipt when applicable.
85
86
## Follow a common terminal workflow
87
88
1. Install and sign in:
89
90
   ```sh
91
   npm install --global @openagentsinc/cli
92
   openagents auth login
93
   openagents auth status
94
   ```
95
96
2. Create or import a repository:
97
98
   ```sh
99
   openagents repo create my-project
100
   # Or copy a GitHub repository once:
101
   openagents repo import OWNER/REPOSITORY
102
   ```
103
104
3. Clone and configure standard Git:
105
106
   ```sh
107
   openagents repo clone OWNER/REPOSITORY
108
   cd REPOSITORY
109
   openagents auth setup-git --local
110
   git pull --ff-only
111
   git push
112
   ```
113
114
## Next steps
115
116
- [Install the CLI](/docs/install-cli)
117
- [Create a repository](/docs/create-repository)
118
- [Import from GitHub](/docs/import-github)
119
- [Clone, push, and pull](/docs/clone-push-pull)
120
- [CLI command reference](/docs/cli-command-reference)
priv/docs/rest-api.md modified +23 -3

@@ -1,7 +1,8 @@

1 1
# REST API
2 2
3
The API is shaped after GitHub's REST API and served under `/api/v3`. An
4
existing client usually needs only a base URL change.
3
The API is shaped after GitHub's REST API and served under `/api/v3`. Check the
4
implemented paths and known differences before you point an existing client at
5
OpenAgents.
5 6
6 7
## Authentication
7 8

@@ -21,6 +22,24 @@ GET /api/v3/repos/:owner/:repo/issues/:issue_number

21 22
PATCH  /api/v3/repos/:owner/:repo/issues/:issue_number
22 23
```
23 24
25
## Repositories
26
27
```text
28
GET    /api/v3/user
29
GET    /api/v3/user/repos
30
GET    /api/v3/repos/:owner/:repo
31
POST   /api/v3/user/repos
32
POST   /api/v3/orgs/:org/repos
33
POST   /api/v3/user/repos/imports
34
POST   /api/v3/orgs/:org/repos/imports
35
GET    /api/v3/repository-imports/:id
36
```
37
38
Repository writes require an `Idempotency-Key` header. The published
39
[`openagents.repositories.v1` contract](/api/contracts/repositories-v1.json)
40
defines request authority, lifecycle states, pagination, and stable error
41
codes. The [OpenAgents CLI](/docs/openagents-cli) implements this contract.
42
24 43
## Comments
25 44
26 45
```

@@ -62,4 +81,5 @@ finds them here rather than in production:

62 81
63 82
## What is not implemented
64 83
65
Pull requests, reviews, webhooks, releases, and Git LFS.
84
Pull requests, reviews, webhooks, releases, SSH Git transport, and Git LFS
85
object storage.
priv/docs/signing-in.md modified +7

@@ -22,3 +22,10 @@ browser.

22 22
**Log out** from the account menu ends the session. It does not revoke the
23 23
GitHub authorization; do that from GitHub's application settings if you want
24 24
the grant removed entirely.
25
26
## Sign in from the CLI
27
28
Run `openagents auth login` to start a browser-assisted device flow. You review
29
and approve that request while signed in to the site. The CLI receives an
30
OpenAgents API token, not your GitHub token. See [Install the
31
CLI](/docs/install-cli) for npm and `npx` instructions.
priv/docs/welcome.md modified +10 -7

@@ -5,10 +5,13 @@ documentation covers the parts you can use today.

5 5
6 6
## What is here now
7 7
8
The issue tracker is the surface furthest along: issues, labels, milestones,
9
assignees, and projects, each with a browser view and a GitHub-compatible REST
10
endpoint. Code browsing renders any file or commit in a repository the forge
11
hosts.
8
OpenAgents hosts Git repositories that you can create, import from GitHub once,
9
clone, push, pull, and browse. Use the browser or the npm-distributed
10
[OpenAgents CLI](/docs/openagents-cli).
11
12
The issue tracker includes issues, labels, milestones, assignees, and projects,
13
each with a browser view and a GitHub-compatible REST endpoint. Code browsing
14
renders files and commits in repositories that you can access.
12 15
13 16
Three surfaces exist to show the system's own work rather than yours. The
14 17
[changelog](/changelog) lists every change with the receipt chain that took it

@@ -24,6 +27,6 @@ unable to tell which half you are reading.

24 27
25 28
## Compatibility
26 29
27
The REST API is shaped after GitHub's, so an existing client usually needs only
28
a base URL change. See [REST API](/docs/rest-api) for what is implemented and
29
what differs.
30
The REST API is shaped after GitHub's. See [REST API](/docs/rest-api) for the
31
implemented repository, issue, and project endpoints and the documented
32
differences.
test/openagents/forge/sync_test.exs modified +1 -2

@@ -40,11 +40,10 @@ defmodule OpenAgents.Forge.SyncTest do

40 40
    sha = source |> git!(["rev-parse", "HEAD"]) |> String.trim()
41 41
    bundle = Path.join(root, "source.bundle")
42 42
    git!(source, ["bundle", "create", bundle, "--all"])
43
    payload = File.read!(bundle)
44 43
    refs = %{"refs/heads/trunk" => sha, "refs/tags/v1" => sha}
45 44
46 45
    index = WAL.new_index()
47
    {:ok, object} = WAL.put_entry("storage-key", 0, payload)
46
    {:ok, object} = WAL.put_entry_file("storage-key", 0, bundle)
48 47
49 48
    entry = %{
50 49
      "seq" => 0,
test/openagents/forge/wal_test.exs modified +27

@@ -111,6 +111,19 @@ defmodule OpenAgents.Forge.WALTest do

111 111
    test "a malformed object key is rejected" do
112 112
      assert {:error, :invalid_object_key} = WAL.get_entry(@repo, "../escape")
113 113
    end
114
115
    test "streams a file into and out of the WAL under the same content key", %{tmp_dir: tmp_dir} do
116
      source = Path.join(tmp_dir, "source.bundle")
117
      destination = Path.join(tmp_dir, "downloaded.bundle")
118
      payload = :crypto.strong_rand_bytes(4 * 1_024 * 1_024)
119
      File.write!(source, payload)
120
121
      assert {:ok, key} = WAL.put_entry_file(@repo, 0, source)
122
      assert key == WAL.entry_key(0, payload)
123
      assert :ok = WAL.get_entry_file(@repo, key, destination)
124
      assert File.stat!(destination).size == byte_size(payload)
125
      assert :crypto.hash(:sha256, File.read!(destination)) == :crypto.hash(:sha256, payload)
126
    end
114 127
  end
115 128
116 129
  describe "digest-addressed artifacts" do

@@ -140,7 +153,11 @@ defmodule OpenAgents.Forge.WALTest do

140 153
        assert {:error, :invalid_repo} = WAL.read_index(bad)
141 154
        assert {:error, :invalid_repo} = WAL.cas_index(bad, :none, WAL.new_index())
142 155
        assert {:error, :invalid_repo} = WAL.put_entry(bad, 0, "x")
156
        assert {:error, :invalid_repo} = WAL.put_entry_file(bad, 0, "/tmp/entry")
143 157
        assert {:error, :invalid_repo} = WAL.get_entry(bad, "entries/00000000-0123456789ab")
158
159
        assert {:error, :invalid_repo} =
160
                 WAL.get_entry_file(bad, "entries/00000000-0123456789ab", "/tmp/entry")
144 161
      end
145 162
    end
146 163
  end

@@ -244,8 +261,18 @@ defmodule OpenAgents.Forge.WALTest do

244 261
245 262
      assert {:error, :not_configured} = OpenAgents.Forge.WAL.Gcs.put_entry(@repo, 0, "x")
246 263
264
      assert {:error, :not_configured} =
265
               OpenAgents.Forge.WAL.Gcs.put_entry_file(@repo, 0, "/tmp/entry")
266
247 267
      assert {:error, :not_configured} =
248 268
               OpenAgents.Forge.WAL.Gcs.get_entry(@repo, "entries/00000000-0123456789ab")
269
270
      assert {:error, :not_configured} =
271
               OpenAgents.Forge.WAL.Gcs.get_entry_file(
272
                 @repo,
273
                 "entries/00000000-0123456789ab",
274
                 "/tmp/entry"
275
               )
249 276
    end
250 277
251 278
    test "object naming helpers" do
test/openagents/repositories/cli_local_e2e_test.exs modified +2 -2

@@ -180,7 +180,7 @@ defmodule OpenAgents.Repositories.CliLocalE2ETest do

180 180
        "-c",
181 181
        "credential.helper=",
182 182
        "clone",
183
        "#{context.api_origin}/git/#{context.user.github_login}/created-project.git",
183
        "#{context.api_origin}/#{context.user.github_login}/created-project.git",
184 184
        anonymous
185 185
      ],
186 186
      [{"GIT_TERMINAL_PROMPT", "0"}]

@@ -203,7 +203,7 @@ defmodule OpenAgents.Repositories.CliLocalE2ETest do

203 203
    private_repository = Repositories.get_by_path!(context.user.github_login, "private-project")
204 204
205 205
    private_url =
206
      "#{context.api_origin}/git/#{context.user.github_login}/private-project.git"
206
      "#{context.api_origin}/#{context.user.github_login}/private-project.git"
207 207
208 208
    {anonymous_private_output, anonymous_private_status} =
209 209
      git_raw(
test/openagents/repositories/provisioner_test.exs modified +48

@@ -304,6 +304,54 @@ defmodule OpenAgents.Repositories.ProvisionerTest do

304 304
    assert "fetch" in arguments
305 305
  end
306 306
307
  test "an import over the configured bundle limit fails without entering the WAL" do
308
    root = Application.fetch_env!(:openagents, :forge_data_dir) |> Path.dirname()
309
    source = Path.join(root, "oversized-source")
310
    File.mkdir_p!(source)
311
    git!(source, ["init", "--initial-branch=main"])
312
    git!(source, ["config", "user.email", "test@example.com"])
313
    git!(source, ["config", "user.name", "Import test"])
314
    File.write!(Path.join(source, "README.md"), "larger than one byte\n")
315
    git!(source, ["add", "README.md"])
316
    git!(source, ["commit", "-m", "Oversized fixture"])
317
318
    sha = source |> git!(["rev-parse", "HEAD"]) |> String.trim()
319
    refs = %{"refs/heads/main" => sha}
320
    user = repository_user_fixture("oversized-import-owner")
321
322
    source_record = %{
323
      source_repository_id: 503,
324
      source_owner_id: user.github_id,
325
      source_full_name: "oversized-import-owner/source",
326
      source_default_branch: "main",
327
      source_ref_digest: ref_digest(source, refs),
328
      source_head_sha: sha,
329
      source_refs: refs,
330
      source_uses_lfs: false
331
    }
332
333
    assert {:ok, repository, repository_import, :created} =
334
             Repositories.create_user_import(
335
               user,
336
               source_record,
337
               %{name: "oversized-import", default_branch: "main"},
338
               "oversized-import-key"
339
             )
340
341
    previous_limit = Application.get_env(:openagents, :repository_import_max_bundle_bytes)
342
    Application.put_env(:openagents, :repository_import_max_bundle_bytes, 1)
343
344
    on_exit(fn ->
345
      restore_env(:repository_import_max_bundle_bytes, previous_limit)
346
    end)
347
348
    assert {:error, :import_too_large} = Importer.import(repository, source_url: source)
349
    failed = OpenAgents.Repo.get!(OpenAgents.Repositories.RepositoryImport, repository_import.id)
350
    assert failed.state == "failed"
351
    assert failed.error_code == "import_too_large"
352
    assert {:error, :not_found} = WAL.read_index(repository.storage_key)
353
  end
354
307 355
  defp bare_git!(storage_key, args) do
308 356
    {output, 0} = Repos.git(Repos.bare_path(storage_key), args)
309 357
    output
test/openagents_web/controllers/repository_controller_test.exs modified +1 -1

@@ -31,7 +31,7 @@ defmodule OpenAgentsWeb.RepositoryControllerTest do

31 31
             "permissions" => %{"admin" => true, "pull" => true, "push" => true}
32 32
           } = json_response(response, 202)
33 33
34
    assert String.ends_with?(clone_url, "/git/octavia/my-project.git")
34
    assert String.ends_with?(clone_url, "/octavia/my-project.git")
35 35
    assert String.ends_with?(html_url, "/octavia/my-project")
36 36
    assert Repositories.get_by_path!("octavia", "my-project").id == id
37 37
  end
test/openagents_web/docs_catalog_test.exs modified +12

@@ -70,6 +70,18 @@ defmodule OpenAgentsWeb.DocsCatalogTest do

70 70
    end
71 71
  end
72 72
73
  test "published CLI docs cover npm, npx, imports, and persistent Git authentication" do
74
    assert {:ok, install} = DocsCatalog.render("install-cli")
75
    assert install.markdown =~ "npm install --global @openagentsinc/cli"
76
    assert install.markdown =~ "npx --yes @openagentsinc/cli@latest"
77
    assert install.markdown =~ "Do not run `auth setup-git` through `npx`"
78
79
    assert {:ok, import} = DocsCatalog.render("import-github")
80
    assert import.markdown =~ "one-time copy"
81
    assert import.markdown =~ "--wait-timeout 0"
82
    assert import.markdown =~ "A client timeout does not cancel"
83
  end
84
73 85
  describe "the docs surface" do
74 86
    test "the index lists every page", %{conn: conn} do
75 87
      {:ok, view, _html} = live(conn, ~p"/docs")
test/openagents_web/live/repository_live_test.exs modified +3 -1

@@ -149,10 +149,12 @@ defmodule OpenAgentsWeb.RepositoryLiveTest do

149 149
150 150
    assert has_element?(view, "#repository-cli")
151 151
    assert has_element?(view, "#repository-cli-copy-0")
152
    assert has_element?(view, ~s(a[href="/docs/openagents-cli"]))
153
    assert html =~ "npx --yes @openagentsinc/cli@latest --version"
152 154
    assert html =~ "npm i -g @openagentsinc/cli"
153 155
    assert html =~ "openagents auth login"
154 156
    assert html =~ "openagents auth setup-git --local"
155
    assert html =~ "/git/&lt;owner&gt;/&lt;name&gt;.git"
157
    assert html =~ "/&lt;owner&gt;/&lt;name&gt;.git"
156 158
  end
157 159
158 160
  test "new repository defaults private and normalizes its name", %{conn: conn} do
test/openagents_web/route_authority_test.exs modified +3 -2

@@ -101,16 +101,17 @@ defmodule OpenAgentsWeb.RouteAuthorityTest do

101 101
           end)
102 102
  end
103 103
104
  test "the Forge smart HTTP mount always runs through credential authentication" do
104
  test "the canonical Forge smart HTTP routes run through credential authentication" do
105 105
    route =
106 106
      Phoenix.Router.route_info(
107 107
        OpenAgentsWeb.Router,
108 108
        "GET",
109
        "/git/openagents.com.git/info/refs",
109
        "/OpenAgentsInc/openagents.com.git/info/refs",
110 110
        "stage.openagents.com"
111 111
      )
112 112
113 113
    assert route.pipe_through == [:forge_git]
114
    assert route.route == "/:owner/:repo/info/refs"
114 115
  end
115 116
116 117
  test "the OAuth callback suppresses router parameter logging at the application boundary" do

This page updates live while a promote is in flight · changelog