push in the durable WAL and serves it. The `origin` remote is the GitHub

mirror; pushing to it directly leaves the forge behind a mirror it does not

know about, and nothing reports the divergence until a clone disagrees with

the site. Automatic mirroring to GitHub is not configured today, so GitHub

stays at whatever was last pushed to it. `ops/ci/push-remote-check.sh` refuses

the site. Production mirrors to GitHub with a force push of every ref, so a

direct GitHub push is not merged later — it is overwritten. GitHub

holds what the forge last mirrored there. `ops/ci/push-remote-check.sh` refuses

a non-forge push, and `mix precommit` installs it into the clone you are

working in, so running precommit before you push is enough. To install it by

hand:

objects, and nothing reports the divergence until a clone disagrees with the

site.

The mirror that would keep GitHub current is not running. `mirror_url/1` reads

`:forge_mirror_urls`, which is empty in `config/config.exs` and set by no

environment, so `MirrorWatch` reports `off` and GitHub receives only what

someone pushes to it. `mirror_now/1` is a `git push --mirror`, a force push of

every ref, so configuring a mirror overwrites whatever direct pushes left on

GitHub rather than merging with it. Configuring that mirror is what makes this

contract complete; until then it keeps the forge authoritative and lets GitHub

go stale, which is the honest trade and not an accident.

A mirror is running. `mirror_url/1` reads `:forge_mirror_urls`, which is empty

in `config/config.exs` but is set from `OPENAGENTS_FORGE_MIRROR_URLS_JSON` in

`config/runtime.exs`, and production sets it for `openagents.com`, so

`MirrorWatch` publishes a per-repo state rather than `off`. `mirror_now/1` is a

`git push --mirror`, a force push of every ref, so the mirror overwrites

whatever a direct push left on GitHub rather than merging with it. That is the

reason this contract admits no direct GitHub push: a push that lands there is

not merged later, it is erased.

This paragraph read the other way — that no mirror was configured — until

2026-08-25, while production had one. A contract that says the destructive

thing is switched off, while it is switched on, is worse than one that says

nothing, so what is configured is checked here against the running node rather

than against `config/config.exs` alone.

`ops/ci/push-remote-check.sh` admits only forge hosts and refuses every other

remote, whatever URL form it takes. `.githooks/pre-push` runs it before the

holding, not an omission.

Two operational facts bound the claim. `:forge_mirror_urls` is empty in

`config/config.exs` and set by no environment, so no mirror runs today and

GitHub holds whatever was last pushed to it directly, which is the trade

`REPOSITORY-002` records. And `mirror_now/1` is a force push of every ref, so

configuring a mirror overwrites what direct pushes left there rather than

merging with it.

`config/config.exs` but set from the environment in `config/runtime.exs`, and

production sets it for `openagents.com`, so a mirror does run and GitHub holds

what the forge last pushed there, which is the trade `REPOSITORY-002` records.

And `mirror_now/1` is a force push of every ref, so the mirror overwrites what a

direct push left there rather than merging with it.

Evidence: `OpenAgents.Forge.Sync`, `OpenAgents.Forge.Pushes`,

`OpenAgents.Forge.PushReceipt`, `OpenAgents.Forge.Verification`, and

defmodule OpenAgents.Forge.Backfill do

  @moduledoc """

  Import the history that predates a repository's seed into that repository's

  own write-ahead log.

  A repository seeded from a shallow fetch records a truthful boundary: the

  seed commit's parents were never fetched, so they were never written to the

  log, and `REPOSITORY-003`'s replay reproduces a grafted history from seq 0

  exactly as recorded. Every ref tip resolves and every clone succeeds, so

  nothing tip-shaped reports a problem. What is missing is everything behind

  the seed.

  Where a mirror holds that history and the log does not, the repository's own

  authority holds less than its copy does, and `EXIT-003`'s claim that the

  forge is the authority is weaker than it reads. This closes that gap the only

  way an append-only log can: by writing the missing objects into the log.

  The bundle is read once, at an operator's instruction, and appended as an

  ordinary `git_bundle` entry that carries the ref map unchanged and an empty

  shallow boundary set — the same entry shape `OpenAgents.Repositories.Importer`

  writes at seq 0, which is why replay needs no new case to apply it. The

  authority boundary is unchanged after it: a mirror supplied bytes on this one

  occasion, the log now holds them, and no read path consults a mirror.

  ## The entry is proven before it is written

  An append-only log cannot retract a bad entry. So the bundle is materialized

  against a throwaway repository that shares the current projection's objects,

  and the import refuses to touch the log unless that union can then be walked

  the way `git upload-pack` walks it — the same check

  `OpenAgents.Forge.Sync` uses to decide a projection is servable. A bundle

  that would leave the repository grafted, or that closes no boundary at all,

  is rejected while the log is still untouched.

  """

  alias OpenAgents.Forge.{Repos, Sync, WAL}

  require Logger

  @doc """

  Append `bundle_path` to `storage_key`'s log as a history import.

  Returns `{:ok, summary}` with the sequence written and the boundary commits

  the import closed, or `{:error, reason}` with the log unchanged.

  `principal` records who authorized the import, and is written to the entry

  the same way a push records its pusher.

  """

  @spec import_history(String.t(), String.t(), String.t()) ::

          {:ok, %{seq: non_neg_integer(), closed: [String.t()]}} | {:error, term()}

  def import_history(storage_key, bundle_path, principal)

      when is_binary(storage_key) and is_binary(bundle_path) and is_binary(principal) do

    with :ok <- validate_principal(principal),

         {:ok, generation, index} <- WAL.read_index(storage_key),

         projection = Repos.bare_path(storage_key),

         {:ok, closed} <- prove_bundle(projection, bundle_path),

         seq = WAL.next_seq(index),

         {:ok, object} <- WAL.put_entry_file(storage_key, seq, bundle_path) do

      entry = %{

        "seq" => seq,

        "object" => object,

        "format" => "git_bundle",

        "refs" => WAL.refs(index),

        "shallow" => [],

        "principal" => principal,

        "pushed_at" => DateTime.to_iso8601(DateTime.utc_now())

      }

      case WAL.cas_index(storage_key, generation, WAL.append_entry(index, entry)) do

        {:ok, _generation} ->

          Logger.info(

            "forge_backfill_imported repo=#{storage_key} seq=#{seq} closed=#{length(closed)}"

          )

          Sync.ensure_fresh(storage_key)

          {:ok, %{seq: seq, closed: closed}}

        {:error, reason} ->

          {:error, {:cas_failed, reason}}

      end

    end

  end

  # The boundary this import must close: the commits the projection holds whose

  # parents it does not. Derived from the objects on disk rather than from what

  # an entry recorded, because a commit whose parent is absent *is* a boundary

  # whatever the log says — the same derivation `OpenAgents.Forge.Sync` uses.

  @doc """

  The commits `storage_key`'s projection holds whose parents it does not.

  An empty list means the projection is already complete and an import has

  nothing to close.

  """

  @spec open_boundaries(String.t()) :: [String.t()]

  def open_boundaries(storage_key) do

    storage_key |> Repos.bare_path() |> boundaries_at()

  end

  defp boundaries_at(path) do

    case Repos.git(path, ["rev-list", "--all", "--max-parents=0"]) do

      {output, 0} ->

        output

        |> String.split("\n", trim: true)

        |> Enum.filter(&Regex.match?(~r/\A[0-9a-f]{40}\z/, &1))

        |> Enum.filter(&grafted?(path, &1))

      {_output, _status} ->

        []

    end

  end

  # A root commit reported by `--max-parents=0` is either a genuine root or a

  # graft boundary, and the two are told apart by the commit object itself:

  # the object records its parents whether or not the repository holds them.

  defp grafted?(path, sha), do: parents_of(path, sha) != []

  # The parents a commit object records, which is not the same question as the

  # parents a repository holds: the object is immutable and names them whether

  # or not they were ever fetched.

  defp parents_of(path, sha) do

    case Repos.git(path, ["cat-file", "commit", sha]) do

      {output, 0} ->

        output

        |> String.split("\n")

        |> Enum.take_while(&(&1 != ""))

        |> Enum.filter(&String.starts_with?(&1, "parent "))

        |> Enum.map(&(&1 |> String.replace_prefix("parent ", "") |> String.trim()))

      {_output, _status} ->

        []

    end

  end

  defp prove_bundle(projection, bundle_path) do

    cond do

      not File.dir?(projection) ->

        {:error, :projection_absent}

      not match?({:ok, %File.Stat{type: :regular}}, File.stat(bundle_path)) ->

        {:error, :bundle_unreadable}

      true ->

        boundaries = boundaries_at(projection)

        if boundaries == [] do

          {:error, :no_open_boundary}

        else

          prove_against_scratch(projection, bundle_path, boundaries)

        end

    end

  end

  # Shares the projection's objects through `objects/info/alternates` rather

  # than copying them, so proving a large repository costs the bundle's own

  # size and nothing more. Nothing here writes to the projection: an alternate

  # is read-only to the borrower, and every new object lands in the scratch

  # directory.

  defp prove_against_scratch(projection, bundle_path, boundaries) do

    scratch =

      Path.join(

        System.tmp_dir!(),

        "openagents-backfill-#{System.unique_integer([:positive, :monotonic])}.git"

      )

    try do

      with :ok <- init_scratch(scratch, projection),

           :ok <- copy_refs(projection, scratch),

           {_output, 0} <- Repos.git(scratch, ["bundle", "unbundle", bundle_path]) do

        # The import claims the boundaries are closed, so prove the walk with

        # no graft in place at all rather than with the old one still excusing

        # it.

        File.rm(Path.join(scratch, "shallow"))

        # Name what must be true rather than inferring it from an exit status.

        # A repository with no refs walks clean, so a walk alone would pass

        # vacuously on a scratch copy that never received them; requiring each

        # boundary's recorded parents to resolve cannot.

        unresolved =

          boundaries

          |> Enum.flat_map(&parents_of(projection, &1))

          |> Enum.uniq()

          |> Enum.reject(&match?({_output, 0}, Repos.git(scratch, ["cat-file", "-e", &1])))

        cond do

          unresolved != [] ->

            {:error, {:still_grafted, unresolved}}

          not match?(

            {_output, 0},

            Repos.git(scratch, ["rev-list", "--objects", "--quiet", "--all"])

          ) ->

            {:error, {:unwalkable, boundaries}}

          true ->

            {:ok, boundaries}

        end

      else

        {:error, reason} -> {:error, reason}

        {_output, _status} -> {:error, :bundle_unreadable}

      end

    after

      File.rm_rf(scratch)

    end

  end

  defp init_scratch(scratch, projection) do

    case Repos.git(scratch, ["init", "--bare", "--quiet", scratch]) do

      {_output, 0} ->

        alternates = Path.join([scratch, "objects", "info", "alternates"])

        File.mkdir_p!(Path.dirname(alternates))

        File.write(alternates, Path.join(projection, "objects") <> "\n")

      {_output, _status} ->

        {:error, :scratch_unavailable}

    end

  end

  defp copy_refs(projection, scratch) do

    Enum.reduce_while(Repos.refs_at(projection), :ok, fn {name, sha}, :ok ->

      case Repos.git(scratch, ["update-ref", name, sha]) do

        {_output, 0} -> {:cont, :ok}

        {_output, _status} -> {:halt, {:error, {:scratch_ref_failed, name}}}

      end

    end)

  end

  defp validate_principal(principal) do

    if String.trim(principal) == "" do

      {:error, :invalid_principal}

    else

      :ok

    end

  end

end

defmodule OpenAgents.Forge.BackfillTest do

  @moduledoc """

  Importing pre-seed history into the log.

  A repository seeded from a shallow fetch serves every ref tip and clones

  without error while holding none of the history behind its seed. These tests

  build exactly that repository, prove it is grafted, import the missing

  history, and then destroy the cache and rebuild from seq 0 — because an

  import that only repairs the projection has repaired a cache, not the

  authority.

  """

  use OpenAgents.DataCase, async: false

  alias OpenAgents.Forge.{Backfill, Repos, Sync, WAL}

  setup do

    base = Path.join(System.tmp_dir!(), "forge-backfill-#{System.unique_integer([:positive])}")

    File.mkdir_p!(base)

    previous_data = Application.get_env(:openagents, :forge_data_dir)

    previous_wal = Application.get_env(:openagents, :forge_wal_dir)

    Application.put_env(:openagents, :forge_data_dir, Path.join(base, "data"))

    Application.put_env(:openagents, :forge_wal_dir, Path.join(base, "wal"))

    on_exit(fn ->

      Application.put_env(:openagents, :forge_data_dir, previous_data)

      Application.put_env(:openagents, :forge_wal_dir, previous_wal)

      File.rm_rf(base)

    end)

    storage_key = "backfill-demo-#{System.unique_integer([:positive])}"

    source = build_source!(base)

    seed_shallow_import!(base, storage_key, source)

    %{base: base, storage_key: storage_key, source: source}

  end

  test "a shallow-seeded repository is grafted and holds none of its own history",

       %{storage_key: storage_key, source: source} do

    :ok = Sync.ensure_fresh!(storage_key)

    path = Repos.bare_path(storage_key)

    assert [boundary] = Backfill.open_boundaries(storage_key)

    assert boundary == rev(source, "HEAD")

    assert count_commits(path) == 1

    assert File.exists?(Path.join(path, "shallow"))

  end

  test "importing the pre-seed bundle closes the boundary and survives a rebuild from seq 0",

       %{base: base, storage_key: storage_key, source: source} do

    :ok = Sync.ensure_fresh!(storage_key)

    path = Repos.bare_path(storage_key)

    total = count_worktree_commits(source)

    assert total > 1

    bundle = pre_seed_bundle!(base, source)

    assert {:ok, %{seq: 1, closed: [_boundary]}} =

             Backfill.import_history(storage_key, bundle, "operator:test")

    assert Backfill.open_boundaries(storage_key) == []

    refute File.exists?(Path.join(path, "shallow"))

    assert count_commits(path) == total

    # The point of the import: the authority now holds the history, so a node

    # that loses its cache rebuilds the whole repository rather than the seed.

    :ok = Sync.rebuild(storage_key)

    rebuilt = Repos.bare_path(storage_key)

    assert Backfill.open_boundaries(storage_key) == []

    assert count_commits(rebuilt) == total

    assert {_output, 0} = Repos.git(rebuilt, ["rev-list", "--objects", "--quiet", "--all"])

  end

  test "the import leaves the ref map exactly as it found it",

       %{base: base, storage_key: storage_key, source: source} do

    :ok = Sync.ensure_fresh!(storage_key)

    {:ok, _generation, before_index} = WAL.read_index(storage_key)

    before_refs = WAL.refs(before_index)

    bundle = pre_seed_bundle!(base, source)

    assert {:ok, _summary} = Backfill.import_history(storage_key, bundle, "operator:test")

    {:ok, _generation, after_index} = WAL.read_index(storage_key)

    assert WAL.refs(after_index) == before_refs

    assert Repos.refs_at(Repos.bare_path(storage_key)) == before_refs

  end

  test "a bundle that closes nothing is refused with the log untouched",

       %{base: base, storage_key: storage_key} do

    :ok = Sync.ensure_fresh!(storage_key)

    {:ok, _generation, before_index} = WAL.read_index(storage_key)

    # A bundle of an unrelated repository: real, readable, and no help.

    unrelated = build_source!(base, "unrelated")

    bundle = Path.join(base, "unrelated.bundle")

    sh!(unrelated, "git", ["bundle", "create", bundle, "--all"])

    assert {:error, {:still_grafted, [_boundary]}} =

             Backfill.import_history(storage_key, bundle, "operator:test")

    {:ok, _generation, after_index} = WAL.read_index(storage_key)

    assert WAL.entries(after_index) == WAL.entries(before_index)

  end

  test "an unreadable bundle is refused with the log untouched",

       %{storage_key: storage_key} do

    :ok = Sync.ensure_fresh!(storage_key)

    {:ok, _generation, before_index} = WAL.read_index(storage_key)

    assert {:error, :bundle_unreadable} =

             Backfill.import_history(storage_key, "/nonexistent/backfill.bundle", "operator:test")

    {:ok, _generation, after_index} = WAL.read_index(storage_key)

    assert WAL.entries(after_index) == WAL.entries(before_index)

  end

  test "an import needs a principal", %{base: base, storage_key: storage_key, source: source} do

    :ok = Sync.ensure_fresh!(storage_key)

    bundle = pre_seed_bundle!(base, source)

    assert {:error, :invalid_principal} = Backfill.import_history(storage_key, bundle, "   ")

  end

  defp build_source!(base, name \\ "source") do

    source = Path.join(base, name)

    File.mkdir_p!(source)

    sh!(source, "git", ["init", "--initial-branch=main", "."])

    sh!(source, "git", ["config", "user.email", "test@example.com"])

    sh!(source, "git", ["config", "user.name", "Forge Test"])

    Enum.each(1..4, fn n ->

      File.write!(Path.join(source, "history-#{n}.txt"), "history #{n}\n")

      sh!(source, "git", ["add", "."])

      sh!(source, "git", ["commit", "-m", "history #{n}"])

    end)

    source

  end

  # Everything behind the seed: the seed's parent and all of its ancestors.

  defp pre_seed_bundle!(base, source) do

    bundle = Path.join(base, "pre-seed-#{System.unique_integer([:positive])}.bundle")

    sh!(source, "git", ["branch", "--force", "pre-seed", "HEAD~1"])

    sh!(source, "git", ["bundle", "create", bundle, "pre-seed"])

    sh!(source, "git", ["branch", "--delete", "--force", "pre-seed"])

    bundle

  end

  # Mirrors `OpenAgents.Repositories.Importer`: a `--depth=1` fetch bundled

  # with `--all`, recorded as the seq 0 `git_bundle` entry with the shallow

  # boundaries the fetch produced.

  defp seed_shallow_import!(base, storage_key, source) do

    snapshot = Path.join(base, "snapshot-#{storage_key}.git")

    sh!(base, "git", ["init", "--bare", "--initial-branch=main", snapshot])

    sh!(base, "git", [

      "--git-dir",

      snapshot,

      "fetch",

      "--depth=1",

      source,

      "refs/heads/main:refs/heads/main"

    ])

    bundle = Path.join(base, "snapshot-#{storage_key}.bundle")

    sh!(base, "git", ["--git-dir", snapshot, "bundle", "create", bundle, "--all"])

    shallow =

      snapshot

      |> Path.join("shallow")

      |> File.read!()

      |> String.split("\n", trim: true)

    assert shallow != []

    head =

      base

      |> sh!("git", ["--git-dir", snapshot, "rev-parse", "refs/heads/main"])

      |> String.trim()

    {:ok, object} = WAL.put_entry_file(storage_key, 0, bundle)

    entry = %{

      "seq" => 0,

      "object" => object,

      "format" => "git_bundle",

      "refs" => %{"refs/heads/main" => head},

      "shallow" => shallow,

      "principal" => "github-import:test",

      "pushed_at" => DateTime.to_iso8601(DateTime.utc_now())

    }

    {:ok, _generation} =

      WAL.cas_index(storage_key, :none, WAL.append_entry(WAL.new_index(), entry))

    :ok

  end

  defp count_worktree_commits(source) do

    source |> sh!("git", ["rev-list", "--count", "--all"]) |> String.trim() |> String.to_integer()

  end

  defp count_commits(path) do

    {output, 0} = Repos.git(path, ["rev-list", "--count", "--all"])

    output |> String.trim() |> String.to_integer()

  end

  defp rev(source, ref) do

    source |> sh!("git", ["rev-parse", ref]) |> String.trim()

  end

  defp sh!(dir, command, args) do

    {output, 0} = System.cmd(command, args, cd: dir, stderr_to_stdout: true)

    output

  end

end

  ## ── EXIT-003: recovery from the WAL, never from the mirror ─────────────

  describe "the mirror contract" do

    # `EXIT-003` and `CLAUDE.md` both stated that no mirror was configured and

    # that none could be, while production had one configured through

    # `OPENAGENTS_FORGE_MIRROR_URLS_JSON`. Since `mirror_now/1` force-pushes

    # every ref, that made both documents promise the destructive thing was

    # switched off while it was switched on. Neither claim can return while

    # runtime still reads that variable.

    test "does not claim a mirror is unconfigured while runtime configures one" do

      assert File.read!("config/runtime.exs") =~ "OPENAGENTS_FORGE_MIRROR_URLS_JSON",

             "the guard below is only meaningful while runtime reads this variable"

      for path <- ["INVARIANTS.md", "CLAUDE.md"] do

        text = File.read!(path)

        for claim <- [

              "set by no environment",

              "no mirror runs today",

              "Automatic mirroring to GitHub is not configured"

            ] do

          refute String.contains?(text, claim),

                 "#{path} claims #{inspect(claim)}, which production contradicts"

        end

      end

    end

  end

  describe "recovery" do

    test "the WAL rebuilds the repository and re-derives receipts the database lost", context do

      seed_history!(context)