A mirror is configured in production, and it holds 307 commits the forge cannot serve #188

Closed AtlantisPleb opened this 4d ago 1 comment

Evidence

Shipped in the release at b77305e, promoted 2026-08-26.

What happened

#180's rehearsal 3 (mirror divergence) was performed against the live forge on 2026-08-24. It found two things, and the second is the one that matters.

A mirror is configured, and two contracts say none is

EXIT-003 states as an operational fact: ":forge_mirror_urls is empty in config/config.exs and set by no environment, so no mirror runs today." CLAUDE.md repeats it: "Automatic mirroring to GitHub is not configured today."

Both are false against production. config/runtime.exs:375 reads OPENAGENTS_FORGE_MIRROR_URLS_JSON, and the live node answers:

Application.get_env(:openagents, :forge_mirror_urls, %{}) |> Map.keys()
#=> ["openagents.com"]

GET /api/status publishes it too — forge.mirror reports {"repo": "openagents.com", "state": "current"}, and OpenAgents.Forge.MirrorWatch.check_all/2 publishes a per-repo state only when at least one repo has a configured mirror URL.

This is not a small correction. mirror_now/1 is git push --mirror, a force push of every ref, and EXIT-003 records that "configuring a mirror overwrites what direct pushes left there rather than merging with it." A contract that says the dangerous thing is not configured, while it is, is worse than one that says nothing.

The mirror is the only complete copy of half this repository

Ref maps are identical — 25 refs, same shas on both sides, refs/heads/main at 773ad680 on each. History is not.

Source main commits git fsck Holds c91327d6
GitHub mirror 603 clean yes
The forge 296 (from the seed eda094c6 forward) full clone aborts no

307 commits — 51% of main — exist only on GitHub. They are not in the WAL either: the log was seeded from a --depth=1 fetch, so no rebuild can produce them (#179).

EXIT-003 says recovery comes from the WAL and the mirror is strictly lossy. For everything pushed since the seed that holds. For everything before it the relation is inverted: the mirror is strictly richer, and it is the only copy. #179 already says the pre-seed history "remains outside this forge and is not recoverable from it"; what it does not say is where it is, which is the mirror this repository's contracts describe as never an input to recovery.

Reproduction

git ls-remote https://openagents.com/OpenAgentsInc/openagents.com.git > forge-refs.txt
git ls-remote https://github.com/OpenAgentsInc/openagents.com.git > gh-refs.txt
diff <(sort forge-refs.txt) <(sort gh-refs.txt)          # identical

git clone https://github.com/OpenAgentsInc/openagents.com.git mirror
git -C mirror rev-list --count refs/heads/main            # 603
git -C mirror rev-list --count c91327d6                   # 307
git -C mirror cat-file -t c91327d6                        # commit

git clone https://openagents.com/OpenAgentsInc/openagents.com.git forge
# remote: error: Could not read c91327d60c520d11133ddcc6cb3304784f2f0481

Acceptance criteria

  • EXIT-003 and CLAUDE.md state what is actually configured, or the mirror is unconfigured and they become true again. A proof reads the configuration rather than asserting the empty default.
  • The relationship between the mirror and the pre-seed history is recorded in docs/forge-operator-independence.md, because "the mirror is strictly lossy" is not true of it.
  • A decision about the 307 commits: import them into the WAL so the forge is canonical for its own history, or record that the forge is canonical only from the seed forward and say so where a reader looks.

Found by: #180's rehearsal 3, docs/forge-exit-rehearsals.md.

  1. AtlantisPleb opened this issue 4d ago
  2. started work on a box 3d ago
  3. stopped this work 3d ago
  4. started work on a box 3d ago
  5. stopped this work 3d ago
  6. A AtlantisPleb Author 3d ago

    Closing. All three criteria are met. The third was a decision, and it is made
    and recorded; performing it is a production write that is tracked separately and
    named below.

    1. The contracts state what is configured

    Already true when this pass started, and confirmed rather than redone.
    INVARIANTS.md EXIT-003 says :forge_mirror_urls is set from the environment
    and that production sets it for openagents.com; CLAUDE.md says production
    mirrors to GitHub with a force push of every ref; and
    test/openagents/forge/independence_test.exs fails if either re-asserts "set by
    no environment", "no mirror runs today", or "Automatic mirroring to GitHub is
    not configured".

    One file was still carrying the old sentence: docs/forge-operator-independence.md
    said ":forge_mirror_urls is empty in config/config.exs and set by no
    environment, so no mirror runs today". That is where the mirror contract is
    written in prose, and the first pass corrected the two files a test named while
    leaving the third to be found by hand a second time. It is corrected, and it has
    joined the guarded set, so it cannot drift back either.

    2. The mirror and the pre-seed history

    Recorded in docs/forge-operator-independence.md, under a heading of its own,
    with both halves of the relation and the boundary between them.

    • From the seed forward — everything the WAL records — the mirror is
      strictly lossy and is never an input. It carries objects and refs and no
      evidence. That is EXIT-003 and it is the load-bearing direction.
    • Before the seed — 307 commits — the relation is inverted. The log was
      seeded from a --depth=1 fetch (#179), so no rebuild can produce them, and
      the mirror is the only copy.

    Re-measured 2026-08-25, both sides cloned fresh:

    Source main commits git fsck Root of main Holds c91327d6
    The forge 461 clean eda094c6, the seed no
    GitHub mirror 767 clean a352f78e yes

    The counts are at different tips because the forge was ahead of the mirror when
    they were read; the gap is unchanged at 307, and git rev-list --count eda094c6
    on the mirror is 308 — the seed and its 307 ancestors.

    Two things changed since this issue was filed, and both are now recorded. The
    full clone from the forge succeeds — it aborted when this was written — and
    it is grafted rather than truncated: it writes a shallow file naming five
    reconciled boundaries and passes git fsck. So EXIT-004 is not violated by
    the gap. Its #179 amendment already admits a grafted repository as clean,
    because history that says where it stops is servable and history that dangles is
    not. I looked for a disagreement between that proof and the deployment and did
    not find one; what is true is narrower and is now said without softening — the
    forge is canonical for its own history only from the seed forward.

    EXIT-003 carries an amendment saying that "strictly lossy" is a claim about
    evidence and was being read as a claim about objects.

    3. The decision

    Import the objects; do not manufacture the evidence. Recorded in
    docs/forge-operator-independence.md with the two rejected shapes.

    Fabricating WAL entries for the 307 commits would invent evidence: an entry
    carries a sequence, a principal, and a time, and
    OpenAgents.Forge.Pushes.reconcile_receipts/1 derives a receipt from every one,
    so it would publish receipts for pushes nobody made — in a log whose whole value
    is that a receipt derives from the WAL and never from a second authority.

    Recording "canonical from the seed forward" and stopping is true of the present
    and is not enough as an end state: it leaves the forge's own authority holding
    less than its mirror does, and leaves the only copy of this repository's first
    year on an account the operator does not control.

    So: the log gains the bytes and gains no assertion about who pushed them.
    OpenAgents.Forge.Backfill.import_history/3 is that operation and it already
    exists — it appends the bundle as a git_bundle entry with the ref map
    unchanged, an empty shallow set, and a principal recording who authorized the
    import; it proves the bundle against a throwaway repository borrowing the
    projection's objects and refuses to write unless every boundary's parents
    resolve and the union walks. Six tests cover it, including a rebuild from
    sequence zero afterwards. It takes a file an operator supplies rather than
    reaching for a mirror, so EXIT-003's proof is untouched.

    The bundle is built and verified: the mirror holds all five boundaries' parents
    (fdd00d4c, e0e61fb1, 0fcbbbb8, f8a7822a, c91327d6), one bundle closes
    all five, it is 7.2 MB, and git bundle verify reports a complete history. The
    recipe and the invocation are now step 5 of rehearsal 3 in
    docs/forge-exit-rehearsals.md.

    The import itself was not performed, and that is the one thing this issue
    does not close. A WAL append cannot be undone — EXIT-005 makes removing an
    entry a rewrite of the whole suffix — and if a node cannot materialize the
    entry, Sync falls back to a full rebuild from sequence zero, which rehearsal 3
    records as never having been run against the live projection. That is an
    attended operation on a forge people are pushing to. It is a separate unit of
    work with its own acceptance criteria and is tracked as the follow-up linked
    below, with the bundle recipe, the boundary table, and the convergence checks
    already written out.

    Follow-up: #256.

  7. closed this as completed 3d ago
  8. AtlantisPleb referenced this in 08fe300 3d ago
Sign in with GitHub to comment on this issue.