The storage key the forge configuration admits is not the one it serves, and a stale bare repository shadows it #190

Closed AtlantisPleb opened this 7d ago 3 comments

Evidence

Shipped in the release at 0bf2706, promoted 2026-08-27.

1 pushes receipt

What happened

#180's rehearsal 2 (receipt verification) was performed against the live forge on 2026-08-24. Its first documented step does not work.

The rehearsal says to run OpenAgents.Forge.Verification.verify("{storage_key}"). The obvious value for {storage_key} is what the forge says it serves:

OpenAgents.Forge.Repos.allowed_repos()
#=> ["openagents.com"]

OpenAgents.Forge.Verification.verify("openagents.com")
#=> {:error, %{entries: 0, repo: "openagents.com",
#=>            findings: [%{code: "wal_unreadable", detail: %{"reason" => ":not_found"}}]}}

The repository people clone is keyed by a UUID instead:

#=> {"OpenAgentsInc", "openagents.com", "ecd89cf6-f602-479f-9f47-266307345aaa"}

And a bare repository under the name exists beside it, holding one stale ref:

Storage key	Bare repo on disk	Refs	`refs/heads/main`	WAL entries
`openagents.com`	yes	1	`5722913cb1b91df3`	none — `:not_found`
`ecd89cf6-f602-479f-9f47-266307345aaa`	yes	24	`773ad68060309963`	275
`sarah`	yes	2	`190383e2c3922 6b0`	55

So /var/lib/openagents/forge/repos/openagents.com.git is a projection of nothing, pinned at a commit the served repository passed long ago, and it is the one an operator following the runbook reaches first.

Why it matters

wal_unreadable is the finding a verifier reports when the log is gone. An operator who runs the documented command on the documented name gets the report that means "your write-ahead log is missing" for a repository whose log is intact — which is the worst possible false positive for a check whose whole purpose is telling an operator whether to trust what the forge serves.

EXIT-002 is not wrong; it verifies what it is pointed at. What is missing is that nothing connects the name a person has to the key the verifier needs.

Acceptance criteria

Verification.verify/1 accepts an owner/name path, or the rehearsal and #179 name the exact query that resolves one to a storage key.
The stale openagents.com.git bare repository is removed or explained. A directory that serves no repository and records no log should not sit in the same directory as the ones that do.
Repos.allowed_repos/0 returning a name that verify/1 cannot use is either reconciled or documented at both ends.

Found by: #180's rehearsal 2, docs/forge-exit-rehearsals.md.

AtlantisPleb opened this issue 7d ago
A AtlantisPleb Author 6d ago

First acceptance criterion landed on main in 183d69e: Verification.verify/1 accepts an owner/name path, resolves it through the serving path's repository mapping, names both the request and the resolved key in the report, and returns a typed repository_not_found distinct from wal_unreadable. Remaining here: the stale bare repo at /var/lib/openagents/forge/repos/openagents.com.git and the allowed_repos reconciliation — both operator/prod work.
AtlantisPleb closed this as completed in bc44604 5d ago
A AtlantisPleb Author 5d ago
Fixed in bc44604.

What the two strings are. A repository has a name — openagents.com, or the OpenAgentsInc/openagents.com path you clone — and a storage key, Repository.storage_key, which is the single path segment the WAL keeps a log under and a node keeps a bare repository under. Repos.allowed_repos/0 returns names. Everything in Repos that reaches the disk takes keys. Nothing connected the two, and because a name is a legal path segment, passing one where a key belonged built a path rather than failing: verify("openagents.com") read a WAL that was never written and reported wal_unreadable about a log of 275 entries that was intact under ecd89cf6-f602-479f-9f47-266307345aaa.

What changed. OpenAgents.Forge.RepoRef is now the one place a name becomes a key, and the order is fixed: a string the WAL holds a log under is a key and resolves to itself with no database read — so a verifier handed a key stays independent of PostgreSQL, and a log older than the repositories table still resolves. Anything else is a name, resolved through the same mapping the serving path uses, namespace aliases included. A reference that settles on no repository, or on two, stops with repository_not_found, repository_name_ambiguous, or repository_lookup_unavailable, and never becomes a path — so an absent repository can no longer look half-alive.

Repos states the distinction in its types and docs. Verification.verify/1 accepts the name people are told to use and reports both the reference asked for and the key checked. The coding lane's workspace (OpenAgents.Tools.Repository) made the same mistake and cloned from the shadow directory; it resolves first now. Pushes.mirror_storage_key/1 delegates, so there is one resolver rather than two orders.

Rehearsal 2 step 1 in docs/forge-exit-rehearsals.md now reads verify("OpenAgentsInc/openagents.com"), EXIT-002 records the rule, and docs/taxonomy.md defines the two words. test/openagents/forge/repo_ref_test.exs proves the documented name verifies clean against the UUID-keyed repository, that a bare repository seeded under the name does not shadow it, and that an unknown name is a typed finding rather than an empty repository.

The stale directory is still there, on purpose. No code deletes it, and the new test seeds one to prove the verifier is not fooled by it and leaves it standing. Cleanup is an operator action. It is safe to remove now that the coding lane no longer clones from it, and it holds nothing the WAL does not: one ref at 5722913cb1b91df3, no log of its own. On the forge host:
```
mv /var/lib/openagents/forge/repos/openagents.com.git /var/lib/openagents/forge/repos/openagents.com.git.retired-190
```
Verify after the move that the served repository is unaffected — OpenAgents.Forge.Verification.verify("OpenAgentsInc/openagents.com") should still report findings: [] — and delete the retired directory once you are satisfied.

One thing this did not fix. REPOSITORY-001 says the first repository "keeps the historical openagents.com key so existing WAL and object storage remain authoritative". The migration seeds it that way and the live row no longer matches: production's key is the UUID. EXIT-002 now records that drift; reconciling REPOSITORY-001 itself is a separate decision about which the invariant should be.
A AtlantisPleb Author 5d ago
Still reproduces on the live forge on 2026-08-25, revision 46cf8a5aea3791936c22e82c145a9a8dd734374d, while performing #180's rehearsal 2 again.

The first acceptance criterion is met: Verification.verify/2 accepts an owner/name path, and verify("OpenAgentsInc/openagents.com") resolves through ReceiptRepository.resolve/1 to ecd89cf6-f602-479f-9f47-266307345aaa and reports findings: [] over 382 entries. That is what made the rehearsal runnable at all.

The other two are unchanged:
```
OpenAgents.Forge.Repos.allowed_repos()
#=> ["openagents.com"]

OpenAgents.Forge.Verification.verify("openagents.com")
#=> {:error, %{entries: 0, storage_key: "openagents.com",
#=>            findings: [%{code: "wal_unreadable", detail: %{"reason" => ":not_found"}}]}}
```
/var/lib/openagents/forge/repos holds seven directories: openagents.com.git plus six UUID-keyed ones. Four repositories are anchored at /.well-known/openagents-forge-anchor.json, and openagents.com is not one of their storage keys. So allowed_repos/0 returns a name that is neither a storage key nor the full set of repositories this forge serves, and an operator following the runbook literally still reaches wal_unreadable for a log that is intact.

Recorded in docs/2026-08-25-forge-exit-rehearsals-2-to-6.md.