defmodule OpenAgents.Forge.Independence do
@moduledoc """
The public disclosure of how far this forge is from operator independence.
`docs/forge-operator-independence.md` states the trust boundary honestly, and
`EXIT-001` through `EXIT-005` bind the parts of it that are checkable. Both
are read by someone who already went looking. This projection is the same
statement placed where a person checking whether the forge is healthy will
see it, because "one operator can read and rewrite everything, and here is
what is not yet provable" is a fact about the service's condition in the same
way a lagging mirror is.
Three properties, and the honest answer to each is derived rather than
restated, so a disclosure cannot drift away from the thing it describes:
* **Export.** Counted from `OpenAgents.DataRights.ExportInventory`, the
ledger `EXIT-001` enforces against the surface. A family that regresses to
`:partial` or `:blocked` appears here without anyone editing this module,
and a gap closed elsewhere disappears from here in the same commit.
* **Verification.** `EXIT-002` and `EXIT-005` make a rewrite of the WAL
evident and total, not impossible. That distinction survives only while a
commitment exists somewhere the operator does not solely control, and
publishing one and having one witnessed are two different facts, so both
are published. `anchor_published` counts the anchors
`OpenAgents.Forge.Anchor` has actually written rather than a config flag,
so a publisher that has stopped reports `false`. `anchor_witnessed` is
`false` because no party other than the operator attests to the document:
the operator serves it and could serve any document, and its value is that
a third party can cheaply keep a copy, not that anyone has. Issue #151
carries the witness.
* **Private data.** Two facts that only mean something together. The account
export can be encrypted to a key the recipient holds (#178), and that is
derived: `OpenAgentsWeb.DataController`'s compiled import table either
reaches `OpenAgents.DataRights.Age` or it does not, so removing the
encryption removes the claim in the same commit. The private store is not
encrypted at rest, and that one is now derived too, from
`OpenAgents.Forge.AtRest`: the store is encrypted exactly when no private
column rests as plaintext, and the columns that do are named and proven
plaintext against the database rather than asserted here. The derivation
can only lower the claim, so an incomplete list understates the store
instead of flattering it; #193 carries what is left.
`operator_reads_source` is derived from the second fact rather than
restated, because it is the same fact: the operator reads the plaintext an
export is built from exactly while the store is plaintext. Publishing the
encryption without it would let a reader conclude the operator cannot read
an export, which is false.
`degraded?` is true while any of the three falls short, so the page does not
need a human to decide when to say so. It is expected to be true today.
A fourth section says how much any of that is worth. Every claim above is
derived from the running node, and the running node's code can be older than
the ledger those claims answer to: #187 found the forge serving a revision 57
commits behind `main`, where the export route, the chain, and this module
itself did not exist, while every proof of them stayed green on `main`. A
disclosure that cannot say how far its own code is from the code its proofs
ran against is asking to be trusted on exactly the point it exists to remove,
so `deployment` publishes that distance: the number of commits on the head
this node serves that the running revision does not carry (#246).
It is a distance and nothing else. Publishing the two revisions it lies
between would put a commit sha on an anonymous surface, which `STATUS-001`
keeps content-free and this section's own proof turns red for. It is also not
an axis of `degraded?`: a node one commit behind is not less independent, and
folding ordinary deploy lag into the independence verdict would make that
verdict mean nothing on the day it mattered.
Nothing here carries content: family names, counts, booleans, issue numbers,
a ref name, and a document path. It answers with the same shape when the
ledger is unreadable, so `STATUS-001`'s rule that the page renders during an
incident still holds.
"""
alias OpenAgents.DataRights.ExportInventory
alias OpenAgents.Forge.Anchor
alias OpenAgents.Forge.AtRest
alias OpenAgents.Forge.Pushes
alias OpenAgents.Forge.Repos
@schema "openagents.forge_independence.v1"
@anchor_issue 168
@witness_issue 151
@at_rest_issue 193
@document "docs/forge-operator-independence.md"
@export_encryption_module OpenAgents.DataRights.Age
@export_controller OpenAgentsWeb.DataController
@account_export OpenAgents.DataRights.AccountExport
# The ref a candidate is proven against before it is pushed: `RELEASE-004`
# binds the full local matrix to the exact candidate sha and
# `.githooks/pre-push` refuses a push without it, so the head of this ref is
# the newest revision the ledger's proofs have run against.
@proven_ref "refs/heads/main"
@sha ~r/\A[0-9a-f]{40}\z/
@doc "The disclosure, in the shape `/api/status` publishes it."
@spec projection() :: map()
def projection do
export = export_section()
verification = verification_section()
private_data = private_data_section()
%{
"schema" => @schema,
"degraded" => degraded?(export, verification, private_data),
"operator" => operator_section(),
"export" => export,
"verification" => verification,
"private_data" => private_data,
"deployment" => deployment_section(),
"document" => @document
}
end
@doc "Whether the forge falls short of independence on any disclosed axis."
@spec degraded?() :: boolean()
def degraded?,
do: degraded?(export_section(), verification_section(), private_data_section())
@doc """
Whether three given sections fall short on any axis.
Public because two of the three axes are constants today: no export is
encrypted and the anchor is witnessed by nobody, so `degraded?/0` would
report `true` even if an axis were dropped from the disjunction entirely.
Varying one section at a time is the only way a proof can show that each
axis is actually load-bearing.
"""
@spec degraded?(map(), map(), map()) :: boolean()
def degraded?(export, verification, private_data) do
export["gaps"] != [] or not verification["anchor_published"] or
not verification["anchor_witnessed"] or
not private_data["export_recipient_encryption"] or
not private_data["encrypted_at_rest"]
end
# Counted from the ledger, so this section cannot claim an export gap is
# closed while `EXIT-001` still records it, and cannot invent one either.
defp export_section do
entries = safely(fn -> ExportInventory.entries() end) || []
gaps =
for entry <- entries, entry.status in [:partial, :blocked] do
%{
"family" => Atom.to_string(entry.family),
"status" => Atom.to_string(entry.status),
"issue" => entry.issue
}
end
%{
"families" => length(entries),
"portable" => count(entries, :portable),
"partial" => count(entries, :partial),
"blocked" => count(entries, :blocked),
"not_user_data" => count(entries, :not_user_data),
"gaps" => Enum.sort_by(gaps, & &1["family"])
}
end
defp count(entries, status), do: Enum.count(entries, &(&1.status == status))
# Publishing an anchor and having one witnessed are two facts, and collapsing
# them into one boolean is how a surface starts claiming more than it can
# show. The operator serves the anchor document, so publication alone leaves
# a consistent rewrite undetectable to anyone who kept no copy of it.
#
# The count comes from the anchors actually written, not from a config flag,
# so a publisher that has stopped reports `false` without anyone editing
# this module. A failed read answers `false` too: the failure direction
# claims less than reality rather than more.
defp verification_section do
published? = safely(fn -> Anchor.published?() end) || false
%{
"property" => if(published?, do: "tamper_evident_published", else: "tamper_evident"),
"chained" => true,
"anchor_published" => published?,
"anchor_witnessed" => false,
"anchor" => Anchor.path(),
"issue" => if(published?, do: @witness_issue, else: @anchor_issue)
}
end
# `encrypted_at_rest` used to be the one stated value left here. It is now
# derived from `OpenAgents.Forge.AtRest`, which answers it from the columns
# that rest as plaintext rather than from a literal beside the disclosure. A
# failed read answers `false`, the same direction every other gather fails
# in: the store is claimed to be less protected than it is, never more.
defp private_data_section do
encrypted_at_rest? = safely(fn -> AtRest.encrypted_at_rest?() end) || false
%{
"export_recipient_encryption" => export_recipient_encryption?(),
"encrypted_at_rest" => encrypted_at_rest?,
"operator_reads_source" => not encrypted_at_rest?,
"access_controlled" => access_controlled?(),
"issue" => @at_rest_issue
}
end
# The export route encrypts to a recipient-held key exactly while it calls
# the module that does it. Reading the compiled import table is the same
# technique `EXIT-002` and `EXIT-003` use to keep a structural claim from
# decaying into a comment.
defp export_recipient_encryption? do
safely(fn -> @export_encryption_module in external_calls(@export_controller) end) || false
end
# "Takes the account and nothing else" is the whole access-control claim for
# this export, and it is a fact about the function's shape.
defp access_controlled? do
safely(fn ->
Code.ensure_loaded?(@account_export) and
function_exported?(@account_export, :build, 1) and
not function_exported?(@account_export, :build, 2)
end) || false
end
# The same read `EXIT-002`'s and `EXIT-003`'s proofs perform: the callee set
# a module was compiled with, which no comment or later refactor can flatter.
defp external_calls(module) do
case :beam_lib.chunks(:code.which(module), [:imports]) do
{:ok, {^module, [imports: imports]}} -> Enum.map(imports, &elem(&1, 0))
_unreadable -> []
end
end
# How far the code answering all of the above is from the code the ledger's
# proofs ran against. The forge hosts its own repository, so no outside
# service is consulted: the running revision is the one this node reports as
# deployed, the proven revision is the head of `refs/heads/main` in the bare
# projection this node serves, and the distance between them is a `git`
# count over objects already on disk.
#
# The claim is deliberately narrow. It is not "how far behind `main` this
# forge is" — nothing here can see a `main` this node declines to serve. It
# is "how many commits on the head this node serves the running revision does
# not carry", which is exactly what the count establishes. A forge that will
# not serve its own repository, a node whose bare projection is empty, and a
# release built from a revision this forge never accepted all report `known:
# false` and no distance, which is the same withholding `EXIT-005` and
# `EXIT-006` already decline to detect.
defp deployment_section, do: deployment(safely(&running_revision/0))
@doc """
The deployment section for a given running revision.
Public with the revision as a parameter for the same reason `degraded?/3` is
public: the node that runs this proof reports a build revision that is not a
commit at all, so the projection alone can only ever exercise the branch that
withholds. A proof that never sees a real distance cannot fail on the day one
stops being computed.
"""
@spec deployment(String.t() | nil) :: map()
def deployment(revision) do
behind = safely(fn -> commits_behind(revision) end)
%{
"proven_ref" => @proven_ref,
"known" => is_integer(behind),
"behind" => behind
}
end
defp commits_behind(running) do
with repo when is_binary(repo) <- List.first(Repos.allowed_repos()),
storage_key = Pushes.mirror_storage_key(repo),
true <- sha?(running),
head when is_binary(head) <- Map.get(Repos.refs(storage_key), @proven_ref),
true <- sha?(head) do
count_between(storage_key, running, head)
else
_unknown -> nil
end
end
# The same revision `/api/status` publishes for this node, so the distance
# and the revision it is measured from can never describe different code.
defp running_revision do
OpenAgents.Forge.DeploymentNode.health()["revision"]
end
defp sha?(value), do: is_binary(value) and Regex.match?(@sha, value)
# `running..head` counts commits reachable from the served head and not from
# the running revision, which stays the right number when the running
# revision is not an ancestor of it. A revision the bare projection has never
# heard of fails the read, and a failed read claims nothing.
defp count_between(storage_key, running, head) do
case Repos.git(Repos.bare_path(storage_key), [
"rev-list",
"--count",
"--end-of-options",
running <> ".." <> head
]) do
{output, 0} ->
case Integer.parse(String.trim(output)) do
{behind, ""} -> behind
_unparsed -> nil
end
_unreadable ->
nil
end
end
defp operator_section do
%{
"model" => "single_operator",
"separation_of_duties" => false,
"operator_reads_audited" => false,
"mirror_is_authority" => false
}
end
defp safely(fun) do
fun.()
rescue
_error -> nil
catch
_kind, _reason -> nil
end
end