Encrypt an account export to a key the operator does not hold

18cb0ca26fda · AtlantisPleb · · parent 9b7e43b37ba6

Encrypt an account export to a key the operator does not hold

An export a recipient can only open with a decryptor the operator wrote
still leaves the operator defining the terms on which you read your own
data, so `GET /data/export/account?recipient=age1…` returns an age v1
document instead. The private half of the identity is generated on the
account's machine and never reaches this forge, and `age`, `rage`, or any
other implementation of the specification opens it.

The decision this settles is #178, and the threat model is the substance
of it. Encrypting to a key the operator holds protects a stolen backup
and nothing else; encrypting to a key only the account holds protects
the artifact from the operator too. What made the export path decidable
where the storage layer is not is that an export is derived: losing the
key costs one repeated download, while a storage key lost the same way
would be permanent. `docs/2026-08-24-private-export-encryption.md`
records that, the four rejected options, and what the operator can still
read — which is everything, because the document is built by reading
plaintext PostgreSQL.

So the claim is published in the shape it is true. EXIT-006's
private-data section now derives `export_recipient_encryption` from
whether the controller was compiled against the encrypting module, and
`access_controlled` from `AccountExport.build/1` existing at arity one
and no other. `operator_reads_source` is derived from
`encrypted_at_rest`, because it is the same fact from the reader's side
and publishing the encryption without it would let someone conclude the
operator cannot read an export. `degraded?` gained the at-rest axis, so
closing the export half cannot make a forge with plaintext columns
report itself independent the moment an anchor appears. #193 carries the
storage half rather than leaving it implied by the half that closed.

The proof is that `age` itself decrypts the output — over one chunk,
over 8.1 MB spanning 124 STREAM chunks, and over an empty document —
with an independent decryptor covering hosts where the binary is absent
and a fixture the real `age` produced pinning that decryptor to the
reference. Mutating the header MAC label and the final-chunk flag both
made `age` refuse the document. Making the file key constant did not
bite at first: two documents differ byte for byte anyway, because the
ephemeral key is fresh per call, so the assertion now reads each file
key back out and compares those.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DTmy4SEXrHXouw5sZbs3f4
Co-Authored-By
Claude Opus 5 (1M context) <noreply@anthropic.com>

Deploy story

What this commit did to the running system — joined from the forge receipt chain, the part a commit page elsewhere cannot show.

Not deployed through the forge lane

No push, promotion, build, or deploy receipt references this commit (receipts are scanned over a bounded recent window). Changes shipped by full node replacement carry their proof in the release gate receipt instead.

Changed files

  • modified INVARIANTS.md
  • added docs/2026-08-24-private-export-encryption.md
  • modified docs/forge-operator-independence.md
  • added lib/openagents/data_rights/age.ex
  • modified lib/openagents/forge/independence.ex
  • modified lib/openagents_web/controllers/data_controller.ex
  • modified lib/openagents_web/live/memory_live.ex
  • added test/fixtures/age/README.md
  • added test/fixtures/age/reference.age
  • modified test/openagents/data_rights/account_export_test.exs
  • added test/openagents/data_rights/age_test.exs
  • modified test/openagents/forge/independence_disclosure_test.exs
  • modified test/openagents/network_status_test.exs
  • added test/support/age_document.ex

Diff

14 files changed, +952 -42

INVARIANTS.md modified +42 -13

@@ -4151,11 +4151,34 @@ true today. Two of the axes are constants, so `degraded?/0` would stay true

4151 4151
even if one were dropped from the disjunction; `degraded?/3` is public for
4152 4152
exactly that reason and the proof varies one axis at a time through it.
4153 4153
4154
One claim is stated rather than derived and it says so: no export is encrypted
4155
to a key the recipient holds and no Ecto column in this repository is encrypted
4156
at rest, which issue #178 carries. There is no registry of encrypted columns to
4157
count, and inventing one so a number could appear would be the kind of claim
4158
this disclosure exists to prevent.
4154
The private-data section is two facts that only mean something together, and
4155
one of them is now derived. `export_recipient_encryption` reads whether
4156
`OpenAgentsWeb.DataController` was compiled against `OpenAgents.DataRights.Age`
4157
— the same compiled-import-table read `EXIT-002` and `EXIT-003` use — so an
4158
export route that stopped encrypting stops being claimed to, in the same
4159
commit. `access_controlled` is derived from `AccountExport.build/1` existing at
4160
arity one and no other, which is the whole access-control claim for this
4161
document: no parameter can widen it. `operator_reads_source` is derived from
4162
`encrypted_at_rest`, because it is the same fact stated from the reader's side,
4163
and publishing the encryption without it would let a reader conclude the
4164
operator cannot read an export.
4165
4166
`encrypted_at_rest` is the one value still stated rather than derived, and it
4167
says so: no Ecto column in this repository is encrypted at rest, which issue
4168
#193 carries. There is no registry of encrypted columns to count, and inventing
4169
one so a number could appear would be the kind of claim this disclosure exists
4170
to prevent.
4171
4172
Amended 2026-08-24 (issue #178). The decision that a private export can be
4173
encrypted to a key the operator does not hold is recorded in
4174
`docs/2026-08-24-private-export-encryption.md` with its threat model and four
4175
rejected options. `degraded?` gained the at-rest axis in the same change,
4176
because closing the export half without it would have let a forge with
4177
plaintext columns report itself independent the moment an anchor appeared. The
4178
published key set moved with the claim, as `STATUS-001` requires:
4179
`independence.private_data.exports_encrypted` was replaced by
4180
`export_recipient_encryption`, and `operator_reads_source` was added. That is
4181
the decision being asked for, not an accident.
4159 4182
4160 4183
`STATUS-001`'s rule holds here: the section carries counts, booleans, family
4161 4184
names, issue numbers, and one document path, and the proof asserts that every

@@ -4164,16 +4187,22 @@ path, an account id, a node name, or a commit sha reaching it turns the proof

4164 4187
red. The whole section degrades to `nil` like every other gather, so a node
4165 4188
that cannot assemble it renders the page without it rather than failing.
4166 4189
4167
Four mutations were confirmed to fail the proof and reverted: publishing an
4190
Six mutations were confirmed to fail the proof and reverted: publishing an
4168 4191
empty gap list while the ledger records gaps; making `degraded` constant;
4169
adding the forge's repository name to the projection; and claiming exports are
4170
encrypted. A fifth was added with the anchor: dropping the witness axis from
4171
the disjunction, which `degraded?/0` cannot detect and `degraded?/3` does.
4192
adding the forge's repository name to the projection; claiming exports are
4193
encrypted; dropping the witness axis from the disjunction, which `degraded?/0`
4194
cannot detect and `degraded?/3` does; dropping the at-rest axis, which the
4195
plaintext-store assertion catches once an anchor is configured; and hardcoding
4196
`export_recipient_encryption` to `true` while removing the encryption from the
4197
route, which the derivation assertion catches even though the projection does
4198
not.
4172 4199
4173 4200
Evidence: `OpenAgents.Forge.Independence`, `OpenAgents.NetworkStatus`,
4174
`OpenAgentsWeb.NetworkStatusLive`,
4175
`test/openagents/forge/independence_disclosure_test.exs`, and
4176
`docs/forge-operator-independence.md`.
4201
`OpenAgentsWeb.NetworkStatusLive`, `OpenAgents.DataRights.Age`,
4202
`test/openagents/forge/independence_disclosure_test.exs`,
4203
`test/openagents/data_rights/age_test.exs`,
4204
`docs/forge-operator-independence.md`, and
4205
`docs/2026-08-24-private-export-encryption.md`.
4177 4206
4178 4207
### STACK-001 — A pull request stack is a durable object, not inferred topology
4179 4208

@@ -4830,7 +4859,7 @@ contract; the invariant prose above defines the assertion, not the filename.

4830 4859
| EXIT-003 | `test/openagents/forge/independence_test.exs` |
4831 4860
| EXIT-004 | `test/openagents/forge/independence_test.exs` |
4832 4861
| EXIT-005 | `test/openagents/forge/independence_test.exs`, `test/openagents/forge/wal_test.exs`, `test/openagents/forge/git_http_test.exs`, `test/openagents_web/controllers/push_receipt_controller_test.exs`, `test/openagents_web/controllers/forge_anchor_controller_test.exs` |
4833
| EXIT-006 | `test/openagents/forge/independence_disclosure_test.exs` |
4862
| EXIT-006 | `test/openagents/forge/independence_disclosure_test.exs`, `test/openagents/data_rights/age_test.exs` |
4834 4863
| STACK-001 | `ops/ci/stack-contracts.sh`, `test/openagents/stacks_test.exs` |
4835 4864
| ISSUE-001 | `test/openagents/forge/commit_references_test.exs`, `test/openagents/issues/closing_references_test.exs`, `test/openagents/forge/push_closes_issues_test.exs` |
4836 4865
| FORUM-001 | `test/openagents/forum/legacy_surface_test.exs`, `test/openagents_web/live/forum_live_test.exs`, `test/openagents_web/route_authority_test.exs`, `test/openagents_web/sidebar_state_test.exs` |
docs/2026-08-24-private-export-encryption.md added +176

@@ -0,0 +1,176 @@

1
# Encrypting a private export to a key the operator does not hold
2
3
**Date:** 2026-08-24
4
**Issue:** #178
5
**Parent:** #94
6
**Companion:** `docs/forge-operator-independence.md`
7
8
#94 asks that private repository exports stay encrypted and access controlled.
9
Access control held. Encryption did not, and `docs/forge-operator-independence.md`
10
recorded that rather than claiming it. This document is the decision that
11
replaces the placeholder.
12
13
## 1. The decision
14
15
**The account export encrypts to an `age` recipient the account supplies, and
16
the storage layer behind it stays plaintext. Both facts are published
17
together.**
18
19
```sh
20
age-keygen -o key.txt
21
age-keygen -y key.txt          # age1…
22
```
23
24
Give the `age1…` value to `GET /data/export/account?recipient=age1…`, or paste
25
it into the field beside the export button on `/memory`. The response is an
26
[age v1](https://age-encryption.org/v1) document. Open it with any
27
implementation of that specification:
28
29
```sh
30
age --decrypt -i key.txt openagents-account-data.json.age > export.json
31
```
32
33
Omitting `recipient` returns today's plain JSON, because an export nobody can
34
open is not portability either. Supplying a recipient that is not an `age`
35
public key returns `422` rather than falling back to plaintext: a request that
36
asked for encryption never silently receives an unencrypted document.
37
38
## 2. What this protects, exactly
39
40
The threat model matters more than the mechanism, because the two obvious
41
designs protect against different adversaries and only one of them is worth
42
naming.
43
44
**Encrypting to a key the operator holds** — the shape of the three existing
45
vaults, which take their keys from the operator's environment — protects
46
against a stolen database dump, a stolen backup, and a stolen bucket. It
47
protects against nothing else. The operator decrypts at will, because the
48
operator has the key.
49
50
**Encrypting to a key only the account holds** protects against the operator,
51
for the artifact it covers. The private half of an `age` identity is generated
52
on the account's machine and never reaches this forge.
53
54
What that buys here is bounded and the bound is the substance:
55
56
- **Protected:** the exported document after it leaves the application
57
  boundary. Its copies, whatever the response passes through in transit, any
58
  proxy or access log that captures a body, any backup of a downloaded file,
59
  and any later re-reading of that file by anyone who obtains it, the operator
60
  included.
61
- **Not protected:** the contents. The forge builds the export by reading
62
  plaintext PostgreSQL. The operator held every record before this ran and
63
  holds them still. Nothing in this decision makes an operator read of the
64
  source auditable, and `ADMIN-001` records that no operator read is audited.
65
66
So this is encryption of an artifact, not confidentiality of a store. Saying
67
otherwise would be exactly the narrowing #178's contract forbids: "Do not
68
narrow the claim to the export path while the storage layer stays plaintext.
69
Either both are stated, or neither is claimed." Both are stated —
70
`GET /api/status` publishes `independence.private_data.export_recipient_encryption`
71
as `true` and, beside it, `encrypted_at_rest` as `false` and
72
`operator_reads_source` as `true`, the last derived from the second so the
73
pair cannot drift apart.
74
75
## 3. Why key loss decides this
76
77
The usual objection to recipient-held keys is that the account loses its data
78
when it loses the key. That objection is decisive for storage and does not
79
apply to an export at all, and noticing the difference is what made this
80
decidable.
81
82
**An export is derived.** An account that loses its `age` identity generates
83
another one and requests the export again. Nothing is lost, because the export
84
was never the record — PostgreSQL is, and it is unchanged. The cost of key
85
loss on this path is one repeated download.
86
87
**Storage is not derived.** A key only the account holds, applied to columns,
88
makes key loss permanent data loss and makes every server-side read
89
impossible: the issue list, search, rendering, and the per-repository
90
projections `TRANSPARENCY-001` publishes all read those columns on the server.
91
92
That asymmetry is the whole reason recipient-held encryption is right on one
93
side of the boundary and wrong on the other, and it is why the storage half is
94
a separate decision under #193 rather than an unfinished part of this one.
95
96
## 4. Options rejected
97
98
**Encrypt the export under an operator-held key.** Rejected. It protects a
99
stolen backup, which the disk and bucket already do, and publishing it would
100
read as protection from the operator while providing none. It is the claim
101
`EXIT-006` exists to keep off the status page.
102
103
**Encrypt every column at rest under an operator-held key.** Rejected for this
104
issue, for the same reason, and separately because it is a large change that
105
buys the threat model nothing it does not already have. `#193` carries the
106
at-rest question with the real trade written down.
107
108
**Encrypt every column at rest under an account-held key.** Rejected. Key loss
109
becomes permanent data loss, and every server-side read of that column stops
110
working — which is most of the product. A serious version of this decides
111
*which* columns stop being server-readable and what the account gives up, and
112
that is a product decision rather than a cryptography one. `#193`.
113
114
**A bespoke sealed-box format with an OpenAgents decryptor.** Rejected, and
115
this is the rejection that matters most. An export a recipient can only open
116
with software the operator wrote leaves the operator defining the terms on
117
which you read your own data. The point of the exercise is independence, so
118
the format has to be one the recipient can read with a tool nobody here
119
controls. `age` has multiple independent implementations and a published
120
specification.
121
122
**PGP/GnuPG.** Rejected on the same axis it wins on. It is more widely
123
installed, and its key model, subkey handling, and armour formats are a large
124
surface to get right in a hand-rolled encoder. `age` is one recipient stanza,
125
one AEAD, and one stream construction — small enough to implement correctly
126
against the specification and check against the reference implementation,
127
which is what `test/openagents/data_rights/age_test.exs` does.
128
129
**Encrypt the two `DATA-004` conversation exports too.** Not rejected, not
130
done. `GET /data/export`, `GET /data/export/atif`, and `GET /memory/export`
131
take the same shape and would take the same parameter. `#178` asked about the
132
account export, and widening the surface in the same change would have widened
133
what needs proving. It is a small follow-on rather than a gap in the decision.
134
135
## 5. What was proven, and how
136
137
`test/openagents/data_rights/age_test.exs` and the route tests in
138
`test/openagents/data_rights/account_export_test.exs`.
139
140
The claim is not "the bytes round trip" — two of our own implementations
141
agreeing would show nothing. It is that the recipient reads the document
142
**without the operator's software**, so three assertions carry it:
143
144
1. The real `age` binary decrypts what `OpenAgents.DataRights.Age` produces.
145
   This ran with `age` v1.3.1 when this decision landed, over a single chunk,
146
   over 8.1 MB spanning 124 `STREAM` chunks, and over an empty document.
147
2. An independent decryptor in `test/support/age_document.ex`, written from
148
   the specification and sharing no code with the module under test, decrypts
149
   it too, so the property is checked where `age` is not installed.
150
3. That decryptor reads `test/fixtures/age/reference.age`, a document the real
151
   `age` binary produced, which pins our reading of the format to the
152
   reference implementation and is what makes assertion 2 mean anything.
153
154
Each proof was mutation-checked: the header MAC's HKDF label was corrupted
155
(`age` refused the document, exit 1), the final-chunk flag was pinned to zero
156
(`age` refused it again), the file key was made constant, the route was made
157
to ignore the recipient, and the route was made to fall back to plaintext on
158
an invalid recipient. Every mutation was confirmed red on the named assertion
159
and reverted.
160
161
The constant-file-key mutation **did not bite at first**, and the gap it
162
exposed is worth recording rather than quietly closing. The original assertion
163
compared two documents and found them different — but the ephemeral X25519 key
164
is fresh per call, so two documents differ byte for byte even when every file
165
key is identical, and a recovered file key would then open every export ever
166
issued. The assertion now reads the file key back out of each document and
167
compares those. The mutation bites.
168
169
## 6. What is still open
170
171
- **The store is plaintext.** `#193`.
172
- **No operator read is audited.** `ADMIN-001` records it as a decision: an
173
  access log the operator writes into the operator's own database is evidence
174
  to the operator and to nobody else. Making it accountable needs the external
175
  anchor `#151` and `#168` carry.
176
- **The conversation exports are not encrypted.** Section 4.
docs/forge-operator-independence.md modified +25 -8

@@ -140,7 +140,9 @@ What is proven portable today:

140 140
  deployment requests and approvals, Box leases and runs, paired computers, and
141 141
  agent links.** `GET /data/export/account`, an account-scoped document
142 142
  published alongside the conversation export rather than inside it. Every
143
  collection states its cap and reports its own truncation.
143
  collection states its cap and reports its own truncation. Adding
144
  `?recipient=age1…` returns the same document encrypted to a key you
145
  generated and this forge never sees (#178).
144 146
- **Pull requests, stacks, and issue dependencies.** The same document's
145 147
  `repository_work` section, which is the one read on this surface that crosses
146 148
  repositories. Enumeration was the easy half: the read filters on the column

@@ -267,12 +269,27 @@ The one omission is the `refs/internal/` namespace, where stack boundary

267 269
commits are retained without being advertised; the proof asserts that this is
268 270
the *only* omission, so withholding a branch would turn it red.
269 271
270
That is exit for source. It is not yet exit for everything: private repository
271
exports are not encrypted, and no commitment to the WAL is published outside
272
operator storage.
273
Those are covered by the gaps above rather than by a claim. Saying so is the
274
point. Five green invariants that assert less than they appear to would be
275
worse than five plus a recorded gap.
272
That is exit for source. It is not yet exit for everything, and the remaining
273
gaps are named rather than softened.
274
275
The account export can now be encrypted to a key the operator does not hold.
276
`GET /data/export/account?recipient=age1…` returns an
277
[age v1](https://age-encryption.org/v1) document, decrypted with `age` or any
278
other implementation of that specification, from a private key generated on
279
your own machine that never reaches this forge. What that protects is the
280
file — its copies, whatever the response passes through, any later reading of
281
it by anyone who obtains it, the operator included. What it does not protect is
282
the contents, because the forge builds the export by reading plaintext
283
PostgreSQL and held every record before the encryption ran.
284
`docs/2026-08-24-private-export-encryption.md` records that boundary, the
285
threat model on both sides of it, and the four options rejected. Both halves
286
are published together at `GET /api/status`, so a reader who sees that an
287
export can be encrypted also sees that the store behind it is not.
288
289
No commitment to the WAL is published outside operator storage, and no column
290
is encrypted at rest. Those are covered by the gaps below rather than by a
291
claim. Saying so is the point. Five green invariants that assert less than they
292
appear to would be worse than five plus a recorded gap.
276 293
277 294
## Invariants
278 295

@@ -327,5 +344,5 @@ the one people clone from.

327 344
| --- | --- |
328 345
| The live forge cannot serve a full clone of its own repository | #179 |
329 346
| The published WAL anchor is served by the operator and witnessed by nobody, so a consistent rewrite is caught only by a reader who kept a copy | #151 |
330
| No export is encrypted to a key the recipient holds, and no column is encrypted at rest | #178 |
347
| No column is encrypted at rest, so the operator reads the source every export is built from | #193 |
331 348
| Five of six exit rehearsals have never been performed | #180 |
lib/openagents/data_rights/age.ex added +210

@@ -0,0 +1,210 @@

1
defmodule OpenAgents.DataRights.Age do
2
  @moduledoc """
3
  Encrypts an export to an `age` recipient the operator does not hold.
4
5
  `EXIT-001` proves an account can obtain its own records. It says nothing
6
  about who else can read the document on the way out, and until now the
7
  answer was "the operator, and anyone the operator's transport touches",
8
  because the export was plain JSON over TLS. This module closes the part of
9
  that which can be closed: the account supplies an X25519 public key, the
10
  forge encrypts to it, and the private half never exists on this side of the
11
  boundary.
12
13
  The format is [age v1](https://age-encryption.org/v1) rather than something
14
  of ours, and that choice is the substance of the claim rather than a
15
  convenience. An export a recipient can only open with a decryptor the
16
  operator wrote is still an export the operator defines the terms of. `age`,
17
  `rage`, and every other implementation of the specification read this
18
  document, so the recipient depends on the operator for neither the key nor
19
  the reader.
20
21
  What it does not do is stated as plainly. The forge builds the export from
22
  plaintext PostgreSQL, so the operator holds the contents before this module
23
  runs and holds them still afterwards. Encryption protects the *artifact* —
24
  the file, its copies, whatever logs or proxies or backups the response
25
  passes through — and it protects nothing about the store it was read from.
26
  `docs/2026-08-24-private-export-encryption.md` records that boundary, the
27
  threat model on both sides of it, and the options rejected for the storage
28
  layer.
29
30
  Losing the key costs nothing here, which is the property that makes this
31
  decidable at all. An export is derived, not stored: an account that loses
32
  its private key requests the export again under a new one. That is why
33
  recipient-held encryption is right for the export path and wrong for the
34
  columns behind it, where the same key loss would be permanent.
35
36
  ## Construction
37
38
  One `X25519` recipient stanza, ChaCha20-Poly1305 over a 16-byte file key,
39
  an HMAC-SHA256 over the header, and the payload in `STREAM` chunks of 64
40
  KiB. All of it comes out of `:crypto`; no dependency is added for it.
41
  """
42
43
  import Bitwise
44
45
  @charset ~c"qpzry9x8gf2tvdw0s3jn54khce6mua7l"
46
  @hrp "age"
47
  @key_bytes 32
48
  @file_key_bytes 16
49
  @chunk_bytes 65_536
50
  @info "age-encryption.org/v1/X25519"
51
  @intro "age-encryption.org/v1\n"
52
53
  @typedoc "A parsed recipient: the raw 32-byte X25519 public key."
54
  @type recipient :: <<_::256>>
55
56
  @doc """
57
  Parses an `age1…` recipient into its raw X25519 public key.
58
59
  Bech32 with the `age` human-readable part, which is what `age-keygen -y`
60
  prints. The checksum is verified, so a transcription error is refused here
61
  rather than producing a document nobody can open.
62
  """
63
  @spec parse_recipient(term()) :: {:ok, recipient()} | {:error, atom()}
64
  def parse_recipient(recipient) when is_binary(recipient) do
65
    with true <- byte_size(recipient) <= 120,
66
         {:ok, @hrp, key} <- bech32_decode(recipient),
67
         @key_bytes <- byte_size(key) do
68
      {:ok, key}
69
    else
70
      _invalid -> {:error, :invalid_recipient}
71
    end
72
  end
73
74
  def parse_recipient(_recipient), do: {:error, :invalid_recipient}
75
76
  @doc """
77
  Encrypts `plaintext` to `recipient`, returning an age v1 document.
78
79
  The recipient is the raw public key from `parse_recipient/1`. An all-zero
80
  X25519 shared secret is refused rather than encrypted to: it means the
81
  recipient key has small order, and the result would be readable by anyone.
82
  """
83
  @spec encrypt(binary(), recipient()) :: {:ok, binary()} | {:error, atom()}
84
  def encrypt(plaintext, <<recipient::binary-size(@key_bytes)>>) when is_binary(plaintext) do
85
    file_key = :crypto.strong_rand_bytes(@file_key_bytes)
86
    {ephemeral_public, ephemeral_secret} = :crypto.generate_key(:ecdh, :x25519)
87
    shared = :crypto.compute_key(:ecdh, recipient, ephemeral_secret, :x25519)
88
89
    if shared == <<0::size(@key_bytes * 8)>> do
90
      {:error, :invalid_recipient}
91
    else
92
      wrap_key = hkdf(shared, ephemeral_public <> recipient, @info)
93
      wrapped = seal(wrap_key, <<0::96>>, file_key)
94
95
      header =
96
        @intro <>
97
          "-> X25519 " <>
98
          encode(ephemeral_public) <> "\n" <> encode(wrapped) <> "\n" <> "---"
99
100
      mac = :crypto.mac(:hmac, :sha256, hkdf(file_key, "", "header"), header)
101
      nonce = :crypto.strong_rand_bytes(16)
102
      stream_key = hkdf(file_key, nonce, "payload")
103
104
      {:ok, header <> " " <> encode(mac) <> "\n" <> nonce <> stream(plaintext, stream_key)}
105
    end
106
  rescue
107
    _error -> {:error, :encryption_failed}
108
  end
109
110
  def encrypt(_plaintext, _recipient), do: {:error, :invalid_recipient}
111
112
  # `STREAM`: each chunk carries its own tag, and the final chunk sets the
113
  # last byte of the nonce, so a truncated document fails to open rather than
114
  # opening short.
115
  defp stream(plaintext, key), do: stream(plaintext, key, 0, [])
116
117
  defp stream(rest, key, counter, acc) do
118
    last? = byte_size(rest) <= @chunk_bytes
119
    size = min(byte_size(rest), @chunk_bytes)
120
    <<chunk::binary-size(^size), remaining::binary>> = rest
121
    flag = if last?, do: 1, else: 0
122
    acc = [seal(key, <<counter::88, flag::8>>, chunk) | acc]
123
124
    if last?,
125
      do: acc |> Enum.reverse() |> IO.iodata_to_binary(),
126
      else: stream(remaining, key, counter + 1, acc)
127
  end
128
129
  defp seal(key, nonce, plaintext) do
130
    {ciphertext, tag} =
131
      :crypto.crypto_one_time_aead(:chacha20_poly1305, key, nonce, plaintext, <<>>, true)
132
133
    ciphertext <> tag
134
  end
135
136
  # HKDF-SHA256 to one 32-byte block. An empty salt is HashLen zeros, which is
137
  # what RFC 5869 says and what every age implementation does.
138
  defp hkdf(ikm, salt, info) do
139
    salt = if salt == "", do: <<0::256>>, else: salt
140
    prk = :crypto.mac(:hmac, :sha256, salt, ikm)
141
    :crypto.mac(:hmac, :sha256, prk, info <> <<1>>)
142
  end
143
144
  defp encode(binary), do: Base.encode64(binary, padding: false)
145
146
  defp bech32_decode(string) do
147
    downcased = String.downcase(string)
148
149
    with [_first, _second | _rest] = parts <- String.split(downcased, "1"),
150
         data = List.last(parts),
151
         hrp = parts |> Enum.drop(-1) |> Enum.join("1"),
152
         true <- hrp != "",
153
         {:ok, values} <- charset_values(data),
154
         true <- length(values) >= 6,
155
         1 <- polymod(hrp_expand(hrp) ++ values),
156
         {:ok, bytes} <- regroup(Enum.drop(values, -6)) do
157
      {:ok, hrp, :binary.list_to_bin(bytes)}
158
    else
159
      _invalid -> {:error, :invalid_recipient}
160
    end
161
  end
162
163
  defp charset_values(data) do
164
    Enum.reduce_while(String.to_charlist(data), {:ok, []}, fn character, {:ok, acc} ->
165
      case Enum.find_index(@charset, &(&1 == character)) do
166
        nil -> {:halt, {:error, :invalid_recipient}}
167
        index -> {:cont, {:ok, acc ++ [index]}}
168
      end
169
    end)
170
  end
171
172
  defp hrp_expand(hrp) do
173
    characters = String.to_charlist(hrp)
174
    Enum.map(characters, &div(&1, 32)) ++ [0] ++ Enum.map(characters, &rem(&1, 32))
175
  end
176
177
  @generator [0x3B6A57B2, 0x26508E6D, 0x1EA119FA, 0x3D4233DD, 0x2A1462B3]
178
179
  defp polymod(values) do
180
    Enum.reduce(values, 1, fn value, checksum ->
181
      top = bsr(checksum, 25)
182
      checksum = bxor(bsl(band(checksum, 0x1FFFFFF), 5), value)
183
184
      @generator
185
      |> Enum.with_index()
186
      |> Enum.reduce(checksum, fn {constant, index}, acc ->
187
        if band(bsr(top, index), 1) == 1, do: bxor(acc, constant), else: acc
188
      end)
189
    end)
190
  end
191
192
  # 5-bit groups back to 8-bit bytes. Trailing bits must be fewer than five and
193
  # zero, which is what makes a padded or truncated recipient fail here.
194
  defp regroup(values) do
195
    {accumulator, bits, bytes} =
196
      Enum.reduce(values, {0, 0, []}, fn value, {accumulator, bits, bytes} ->
197
        drain(bor(bsl(accumulator, 5), value), bits + 5, bytes)
198
      end)
199
200
    if bits < 5 and band(bsl(accumulator, 8 - bits), 0xFF) == 0,
201
      do: {:ok, bytes},
202
      else: {:error, :invalid_recipient}
203
  end
204
205
  defp drain(accumulator, bits, bytes) when bits >= 8 do
206
    drain(accumulator, bits - 8, bytes ++ [band(bsr(accumulator, bits - 8), 0xFF)])
207
  end
208
209
  defp drain(accumulator, bits, bytes), do: {accumulator, bits, bytes}
210
end
lib/openagents/forge/independence.ex modified +56 -11

@@ -28,11 +28,19 @@ defmodule OpenAgents.Forge.Independence do

28 28
    the operator serves it and could serve any document, and its value is that
29 29
    a third party can cheaply keep a copy, not that anyone has. Issue #151
30 30
    carries the witness.
31
  * **Private data.** No export is encrypted to a key the recipient holds, and
32
    no Ecto column in this repository is encrypted at rest. Issue #178 carries
33
    that decision. This one is stated rather than derived, because there is no
34
    registry of encrypted columns to count and inventing one to make a number
35
    appear would be the kind of claim this disclosure exists to avoid.
31
  * **Private data.** Two facts that only mean something together. The account
32
    export can be encrypted to a key the recipient holds (#178), and that is
33
    derived: `OpenAgentsWeb.DataController`'s compiled import table either
34
    reaches `OpenAgents.DataRights.Age` or it does not, so removing the
35
    encryption removes the claim in the same commit. Nothing in PostgreSQL is
36
    encrypted at rest, and that one is stated, because there is no registry of
37
    encrypted columns to count and inventing one to make a number appear would
38
    be the kind of claim this disclosure exists to avoid; #193 carries it.
39
    `operator_reads_source` is derived from the second fact rather than
40
    restated, because it is the same fact: the operator reads the plaintext an
41
    export is built from exactly while the store is plaintext. Publishing the
42
    encryption without it would let a reader conclude the operator cannot read
43
    an export, which is false.
36 44
37 45
  `degraded?` is true while any of the three falls short, so the page does not
38 46
  need a human to decide when to say so. It is expected to be true today.

@@ -50,8 +58,11 @@ defmodule OpenAgents.Forge.Independence do

50 58
51 59
  @anchor_issue 168
52 60
  @witness_issue 151
53
  @encryption_issue 178
61
  @at_rest_issue 193
54 62
  @document "docs/forge-operator-independence.md"
63
  @export_encryption_module OpenAgents.DataRights.Age
64
  @export_controller OpenAgentsWeb.DataController
65
  @account_export OpenAgents.DataRights.AccountExport
55 66
56 67
  @doc "The disclosure, in the shape `/api/status` publishes it."
57 68
  @spec projection() :: map()

@@ -88,7 +99,9 @@ defmodule OpenAgents.Forge.Independence do

88 99
  @spec degraded?(map(), map(), map()) :: boolean()
89 100
  def degraded?(export, verification, private_data) do
90 101
    export["gaps"] != [] or not verification["anchor_published"] or
91
      not verification["anchor_witnessed"] or not private_data["exports_encrypted"]
102
      not verification["anchor_witnessed"] or
103
      not private_data["export_recipient_encryption"] or
104
      not private_data["encrypted_at_rest"]
92 105
  end
93 106
94 107
  # Counted from the ledger, so this section cannot claim an export gap is

@@ -139,15 +152,47 @@ defmodule OpenAgents.Forge.Independence do

139 152
    }
140 153
  end
141 154
155
  # `encrypted_at_rest` is the one stated value left here, and everything
156
  # around it is read from the code rather than asserted beside it.
142 157
  defp private_data_section do
158
    encrypted_at_rest? = false
159
143 160
    %{
144
      "exports_encrypted" => false,
145
      "encrypted_at_rest" => false,
146
      "access_controlled" => true,
147
      "issue" => @encryption_issue
161
      "export_recipient_encryption" => export_recipient_encryption?(),
162
      "encrypted_at_rest" => encrypted_at_rest?,
163
      "operator_reads_source" => not encrypted_at_rest?,
164
      "access_controlled" => access_controlled?(),
165
      "issue" => @at_rest_issue
148 166
    }
149 167
  end
150 168
169
  # The export route encrypts to a recipient-held key exactly while it calls
170
  # the module that does it. Reading the compiled import table is the same
171
  # technique `EXIT-002` and `EXIT-003` use to keep a structural claim from
172
  # decaying into a comment.
173
  defp export_recipient_encryption? do
174
    safely(fn -> @export_encryption_module in external_calls(@export_controller) end) || false
175
  end
176
177
  # "Takes the account and nothing else" is the whole access-control claim for
178
  # this export, and it is a fact about the function's shape.
179
  defp access_controlled? do
180
    safely(fn ->
181
      Code.ensure_loaded?(@account_export) and
182
        function_exported?(@account_export, :build, 1) and
183
        not function_exported?(@account_export, :build, 2)
184
    end) || false
185
  end
186
187
  # The same read `EXIT-002`'s and `EXIT-003`'s proofs perform: the callee set
188
  # a module was compiled with, which no comment or later refactor can flatter.
189
  defp external_calls(module) do
190
    case :beam_lib.chunks(:code.which(module), [:imports]) do
191
      {:ok, {^module, [imports: imports]}} -> Enum.map(imports, &elem(&1, 0))
192
      _unreadable -> []
193
    end
194
  end
195
151 196
  defp operator_section do
152 197
    %{
153 198
      "model" => "single_operator",
lib/openagents_web/controllers/data_controller.ex modified +37 -6

@@ -1,7 +1,7 @@

1 1
defmodule OpenAgentsWeb.DataController do
2 2
  use OpenAgentsWeb, :controller
3 3
4
  alias OpenAgents.DataRights.AccountExport
4
  alias OpenAgents.DataRights.{AccountExport, Age}
5 5
  alias OpenAgents.{Conversations, DataRights}
6 6
7 7
  def show(conn, _params) do

@@ -48,20 +48,51 @@ defmodule OpenAgentsWeb.DataController do

48 48
  authenticated account and nothing else, so no parameter can widen it to
49 49
  another account's records.
50 50
  """
51
  def export_account(conn, _params) do
51
  # The `recipient` parameter is an `age1…` public key the account generated
52
  # itself. Supplying one costs the operator the ability to read the file it
53
  # sends; supplying nothing keeps today's plain JSON, because an export
54
  # nobody can open is not portability either. Neither branch widens what the
55
  # document contains: `AccountExport.build/1` still takes the account and
56
  # nothing else (#178, EXIT-001).
57
  def export_account(conn, params) do
52 58
    with %{status: "active"} = user <- conn.assigns.current_user,
59
         {:ok, recipient} <- recipient(params),
53 60
         {:ok, export} <- AccountExport.build(user),
54
         {:ok, body} <- Jason.encode(export) do
61
         {:ok, body} <- Jason.encode(export),
62
         {:ok, body, extension, type} <- encrypt(body, recipient) do
55 63
      conn
56
      |> put_resp_content_type("application/json")
64
      |> put_resp_content_type(type)
57 65
      |> put_resp_header(
58 66
        "content-disposition",
59
        ~s(attachment; filename="openagents-account-data.json")
67
        ~s(attachment; filename="openagents-account-data.json#{extension}")
60 68
      )
61 69
      |> put_resp_header("cache-control", "no-store")
62 70
      |> send_resp(:ok, body)
63 71
    else
64
      _unavailable -> send_resp(conn, :not_found, "Account export is unavailable.")
72
      {:error, :invalid_recipient} ->
73
        conn
74
        |> put_status(:unprocessable_entity)
75
        |> text("Supply an age recipient public key, the age1… value age-keygen -y prints.")
76
77
      _unavailable ->
78
        send_resp(conn, :not_found, "Account export is unavailable.")
79
    end
80
  end
81
82
  defp recipient(%{"recipient" => recipient}) when is_binary(recipient) do
83
    case String.trim(recipient) do
84
      "" -> {:ok, :none}
85
      trimmed -> Age.parse_recipient(trimmed)
86
    end
87
  end
88
89
  defp recipient(_params), do: {:ok, :none}
90
91
  defp encrypt(body, :none), do: {:ok, body, "", "application/json"}
92
93
  defp encrypt(body, recipient) do
94
    with {:ok, sealed} <- Age.encrypt(body, recipient) do
95
      {:ok, sealed, ".age", "application/octet-stream"}
65 96
    end
66 97
  end
67 98
lib/openagents_web/live/memory_live.ex modified +24

@@ -309,6 +309,30 @@ defmodule OpenAgentsWeb.MemoryLive do

309 309
          <.text_button id="export-account-data" href="/data/export/account" download>
310 310
            <.icon name="download" /> Export Forge and Forum Data
311 311
          </.text_button>
312
          <%!-- Encrypting that document to a key you generated is the one
313
          place the operator's reach over an export can actually be removed
314
          (#178). It protects the file, not the database it was read from,
315
          and losing the key costs you nothing: ask for the export again. --%>
316
          <form
317
            method="get"
318
            action="/data/export/account"
319
            id="export-account-data-encrypted"
320
            class="flex flex-wrap items-center gap-2"
321
          >
322
            <.input
323
              type="text"
324
              name="recipient"
325
              id="export-recipient"
326
              value=""
327
              placeholder="age1… (age-keygen -y)"
328
              autocomplete="off"
329
              spellcheck="false"
330
              aria-label="age recipient public key"
331
            />
332
            <.button type="submit" id="export-account-data-encrypted-submit" data-size="sm">
333
              <.icon name="download" /> Export encrypted to my key
334
            </.button>
335
          </form>
312 336
          <.text_button
313 337
            id="forget-all-memory"
314 338
            tone={:danger}
test/fixtures/age/README.md added +19

@@ -0,0 +1,19 @@

1
# age reference fixture
2
3
`reference.age` was produced by the reference `age` implementation (v1.3.1):
4
5
```sh
6
printf 'openagents account export fixture\n' > plain.txt
7
age --encrypt -r age1ja3nzz0wlmuqvsfhrx7w4kq00knnmu3ur32y47734k5javp4cqnqumg72a \
8
  -o reference.age plain.txt
9
```
10
11
`test/openagents/data_rights/age_test.exs` decrypts it with
12
`OpenAgents.Test.AgeDocument`, the decryptor written from the specification.
13
That direction is what pins our reading of the format to the reference
14
implementation, so the encryption round trip means something where the `age`
15
binary is not installed.
16
17
The identity is a throwaway generated for this fixture and protects nothing.
18
It lives in the test as `@fixture_secret` rather than in a file, so nothing
19
here looks like a credential in use.
test/fixtures/age/reference.age added

Binary file. Nothing to show as text.

test/openagents/data_rights/account_export_test.exs modified +63

@@ -10,6 +10,12 @@ defmodule OpenAgents.DataRights.AccountExportTest do

10 10
11 11
  use OpenAgentsWeb.ConnCase, async: false
12 12
13
  # The account export's encryption keypair for these tests. The secret half
14
  # exists only here, which is the shape of the claim: the forge holds the
15
  # public half and cannot read what it produced.
16
  @recipient "age1ja3nzz0wlmuqvsfhrx7w4kq00knnmu3ur32y47734k5javp4cqnqumg72a"
17
  @identity "AGE-SECRET-KEY-1XDUAFRZQALRUZ2G4UK6NDASFWPUFRTJ46359KD974PUPZ2UYXK2Q6G8QT3"
18
13 19
  import Ecto.Query
14 20
15 21
  alias OpenAgents.DataRights.AccountExport

@@ -418,6 +424,63 @@ defmodule OpenAgents.DataRights.AccountExportTest do

418 424
      refute response.resp_body =~ "openagents.account_export.v1"
419 425
    end
420 426
427
    test "a recipient key encrypts the same document to a key the operator does not hold", %{
428
      conn: conn,
429
      board: board
430
    } do
431
      # #178. The `recipient` parameter changes who can read the response and
432
      # nothing about what it contains: the same post is inside, and the
433
      # operator's own decryptor is not required to get it out.
434
      user = github_user("account-export-sealed", "export-sealed")
435
      topic = topic_fixture(board, "user:" <> user.id, "Sealed")
436
      _post = post_fixture(topic, "user:" <> user.id, "Through the sealed route.")
437
438
      response =
439
        conn
440
        |> Plug.Test.init_test_session(%{"user_id" => user.id})
441
        |> get(~p"/data/export/account?recipient=#{@recipient}")
442
443
      assert response.status == 200
444
445
      assert ["attachment; filename=\"openagents-account-data.json.age\""] =
446
               Plug.Conn.get_resp_header(response, "content-disposition")
447
448
      assert String.starts_with?(response.resp_body, "age-encryption.org/v1\n")
449
      refute response.resp_body =~ "Through the sealed route."
450
451
      body = Jason.decode!(OpenAgents.Test.AgeDocument.decrypt(response.resp_body, @identity))
452
      assert body["account"]["id"] == user.id
453
      assert [%{"body_text" => "Through the sealed route."}] = body["forum"]["posts"]
454
    end
455
456
    test "a recipient that is not an age public key is refused rather than sent in the clear", %{
457
      conn: conn
458
    } do
459
      user = github_user("account-export-bad-key", "export-bad-key")
460
461
      for recipient <- ["age1notarealkey", "ssh-ed25519 AAAA", @identity] do
462
        response =
463
          conn
464
          |> Plug.Test.init_test_session(%{"user_id" => user.id})
465
          |> get(~p"/data/export/account?recipient=#{recipient}")
466
467
        assert response.status == 422
468
        refute response.resp_body =~ "openagents.account_export.v1"
469
      end
470
    end
471
472
    test "an empty recipient keeps the plain document rather than failing", %{conn: conn} do
473
      user = github_user("account-export-blank-key", "export-blank-key")
474
475
      response =
476
        conn
477
        |> Plug.Test.init_test_session(%{"user_id" => user.id})
478
        |> get(~p"/data/export/account?recipient=")
479
480
      assert response.status == 200
481
      assert Jason.decode!(response.resp_body)["account"]["id"] == user.id
482
    end
483
421 484
    test "the route reads the session, so no parameter reaches another account", %{
422 485
      conn: conn,
423 486
      board: board
test/openagents/data_rights/age_test.exs added +142

@@ -0,0 +1,142 @@

1
defmodule OpenAgents.DataRights.AgeTest do
2
  @moduledoc """
3
  The claim under test is not "the bytes round trip". It is that a recipient
4
  holding a key the operator never saw can read the export **without the
5
  operator's software**, which is the only version of the claim worth
6
  publishing on `/status` (#178, `EXIT-006`).
7
8
  Three assertions carry it, and each covers what the others cannot:
9
10
  1. `age` itself decrypts what this module produces. This is the claim
11
     stated exactly. It runs wherever the binary is installed, and it is the
12
     assertion that ran when the decision landed.
13
  2. An independent decryptor written here from the specification decrypts it
14
     too, so the property is checked where `age` is not installed. On its own
15
     this would prove only that two of our own implementations agree.
16
  3. That decryptor reads a document the real `age` binary produced,
17
     checked in as a fixture. This pins our reading of the format to the
18
     reference implementation, which is what makes assertion 2 mean something
19
     when assertion 1 does not run.
20
  """
21
22
  use ExUnit.Case, async: true
23
24
  alias OpenAgents.DataRights.Age
25
26
  @fixture_directory Path.expand("../../fixtures/age", __DIR__)
27
  @fixture_recipient "age1ja3nzz0wlmuqvsfhrx7w4kq00knnmu3ur32y47734k5javp4cqnqumg72a"
28
  @fixture_secret "AGE-SECRET-KEY-1XDUAFRZQALRUZ2G4UK6NDASFWPUFRTJ46359KD974PUPZ2UYXK2Q6G8QT3"
29
  @fixture_plaintext "openagents account export fixture\n"
30
31
  describe "parse_recipient/1" do
32
    test "accepts the age1… value age-keygen -y prints" do
33
      assert {:ok, key} = Age.parse_recipient(@fixture_recipient)
34
      assert byte_size(key) == 32
35
    end
36
37
    test "refuses a recipient whose checksum does not hold" do
38
      <<head::binary-size(byte_size(@fixture_recipient) - 1), last::binary>> = @fixture_recipient
39
      mutated = head <> if(last == "a", do: "q", else: "a")
40
41
      assert {:error, :invalid_recipient} = Age.parse_recipient(mutated)
42
    end
43
44
    test "refuses an identity, a bech32 string of another kind, and a non-binary" do
45
      assert {:error, :invalid_recipient} = Age.parse_recipient(@fixture_secret)
46
47
      assert {:error, :invalid_recipient} =
48
               Age.parse_recipient("bc1qw508d6qejxtdg4y5r3zarvary0c5xw7kv8f3t4")
49
50
      assert {:error, :invalid_recipient} = Age.parse_recipient("")
51
      assert {:error, :invalid_recipient} = Age.parse_recipient(nil)
52
    end
53
  end
54
55
  describe "encrypt/2" do
56
    test "the age binary decrypts what this module produces" do
57
      case System.find_executable("age") do
58
        nil ->
59
          # Not a silent skip: the property is still asserted by the two tests
60
          # below, and this line says which half did not run.
61
          IO.puts(:stderr, "age binary absent: third-party decryption not exercised here")
62
63
        _path ->
64
          {:ok, recipient} = Age.parse_recipient(@fixture_recipient)
65
          plaintext = Jason.encode!(%{"account" => "export", "items" => Enum.to_list(1..2_000)})
66
          {:ok, sealed} = Age.encrypt(plaintext, recipient)
67
68
          directory = temporary_directory()
69
          identity = Path.join(directory, "identity.txt")
70
          document = Path.join(directory, "export.age")
71
          File.write!(identity, @fixture_secret <> "\n")
72
          File.write!(document, sealed)
73
74
          assert {decrypted, 0} =
75
                   System.cmd("age", ["--decrypt", "-i", identity, document])
76
77
          assert decrypted == plaintext
78
      end
79
    end
80
81
    test "an independent decryptor reads a single chunk, many chunks, and an empty document" do
82
      {:ok, recipient} = Age.parse_recipient(@fixture_recipient)
83
84
      for plaintext <- ["", "one line\n", :crypto.strong_rand_bytes(200_000)] do
85
        assert {:ok, sealed} = Age.encrypt(plaintext, recipient)
86
        assert decrypt(sealed, @fixture_secret) == plaintext
87
      end
88
    end
89
90
    test "the decryptor above reads a document the age binary produced" do
91
      reference = File.read!(Path.join(@fixture_directory, "reference.age"))
92
93
      assert decrypt(reference, @fixture_secret) == @fixture_plaintext
94
    end
95
96
    test "a truncated document does not open short" do
97
      {:ok, recipient} = Age.parse_recipient(@fixture_recipient)
98
      {:ok, sealed} = Age.encrypt(:crypto.strong_rand_bytes(200_000), recipient)
99
      truncated = binary_part(sealed, 0, byte_size(sealed) - 40)
100
101
      assert catch_error(decrypt(truncated, @fixture_secret))
102
    end
103
104
    test "two exports do not share a file key" do
105
      # Comparing the documents is not this assertion: the ephemeral X25519
106
      # key is fresh per call, so two documents differ byte for byte even
107
      # when every file key is identical. Recovering one file key would then
108
      # open every export ever issued, so the file key itself is what has to
109
      # be compared, and it is read back out of each document.
110
      {:ok, recipient} = Age.parse_recipient(@fixture_recipient)
111
      {:ok, first} = Age.encrypt("same plaintext", recipient)
112
      {:ok, second} = Age.encrypt("same plaintext", recipient)
113
114
      refute first == second
115
      refute file_key(first, @fixture_secret) == file_key(second, @fixture_secret)
116
    end
117
118
    test "refuses a recipient that is not 32 bytes" do
119
      assert {:error, :invalid_recipient} = Age.encrypt("body", <<0::128>>)
120
      assert {:error, :invalid_recipient} = Age.encrypt("body", "age1…")
121
    end
122
123
    test "refuses a low-order recipient, whose ciphertext anyone could read" do
124
      # `:crypto` raises on an all-zero peer key before the explicit
125
      # all-zero-shared-secret check can answer, so the refusal arrives as
126
      # `:encryption_failed`. Both guards are kept: the explicit one covers
127
      # the low-order points `:crypto` accepts.
128
      assert {:error, reason} = Age.encrypt("body", <<0::256>>)
129
      assert reason in [:invalid_recipient, :encryption_failed]
130
    end
131
  end
132
133
  defp temporary_directory do
134
    directory = Path.join(System.tmp_dir!(), "age-test-#{System.unique_integer([:positive])}")
135
    File.mkdir_p!(directory)
136
    on_exit(fn -> File.rm_rf(directory) end)
137
    directory
138
  end
139
140
  defdelegate decrypt(document, identity), to: OpenAgents.Test.AgeDocument
141
  defdelegate file_key(document, identity), to: OpenAgents.Test.AgeDocument
142
end
test/openagents/forge/independence_disclosure_test.exs modified +50 -3

@@ -76,6 +76,26 @@ defmodule OpenAgents.Forge.IndependenceDisclosureTest do

76 76
      assert Independence.degraded?()
77 77
    end
78 78
79
    test "a plaintext store is enough on its own to report degraded" do
80
      # #178 closed the export half of the private-data axis. If closing it
81
      # had quietly taken that axis out of the disjunction, a forge with
82
      # plaintext columns would report itself independent as soon as the
83
      # verification axis cleared. Every other axis is handed in clean here,
84
      # which `degraded?/3` exists for: the projection alone cannot show that
85
      # this axis is load-bearing, because the others are never clean.
86
      clean_export = %{"gaps" => []}
87
      clean_verification = %{"anchor_published" => true, "anchor_witnessed" => true}
88
      private_data = Independence.projection()["private_data"]
89
90
      refute Independence.degraded?(clean_export, clean_verification, %{
91
               private_data
92
               | "encrypted_at_rest" => true,
93
                 "operator_reads_source" => false
94
             })
95
96
      assert Independence.degraded?(clean_export, clean_verification, private_data)
97
    end
98
79 99
    # The publication state is read from the anchors that exist, not from a
80 100
    # flag someone can set, so this test publishes a real one.
81 101
    test "publishing an anchor changes the verification claim" do

@@ -104,13 +124,40 @@ defmodule OpenAgents.Forge.IndependenceDisclosureTest do

104 124
      assert Independence.degraded?()
105 125
    end
106 126
107
    test "an unencrypted private export is disclosed rather than softened" do
127
    test "the export encryption and the plaintext store are published together" do
128
      # #178 decided that an export can be encrypted to a recipient-held key
129
      # and that the columns behind it stay plaintext. Publishing the first
130
      # without the second would let a reader conclude the operator cannot
131
      # read an export, so the second is what `operator_reads_source` says
132
      # and it is derived from the first rather than asserted beside it.
108 133
      private_data = Independence.projection()["private_data"]
109 134
110
      refute private_data["exports_encrypted"]
135
      assert private_data["export_recipient_encryption"]
111 136
      refute private_data["encrypted_at_rest"]
137
      assert private_data["operator_reads_source"]
112 138
      assert private_data["access_controlled"]
113
      assert private_data["issue"] == 178
139
      assert private_data["issue"] == 193
140
    end
141
142
    test "the export encryption claim is read from the route rather than asserted" do
143
      # The claim is true exactly while `OpenAgentsWeb.DataController` was
144
      # compiled against the module that encrypts. Nothing here restates it,
145
      # so removing the encryption removes the claim in the same commit.
146
      {:ok, {module, [imports: imports]}} =
147
        :beam_lib.chunks(:code.which(OpenAgentsWeb.DataController), [:imports])
148
149
      assert module == OpenAgentsWeb.DataController
150
151
      assert OpenAgents.DataRights.Age in Enum.map(imports, &elem(&1, 0)),
152
             "EXIT-006 derives `export_recipient_encryption` from this call. If the export " <>
153
               "route stopped encrypting, the disclosure must stop claiming it."
154
    end
155
156
    test "the account export still takes the account and nothing else" do
157
      # `access_controlled` is derived from this shape, which is the whole
158
      # access-control claim for this document: no parameter can widen it.
159
      assert function_exported?(OpenAgents.DataRights.AccountExport, :build, 1)
160
      refute function_exported?(OpenAgents.DataRights.AccountExport, :build, 2)
114 161
    end
115 162
  end
116 163
test/openagents/network_status_test.exs modified +2 -1

@@ -113,8 +113,9 @@ defmodule OpenAgents.NetworkStatusTest do

113 113
    "independence.private_data",
114 114
    "independence.private_data.access_controlled",
115 115
    "independence.private_data.encrypted_at_rest",
116
    "independence.private_data.exports_encrypted",
116
    "independence.private_data.export_recipient_encryption",
117 117
    "independence.private_data.issue",
118
    "independence.private_data.operator_reads_source",
118 119
    "independence.schema",
119 120
    "independence.verification",
120 121
    # The anchor's address, its publication state, and whether anybody outside
test/support/age_document.ex added +106

@@ -0,0 +1,106 @@

1
defmodule OpenAgents.Test.AgeDocument do
2
  @moduledoc """
3
  An age v1 decryptor written from the specification, sharing no code with
4
  `OpenAgents.DataRights.Age`.
5
6
  It exists so the export encryption #178 decided can be checked where the
7
  `age` binary is not installed, and it is itself pinned to the reference
8
  implementation by `test/fixtures/age/reference.age`, a document the real
9
  `age` produced. Two of our own implementations agreeing would prove nothing
10
  on its own; that fixture is what makes the agreement mean something.
11
12
  It handles exactly one X25519 stanza, which is all the export path produces.
13
  """
14
15
  import Bitwise
16
17
  def file_key(document, identity) do
18
    {"age-secret-key-", secret} = bech32_decode(String.downcase(identity))
19
    {recipient, ^secret} = :crypto.generate_key(:ecdh, :x25519, secret)
20
    [_intro, stanza, wrapped | _rest] = String.split(document, "\n", parts: 5)
21
    "-> X25519 " <> ephemeral = stanza
22
23
    shared = :crypto.compute_key(:ecdh, decode(ephemeral), secret, :x25519)
24
    salt = decode(ephemeral) <> recipient
25
26
    open!(hkdf(shared, salt, "age-encryption.org/v1/X25519"), <<0::96>>, decode(wrapped))
27
  end
28
29
  def decrypt(document, identity) do
30
    [intro, stanza, wrapped, mac_line | _rest] = String.split(document, "\n", parts: 5)
31
    true = intro == "age-encryption.org/v1"
32
    "--- " <> mac = mac_line
33
    header = Enum.join([intro, stanza, wrapped, "---"], "\n")
34
    file_key = file_key(document, identity)
35
36
    true = :crypto.mac(:hmac, :sha256, hkdf(file_key, "", "header"), header) == decode(mac)
37
38
    prefix = byte_size(header) + byte_size(" " <> mac <> "\n")
39
    <<_consumed::binary-size(^prefix), nonce::binary-size(16), payload::binary>> = document
40
    unstream(payload, hkdf(file_key, nonce, "payload"), 0, [])
41
  end
42
43
  defp unstream(payload, key, counter, acc) do
44
    last? = byte_size(payload) <= 65_536 + 16
45
    size = min(byte_size(payload), 65_536 + 16)
46
    <<chunk::binary-size(^size), rest::binary>> = payload
47
    flag = if last?, do: 1, else: 0
48
    acc = [open!(key, <<counter::88, flag::8>>, chunk) | acc]
49
50
    if last?,
51
      do: acc |> Enum.reverse() |> IO.iodata_to_binary(),
52
      else: unstream(rest, key, counter + 1, acc)
53
  end
54
55
  defp open!(key, nonce, sealed) do
56
    size = byte_size(sealed) - 16
57
    <<ciphertext::binary-size(^size), tag::binary-size(16)>> = sealed
58
59
    case :crypto.crypto_one_time_aead(
60
           :chacha20_poly1305,
61
           key,
62
           nonce,
63
           ciphertext,
64
           <<>>,
65
           tag,
66
           false
67
         ) do
68
      plaintext when is_binary(plaintext) -> plaintext
69
      :error -> raise "chunk did not authenticate"
70
    end
71
  end
72
73
  defp hkdf(ikm, salt, info) do
74
    salt = if salt == "", do: <<0::256>>, else: salt
75
    prk = :crypto.mac(:hmac, :sha256, salt, ikm)
76
    :crypto.mac(:hmac, :sha256, prk, info <> <<1>>)
77
  end
78
79
  defp decode(value), do: Base.decode64!(value, padding: false)
80
81
  @charset ~c"qpzry9x8gf2tvdw0s3jn54khce6mua7l"
82
83
  defp bech32_decode(string) do
84
    parts = String.split(string, "1")
85
    data = List.last(parts)
86
    hrp = parts |> Enum.drop(-1) |> Enum.join("1")
87
88
    bytes =
89
      data
90
      |> String.to_charlist()
91
      |> Enum.map(fn character -> Enum.find_index(@charset, &(&1 == character)) end)
92
      |> Enum.drop(-6)
93
      |> Enum.reduce({0, 0, []}, fn value, {accumulator, bits, bytes} ->
94
        drain(bor(bsl(accumulator, 5), value), bits + 5, bytes)
95
      end)
96
      |> elem(2)
97
98
    {hrp, :binary.list_to_bin(bytes)}
99
  end
100
101
  defp drain(accumulator, bits, bytes) when bits >= 8 do
102
    drain(accumulator, bits - 8, bytes ++ [band(bsr(accumulator, bits - 8), 0xFF)])
103
  end
104
105
  defp drain(accumulator, bits, bytes), do: {accumulator, bits, bytes}
106
end

This page updates live while a promote is in flight · changelog