Draw the knowledge-base boundary at promotion

8c65caa0fd7f · AtlantisPleb · · parent b02c8e070b65

Draw the knowledge-base boundary at promotion

System memory and the knowledge base both put network-level knowledge in
front of every session, and specification section 8 keeps them from becoming
rival stores of one claim with two rules: promotion drains memory into the
knowledge base, and the knowledge base wins a recall collision. The cloud
re-base took away the place the second rule was going to be enforced. The
knowledge base is retrieved in the client from a corpus compiled into a
WebAssembly plugin; memory recall runs here inside POST /api/v1/responses.
The two notes reach the model through different fields of the same request
and no process holds both.

Enforce the boundary at promotion instead, and define the collision test as
identity rather than text: a memory and a stance are the same claim exactly
when a promotion tombstone on that memory names that stance. That link is a
steward's recorded judgment, and it is the only identifier the two rails can
share — the corpus gives each stance a stable id, but the plugin does not
deserialize it, so it is absent from the hit, the output schema, and the
injected note. Either recall-time shape would therefore have to compare a
stance's prose to a memory's, which is a heuristic; shipping one as "the
knowledge base wins" would read as a guarantee while dropping true memories
on a false positive, and duplication is the benign failure where suppression
is the destructive one.

Given that test, a recall-time rule has nothing left to do. Promotions.promote
writes a superseding row on the claim's slug — a promotion tombstone — so the
claim is no longer live, and the tombstone can never be admitted, so it is
never eligible. Recall surfaces live admitted rows, so a promoted claim's one
live home is the stance.

Two things are database predicates rather than validations. A tombstone names
its stance: position(stance in body) > 0 in memories_system_shape, so one
pointing nowhere is unrepresentable. And no record may name a tombstone: the
composite foreign key (memory_id, memory_promoted) -> memories (id, promoted)
refuses an admission, a challenge, and a refutation alike, with the literal
false pinned by memory_admissions_shape, the device challenge_role already
uses. promoted is generated from stance, so the two cannot disagree.

Two things are deliberately not enforced, and MEMORY-012 names both rather
than letting the invariant read as covering them: an unlinked coincidence, a
stance and an admitted memory nobody recorded as one claim; and a fresh claim
written on a promoted slug, which would take a read of memories by slug across
accounts that MEMORY-010 exists to keep out of this store.

docs/memory/knowledge-base-boundary.md carries the reasoning and the promotion
procedure, so the drain direction is operable rather than folklore.

Contract: docs/memory/2026-08-25-system-memory-spec.md section 8.
Closes OpenAgentsInc/openagents#65.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SoZMfWRSGnf6FZX2Ar9rQ2
Co-Authored-By
Claude Fable 5 <noreply@anthropic.com>
Closes
OpenAgentsInc/openagents#65 (another repository — recorded, not closed)

Deploy story

What this commit did to the running system — joined from the forge receipt chain, the part a commit page elsewhere cannot show.

pushed
by user · WAL seq 433 · 2026-08-26T01:40:21.532645Z

Changed files

  • modified INVARIANTS.md
  • added docs/memory/knowledge-base-boundary.md
  • modified docs/taxonomy.md
  • modified lib/openagents/memories/admission.ex
  • modified lib/openagents/memories/memory.ex
  • added lib/openagents/memories/promotions.ex
  • modified priv/migration_lineages/prior-2026-08-19.json
  • added priv/repo/migrations/20260826010000_promote_system_memories.exs
  • added test/openagents/memories/promotion_test.exs
  • modified test/openagents/memories_test.exs

Diff

10 files changed, +1268 -12

INVARIANTS.md modified +77

@@ -1295,6 +1295,74 @@ composite foreign keys `memory_admissions_memory_fkey` and

1295 1295
`test/openagents/memories/challenge_test.exs`, and
1296 1296
`test/openagents/memories/system_recall_test.exs`.
1297 1297
1298
### MEMORY-012 — A promoted claim has one live home, and the boundary is drawn where it crosses
1299
1300
Status: Current
1301
1302
The knowledge base owns what the project has reviewed and decided; system
1303
memory owns what the network has observed and can evidence. A stance is
1304
editorial and a memory row is evidentiary, and specification section 8 keeps the
1305
two from becoming rival stores of one claim with two rules: promotion drains
1306
memory into the knowledge base, and the knowledge base wins a recall collision.
1307
1308
**Promotion is where the boundary is enforced, and it is the only place it can
1309
be.** The knowledge base is retrieved in the client from a corpus compiled into
1310
a WebAssembly plugin; memory recall runs on this server inside
1311
`POST /api/v1/responses`, and the two notes reach the model through different
1312
fields of the same request. No process holds both. A precedence rule between
1313
them needs a decidable test for "the same claim", and two rails that retrieve by
1314
different methods over different corpora share no identifier unless somebody
1315
records one. **A memory and a stance are the same claim exactly when a promotion
1316
tombstone on that memory names that stance** — a steward's recorded judgment,
1317
never an overlap of prose. Nothing else is a collision here, so a memory that
1318
shares a stance's vocabulary, or quotes its id, is a different claim and keeps
1319
its place in recall.
1320
1321
Given that link, precedence needs no second mechanism. `promote/3` writes a
1322
superseding row on the claim's slug — a **promotion tombstone** — and points the
1323
old row at it, so the claim is no longer live and the chain is kept rather than
1324
deleted. The tombstone carries the stance id, a body this server composes rather
1325
than the caller, and `admission: "candidate"`, which is all it can ever hold: no
1326
admission, challenge, or refutation may name a promotion tombstone, refused by
1327
the composite foreign key `(memory_id, memory_promoted) -> memories (id,
1328
promoted)` with the literal `false` pinned by `memory_admissions_shape`, the
1329
same device `challenge_role` uses. `promoted` is generated from `stance`, so it
1330
cannot disagree with it. Recall surfaces live rows whose derived status is
1331
admitted (specification 7.1), so neither half of a promoted claim can reach a
1332
note by any route, and the reviewed stance is its one live home.
1333
1334
"A tombstone whose body names the stance" is a shape rather than a habit:
1335
`position(stance in body) > 0` is part of `memories_system_shape`, so a
1336
tombstone pointing nowhere is unrepresentable rather than merely unwritten by
1337
the code that exists today. A stance id is lowercase words joined by hyphens, as
1338
the corpus writes them, and a `user` or `learned` row carrying one is refused in
1339
both directions the way the other system columns are.
1340
1341
Only a steward promotes. Promotion records the outcome of a review, and the
1342
review is the knowledge base's authority; an author draining their own claim
1343
into the corpus would assert the review rather than record it. A promotion is a
1344
steward's correction on the target's slug, so it resolves the target's open
1345
challenges in the same transaction, as any steward correction does
1346
(MEMORY-011).
1347
1348
Two things are deliberately not enforced, and calling them enforced would be
1349
worse than leaving them open. **An unlinked coincidence** — a stance and an
1350
admitted memory a reader would call one claim, with no promotion between them —
1351
is not suppressed, because no decidable test separates it from two claims that
1352
merely share words; a similarity threshold shipped as "the knowledge base wins"
1353
would read as a guarantee while dropping true memories on a false positive,
1354
and duplication is the benign failure where suppression is the destructive one.
1355
The honest answer to a real duplicate is to promote it. **A fresh claim written
1356
on a promoted slug** is not refused either: draining by slug would take a read
1357
of `memories` across accounts, which is the predicate MEMORY-010 exists to keep
1358
out of this store.
1359
1360
Evidence: `OpenAgents.Memories.Promotions`, `OpenAgents.Memories.Memory`, the
1361
`memories_system_shape` stance clauses, the generated `memories.promoted`
1362
column, the composite foreign key `memory_admissions_promotion_fkey`,
1363
`docs/memory/knowledge-base-boundary.md`, and
1364
`test/openagents/memories/promotion_test.exs`.
1365
1298 1366
### PRIVACY-001 — Secret-bearing profile memory is rejected, never scrub-stored
1299 1367
1300 1368
Status: Current

@@ -3445,6 +3513,14 @@ sentence:

3445 3513
  is a second record beside the first rather than a verdict that quietly
3446 3514
  changed. The authority buys no read — the operator sees no memory of another
3447 3515
  account through this module.
3516
- Promoting an admitted system memory to a knowledge-base stance, through
3517
  `OpenAgents.Memories.Promotions` (MEMORY-012). Only a steward promotes,
3518
  because a promotion records the outcome of a review rather than asserts one,
3519
  and the steward set is this same allowlist. The write appends: the promotion
3520
  is a superseding tombstone on the claim's slug, so the claim, its evidence,
3521
  and its admission record all stay readable underneath it. The authority buys
3522
  no read here either — the target is named by id and refused as absent when it
3523
  is not a live system row.
3448 3524
- Recording Gym runs and trials under `POST /api/v1/gym/runs`, the lifecycle
3449 3525
  routes `POST /api/v1/gym/runs/start`, `POST /api/v1/gym/runs/:id/trials`,
3450 3526
  and `PATCH /api/v1/gym/runs/:id`

@@ -6194,6 +6270,7 @@ contract; the invariant prose above defines the assertion, not the filename.

6194 6270
| MEMORY-009 | `test/openagents/graph_memory_test.exs` |
6195 6271
| MEMORY-010 | `test/openagents/memories_test.exs`, `test/openagents_web/controllers/memory_controller_test.exs`, `test/openagents_web/controllers/responses_controller_test.exs` |
6196 6272
| MEMORY-011 | `test/openagents/memories/system_memory_test.exs`, `test/openagents/memories/challenge_test.exs`, `test/openagents/memories/system_recall_test.exs` |
6273
| MEMORY-012 | `test/openagents/memories/promotion_test.exs` |
6197 6274
| PRIVACY-001 | `test/openagents/memory/policy_and_redaction_test.exs`, `test/openagents/memory/scope_boundary_test.exs` |
6198 6275
| TURN-001 | `test/openagents/conversations_test.exs` |
6199 6276
| TURN-002 | `test/openagents/conversations_test.exs` |
docs/memory/knowledge-base-boundary.md added +209

@@ -0,0 +1,209 @@

1
# The knowledge-base boundary
2
3
The knowledge base and system memory both put network-level knowledge in front
4
of every session. This page records the line between them, where that line is
5
enforced, why it is enforced there rather than at recall, and how to run a
6
promotion.
7
8
Contract: `docs/memory/2026-08-25-system-memory-spec.md` section 8 in
9
`OpenAgentsInc/openagents`. Invariant: `INVARIANTS.md`, MEMORY-012.
10
11
## The line
12
13
The knowledge base owns what the project has reviewed and decided. Memory owns
14
what the network has observed and can evidence. A stance is editorial; a memory
15
row is evidentiary.
16
17
| | Knowledge base | System memory |
18
| --- | --- | --- |
19
| Unit | Stance or doc summary | Memory row |
20
| Authority | Human review; content checked into git | Evidence refs; steward admission receipt |
21
| Change | Regenerated from docs; edited like content | Superseded by later rows; never edited |
22
| Provenance | Source doc and review date | Writing account, `as_of`, evidence refs |
23
| Dispute | Documentation change | Challenge and refutation records |
24
| Note label | `[From the OpenAgents knowledge base — …]` | `[From memory: system, …]` |
25
26
The knowledge base lives in `OpenAgentsInc/openagents`: the corpus of stances is
27
`plugins/knowledge-base/kb/stances.json`, compiled by `build-kb.mjs` into
28
`kb.json` and embedded in a WebAssembly plugin the CLI loads. System memory
29
lives here, in the `system` bucket of `OpenAgents.Memories`.
30
31
## Where the boundary is enforced
32
33
**At promotion, in `OpenAgents.Memories.Promotions`.** Not at recall.
34
35
The two rails no longer meet in one process. The client retrieves the knowledge
36
base and concatenates its note into the outgoing turn text; this server recalls
37
memory inside `POST /api/v1/responses` and appends its note below the caller's
38
`instructions`. Both reach the model, neither sees the other.
39
40
The specification's second rule — the knowledge base wins a recall collision —
41
therefore needs an enforcement point chosen deliberately. This is the choice and
42
the reasoning.
43
44
### The collision test
45
46
**A memory and a stance are the same claim exactly when a promotion tombstone on
47
that memory names that stance.** Nothing else is a collision.
48
49
That is decidable, it is recorded by a person who checked, and it is the only
50
identity the two rails can share. The corpus gives each stance a stable
51
kebab-case `id` (`earning-bitcoin`, `coder-tiers`); a promotion writes that id
52
into the `stance` column of a tombstone. Two claims that merely share
53
vocabulary are two claims. A memory that quotes a stance id in its prose — a
54
claim *about* the corpus, say — is a different claim and keeps its place in
55
recall.
56
57
### Why not enforce at recall
58
59
Both recall-time shapes the specification names need the client to say "my
60
knowledge-base hit covers this memory", and neither can say it.
61
62
* **The identifier does not survive.** The corpus has stance ids, but the
63
  plugin's `Entry::Stance` does not deserialize `id`, so it is absent from the
64
  hit, from the plugin's declared output schema, from the TypeScript
65
  `KnowledgeHit`, and from the injected note. A client-side rule would compare a
66
  stance's prose to a memory's prose, and a server-side rule would compare
67
  prose the client forwarded. Either is a similarity heuristic.
68
69
* **A heuristic shipped as a rule is worse than no rule.** "The knowledge base
70
  wins collisions" reads as a guarantee. A false positive drops a true memory
71
  the reader never learns was withheld; a false negative lets the duplicate
72
  through anyway. Duplication is the benign failure — the two notes are labelled
73
  differently and a model can weigh them — and suppression is the destructive
74
  one, so a rule that trades the benign failure for the destructive one is a bad
75
  trade even when it is usually right.
76
77
* **Sending the hit to the server also widens the request.** A client-supplied
78
  "the knowledge base covers this" field is an untrusted claim that mutes the
79
  memory rail, and the server would depend on corpus state it cannot check.
80
81
* **The rule's purpose is served by draining.** When a stance and a memory
82
  really do speak to one claim, the correct response is not to silence one every
83
  turn: it is to promote, so the claim has one home. That is the specification's
84
  first rule, and making it operable is what this page and
85
  `Promotions.promote/3` are for.
86
87
Given the collision test above, a recall-time rule would also have nothing left
88
to do. A promoted claim is superseded, so it is not live; its tombstone can never
89
be admitted, so it is not eligible. Recall surfaces live admitted rows
90
(specification 7.1), so the promoted claim's one live home is the stance, and
91
there is no second speaker to silence.
92
93
### What is deliberately not enforced
94
95
* **An unlinked coincidence.** A stance and an admitted memory that a reader
96
  would call one claim, with no promotion between them, both attach. Nothing
97
  decides that today, for the reasons above. If it becomes a real problem, the
98
  fix is to promote the memory, not to add a threshold.
99
100
* **A fresh claim written on a promoted slug.** After a promotion the slug's live
101
  head is a tombstone; a steward who admits a new row on that slug is re-opening
102
  a claim the project already drained. Refusing it would take a read of
103
  `memories` by slug across accounts, which is the predicate MEMORY-010 exists to
104
  keep out of this store.
105
106
Both are named in MEMORY-012 so that nobody reads the invariant as covering
107
them.
108
109
## What a promotion writes
110
111
`Promotions.promote/3` is the ordinary correction path with a fixed shape. It
112
writes a superseding row on the claim's slug — a **promotion tombstone** — and
113
points the old row at it.
114
115
The tombstone carries:
116
117
* `stance` — the knowledge-base stance id. Lowercase words joined by hyphens.
118
* a body this server composes, naming that stance. The table requires the stance
119
  to appear in the body (`position(stance in body) > 0` in
120
  `memories_system_shape`), so a tombstone that points nowhere cannot exist.
121
* `evidence_refs` — where the stance lives and the digest of what was reviewed.
122
* `admission: "candidate"`, which is all it can ever hold. No admission,
123
  challenge, or refutation may name a promotion tombstone: the composite foreign
124
  key `(memory_id, memory_promoted) -> memories (id, promoted)` refuses it, with
125
  the literal `false` pinned by `memory_admissions_shape`.
126
127
Nothing is deleted. The claim, its evidence, and its admission record stay
128
readable underneath the tombstone, and a promotion by a steward resolves the
129
claim's open challenges in the same transaction, as any steward correction does.
130
131
## Running a promotion
132
133
A promotion has two halves, and they land in two repositories. Do the review
134
half first: until the stance exists, the tombstone would point at nothing.
135
136
### 1. Add the stance to the corpus
137
138
In `OpenAgentsInc/openagents`:
139
140
1. Add a record to `plugins/knowledge-base/kb/stances.json` with an `id`
141
   (lowercase words joined by hyphens), a `title`, the `questions` a reader
142
   would ask, a `state`, the `answer`, the `sources` it rests on, and today's
143
   `date`. Cite the memory you are draining among the sources.
144
2. Rebuild the corpus and the artifact:
145
146
   ```sh
147
   cd plugins/knowledge-base
148
   node build-kb.mjs
149
   cargo build --release --target wasm32-unknown-unknown -p knowledge-base
150
   ```
151
152
3. Re-pin `artifact.digest` in `plugins/knowledge-base/manifest.json` to the
153
   digest of the rebuilt `knowledge_base.wasm`.
154
4. Review and land the change the way any documentation change lands. The
155
   stance is only reviewed once a person has reviewed it.
156
157
### 2. Drain the memory
158
159
In this repository, as a steward:
160
161
```elixir
162
OpenAgents.Memories.Promotions.promote(steward, memory_id, %{
163
  "stance" => "gateway-402-retired-model",
164
  "slug" => "sys:gateway-402-retired-model",
165
  "evidence_refs" => [
166
    %{
167
      "kind" => "url",
168
      "ref" => "https://openagents.com/OpenAgentsInc/openagents/plugins/knowledge-base/kb/stances.json",
169
      "digest" => "sha256:…"
170
    }
171
  ]
172
})
173
```
174
175
* `slug` is the target's slug. It binds the tombstone to the claim it drains,
176
  the same way `Admissions.supersede/3` requires it. Read it off the claim
177
  before promoting.
178
* `digest` is the digest of the reviewed stance, so the evidence behind the
179
  promotion cannot be swapped afterwards.
180
* `as_of` defaults to today — the date the claim became a reviewed position.
181
182
Refusals: `:steward_required` when the account is not an operator,
183
`:stance_required` when no stance is named, `:not_supersedable` when the target
184
is not a live system memory, and a changeset for a malformed stance id or
185
missing evidence.
186
187
There is no HTTP route for this, exactly as there is none for admission or
188
challenge. Promotion is a steward's action on the network's store, taken from a
189
console with the same authority `OpenAgents.Memories.Admissions` requires.
190
191
### 3. Check the drain
192
193
The claim should have left recall without leaving the store:
194
195
```elixir
196
claim = OpenAgents.Repo.get!(OpenAgents.Memories.Memory, memory_id)
197
claim.superseded_by_id                         # the tombstone
198
OpenAgents.Memories.Admissions.status(claim)   # still "admitted" — nothing was erased
199
```
200
201
## When to promote
202
203
A system memory is ready to drain when it has stabilized: admitted, unchallenged
204
for a sustained period, and repeatedly recalled. Promoting earlier turns an
205
observation into a reviewed position before the review has anything to review.
206
207
Promote sooner than that in one case: when a stance already covers the claim.
208
That is the duplicate the recall-time rule was written for, and draining it is
209
how this store answers it.
docs/taxonomy.md modified +17

@@ -335,6 +335,23 @@ candidate, and a suspension is a claim that was admitted and is now contested.

335 335
Say "queued" for a challenge held back by the flood cap: it is recorded and
336 336
attributed, and it has no effect on any status.
337 337
338
**Stance** — a reviewed editorial position in the knowledge base, which lives in
339
`OpenAgentsInc/openagents` at `plugins/knowledge-base/kb/stances.json` and is
340
retrieved in the client. A stance is what the project has reviewed and decided;
341
a system memory is what the network has observed and can evidence. Say "stance"
342
only for the knowledge-base record and never for a memory row, and name a stance
343
by its corpus `id` — lowercase words joined by hyphens.
344
345
**Promotion** and **promotion tombstone** — a *promotion* is a steward draining
346
a stabilized system memory into a stance: the claim becomes a reviewed position
347
and stops being a live memory. The superseding row it leaves behind is a
348
*promotion tombstone*, and it names the stance in its body. Say "promotion
349
tombstone" in full — a bare "tombstone" means the portable-export tombstone in
350
`OpenAgents.Memory.Portability`, which is a different thing. A promotion is the
351
one place the knowledge-base boundary is enforced, so do not describe recall as
352
preferring the knowledge base: it does not compare the two rails
353
(`docs/memory/knowledge-base-boundary.md`, MEMORY-012).
354
338 355
### Threads
339 356
340 357
A **thread** is the unit of agent work. Everything in this section describes a
lib/openagents/memories/admission.ex modified +17 -1

@@ -39,8 +39,15 @@ defmodule OpenAgents.Memories.Admission do

39 39
  `user` or `learned` row without reading `memories` at all, which is how these
40 40
  paths stay clear of the account boundary MEMORY-010 draws.
41 41
42
  `memory_promoted` is always the literal `false`, and the composite foreign key
43
  `(memory_id, memory_promoted) -> memories (id, promoted)` is what refuses a
44
  record naming a promotion tombstone. A tombstone is a pointer at a reviewed
45
  knowledge-base stance, not a claim: there is nothing to admit, nothing to
46
  challenge, and the knowledge base's own dispute path is a documentation
47
  change. See `OpenAgents.Memories.Promotions`.
48
42 49
  `challenge_role` is always the literal `challenge` on a refutation and null
43
  elsewhere. It exists so the second composite foreign key,
50
  elsewhere. It exists so the third composite foreign key,
44 51
  `(challenge_id, memory_id, challenge_role) -> (id, memory_id, role)`, can
45 52
  insist that a refutation names a challenge — and one against the same memory
46 53
  it claims to restore. PostgreSQL will not put a literal in a foreign key, so

@@ -65,6 +72,7 @@ defmodule OpenAgents.Memories.Admission do

65 72
  schema "memory_admissions" do
66 73
    belongs_to :memory, Memory
67 74
    field :memory_bucket, :string, default: "system"
75
    field :memory_promoted, :boolean, default: false
68 76
    belongs_to :author, User
69 77
    belongs_to :challenge, __MODULE__
70 78
    field :challenge_role, :string

@@ -169,6 +177,7 @@ defmodule OpenAgents.Memories.Admission do

169 177
    |> update_change(:ground, &trim/1)
170 178
    |> put_change(:role, role)
171 179
    |> put_change(:memory_bucket, "system")
180
    |> put_change(:memory_promoted, false)
172 181
  end
173 182
174 183
  defp validate_shape(changeset) do

@@ -184,6 +193,13 @@ defmodule OpenAgents.Memories.Admission do

184 193
      name: :memory_admissions_memory_fkey,
185 194
      message: "names no system memory"
186 195
    )
196
    # The second composite key. A promotion tombstone is a pointer at a reviewed
197
    # stance rather than a claim, so no record may name one, and the refusal is
198
    # the foreign key rather than a read of the row.
199
    |> foreign_key_constraint(:memory_id,
200
      name: :memory_admissions_promotion_fkey,
201
      message: "names a promoted memory, which is a tombstone rather than a claim"
202
    )
187 203
    |> check_constraint(:verdict, name: :memory_admissions_shape)
188 204
    |> check_constraint(:evidence_refs,
189 205
      name: :memory_admissions_shape,
lib/openagents/memories/memory.ex modified +81 -9

@@ -32,6 +32,11 @@ defmodule OpenAgents.Memories.Memory do

32 32
  records, so a row that says `admitted` with no steward record behind it still
33 33
  reads as a candidate.
34 34
35
  A system row may also carry a `stance`, and one that does is a **promotion
36
  tombstone**: the claim was promoted to a reviewed knowledge-base stance, this
37
  row supersedes the claim, and its body names the stance that replaced it. See
38
  `OpenAgents.Memories.Promotions`.
39
35 40
  `superseded_by_id` is how a correction lands. The replacement is a new row
36 41
  and the old row points at it, so the store keeps the chain rather than
37 42
  overwriting the mistake. Nothing here updates `body`: a memory's text is

@@ -61,6 +66,10 @@ defmodule OpenAgents.Memories.Memory do

61 66
  @slug_prefix "sys:"
62 67
  @tiers ~w(ledger glass)
63 68
  @admissions ~w(candidate admitted rejected)
69
  @stance_characters 200
70
  # A knowledge-base stance id: lowercase words joined by hyphens, as the
71
  # corpus writes them (`earning-bitcoin`, `coder-tiers`).
72
  @stance_format ~r/^[a-z0-9]+(-[a-z0-9]+)*$/
64 73
65 74
  schema "memories" do
66 75
    belongs_to :user, User

@@ -77,6 +86,18 @@ defmodule OpenAgents.Memories.Memory do

77 86
    field :as_of, :date
78 87
    field :admission, :string
79 88
    field :evidence_refs, {:array, :map}
89
90
    # The knowledge-base stance this claim was promoted to. A row carrying one
91
    # is a promotion tombstone: the claim's live home is the reviewed stance,
92
    # and this row exists to say so. Null on every other row.
93
    field :stance, :string
94
95
    # Whether this row is a promotion tombstone. PostgreSQL generates it from
96
    # `stance`, so it cannot disagree; it exists as a column because a foreign
97
    # key can reference one and an expression cannot. Read back after a write,
98
    # never written.
99
    field :promoted, :boolean, read_after_writes: true
100
80 101
    # The generated `tsvector` the lexical stand-in ranks over. PostgreSQL
81 102
    # writes it; nothing here reads it back, so it never rides a select.
82 103
    field :search_vector, :string, load_in_query: false

@@ -123,6 +144,20 @@ defmodule OpenAgents.Memories.Memory do

123 144
  @spec evidence_kinds() :: [String.t()]
124 145
  def evidence_kinds, do: Evidence.kinds()
125 146
147
  @doc """
148
  Whether this row is a promotion tombstone.
149
150
  A tombstone is a pointer at a reviewed stance, not a claim of its own: no
151
  admission, challenge, or refutation may name one, and nothing can admit it, so
152
  it reaches no session's recall.
153
  """
154
  @spec promoted?(t()) :: boolean()
155
  def promoted?(%__MODULE__{stance: stance}), do: is_binary(stance)
156
157
  @doc "The longest stance id the store accepts, in characters."
158
  @spec stance_characters() :: pos_integer()
159
  def stance_characters, do: @stance_characters
160
126 161
  @doc "The prefix every system slug carries."
127 162
  @spec slug_prefix() :: String.t()
128 163
  def slug_prefix, do: @slug_prefix

@@ -152,12 +187,14 @@ defmodule OpenAgents.Memories.Memory do

152 187
      :tier,
153 188
      :as_of,
154 189
      :admission,
155
      :evidence_refs
190
      :evidence_refs,
191
      :stance
156 192
    ])
157 193
    |> update_change(:body, &trim/1)
158 194
    |> update_change(:source_ref, &trim/1)
159 195
    |> update_change(:slug, &trim/1)
160 196
    |> update_change(:entity, &trim/1)
197
    |> update_change(:stance, &trim/1)
161 198
    |> validate_required([:bucket, :body])
162 199
    |> validate_inclusion(:bucket, @buckets)
163 200
    |> validate_length(:body, min: 1, max: @body_characters, count: :graphemes)

@@ -191,17 +228,52 @@ defmodule OpenAgents.Memories.Memory do

191 228
    |> validate_inclusion(:tier, @tiers)
192 229
    |> validate_inclusion(:admission, @admissions)
193 230
    |> Evidence.validate(:evidence_refs)
231
    |> validate_stance()
232
  end
233
234
  # A stance is optional — most system rows carry none — but a row that names
235
  # one is a promotion tombstone, and both halves of that shape are checked
236
  # here and again at the table. `position(stance in body) > 0` is the database
237
  # half of "a tombstone whose body names the stance"; this is the half that can
238
  # explain itself to the caller.
239
  defp validate_stance(changeset) do
240
    case get_field(changeset, :stance) do
241
      nil ->
242
        changeset
243
244
      stance ->
245
        changeset
246
        |> validate_length(:stance, min: 1, max: @stance_characters, count: :graphemes)
247
        |> validate_format(:stance, @stance_format,
248
          message: "must be a knowledge-base stance id, in lowercase words joined by hyphens"
249
        )
250
        |> validate_body_names(stance)
251
    end
252
  end
253
254
  defp validate_body_names(changeset, stance) do
255
    body = get_field(changeset, :body)
256
257
    if is_binary(body) and String.contains?(body, stance) do
258
      changeset
259
    else
260
      add_error(changeset, :body, "must name the stance this claim was promoted to")
261
    end
194 262
  end
195 263
196 264
  defp refuse_system_fields(changeset) do
197
    Enum.reduce([:slug, :entity, :tier, :as_of, :admission, :evidence_refs], changeset, fn
198
      field, acc ->
199
        if is_nil(get_field(acc, field)) do
200
          acc
201
        else
202
          add_error(acc, field, "belongs only to a system memory")
203
        end
204
    end)
265
    Enum.reduce(
266
      [:slug, :entity, :tier, :as_of, :admission, :evidence_refs, :stance],
267
      changeset,
268
      fn
269
        field, acc ->
270
          if is_nil(get_field(acc, field)) do
271
            acc
272
          else
273
            add_error(acc, field, "belongs only to a system memory")
274
          end
275
      end
276
    )
205 277
  end
206 278
207 279
  @doc "Points a memory at the memory that replaced it."
lib/openagents/memories/promotions.ex added +176

@@ -0,0 +1,176 @@

1
defmodule OpenAgents.Memories.Promotions do
2
  @moduledoc """
3
  The drain from the memory store into the knowledge base, and the place the
4
  boundary between the two is enforced.
5
6
  ## The line, and where it is drawn
7
8
  Specification section 8 draws the line: the knowledge base owns what the
9
  project has reviewed and decided, memory owns what the network has observed
10
  and can evidence. A stance is editorial, a memory row is evidentiary. Two
11
  rules keep the two from becoming rival stores of one claim — promotion drains
12
  memory into the knowledge base, and the knowledge base wins a recall
13
  collision.
14
15
  The second rule needs a place to stand, and the cloud re-base took it away.
16
  The knowledge base is retrieved in the client, from a corpus compiled into a
17
  WebAssembly plugin; memory recall runs here, inside `POST /api/v1/responses`.
18
  The two notes are assembled in different processes and arrive in different
19
  parts of the request, so no process sees both.
20
21
  **This module is the enforcement point.** The boundary is drawn where a claim
22
  crosses it — at promotion — rather than at recall, and the reasoning is in
23
  `docs/memory/knowledge-base-boundary.md`. The short form:
24
25
  * A precedence rule needs a decidable notion of "the same claim". Two rails
26
    that retrieve by different methods over different corpora share no
27
    identifier unless somebody records one, and promotion is the only moment
28
    anybody does: a steward states that this row is now that stance.
29
30
  * Given that link, a recall-time rule would have nothing left to do. A
31
    promoted row is superseded, so it is not live; the tombstone that replaced
32
    it can never be admitted, so it is not eligible. Recall surfaces live
33
    admitted rows (specification 7.1), so a promoted claim's one live home is
34
    the stance, and there is no second speaker for the collision rule to
35
    silence.
36
37
  * For a pair no promotion links, no enforcement point could decide anything
38
    either. The client would have to judge "covers the same claim" by comparing
39
    a stance's prose to a memory's, which is a heuristic; shipping one as
40
    "the knowledge base wins" would read as a guarantee while dropping true
41
    memories on a false positive. Duplication is the benign failure and
42
    suppression is the destructive one, so the honest answer to a real duplicate
43
    is to promote it, which is what this module is for.
44
45
  ## What a promotion is
46
47
  `promote/3` is the existing correction path with a fixed shape. It writes a
48
  superseding row on the claim's slug — a **promotion tombstone** — and points
49
  the old row at it, so the store keeps the chain rather than deleting the
50
  claim. The tombstone carries:
51
52
  * `stance`, the knowledge-base stance id the claim now lives as. It is the
53
    `id` field of a record in the corpus at
54
    `plugins/knowledge-base/kb/stances.json` in `OpenAgentsInc/openagents`.
55
  * a body this module writes rather than the caller, naming that stance. The
56
    table requires the stance to appear in the body, so "a tombstone whose body
57
    names the stance" is a shape rather than a habit.
58
  * `evidence_refs` the steward cites for the promotion — where the stance
59
    lives and the digest of what was reviewed.
60
  * `admission: "candidate"`, which is all it can ever be. Nothing can admit a
61
    tombstone: the composite foreign key
62
    `(memory_id, memory_promoted) -> memories (id, promoted)` refuses an
63
    admission, a challenge, and a refutation alike.
64
65
  Only a steward promotes. Promotion records the outcome of a review, and the
66
  review is the knowledge base's authority; an author draining their own claim
67
  into the corpus would be asserting the review rather than recording it.
68
69
  ## What this module does not enforce
70
71
  Two things, both named rather than papered over.
72
73
  * **An unlinked coincidence.** A stance and an admitted system memory that a
74
    reader would call the same claim, with no promotion between them, both
75
    attach. Nothing here decides that, because nothing can decide it: the two
76
    rails share no identifier for the pair.
77
78
  * **A new claim written on a promoted slug.** After a promotion the slug's
79
    live head is a tombstone, and a steward who admits a fresh row on that slug
80
    is re-opening a claim the project already drained. Refusing it would take a
81
    read of `memories` by slug across accounts, which is the predicate
82
    MEMORY-010 exists to keep out of this store.
83
  """
84
85
  alias OpenAgents.Accounts.User
86
  alias OpenAgents.Memories.{Admissions, Memory}
87
88
  @tier "ledger"
89
90
  @doc """
91
  Promotes one system memory to a knowledge-base stance.
92
93
  Attributes: `stance` (required) — the stance id the claim now lives as — and
94
  `slug` (required) — the target's slug, which is what binds the tombstone to
95
  the claim it drains, exactly as `OpenAgents.Memories.Admissions.supersede/3`
96
  requires it. `evidence_refs` (required) cites where the stance lives and the
97
  digest of what was reviewed. `as_of` defaults to today, the date the claim
98
  became a reviewed position, and `entity` carries over from the claim when the
99
  caller names it.
100
101
  The body is not a caller's to write: `body/1` composes it so the stance is
102
  named the same way every time, and the table refuses a tombstone whose body
103
  does not name its stance.
104
105
  When the target is under open challenges, the same transaction records a
106
  refutation of each — a promotion is a steward's correction, and the
107
  correction is the resolution.
108
109
  Refuses `:steward_required` for an account without the role,
110
  `:stance_required` when no stance is named or the name is blank — there is
111
  nowhere for the claim to go, so there is no promotion to write — and
112
  `:not_supersedable` when the target is not a live system memory. A caller with no standing learns nothing
113
  about the row from any of them.
114
  """
115
  @spec promote(User.t(), String.t(), map()) ::
116
          {:ok, Memory.t()}
117
          | {:error, Ecto.Changeset.t()}
118
          | {:error, :steward_required}
119
          | {:error, :stance_required}
120
          | {:error, :not_supersedable}
121
  def promote(%User{} = steward, memory_id, attrs) when is_map(attrs) do
122
    attrs = normalize(attrs)
123
124
    cond do
125
      not Admissions.steward?(steward) -> {:error, :steward_required}
126
      not named?(Map.get(attrs, "stance")) -> {:error, :stance_required}
127
      true -> Admissions.supersede(steward, memory_id, tombstone(attrs))
128
    end
129
  end
130
131
  # A blank stance is refused here rather than left to the changeset, which
132
  # casts an empty string to `nil` and would write an ordinary supersession
133
  # under a body announcing a promotion to nowhere.
134
  defp named?(stance) when is_binary(stance), do: String.trim(stance) != ""
135
  defp named?(_absent), do: false
136
137
  @doc """
138
  The body a promotion tombstone carries.
139
140
  Fixed rather than free text, so the stance is named identically on every
141
  tombstone and a reader who meets one knows where the claim went. The table
142
  requires the stance to appear here.
143
  """
144
  @spec body(String.t()) :: String.t()
145
  def body(stance) when is_binary(stance) do
146
    "Promoted to the OpenAgents knowledge base stance `#{stance}`. " <>
147
      "The reviewed stance is the live home for this claim; this row is a " <>
148
      "tombstone and surfaces to nobody."
149
  end
150
151
  @doc "The transparency tier every promotion tombstone carries."
152
  @spec tier() :: String.t()
153
  def tier, do: @tier
154
155
  # The tombstone's shape. Everything the caller may name is read from `attrs`;
156
  # everything that makes this a tombstone rather than a claim is put here, so a
157
  # request body cannot ask for a promotion that admits itself or for a body
158
  # that points somewhere other than the stance.
159
  defp tombstone(attrs) do
160
    stance = attrs |> Map.fetch!("stance") |> String.trim()
161
162
    attrs
163
    |> Map.take(["slug", "entity", "evidence_refs"])
164
    |> Map.merge(%{
165
      "stance" => stance,
166
      "body" => body(stance),
167
      "tier" => @tier,
168
      "admission" => "candidate",
169
      "as_of" => Map.get(attrs, "as_of") || Date.utc_today()
170
    })
171
  end
172
173
  defp normalize(attrs) do
174
    Map.new(attrs, fn {key, value} -> {to_string(key), value} end)
175
  end
176
end
priv/migration_lineages/prior-2026-08-19.json modified +2 -1

@@ -311,7 +311,8 @@

311 311
    20260825170100,
312 312
    20260825195623,
313 313
    20260825220000,
314
    20260825230000
314
    20260825230000,
315
    20260826010000
315 316
  ],
316 317
  "required_tables": [
317 318
    "users",
priv/repo/migrations/20260826010000_promote_system_memories.exs added +278

@@ -0,0 +1,278 @@

1
defmodule OpenAgents.Repo.Migrations.PromoteSystemMemories do
2
  use Ecto.Migration
3
4
  # The drain from the memory store into the knowledge base, and the line that
5
  # keeps the two from becoming rival stores of one claim.
6
  #
7
  # Specification section 8 draws the line — the knowledge base owns what the
8
  # project has reviewed and decided, memory owns what the network has observed
9
  # and can evidence — and states two rules over it. The first is that promotion
10
  # drains memory into the knowledge base: a stabilized claim becomes a reviewed
11
  # stance, and the row it came from is superseded by a **promotion tombstone**
12
  # whose body names that stance, so the claim has exactly one live home. The
13
  # second is that the knowledge base wins a recall collision.
14
  #
15
  # This migration implements the first rule and makes the second one moot for
16
  # every claim the first covers. `docs/memory/knowledge-base-boundary.md`
17
  # carries the reasoning; the short version is that a claim is "the same claim"
18
  # across the two rails only when a promotion recorded the link, and once a
19
  # promotion has recorded it the memory half is no longer a live admitted row
20
  # for recall to collide with.
21
  #
22
  # Two decisions are database predicates rather than changeset validations
23
  # (MEMORY-004):
24
  #
25
  #   * **A tombstone names its stance.** `position(stance in body) > 0` is what
26
  #     makes "whose body names the stance" a shape rather than a convention the
27
  #     writing code happens to follow today. A tombstone that pointed nowhere
28
  #     would leave a reader holding a claim with no live home at all.
29
  #
30
  #   * **A tombstone is not a claim, so no record may name one.** The composite
31
  #     foreign key `(memory_id, memory_promoted) -> memories (id, promoted)`,
32
  #     with `memory_promoted` pinned to `false` by the shape constraint, is
33
  #     what makes an admission, a challenge, or a refutation against a
34
  #     tombstone unrepresentable. That is the load-bearing half: recall
35
  #     surfaces admitted rows only (specification 7.1), so a row that can never
36
  #     be admitted can never be recalled, and the promoted claim's one live
37
  #     home is the stance.
38
  #
39
  # The pinned literal is the same device `challenge_role` already uses:
40
  # PostgreSQL will not put a literal in a foreign key, so the literal is a
41
  # column the check constraint holds down.
42
  def up do
43
    alter table(:memories) do
44
      # The knowledge-base stance this claim was promoted to, and the only
45
      # cross-rail identifier either rail has. It is the `id` field of a record
46
      # in the knowledge-base corpus (`plugins/knowledge-base/kb/stances.json`
47
      # in `OpenAgentsInc/openagents`), which is kebab-case and stable across
48
      # regenerations of the compiled plugin.
49
      #
50
      # A row carrying one is a promotion tombstone. Null on every other row,
51
      # system or otherwise.
52
      add :stance, :string
53
    end
54
55
    # Whether this row is a promotion tombstone, as a column a foreign key can
56
    # reference. Generated rather than written, so it cannot disagree with
57
    # `stance`, and stored rather than virtual, so it can be indexed.
58
    execute(
59
      """
60
      ALTER TABLE memories
61
      ADD COLUMN promoted boolean
62
      GENERATED ALWAYS AS (stance IS NOT NULL) STORED
63
      """,
64
      "ALTER TABLE memories DROP COLUMN promoted"
65
    )
66
67
    drop constraint(:memories, :memories_system_shape)
68
69
    # The system shape, extended in both directions: `stance` is refused
70
    # outright on a `user` or `learned` row, and on a system row it is either
71
    # absent or a well-formed stance id that the body names.
72
    #
73
    # Every column is asserted `IS NOT NULL` before it is compared, for the
74
    # reason the constraint this replaces gives: a check constraint passes when
75
    # it evaluates to NULL, so a format test alone would admit a tombstone whose
76
    # stance is absent rather than malformed.
77
    create constraint(:memories, :memories_system_shape,
78
             check: """
79
             (
80
               bucket <> 'system'
81
               AND slug IS NULL
82
               AND entity IS NULL
83
               AND tier IS NULL
84
               AND as_of IS NULL
85
               AND admission IS NULL
86
               AND evidence_refs IS NULL
87
               AND stance IS NULL
88
             ) OR (
89
               bucket = 'system'
90
               AND slug IS NOT NULL
91
               AND slug LIKE 'sys:%'
92
               AND char_length(slug) BETWEEN 5 AND 200
93
               AND (entity IS NULL OR char_length(entity) BETWEEN 1 AND 200)
94
               AND tier IS NOT NULL
95
               AND tier IN ('ledger','glass')
96
               AND as_of IS NOT NULL
97
               AND admission IS NOT NULL
98
               AND admission IN ('candidate','admitted','rejected')
99
               AND evidence_refs IS NOT NULL
100
               AND jsonb_typeof(evidence_refs) = 'array'
101
               AND jsonb_array_length(evidence_refs) BETWEEN 1 AND 20
102
               AND NOT jsonb_path_exists(evidence_refs, '$[*] ? (!(@.type() == "object"
103
                     && exists(@.kind ? (@ == "receipt" || @ == "memory" || @ == "url"))
104
                     && exists(@.ref ? (@.type() == "string" && @ != ""))
105
                     && exists(@.digest ? (@.type() == "string" && @ != ""))))')
106
               AND (
107
                 stance IS NULL
108
                 OR (
109
                   char_length(stance) BETWEEN 1 AND 200
110
                   AND stance ~ '^[a-z0-9]+(-[a-z0-9]+)*$'
111
                   AND position(stance in body) > 0
112
                 )
113
               )
114
             )
115
             """
116
           )
117
118
    # What the composite foreign key below points at.
119
    create unique_index(:memories, [:id, :promoted], name: :memories_id_promoted_index)
120
121
    # Reading the store's promotion tombstones without scanning it.
122
    create index(:memories, [:stance], where: "stance IS NOT NULL")
123
124
    alter table(:memory_admissions) do
125
      # Always `false`. It exists so the foreign key below can insist that
126
      # `memory_id` names a memory that is not a promotion tombstone; the shape
127
      # constraint pins the literal.
128
      add :memory_promoted, :boolean, null: false, default: false
129
    end
130
131
    execute(
132
      """
133
      ALTER TABLE memory_admissions
134
      ADD CONSTRAINT memory_admissions_promotion_fkey
135
      FOREIGN KEY (memory_id, memory_promoted)
136
      REFERENCES memories (id, promoted) ON DELETE CASCADE
137
      """,
138
      "ALTER TABLE memory_admissions DROP CONSTRAINT memory_admissions_promotion_fkey"
139
    )
140
141
    drop constraint(:memory_admissions, :memory_admissions_shape)
142
143
    create constraint(:memory_admissions, :memory_admissions_shape,
144
             check: """
145
             memory_bucket = 'system'
146
             AND memory_promoted = false
147
             AND role IN ('admission','challenge','refutation')
148
             AND char_length(slug) BETWEEN 1 AND 200
149
             AND char_length(ground) BETWEEN 1 AND 2000
150
             AND (
151
               (role = 'admission'
152
                AND verdict IS NOT NULL
153
                AND verdict IN ('admitted','rejected')
154
                AND slug = 'adm:' || memory_id::text
155
                AND challenge_id IS NULL
156
                AND challenge_role IS NULL
157
                AND evidence_refs IS NULL)
158
               OR (role = 'challenge'
159
                AND verdict IS NULL
160
                AND slug = 'chl:' || memory_id::text
161
                AND challenge_id IS NULL
162
                AND challenge_role IS NULL
163
                AND (
164
                  evidence_refs IS NULL
165
                  OR (
166
                    jsonb_typeof(evidence_refs) = 'array'
167
                    AND jsonb_array_length(evidence_refs) BETWEEN 1 AND 20
168
                    AND NOT jsonb_path_exists(evidence_refs, '$[*] ? (!(@.type() == "object"
169
                          && exists(@.kind ? (@ == "receipt" || @ == "memory" || @ == "url"))
170
                          && exists(@.ref ? (@.type() == "string" && @ != ""))
171
                          && exists(@.digest ? (@.type() == "string" && @ != ""))))')
172
                  )
173
                ))
174
               OR (role = 'refutation'
175
                AND verdict IS NULL
176
                AND challenge_id IS NOT NULL
177
                AND challenge_role IS NOT NULL
178
                AND challenge_role = 'challenge'
179
                AND slug = 'ref:' || challenge_id::text
180
                AND evidence_refs IS NULL)
181
             )
182
             """
183
           )
184
  end
185
186
  def down do
187
    drop constraint(:memory_admissions, :memory_admissions_shape)
188
189
    create constraint(:memory_admissions, :memory_admissions_shape,
190
             check: """
191
             memory_bucket = 'system'
192
             AND role IN ('admission','challenge','refutation')
193
             AND char_length(slug) BETWEEN 1 AND 200
194
             AND char_length(ground) BETWEEN 1 AND 2000
195
             AND (
196
               (role = 'admission'
197
                AND verdict IS NOT NULL
198
                AND verdict IN ('admitted','rejected')
199
                AND slug = 'adm:' || memory_id::text
200
                AND challenge_id IS NULL
201
                AND challenge_role IS NULL
202
                AND evidence_refs IS NULL)
203
               OR (role = 'challenge'
204
                AND verdict IS NULL
205
                AND slug = 'chl:' || memory_id::text
206
                AND challenge_id IS NULL
207
                AND challenge_role IS NULL
208
                AND (
209
                  evidence_refs IS NULL
210
                  OR (
211
                    jsonb_typeof(evidence_refs) = 'array'
212
                    AND jsonb_array_length(evidence_refs) BETWEEN 1 AND 20
213
                    AND NOT jsonb_path_exists(evidence_refs, '$[*] ? (!(@.type() == "object"
214
                          && exists(@.kind ? (@ == "receipt" || @ == "memory" || @ == "url"))
215
                          && exists(@.ref ? (@.type() == "string" && @ != ""))
216
                          && exists(@.digest ? (@.type() == "string" && @ != ""))))')
217
                  )
218
                ))
219
               OR (role = 'refutation'
220
                AND verdict IS NULL
221
                AND challenge_id IS NOT NULL
222
                AND challenge_role IS NOT NULL
223
                AND challenge_role = 'challenge'
224
                AND slug = 'ref:' || challenge_id::text
225
                AND evidence_refs IS NULL)
226
             )
227
             """
228
           )
229
230
    execute("ALTER TABLE memory_admissions DROP CONSTRAINT memory_admissions_promotion_fkey")
231
232
    alter table(:memory_admissions) do
233
      remove :memory_promoted
234
    end
235
236
    drop index(:memories, [:stance], where: "stance IS NOT NULL")
237
    drop index(:memories, [:id, :promoted], name: :memories_id_promoted_index)
238
    drop constraint(:memories, :memories_system_shape)
239
240
    create constraint(:memories, :memories_system_shape,
241
             check: """
242
             (
243
               bucket <> 'system'
244
               AND slug IS NULL
245
               AND entity IS NULL
246
               AND tier IS NULL
247
               AND as_of IS NULL
248
               AND admission IS NULL
249
               AND evidence_refs IS NULL
250
             ) OR (
251
               bucket = 'system'
252
               AND slug IS NOT NULL
253
               AND slug LIKE 'sys:%'
254
               AND char_length(slug) BETWEEN 5 AND 200
255
               AND (entity IS NULL OR char_length(entity) BETWEEN 1 AND 200)
256
               AND tier IS NOT NULL
257
               AND tier IN ('ledger','glass')
258
               AND as_of IS NOT NULL
259
               AND admission IS NOT NULL
260
               AND admission IN ('candidate','admitted','rejected')
261
               AND evidence_refs IS NOT NULL
262
               AND jsonb_typeof(evidence_refs) = 'array'
263
               AND jsonb_array_length(evidence_refs) BETWEEN 1 AND 20
264
               AND NOT jsonb_path_exists(evidence_refs, '$[*] ? (!(@.type() == "object"
265
                     && exists(@.kind ? (@ == "receipt" || @ == "memory" || @ == "url"))
266
                     && exists(@.ref ? (@.type() == "string" && @ != ""))
267
                     && exists(@.digest ? (@.type() == "string" && @ != ""))))')
268
             )
269
             """
270
           )
271
272
    execute("ALTER TABLE memories DROP COLUMN promoted")
273
274
    alter table(:memories) do
275
      remove :stance
276
    end
277
  end
278
end
test/openagents/memories/promotion_test.exs added +404

@@ -0,0 +1,404 @@

1
defmodule OpenAgents.Memories.PromotionTest do
2
  @moduledoc """
3
  The knowledge-base boundary, and the one place it is enforced.
4
5
  Specification section 8 says the knowledge base wins when both rails speak
6
  about the same claim. Since the cloud re-base the two rails are assembled in
7
  different processes, so this store draws the line where a claim crosses it —
8
  at promotion — rather than where the notes are assembled. The reasoning is in
9
  `docs/memory/knowledge-base-boundary.md`; what these tests pin is that the
10
  line actually holds.
11
12
  **The collision test is identity, never text.** Two claims are the same claim
13
  across the rails exactly when a promotion tombstone on the memory's slug names
14
  the stance, and that link is recorded by a steward at promotion. Prose overlap
15
  is not a collision here, and the near-miss tests below are the half that says
16
  so: a memory whose words match a stance's, and even one whose body quotes the
17
  stance id, stays live and stays eligible, because nobody recorded that they
18
  are the same claim.
19
20
  What makes the resolution real rather than advisory is that a promoted claim
21
  cannot reach recall by any route. The row it came from is superseded, so it is
22
  not live. The tombstone that replaced it can never be admitted, because the
23
  composite foreign key `(memory_id, memory_promoted)` refuses every record that
24
  names it — and recall surfaces admitted rows only (specification 7.1). Both
25
  halves are database predicates, so they hold for a second write path as well
26
  as for the one this module offers, which is why the constraint tests insert
27
  around the changeset.
28
  """
29
  use OpenAgents.DataCase, async: true
30
31
  alias OpenAgents.Memories
32
  alias OpenAgents.Memories.{Admissions, Memory, Promotions}
33
34
  # The owner account is an operator by definition (`@owner_github_id`), so a
35
  # steward needs no configuration change and these tests stay async.
36
  @owner_github_id 14_167_547
37
38
  @stance "gateway-402-retired-model"
39
40
  defp account(key) do
41
    digest = :crypto.hash(:sha256, key)
42
    github_id = digest |> binary_part(0, 7) |> :binary.decode_unsigned()
43
44
    upsert(github_id, "promote-" <> (digest |> Base.encode16(case: :lower) |> binary_part(0, 12)))
45
  end
46
47
  defp steward, do: upsert(@owner_github_id, "AtlantisPleb")
48
49
  defp upsert(github_id, login) do
50
    {:ok, user} =
51
      OpenAgents.Accounts.upsert_github_user(%{
52
        github_id: github_id,
53
        github_login: login,
54
        github_avatar_url: "https://avatars.githubusercontent.com/u/#{github_id}?v=4"
55
      })
56
57
    user
58
  end
59
60
  defp evidence do
61
    [%{"kind" => "receipt", "ref" => "receipt:4f1c", "digest" => "sha256:9ab3"}]
62
  end
63
64
  defp stance_evidence do
65
    [
66
      %{
67
        "kind" => "url",
68
        "ref" => "https://openagents.com/OpenAgentsInc/openagents/kb/stances.json",
69
        "digest" => "sha256:7897da0a"
70
      }
71
    ]
72
  end
73
74
  defp candidate(overrides \\ %{}) do
75
    Map.merge(
76
      %{
77
        "bucket" => "system",
78
        "slug" => "sys:gateway-402-retired-model",
79
        "body" =>
80
          "A 402 from the inference gateway usually means the default model was " <>
81
            "retired upstream. Check gateway status before bisecting local lanes.",
82
        "entity" => "inference-gateway",
83
        "tier" => "ledger",
84
        "as_of" => ~D[2026-08-25],
85
        "admission" => "candidate",
86
        "evidence_refs" => evidence()
87
      },
88
      overrides
89
    )
90
  end
91
92
  # An admitted system claim: written by an author, admitted by a steward.
93
  defp admitted_claim(key, overrides \\ %{}) do
94
    author = account(key)
95
    {:ok, memory} = Memories.create(author, candidate(overrides))
96
97
    {:ok, _verdict} =
98
      Admissions.record(steward(), memory.id, %{
99
        "verdict" => "admitted",
100
        "ground" => "The receipt shows the retirement and the 402 together."
101
      })
102
103
    memory
104
  end
105
106
  defp promotion(overrides \\ %{}) do
107
    Map.merge(
108
      %{
109
        "stance" => @stance,
110
        "slug" => "sys:gateway-402-retired-model",
111
        "evidence_refs" => stance_evidence()
112
      },
113
      overrides
114
    )
115
  end
116
117
  # Specification 7.1's eligibility filter, written here rather than imported.
118
  #
119
  # Recall does not read the system bucket yet — the issue that builds the
120
  # ranking owns that — so these tests model the predicate the specification
121
  # states: a system memory reaches a note when it is live and its derived
122
  # status is `admitted`. Writing it out is the point. A test that asserted
123
  # "the tombstone is not admitted" without saying what that buys would pin a
124
  # fact rather than the rule the fact serves.
125
  defp eligible?(%Memory{} = memory) do
126
    reread = Repo.get!(Memory, memory.id)
127
128
    is_nil(reread.superseded_by_id) and Admissions.status(reread) == "admitted"
129
  end
130
131
  # A row assembled as a struct rather than through `changeset/2`. The second
132
  # route the constraints exist for.
133
  defp around_the_changeset(user, overrides) do
134
    fields =
135
      Map.merge(
136
        %{
137
          user_id: user.id,
138
          bucket: "system",
139
          body: "Written around the write path.",
140
          slug: "sys:around-the-write-path",
141
          tier: "ledger",
142
          as_of: ~D[2026-08-25],
143
          admission: "candidate",
144
          evidence_refs: evidence()
145
        },
146
        overrides
147
      )
148
149
    Repo.insert(struct(Memory, fields))
150
  end
151
152
  describe "a promotion tombstone" do
153
    test "supersedes the claim and names the stance it went to" do
154
      claim = admitted_claim("promote-shape")
155
156
      assert {:ok, tombstone} = Promotions.promote(steward(), claim.id, promotion())
157
158
      assert tombstone.bucket == "system"
159
      assert tombstone.stance == @stance
160
      assert tombstone.promoted == true
161
      assert tombstone.body =~ @stance
162
      assert tombstone.body == Promotions.body(@stance)
163
      assert tombstone.admission == "candidate"
164
      assert tombstone.tier == Promotions.tier()
165
      assert Memory.promoted?(tombstone)
166
167
      assert Repo.get!(Memory, claim.id).superseded_by_id == tombstone.id
168
    end
169
170
    test "a body that does not name its stance is refused at the table" do
171
      author = account("promote-body-names-stance")
172
173
      assert_raise Ecto.ConstraintError, ~r/memories_system_shape/, fn ->
174
        around_the_changeset(author, %{
175
          stance: @stance,
176
          body: "Promoted somewhere, but this body never says where."
177
        })
178
      end
179
    end
180
181
    test "a malformed stance id is refused" do
182
      claim = admitted_claim("promote-stance-format")
183
184
      for malformed <- ["Gateway-402", "gateway 402", "gateway--402", "-gateway", "kb/stance"] do
185
        assert {:error, changeset} =
186
                 Promotions.promote(steward(), claim.id, promotion(%{"stance" => malformed}))
187
188
        assert Map.has_key?(errors_on(changeset), :stance),
189
               "#{inspect(malformed)} was accepted as a stance id"
190
      end
191
192
      # A blank name is refused before the changeset, which would cast it to
193
      # absence and write an ordinary supersession under a promotion's body.
194
      assert {:error, :stance_required} =
195
               Promotions.promote(steward(), claim.id, promotion(%{"stance" => "   "}))
196
197
      assert Repo.get!(Memory, claim.id).superseded_by_id == nil
198
    end
199
200
    test "a stance belongs only to a system memory" do
201
      author = account("promote-account-scoped")
202
203
      assert {:error, changeset} =
204
               Memories.create(author, %{
205
                 "body" => "I prefer pnpm, and the stance is gateway-402-retired-model.",
206
                 "stance" => @stance
207
               })
208
209
      assert Map.has_key?(errors_on(changeset), :stance)
210
211
      assert_raise Ecto.ConstraintError, ~r/memories_system_shape/, fn ->
212
        around_the_changeset(author, %{
213
          bucket: "user",
214
          slug: nil,
215
          tier: nil,
216
          as_of: nil,
217
          admission: nil,
218
          evidence_refs: nil,
219
          stance: @stance
220
        })
221
      end
222
    end
223
  end
224
225
  describe "the knowledge base wins the claims it was promoted for" do
226
    # The whole rule, in one test: before promotion the memory rail speaks,
227
    # after promotion it does not, and nothing was deleted to make that true.
228
    test "a promoted claim leaves recall and the claim's chain is kept" do
229
      claim = admitted_claim("promote-drains")
230
231
      assert eligible?(claim), "an admitted live claim should reach recall"
232
233
      {:ok, tombstone} = Promotions.promote(steward(), claim.id, promotion())
234
235
      refute eligible?(claim), "a promoted claim must not reach recall"
236
      refute eligible?(tombstone), "a tombstone must not reach recall"
237
238
      # Superseded, not deleted: the claim, its evidence, and its admission
239
      # record are all still readable.
240
      assert %Memory{} = Repo.get!(Memory, claim.id)
241
      assert Admissions.status(claim) == "admitted"
242
      assert Admissions.list(claim) != []
243
    end
244
245
    # `:not_found` rather than a changeset error, because the composite foreign
246
    # key is what refuses this and the module reports a key violation as
247
    # absence: a steward who names a tombstone learns that it is not admissible,
248
    # and nothing more.
249
    test "nothing can admit a tombstone, so nothing can make one eligible" do
250
      claim = admitted_claim("promote-unadmittable")
251
      {:ok, tombstone} = Promotions.promote(steward(), claim.id, promotion())
252
253
      assert {:error, :not_found} =
254
               Admissions.record(steward(), tombstone.id, %{
255
                 "verdict" => "admitted",
256
                 "ground" => "Trying to admit the tombstone itself."
257
               })
258
259
      assert Admissions.status(tombstone) == "candidate"
260
      refute eligible?(tombstone)
261
    end
262
263
    test "a tombstone is a pointer rather than a claim, so it takes no records" do
264
      claim = admitted_claim("promote-no-records")
265
      {:ok, tombstone} = Promotions.promote(steward(), claim.id, promotion())
266
267
      assert {:error, :not_found} =
268
               Admissions.challenge(account("promote-challenger"), tombstone.id, %{
269
                 "ground" => "The stance is wrong.",
270
                 "evidence_refs" => evidence()
271
               })
272
273
      assert Admissions.list(tombstone) == []
274
    end
275
276
    # The record path is refused by the database, so a second write path is
277
    # refused too rather than merely unattempted by the code that exists today.
278
    test "a record naming a tombstone is refused around the changeset" do
279
      claim = admitted_claim("promote-around-the-record-path")
280
      {:ok, tombstone} = Promotions.promote(steward(), claim.id, promotion())
281
282
      assert_raise Ecto.ConstraintError, ~r/memory_admissions_promotion_fkey/, fn ->
283
        Repo.insert(
284
          struct(OpenAgents.Memories.Admission, %{
285
            memory_id: tombstone.id,
286
            memory_bucket: "system",
287
            memory_promoted: false,
288
            author_id: steward().id,
289
            slug: "adm:" <> tombstone.id,
290
            role: "admission",
291
            verdict: "admitted",
292
            ground: "Written around the record path.",
293
            inserted_at: DateTime.utc_now()
294
          })
295
        )
296
      end
297
    end
298
  end
299
300
  describe "a near miss is not a collision" do
301
    # The claim this store would call "the same claim" is the one a promotion
302
    # names. Shared vocabulary is not that, and neither is quoting the stance id
303
    # in prose — otherwise a memory *about* a stance would silence itself.
304
    test "a memory that shares the promoted claim's words stays eligible" do
305
      promoted = admitted_claim("promote-near-miss-target")
306
      {:ok, _tombstone} = Promotions.promote(steward(), promoted.id, promotion())
307
308
      neighbour =
309
        admitted_claim("promote-near-miss-neighbour", %{
310
          "slug" => "sys:gateway-402-quota-exhausted",
311
          "body" =>
312
            "A 402 from the inference gateway can also mean the account's " <>
313
              "gateway quota is exhausted. Check the balance before the model status."
314
        })
315
316
      assert neighbour.stance == nil
317
      refute Memory.promoted?(neighbour)
318
      assert eligible?(neighbour), "a distinct claim must survive its neighbour's promotion"
319
    end
320
321
    test "a memory that quotes the stance id is not thereby promoted" do
322
      promoted = admitted_claim("promote-near-miss-quote-target")
323
      {:ok, _tombstone} = Promotions.promote(steward(), promoted.id, promotion())
324
325
      quoting =
326
        admitted_claim("promote-near-miss-quote", %{
327
          "slug" => "sys:kb-corpus-rebuild",
328
          "body" =>
329
            "Editing the #{@stance} stance needs `node build-kb.mjs` and a " <>
330
              "rebuild before the digest is re-pinned; the corpus is compiled, not read."
331
        })
332
333
      assert quoting.body =~ @stance
334
      assert quoting.stance == nil
335
      assert eligible?(quoting), "quoting a stance id is not a promotion"
336
    end
337
338
    # Same claim, same slug, one promotion: the tombstone drains the row it
339
    # names and nothing else. Draining by slug would take a cross-account read
340
    # of `memories`, which MEMORY-010 keeps out of this store.
341
    test "promotion drains the row it names, not every row sharing its slug" do
342
      promoted = admitted_claim("promote-slug-a")
343
      sibling = admitted_claim("promote-slug-b", %{"as_of" => ~D[2026-08-24]})
344
345
      assert promoted.slug == sibling.slug
346
347
      {:ok, _tombstone} = Promotions.promote(steward(), promoted.id, promotion())
348
349
      refute eligible?(promoted)
350
      assert eligible?(sibling), "only the named row is drained"
351
    end
352
  end
353
354
  describe "who promotes" do
355
    test "only a steward" do
356
      author = account("promote-author")
357
      {:ok, memory} = Memories.create(author, candidate())
358
359
      assert {:error, :steward_required} = Promotions.promote(author, memory.id, promotion())
360
      assert Repo.get!(Memory, memory.id).superseded_by_id == nil
361
    end
362
363
    test "a promotion names a stance or it is not a promotion" do
364
      claim = admitted_claim("promote-stance-required")
365
366
      assert {:error, :stance_required} =
367
               Promotions.promote(steward(), claim.id, Map.delete(promotion(), "stance"))
368
369
      assert Repo.get!(Memory, claim.id).superseded_by_id == nil
370
    end
371
372
    test "a target that is not a live system memory is refused without a read" do
373
      claim = admitted_claim("promote-twice")
374
      {:ok, _tombstone} = Promotions.promote(steward(), claim.id, promotion())
375
376
      assert {:error, :not_supersedable} = Promotions.promote(steward(), claim.id, promotion())
377
378
      assert {:error, :not_supersedable} =
379
               Promotions.promote(steward(), Ecto.UUID.generate(), promotion())
380
381
      assert {:error, :not_supersedable} =
382
               Promotions.promote(steward(), "not-a-uuid", promotion())
383
    end
384
385
    # A promotion is a steward's correction on the target's slug, and the
386
    # specification names that as the second resolution path for a challenge.
387
    test "promoting resolves the open challenges on the claim" do
388
      claim = admitted_claim("promote-resolves-challenges")
389
390
      {:ok, _challenge} =
391
        Admissions.challenge(account("promote-challenger-two"), claim.id, %{
392
          "ground" => "The 402 is a quota error, not a retirement.",
393
          "evidence_refs" => evidence()
394
        })
395
396
      assert Admissions.status(claim) == "suspended"
397
398
      assert {:ok, _tombstone} = Promotions.promote(steward(), claim.id, promotion())
399
400
      assert Admissions.status(claim) == "admitted"
401
      assert Enum.any?(Admissions.list(claim), &(&1.role == "refutation"))
402
    end
403
  end
404
end
test/openagents/memories_test.exs modified +7 -1

@@ -328,7 +328,13 @@ defmodule OpenAgents.MemoriesTest do

328 328
      # out, so a caller with no standing learns nothing from the refusal.
329 329
      "lib/openagents/memories/admissions.ex",
330 330
      # The system bucket's read path, which names no account at all.
331
      "lib/openagents/memories/system_recall.ex"
331
      "lib/openagents/memories/system_recall.ex",
332
      # Promotion holds no query of its own — it composes a tombstone and hands
333
      # it to `supersede/3` — and it is named here so that a lookup added
334
      # beside it, by slug or by stance, has to carry the account boundary or
335
      # fail. Draining a claim by slug would be exactly such a lookup
336
      # (MEMORY-012).
337
      "lib/openagents/memories/promotions.ex"
332 338
    ]
333 339
334 340
    # MEMORY-001's amendment, written as a budget. Two queries in the plane

This page updates live while a promote is in flight · changelog