`test/openagents_web/controllers/memory_controller_test.exs`, and

`test/openagents_web/controllers/responses_controller_test.exs`.

### MEMORY-011 — A system memory is evidenced, admitted by receipt, and surfaced to nobody

Status: Current

The `system` bucket of `OpenAgents.Memories` holds what the network as a whole

has learned rather than what one account asked to have remembered. A wrong

`user` memory misleads one session; a wrong `system` memory would reach every

session, so the row carries what makes a claim answerable and the store refuses

one that does not.

A system row carries fields the other two buckets carry none of: a `sys:` slug,

a transparency `tier` of `ledger` or `glass` and never lower, an `as_of` date

distinct from the insert time, an `admission` the author claims, and a

non-empty `evidence_refs` list whose every entry names a `kind` of `receipt`,

`memory`, or `url`, a `ref`, and a digest. Both halves are the

`memories_system_shape` constraint as well as the changeset: an evidence-free

or sub-`ledger` candidate is unrepresentable at the table, not merely unwritten

by the code that exists today, and a `user` or `learned` row carrying a tier is

refused the same way. Every column is asserted `IS NOT NULL` before it is

compared, because a check constraint passes when it evaluates to NULL and an

absent evidence list is exactly the hole the constraint exists to close.

Admission is a receipt, never a field. A verdict is a row in

`memory_admissions` — attributed to the steward who wrote it, dated, and

append-only, with no `updated_at` and no path that updates one — and

`OpenAgents.Memories.Admissions.status/1` derives the effective status from

those records. The `admission` column on the candidate is the author's claim

and nothing more, so a row that says `admitted` with no steward record behind

it reads as a candidate. A steward who changes their mind writes a second

record; both stay readable.

Only a steward admits, and the steward set is ADMIN-001's operator allowlist,

bootstrapped to the owner's account. Only the original author or a steward

writes a superseding row on a system slug; anyone else is refused, and

supersession is the only correction path there is.

Neither path reads another account's memory. An admission record proves its

candidate is a system row through the composite foreign key

`(memory_id, memory_bucket) -> memories (id, bucket)` rather than through a

lookup, and a correction authorizes inside the `UPDATE` predicate — the actor's

own `user_id` or the steward role — so a caller with no standing is refused

without learning that the row exists. MEMORY-010's rule that every query rooted

at `OpenAgents.Memories.Memory` names `user_id` holds unchanged, and the AST

proof reads this module too.

Nothing surfaces. `OpenAgents.Memories.recall/3` reads the `user` and `learned`

buckets, named as a predicate in the query rather than filtered out of its

result, and no session sees a system memory — not another account's, and not

its own author's. That is deliberate and it is the whole reason this invariant

can stand beside MEMORY-001 and MEMORY-010 rather than amending them: an

admitted row read into every account's turn is cross-account recall by

construction, so surfacing the bucket is a privacy decision with an eligibility

filter of its own, not a ranking change. A system memory that is stored,

evidenced, and admitted but recalled by nobody is a coherent state; a quietly

widened recall predicate is not.

Evidence: `OpenAgents.Memories.Admissions`, `OpenAgents.Memories.Admission`,

`OpenAgents.Memories.Memory`, the `memories_system_shape` and

`memory_admissions_shape` constraints and the composite foreign key

`memory_admissions_memory_fkey`, and

`test/openagents/memories/system_memory_test.exs`.

### PRIVACY-001 — Secret-bearing profile memory is rejected, never scrub-stored

Status: Current

  `OpenAgents.ProfileMemory.forget_active/2`, which supersedes rather than

  deletes, so a retraction is another entry in the audit trail the same

  surface renders and never a row that quietly stops existing.

- Admitting or rejecting a candidate system memory, and correcting an admitted

  one, through `OpenAgents.Memories.Admissions` (MEMORY-011). Only a steward

  admits, and the steward set is this allowlist, bootstrapped to the owner's

  account. The write appends: an admission record is inserted and never

  updated, and a correction supersedes rather than edits, so a reversal is a

  second record beside the first rather than a verdict that quietly changed.

  The authority buys no read — the operator sees no memory of another account

  through this module.

- Recording Gym runs and trials under `POST /api/v1/gym/runs`, the lifecycle

  routes `POST /api/v1/gym/runs/start`, `POST /api/v1/gym/runs/:id/trials`,

  and `PATCH /api/v1/gym/runs/:id`

| MEMORY-008 | `test/openagents/experience_memory_test.exs` |

| MEMORY-009 | `test/openagents/graph_memory_test.exs` |

| MEMORY-010 | `test/openagents/memories_test.exs`, `test/openagents_web/controllers/memory_controller_test.exs`, `test/openagents_web/controllers/responses_controller_test.exs` |

| MEMORY-011 | `test/openagents/memories/system_memory_test.exs` |

| PRIVACY-001 | `test/openagents/memory/policy_and_redaction_test.exs`, `test/openagents/memory/scope_boundary_test.exs` |

| TURN-001 | `test/openagents/conversations_test.exs` |

| TURN-002 | `test/openagents/conversations_test.exs` |

**Memories** — `OpenAgents.Memories`, and a different thing from the memory

planes above. A memory is account-scoped (`memories.user_id`, not a visitor),

thread-sourced, and authoritative rather than derived: nothing can rebuild it,

because the sentence a reader asked to have remembered is the only copy. Two

buckets, `user` and `learned`. Recall runs server-side inside `POST

because the sentence a reader asked to have remembered is the only copy. Three

buckets, `user`, `learned`, and `system`. Recall runs server-side inside `POST

/api/v1/responses`, so every client gets it without implementing retrieval.

Use "memories" for this store and "memory planes" for the projections; a

durable fact a reader states in the web conversation is still profile memory.

**System memory** — the `system` bucket of that store: what the network as a

whole has learned, rather than what one account asked to have remembered. Say

"system memory" for a row and "admission record" for the steward's verdict on

it, and keep the two apart — a candidate's status is derived from the records,

never read from the candidate. Recall does not read this bucket: a system

memory is stored, evidenced, and admitted, and surfaced to no session yet

(MEMORY-011).

### Threads

A **thread** is the unit of agent work. Everything in this section describes a

  ## What it holds

  Two buckets, described on `OpenAgents.Memories.Memory`. `user` memories are

  Three buckets, described on `OpenAgents.Memories.Memory`. `user` memories are

  explicit: a reader said "remember that I prefer X" and something called

  `create/2`. Nothing here infers a memory from what a turn contained, and

  nothing should — a store that fills itself is a store nobody trusts.

  `learned` memories come from server-side consolidation over thread events.

  `system` memories are what the network as a whole has learned, and they pass

  an evidence-backed admission gate before they mean anything

  (`OpenAgents.Memories.Admissions`). `recall/3` reads the first two only.

  ## Corrections supersede

          | {:error, :supersedes_not_found}

  def create(%User{} = user, attrs) when is_map(attrs) do

    attrs = normalize(attrs)

    body = Map.get(attrs, "body")

    embedding =

      case body do

        text when is_binary(text) and text != "" -> Semantic.embedding_for(text)

        _absent -> nil

      end

    attrs =

      case embedding do

        {vector, model} -> Map.merge(attrs, %{"embedding" => vector, "embedding_model" => model})

        nil -> attrs

      end

    changeset = Memory.changeset(%Memory{user_id: user.id}, attrs)

    changeset = build(user, attrs)

    Multi.new()

    |> Multi.run(:supersedes, fn _repo, _changes -> superseded(user, attrs) end)

    end

  end

  @doc """

  The changeset one write of `attrs` for `user` produces, embedding included.

  `create/2` is the ordinary way in. This is here for a caller that has to

  write a memory inside a transaction of its own —

  `OpenAgents.Memories.Admissions.supersede/3` corrects a system claim and

  points the old row at the new one, and a nested `Repo.transaction/1` would

  make its refusal path roll back more than it meant to.

  The owner is set on the struct and never cast, here as in `create/2`.

  """

  @spec build(User.t(), map()) :: Ecto.Changeset.t()

  def build(%User{} = user, attrs) when is_map(attrs) do

    attrs = normalize(attrs)

    embedding =

      case Map.get(attrs, "body") do

        text when is_binary(text) and text != "" -> Semantic.embedding_for(text)

        _absent -> nil

      end

    attrs =

      case embedding do

        {vector, model} -> Map.merge(attrs, %{"embedding" => vector, "embedding_model" => model})

        nil -> attrs

      end

    Memory.changeset(%Memory{user_id: user.id}, attrs)

  end

  @doc """

  The account's memories, newest first.

  Live only unless `include_superseded: true`. Options: `bucket` to narrow to

  one bucket, and `limit`, capped at #{@maximum_listed}.

  one bucket, `buckets` to narrow to several, and `limit`, capped at

  #{@maximum_listed}. Every one of them is a predicate in the query rather than

  a filter over its result.

  """

  @spec list(User.t(), keyword()) :: [Memory.t()]

  def list(%User{} = user, opts \\ []) do

  @doc """

  What this turn should be told, bounded.

  `query` is the incoming input. Every live memory the account holds is ranked

  against it; `user` memories are kept regardless of score and `learned` ones

  only above the backend's floor; the result is cut to `maximum_attached`

  memories and `maximum_attached_characters`, and what the cut excluded is

  counted rather than dropped in silence.

  `query` is the incoming input. Every live `user` or `learned` memory the

  account holds is ranked against it; `user` memories are kept regardless of

  score and `learned` ones only above the backend's floor; the result is cut to

  `maximum_attached` memories and `maximum_attached_characters`, and what the

  cut excluded is counted rather than dropped in silence.

  The `system` bucket is not read here, by anyone, including its own author.

  A system memory is stored and admitted (`OpenAgents.Memories.Admissions`) and

  surfaced to nobody: an admitted row reaches every account's turn or none, and

  the first is cross-account recall, which MEMORY-001 and MEMORY-010 forbid.

  The bucket list is a predicate in the query rather than a filter applied to

  its results, so widening it is a deliberate edit to the recall issue's

  eligibility filter and not something a ranking change can do by accident.

  Never raises. An unreadable store or an unavailable backend recalls nothing.

  """

  def recall(user, query, opts \\ [])

  def recall(%User{} = user, query, opts) when is_binary(query) and query != "" do

    candidates = list(user, limit: maximum_live_memories())

    candidates =

      list(user, limit: maximum_live_memories(), buckets: Memory.recallable_buckets())

    {backend, ranked, floor} = Retrieval.rank(user.id, query, candidates)

    eligible =

        where(query, [memory], is_nil(memory.superseded_by_id))

      end

    query =

      case Keyword.get(opts, :buckets) do

        [_first | _rest] = buckets -> where(query, [m], m.bucket in ^buckets)

        _all -> query

      end

    case Keyword.get(opts, :bucket) do

      bucket when bucket in ["user", "learned"] -> where(query, [m], m.bucket == ^bucket)

      _all -> query

      bucket when is_binary(bucket) ->

        if bucket in Memory.buckets(), do: where(query, [m], m.bucket == ^bucket), else: query

      _all ->

        query

    end

  end

defmodule OpenAgents.Memories.Admission do

  @moduledoc """

  One verdict on one candidate system memory, and the ground for it.

  Admission is a receipt, not an assertion. The registry does not turn a

  promise green because someone said so, and the memory store does not admit a

  claim because its author wrote `admitted` on it: the record here is what a

  status is derived from, and the `admission` field on the candidate is only

  what the author claimed.

  The record is append-only. There is no `updated_at` and nothing updates one,

  so a steward who changes their mind writes a second record rather than

  editing the first, and both stay readable.

  `memory_bucket` rides the row so the composite foreign key

  `(memory_id, memory_bucket) -> memories (id, bucket)` can pin the candidate

  to the `system` bucket. That is what lets the write path refuse an admission

  record naming a `user` or `learned` row without reading `memories` at all,

  which is how the admission path stays clear of the account boundary

  MEMORY-010 draws.

  `role` names the three record roles the specification adds. Only `admission`

  is written today; `challenge` and `refutation` are the same enum and land

  with the issue that owns them.

  """

  use Ecto.Schema

  import Ecto.Changeset

  alias OpenAgents.Accounts.User

  alias OpenAgents.Memories.Memory

  @primary_key {:id, :binary_id, autogenerate: true}

  @foreign_key_type :binary_id

  @timestamps_opts [type: :utc_datetime_usec]

  @roles ~w(admission challenge refutation)

  @verdicts ~w(admitted rejected)

  @ground_characters 2_000

  schema "memory_admissions" do

    belongs_to :memory, Memory

    field :memory_bucket, :string, default: "system"

    belongs_to :steward, User

    field :slug, :string

    field :role, :string, default: "admission"

    field :verdict, :string

    field :ground, :string

    timestamps(updated_at: false)

  end

  @type t :: %__MODULE__{}

  @doc "The record roles the memory store recognises."

  @spec roles() :: [String.t()]

  def roles, do: @roles

  @doc "The verdicts an admission record may carry."

  @spec verdicts() :: [String.t()]

  def verdicts, do: @verdicts

  @doc "The longest ground the store accepts, in characters."

  @spec ground_characters() :: pos_integer()

  def ground_characters, do: @ground_characters

  @doc """

  Validates one admission record.

  The steward and the candidate are set on the struct rather than cast, so a

  request body can name neither who admitted nor, by extension, on whose

  authority.

  """

  @spec changeset(t(), map()) :: Ecto.Changeset.t()

  def changeset(record, attrs) do

    record

    |> cast(attrs, [:verdict, :ground])

    |> update_change(:ground, &trim/1)

    |> put_change(:role, "admission")

    |> put_change(:memory_bucket, "system")

    |> put_slug()

    |> validate_required([:memory_id, :steward_id, :slug, :role, :verdict, :ground])

    |> validate_inclusion(:role, @roles)

    |> validate_inclusion(:verdict, @verdicts)

    |> validate_length(:ground, min: 1, max: @ground_characters, count: :graphemes)

    |> foreign_key_constraint(:steward_id)

    # The composite key, named as the database names it. A record for a `user`

    # or `learned` row fails here rather than in a read that had to cross an

    # account to check.

    |> foreign_key_constraint(:memory_id,

      name: :memory_admissions_memory_fkey,

      message: "names no system memory"

    )

    |> check_constraint(:verdict, name: :memory_admissions_shape)

  end

  defp put_slug(changeset) do

    case get_field(changeset, :memory_id) do

      id when is_binary(id) -> put_change(changeset, :slug, "adm:" <> id)

      _absent -> changeset

    end

  end

  defp trim(value) when is_binary(value), do: String.trim(value)

  defp trim(value), do: value

end

defmodule OpenAgents.Memories.Admissions do

  @moduledoc """

  The gate in front of the system bucket: anyone can propose, only evidence

  admits, and only a steward writes the receipt.

  ## Why admission is a record rather than a field

  A wrong `user` memory misleads one session. A wrong `system` memory would

  reach every session on the network, so the store treats a status the way the

  promise registry treats a green promise: it is derived from a receipt

  somebody is answerable for, never read from a flag the claimant set. An

  author who writes `admission: "admitted"` on their own row has claimed

  something, and `status/1` still answers `"candidate"` until a steward records

  a verdict.

  ## Who admits

  A steward, and nobody else. The specification called for a published, signed

  allowlist of pubkeys evaluated as of an event's timestamp; on this substrate

  a single trusted server holds the role and checks it where the row is

  created, so what survives is the rule the machinery existed to enforce.

  The role is `OpenAgents.Accounts.admin?/1` — the operator allowlist of

  immutable GitHub numeric IDs, bootstrapped to the owner's account. That is

  the honest reading of "accounts the operator has marked as stewards,

  bootstrapped to the operator's own account", and it is the only account-level

  authority this server has: there is no `role` column on `users`, and the one

  per-account grant table in the repository grants roles on a repository rather

  than on the network. Broadening the steward set later touches the role

  assignment, not these record shapes. ADMIN-001 enumerates this module as an

  operator gate.

  ## What this module deliberately does not do

  It does not surface anything. An admitted system memory is stored, derived,

  and read by nobody: `OpenAgents.Memories.recall/3` reads the `user` and

  `learned` buckets only, and MEMORY-001 and MEMORY-010 confine recall to the

  acting account with no unscoped fallback. Reading an admitted row into every

  account's turn is cross-account recall by construction, so it is a privacy

  decision that belongs to the recall issue rather than a ranking detail this

  one can settle.

  It also never reads another account's memory. The composite foreign key

  `(memory_id, memory_bucket)` is what proves a candidate is a `system` row, so

  `record/3` writes without a lookup, and `supersede/3` authorizes inside the

  `UPDATE` predicate and learns nothing from a refusal.

  """

  import Ecto.Query

  alias Ecto.Multi

  alias OpenAgents.Accounts

  alias OpenAgents.Accounts.User

  alias OpenAgents.Memories

  alias OpenAgents.Memories.{Admission, Memory}

  alias OpenAgents.Repo

  @doc """

  Whether `user` may admit.

  A steward is an operator account. `admin?/1` refuses a banned account and

  reads the GitHub numeric ID rather than the login, so a renamed account keeps

  its authority and a transferred login does not inherit it.

  """

  @spec steward?(User.t() | nil) :: boolean()

  def steward?(user), do: Accounts.admin?(user)

  @doc """

  Writes one admission record against a candidate system memory.

  Attributes: `verdict` (`admitted` or `rejected`) and `ground` (why). The

  steward and the candidate are set on the struct, so a request body can name

  neither.

  Refuses `:steward_required` for an account without the role, and

  `:not_found` when `memory_id` does not name a system memory — the composite

  foreign key decides that, so a caller learns nothing about a row in another

  bucket beyond the fact that it is not admissible.

  """

  @spec record(User.t(), String.t(), map()) ::

          {:ok, Admission.t()}

          | {:error, Ecto.Changeset.t()}

          | {:error, :steward_required}

          | {:error, :not_found}

  def record(%User{} = steward, memory_id, attrs) when is_map(attrs) do

    if steward?(steward) do

      with {:ok, id} <- cast_id(memory_id) do

        %Admission{memory_id: id, steward_id: steward.id}

        |> Admission.changeset(normalize(attrs))

        |> Repo.insert()

        |> case do

          {:ok, admission} -> {:ok, admission}

          {:error, changeset} -> refusal(changeset)

        end

      end

    else

      {:error, :steward_required}

    end

  end

  @doc """

  A candidate's effective admission status.

  Derived from the admission records that reference it, newest verdict first,

  and never from the candidate's own `admission` field. A memory with no

  admission record behind it is a `candidate`, whatever it says about itself.

  Answers `nil` for a memory outside the system bucket, which has no admission

  status to have.

  """

  @spec status(Memory.t() | String.t()) :: String.t() | nil

  def status(%Memory{bucket: "system", id: id}), do: status(id)

  def status(%Memory{}), do: nil

  def status(memory_id) when is_binary(memory_id) do

    case cast_id(memory_id) do

      {:ok, id} -> Repo.one(latest(id)) || "candidate"

      {:error, :not_found} -> nil

    end

  end

  @doc """

  Every admission record against one candidate, oldest first.

  Append-only, so this is the whole history rather than the current state, and

  `status/1` is what reads a state out of it.

  """

  @spec list(Memory.t() | String.t()) :: [Admission.t()]

  def list(%Memory{id: id}), do: list(id)

  def list(memory_id) when is_binary(memory_id) do

    case cast_id(memory_id) do

      {:ok, id} ->

        Repo.all(

          from(record in Admission,

            where: record.memory_id == ^id,

            order_by: [asc: record.inserted_at, asc: record.id]

          )

        )

      {:error, :not_found} ->

        []

    end

  end

  @doc """

  Corrects a system memory by writing a replacement and pointing the old row at

  it.

  Only the original author or a steward may correct a system slug. Anyone else

  who disagrees files a challenge; the store has no path for editing somebody

  else's claim, and supersession is the only correction path there is.

  `attrs` describe the replacement, which is written under `user`'s account

  through the ordinary write path — same evidence requirement, same tier floor,

  same constraint. Name the target's slug on it: the slug is what binds the

  correction to the claim it corrects. A correction is admitted at the account

  ceiling, as `OpenAgents.Memories.create/2` admits one, because it replaces a

  live row with a live row.

  The authorization is the `UPDATE` predicate rather than a read followed by a

  decision. A caller with no standing gets `:not_supersedable` and learns

  nothing about the row, including whether it exists.

  """

  @spec supersede(User.t(), String.t(), map()) ::

          {:ok, Memory.t()}

          | {:error, Ecto.Changeset.t()}

          | {:error, :not_supersedable}

  def supersede(%User{} = user, target_id, attrs) when is_map(attrs) do

    with {:ok, id} <- cast_target(target_id) do

      replacement = Memories.build(user, Map.put(normalize(attrs), "bucket", "system"))

      Multi.new()

      |> Multi.insert(:replacement, replacement)

      |> Multi.run(:target, fn repo, %{replacement: written} ->

        updates = [superseded_by_id: written.id, updated_at: DateTime.utc_now()]

        case repo.update_all(correctable(user, id), set: updates) do

          {1, _rows} -> {:ok, written}

          {0, _rows} -> {:error, :not_supersedable}

        end

      end)

      |> Repo.transaction()

      |> case do

        {:ok, %{replacement: written}} -> {:ok, written}

        {:error, :replacement, changeset, _changes} -> {:error, changeset}

        {:error, _step, reason, _changes} -> {:error, reason}

      end

    end

  end

  # ── internal ───────────────────────────────────────────────────────────────

  # MEMORY-010: the account boundary is a predicate in the query. Here it is

  # one half of the authorization — the row is the actor's own — and the other

  # half is the steward role, which lives in the operator allowlist rather than

  # in a column and so arrives as a bound boolean. A caller who is neither

  # matches no row, so the update reports zero and nothing is read out.

  defp correctable(%User{id: user_id} = user, target_id) do

    steward = steward?(user)

    from(memory in Memory,

      where: memory.id == ^target_id,

      where: memory.bucket == "system",

      where: is_nil(memory.superseded_by_id),

      where: memory.user_id == ^user_id or type(^steward, :boolean)

    )

  end

  defp latest(memory_id) do

    from(record in Admission,

      where: record.memory_id == ^memory_id,

      where: record.role == "admission",

      order_by: [desc: record.inserted_at, desc: record.id],

      limit: 1,

      select: record.verdict

    )

  end

  # The composite foreign key is what refuses a candidate outside the system

  # bucket, so its violation is reported as absence rather than as a changeset

  # error about a column the caller never named.

  defp refusal(changeset) do

    if Enum.any?(changeset.errors, fn {field, _error} -> field == :memory_id end) do

      {:error, :not_found}

    else

      {:error, changeset}

    end

  end

  defp cast_id(value) when is_binary(value) do

    case Ecto.UUID.cast(value) do

      {:ok, id} -> {:ok, id}

      :error -> {:error, :not_found}

    end

  end

  defp cast_id(_value), do: {:error, :not_found}

  defp cast_target(value) do

    case cast_id(value) do

      {:ok, id} -> {:ok, id}

      {:error, :not_found} -> {:error, :not_supersedable}

    end

  end

  defp normalize(attrs) do

    Map.new(attrs, fn {key, value} -> {to_string(key), value} end)

  end

end

  a memory and nothing can rebuild it, because the sentence a reader typed once

  is the only copy.

  Two buckets, kept distinct because they earn attention differently:

  Three buckets, kept distinct because they earn attention differently:

  * `user` — the reader said "remember that I prefer X". Explicit only, never

    inferred from what a turn happened to contain.

  * `learned` — server-side consolidation over thread events produced it. It

    carries `source_ref` so a wrong learning is traced back to the work that

    taught it.

  * `system` — what the network as a whole has learned. See

    `OpenAgents.Memories.Admissions`.

  A `system` row carries fields the other two do not, and they are required

  together: a `sys:` slug, a transparency `tier` of `ledger` or `glass`, an

  `as_of` date for the claim, an `admission` the author claims, and a non-empty

  `evidence_refs` list. A `user` or `learned` row carries none of them. Both

  halves of that rule are a database constraint (`memories_system_shape`) as

  well as a validation here, so an evidence-free candidate is unrepresentable

  rather than merely unwritten by the code that exists today.

  `admission` is the author's claim and nothing more. Effective status comes

  from `OpenAgents.Memories.Admissions.status/1`, which reads the admission

  records, so a row that says `admitted` with no steward record behind it still

  reads as a candidate.

  `superseded_by_id` is how a correction lands. The replacement is a new row

  and the old row points at it, so the store keeps the chain rather than

  @foreign_key_type :binary_id

  @timestamps_opts [type: :utc_datetime_usec]

  @buckets ~w(user learned)

  @buckets ~w(user learned system)

  @recallable_buckets ~w(user learned)

  @default_bucket "user"

  @body_characters 2_000

  @source_ref_characters 200

  @slug_characters 200

  @slug_prefix "sys:"

  @tiers ~w(ledger glass)

  @admissions ~w(candidate admitted rejected)

  @evidence_kinds ~w(receipt memory url)

  @evidence_refs 20

  schema "memories" do

    belongs_to :user, User

    field :source_ref, :string

    field :embedding, {:array, :float}

    field :embedding_model, :string

    # The system bucket's fields. Null on every other row.

    field :slug, :string

    field :entity, :string

    field :tier, :string

    field :as_of, :date

    field :admission, :string

    field :evidence_refs, {:array, :map}

    # The generated `tsvector` the lexical stand-in ranks over. PostgreSQL

    # writes it; nothing here reads it back, so it never rides a select.

    field :search_vector, :string, load_in_query: false

  @spec buckets() :: [String.t()]

  def buckets, do: @buckets

  @doc """

  The buckets recall reads.

  `system` is stored and admitted but surfaced to nobody. Reading an admitted

  system row into every account's turn is cross-account recall by construction,

  which MEMORY-001 and MEMORY-010 forbid, so widening this list is a privacy

  decision rather than a ranking change. It belongs to the recall issue that

  owns the eligibility filter, not to the store.

  """

  @spec recallable_buckets() :: [String.t()]

  def recallable_buckets, do: @recallable_buckets

  @doc "The transparency tiers a system memory may carry."

  @spec tiers() :: [String.t()]

  def tiers, do: @tiers

  @doc "The admission states an author may claim."

  @spec admissions() :: [String.t()]

  def admissions, do: @admissions

  @doc "The kinds of evidence a system memory may cite."

  @spec evidence_kinds() :: [String.t()]

  def evidence_kinds, do: @evidence_kinds

  @doc "The prefix every system slug carries."

  @spec slug_prefix() :: String.t()

  def slug_prefix, do: @slug_prefix

  @doc "The bucket a write lands in when it names none."

  @spec default_bucket() :: String.t()

  def default_bucket, do: @default_bucket

  @spec changeset(t(), map()) :: Ecto.Changeset.t()

  def changeset(memory, attrs) do

    memory

    |> cast(attrs, [:bucket, :body, :source_ref, :embedding, :embedding_model])

    |> cast(attrs, [

      :bucket,

      :body,

      :source_ref,

      :embedding,

      :embedding_model,

      :slug,

      :entity,

      :tier,

      :as_of,

      :admission,

      :evidence_refs

    ])

    |> update_change(:body, &trim/1)

    |> update_change(:source_ref, &trim/1)

    |> update_change(:slug, &trim/1)

    |> update_change(:entity, &trim/1)

    |> validate_required([:bucket, :body])

    |> validate_inclusion(:bucket, @buckets)

    |> validate_length(:body, min: 1, max: @body_characters, count: :graphemes)

    |> validate_length(:source_ref, min: 1, max: @source_ref_characters, count: :graphemes)

    |> validate_bucket_fields()

    |> foreign_key_constraint(:user_id)

    |> check_constraint(:body, name: :memories_shape)

    |> check_constraint(:evidence_refs,

      name: :memories_system_shape,

      message: "does not satisfy the system-memory shape"

    )

  end

  # The system fields, required together on a system row and refused outright

  # on the other two. The database says the same thing in

  # `memories_system_shape`; this is the half that can explain itself to the

  # caller.

  defp validate_bucket_fields(changeset) do

    case get_field(changeset, :bucket) do

      "system" -> validate_system(changeset)

      _account_scoped -> refuse_system_fields(changeset)

    end

  end

  defp validate_system(changeset) do

    changeset

    |> validate_required([:slug, :tier, :as_of, :admission, :evidence_refs])

    |> validate_length(:slug, min: 1, max: @slug_characters, count: :graphemes)

    |> validate_format(:slug, ~r/^sys:/, message: "must start with #{@slug_prefix}")

    |> validate_length(:entity, min: 1, max: @slug_characters, count: :graphemes)

    |> validate_inclusion(:tier, @tiers)

    |> validate_inclusion(:admission, @admissions)

    |> validate_evidence_refs()

  end

  defp refuse_system_fields(changeset) do

    Enum.reduce([:slug, :entity, :tier, :as_of, :admission, :evidence_refs], changeset, fn

      field, acc ->

        if is_nil(get_field(acc, field)) do

          acc

        else

          add_error(acc, field, "belongs only to a system memory")

        end

    end)

  end

  # A system memory without evidence is an assertion, and assertions do not

  # enter the shared bucket. The list is required, non-empty, and every entry

  # names a kind, a ref, and a digest — the digest so the evidence behind an

  # admitted claim cannot be swapped afterwards.

  defp validate_evidence_refs(changeset) do

    case get_change(changeset, :evidence_refs, get_field(changeset, :evidence_refs)) do

      nil ->

        changeset

      [] ->

        add_error(changeset, :evidence_refs, "must name at least one piece of evidence")

      refs when is_list(refs) and length(refs) > @evidence_refs ->

        add_error(

          changeset,

          :evidence_refs,

          "names more than #{@evidence_refs} pieces of evidence"

        )

      refs when is_list(refs) ->

        if Enum.all?(refs, &evidence_ref?/1) do

          put_change(changeset, :evidence_refs, Enum.map(refs, &normalize_ref/1))

        else

          add_error(

            changeset,

            :evidence_refs,

            "each entry needs a kind of #{Enum.join(@evidence_kinds, ", ")}, a ref, and a digest"

          )

        end

      _not_a_list ->

        add_error(changeset, :evidence_refs, "must be a list")

    end

  end

  defp evidence_ref?(ref) when is_map(ref) do

    kind(ref) in @evidence_kinds and present?(entry(ref, "ref", :ref)) and

      present?(entry(ref, "digest", :digest))

  end

  defp evidence_ref?(_ref), do: false

  defp normalize_ref(ref) do

    %{

      "kind" => kind(ref),

      "ref" => String.trim(entry(ref, "ref", :ref)),

      "digest" => String.trim(entry(ref, "digest", :digest))

    }

  end

  defp kind(ref), do: entry(ref, "kind", :kind)

  # A caller writes `%{"kind" => …}` over the API and `%{kind: …}` in Elixir,

  # and both mean the same evidence ref.

  defp entry(ref, string_key, atom_key) do

    case Map.get(ref, string_key, Map.get(ref, atom_key)) do

      value when is_binary(value) -> value

      _absent -> nil

    end

  end

  defp present?(value) when is_binary(value), do: String.trim(value) != ""

  defp present?(_value), do: false

  @doc "Points a memory at the memory that replaced it."

  @spec supersede_changeset(t(), t()) :: Ecto.Changeset.t()

  def supersede_changeset(memory, replacement) do

    20260825160000,

    20260825170000,

    20260825170100,

    20260825195623

    20260825195623,

    20260825220000

  ],

  "required_tables": [

    "users",

defmodule OpenAgents.Repo.Migrations.CreateSystemMemories do

  use Ecto.Migration

  # The third memory bucket: what the network as a whole has learned, rather

  # than what one account asked to have remembered.

  #

  # A wrong `user` memory misleads one session. A wrong `system` memory would

  # reach every session, so the row carries the things that make a claim

  # answerable — who wrote it, when it was observed true, and what evidence

  # stands behind it — and the table refuses a row that carries none of them.

  #

  # Two decisions are load-bearing here, and both are database predicates

  # rather than changeset validations (MEMORY-004):

  #

  #   * A system candidate with an empty evidence list cannot exist. The write

  #     path refuses it too, but a constraint is what makes it unrepresentable

  #     rather than merely unwritten by the code that exists today.

  #

  #   * An admission record can only name a `system` row. The composite foreign

  #     key on `(id, bucket)` enforces that without any read of `memories`, so

  #     the admission path never issues a query across the account boundary

  #     MEMORY-010 draws.

  #

  # Rows in the `user` and `learned` buckets carry none of the system columns,

  # and the constraint says so in both directions.

  def up do

    alter table(:memories) do

      # `sys:` prefixed. The prefix is a routing convention, not a boundary —

      # the boundary is the admission record and the write authorization.

      add :slug, :string

      add :entity, :string

      # `ledger` or `glass`, never lower. A system memory's body reaches every

      # agent by definition, which is content plus metadata; a claim that

      # cannot ship its content is not a system memory.

      add :tier, :string

      # The date the claim was observed true, distinct from `inserted_at`.

      # `inserted_at` orders the chain; `as_of` dates the claim, so a stale

      # truth reads as dated rather than as current.

      add :as_of, :date

      # The author's own claim, and nothing more. Effective status is derived

      # from `memory_admissions`, so a row that says `admitted` with no steward

      # record behind it still reads as a candidate.

      add :admission, :string

      # A non-empty list of `{kind, ref, digest}`. `receipt` points at a forge

      # receipt, `memory` at a prior admitted row, `url` at public material;

      # the digest is what keeps evidence from being swapped after admission.

      add :evidence_refs, :jsonb

    end

    # The original shape constraint named two buckets. Recreating it is how the

    # third one becomes writable at all.

    drop constraint(:memories, :memories_shape)

    create constraint(:memories, :memories_shape,

             check: """

             bucket IN ('user','learned','system')

             AND char_length(body) BETWEEN 1 AND 2000

             AND (source_ref IS NULL OR char_length(source_ref) BETWEEN 1 AND 200)

             AND (superseded_by_id IS NULL OR superseded_by_id <> id)

             """

           )

    # The system columns, present together on a system row and absent together

    # on every other row. Stating the absent half matters as much as the

    # present half: it keeps a `user` row from quietly carrying a tier nothing

    # reads and nobody set deliberately.

    #

    # Every column is asserted `IS NOT NULL` before it is compared, and that is

    # not belt and braces. A check constraint passes when it evaluates to NULL,

    # so `tier IN ('ledger','glass')` alone admits a row with no tier at all,

    # and a length test alone admits a candidate whose evidence list is absent

    # rather than empty — which is the exact hole this constraint exists to

    # close.

    create constraint(:memories, :memories_system_shape,

             check: """

             (

               bucket <> 'system'

               AND slug IS NULL

               AND entity IS NULL

               AND tier IS NULL

               AND as_of IS NULL

               AND admission IS NULL

               AND evidence_refs IS NULL

             ) OR (

               bucket = 'system'

               AND slug IS NOT NULL

               AND slug LIKE 'sys:%'

               AND char_length(slug) BETWEEN 5 AND 200

               AND (entity IS NULL OR char_length(entity) BETWEEN 1 AND 200)

               AND tier IS NOT NULL

               AND tier IN ('ledger','glass')

               AND as_of IS NOT NULL

               AND admission IS NOT NULL

               AND admission IN ('candidate','admitted','rejected')

               AND evidence_refs IS NOT NULL

               AND jsonb_typeof(evidence_refs) = 'array'

               AND jsonb_array_length(evidence_refs) BETWEEN 1 AND 20

               AND NOT jsonb_path_exists(evidence_refs, '$[*] ? (!(@.type() == "object"

                     && exists(@.kind ? (@ == "receipt" || @ == "memory" || @ == "url"))

                     && exists(@.ref ? (@.type() == "string" && @ != ""))

                     && exists(@.digest ? (@.type() == "string" && @ != ""))))')

             )

             """

           )

    # What the composite foreign key below points at. A memory's bucket cannot

    # change out from under an admission record while one references it.

    create unique_index(:memories, [:id, :bucket], name: :memories_id_bucket_index)

    # Admission is a receipt, not a field the author sets.

    #

    # The record is a row of its own rather than a column on the candidate for

    # the reason a promise flip is a receipt rather than a flag: the account

    # that wrote the claim is not the account that may admit it, and the store

    # keeps every verdict rather than the last one written over the others.

    create table(:memory_admissions, primary_key: false) do

      add :id, :binary_id, primary_key: true

      # The candidate this record judges, and the bucket it must be in. The

      # bucket rides the row so the composite foreign key can pin it.

      add :memory_id, :binary_id, null: false

      add :memory_bucket, :string, null: false

      # The account that wrote the record. A steward at the time of the write;

      # the row records who, and the role check runs where the row is created.