Close the reported Sarah memory, voice, and control gaps

c896ce88fb02 · Devin AI · · parent 00b31e35b6cc

Close the reported Sarah memory, voice, and control gaps

Voice tool steps were durable but only projected while the session was live,
so a reload showed the spoken answer with no record of the tools behind it.
Tool steps now join to their assistant message through the voice response
receipt, and the transcript merges text and voice activity into one list
ordered by step sequence.

The voice tool-call budget sent a typed refusal to the provider and wrote
nothing durable, so a truncated call read back as complete. The refusal is now
a terminal step carrying tool_call_limit_reached and the host disclosure.

public_leaderboard_opted_out filtered the board but had no writer, no control,
and no cache invalidation, so the column was unreachable. /memory now carries
the preference and every change invalidates the cached projection.

The /memory return control still sent toggle_memory, an event from when memory
was a panel inside the conversation; it navigates to /sarah now.

Memory authorization and consent held already. Tests now walk a second owner
through every read and mutation of another owner's record and its superseded
versions. docs/2026-08-23-sarah-memory-voice-session-control-audit.md records
the evidence per reported gap.

Co-Authored-By: Christopher David <chris@openagents.com>
Co-Authored-By
Christopher David <chris@openagents.com>

Deploy story

What this commit did to the running system — joined from the forge receipt chain, the part a commit page elsewhere cannot show.

Not deployed through the forge lane

No push, promotion, build, or deploy receipt references this commit (receipts are scanned over a bounded recent window). Changes shipped by full node replacement carry their proof in the release gate receipt instead.

Changed files

  • added docs/2026-08-23-sarah-memory-voice-session-control-audit.md
  • modified lib/openagents/accounts.ex
  • modified lib/openagents/accounts/user.ex
  • modified lib/openagents/voice.ex
  • modified lib/openagents/voice_sessions/session_server.ex
  • modified lib/openagents_web/live/chat_live.ex
  • modified lib/openagents_web/live/memory_live.ex
  • modified ops/ci/allowed-sarah-references.txt
  • modified test/openagents/leaderboard_test.exs
  • modified test/openagents/profile_memory_test.exs
  • modified test/openagents/voice_sessions_test.exs
  • modified test/openagents/voice_test.exs
  • modified test/openagents_web/live/chat_live_test.exs
  • modified test/openagents_web/live/memory_live_test.exs

Diff

14 files changed, +636 -16

docs/2026-08-23-sarah-memory-voice-session-control-audit.md added +110

@@ -0,0 +1,110 @@

1
# Sarah memory, voice, session, and control audit
2
3
**Date:** 2026-08-23
4
**Issue:** `OpenAgentsInc/openagents.com` #89
5
**Question:** Of the gaps reported after the launch episode, which still exist
6
in `main`, and what is the evidence either way?
7
8
Each gap below states its current status and the exact test that holds it. A
9
gap marked *held* was already closed in code but had no test naming the
10
behavior the report asked about; a gap marked *fixed* needed code in this
11
change.
12
13
## 1. Automatic memory saves, consent, and version history
14
15
**Held.** A profile-memory write requires consent evidence in the current
16
message, and `OpenAgents.Memory.Consent` returns
17
`:memory_consent_required` or `:memory_consent_mismatch` rather than writing
18
silently. Correction supersedes instead of overwriting, so the prior claim
19
stays readable as history and `/memory` shows the superseded row.
20
21
Evidence: `test/openagents/tools/profile_memory_tools_test.exs`,
22
`test/openagents/profile_memory_test.exs`,
23
`test/openagents_web/live/memory_live_test.exs`.
24
25
## 2. Voice tool calls in the same ordered activity stream as text
26
27
**Fixed.** Voice tool steps were durable in `voice_tool_steps`, but the
28
transcript only projected the steps of a live session. After the call ended, a
29
reload showed the spoken answer with no record of the tools that produced it,
30
while a text turn kept its activity attached to its assistant message.
31
32
`Voice.list_tool_step_activity_by_message/1` now joins tool steps to the
33
assistant message through the voice response receipt, and `ChatLive` merges
34
that projection with the text projection and orders both by step sequence. One
35
assistant message therefore carries one ordered activity list regardless of
36
modality.
37
38
Evidence: `test/openagents_web/live/chat_live_test.exs`, "voice tool activity
39
stays in the ordered transcript after the call ends";
40
`test/openagents/voice_test.exs`, "a refused host limit is durable and keyed to
41
the assistant message it belongs to".
42
43
## 3. Tool-call budget, visible limit, and typed terminal reason
44
45
**Fixed on the voice side.** The text runtime already budgets the loop and
46
records a typed `tool_call_limit_reached` outcome on the step that exceeded it,
47
with a runaway backstop above the budget
48
(`test/openagents/turn_tool_loop_test.exs`). The voice runtime sent the same
49
typed refusal to the provider but wrote nothing durable, so a truncated call
50
read back as a complete one.
51
52
`Voice.refuse_tool_step/4` now records the refusal as a terminal step with
53
status `refused`, the typed code, and the host executor disclosure, so the
54
refusal is evidence in the transcript rather than only a provider-side message.
55
56
Evidence: `test/openagents/voice_sessions_test.exs`, "the tool-call limit
57
refuses the next call and drives one tool-free report".
58
59
## 4. Voice session timeout, reconnect, resume, and failure
60
61
**Held.** The session budget warns at 80 percent and ends with a visible
62
reason at the ceiling; sideband loss reconnects under the same generation;
63
admission times out; a runtime restart fails the admitted generation
64
deterministically without inventing transcript or usage; a new generation is
65
admitted with prior conversation and tool evidence.
66
67
Evidence: `test/openagents/voice_sessions_test.exs` and
68
`test/openagents/voice_test.exs`, tests named for each of those behaviors.
69
70
## 5. Memory authorization across owners and versions
71
72
**Held, now proven.** `OpenAgents.ProfileMemory` scopes every read and
73
mutation by `owner_visitor_id`, including the locked queries that back forget
74
and purge. The report asked for a reproduction, so the test now walks a second
75
owner through the whole surface — list, get, correct, purge, transition, and
76
each forget mode — against another owner's record and its superseded versions,
77
and asserts the owner cannot read it, change it, delete it, or learn from the
78
result that it exists. A forget that targets nothing reports
79
`already_absent`, which is the same answer an empty account gets.
80
81
Evidence: `test/openagents/profile_memory_test.exs`.
82
83
## 6. Leaderboard opt-out
84
85
**Fixed.** `Leaderboard.ranked_query/1` already excluded accounts with
86
`public_leaderboard_opted_out`, but nothing let an account set the field: there
87
was no writer, no control, and no cache invalidation, so the column was
88
unreachable and the cached projection could serve a withheld account until an
89
unrelated recompute.
90
91
`Accounts.set_public_leaderboard_opt_out/2` writes the preference and
92
invalidates the cached projection on every change, including the change that
93
removes the account. `/memory` carries the control and states the current
94
state. Every public read of the board — the LiveView, the cached server, and
95
the recompute behind it — goes through `Leaderboard.ranked_query/1`, and no
96
other module projects the board, so one write covers every public surface.
97
98
Evidence: `test/openagents/leaderboard_test.exs`, "an account can withhold
99
itself from the board and publish itself again";
100
`test/openagents_web/live/memory_live_test.exs`, "the way back is a link, and
101
the board preference is the account's to set".
102
103
## 7. Return control on the memory page
104
105
**Fixed.** `/memory` is a page with an address, but its return control still
106
sent `toggle_memory`, an event from when memory was a panel inside the
107
conversation. No handler existed, so the control raised instead of returning.
108
It navigates to `/sarah` now.
109
110
Evidence: `test/openagents_web/live/memory_live_test.exs`.
lib/openagents/accounts.ex modified +22

@@ -112,6 +112,28 @@ defmodule OpenAgents.Accounts do

112 112
    }
113 113
  end
114 114
115
  @doc """
116
  Sets whether this account is withheld from the public leaderboard.
117
118
  The board is the one projection that crosses the account boundary
119
  (`INVARIANTS.md` LEADERBOARD-001), so the account it is about decides whether
120
  it appears. The cached projection is invalidated on every change, including a
121
  change that removes the account, so opting out takes effect on the next push
122
  to connected viewers rather than at the next unrelated recompute.
123
  """
124
  @spec set_public_leaderboard_opt_out(User.t(), boolean()) ::
125
          {:ok, User.t()} | {:error, Ecto.Changeset.t()}
126
  def set_public_leaderboard_opt_out(%User{} = user, opted_out?) when is_boolean(opted_out?) do
127
    case user |> User.leaderboard_changeset(opted_out?) |> Repo.update() do
128
      {:ok, updated} ->
129
        :ok = OpenAgents.Leaderboard.invalidate()
130
        {:ok, updated}
131
132
      {:error, changeset} ->
133
        {:error, changeset}
134
    end
135
  end
136
115 137
  @doc "Rewraps every retained GitHub token atomically; returns only the rotated count."
116 138
  @spec rotate_github_tokens!() :: non_neg_integer()
117 139
  def rotate_github_tokens! do
lib/openagents/accounts/user.ex modified +6

@@ -81,6 +81,12 @@ defmodule OpenAgents.Accounts.User do

81 81
    |> unique_constraint(:github_id)
82 82
  end
83 83
84
  def leaderboard_changeset(user, opted_out?) when is_boolean(opted_out?) do
85
    user
86
    |> cast(%{public_leaderboard_opted_out: opted_out?}, [:public_leaderboard_opted_out])
87
    |> validate_required([:public_leaderboard_opted_out])
88
  end
89
84 90
  def ban_changeset(user, reason_code) do
85 91
    user
86 92
    |> cast(
lib/openagents/voice.ex modified +73

@@ -619,6 +619,50 @@ defmodule OpenAgents.Voice do

619 619
    )
620 620
  end
621 621
622
  @doc """
623
  Voice tool-step activity keyed by the assistant message each step belongs to.
624
625
  A voice tool call is durable evidence of the same authority a text tool call
626
  has, so it belongs beside the assistant message it produced rather than only
627
  in the live session panel, which empties when the call ends or the page
628
  reloads. The response receipt carries the assistant message, so the join is
629
  the receipt.
630
631
  Selects the same bounded projection as `list_tool_step_activity/1`, so
632
  INVARIANTS.md UI-002 holds: already-scrubbed durable values and never a
633
  provider identifier.
634
  """
635
  @spec list_tool_step_activity_by_message([String.t()]) :: %{String.t() => [map()]}
636
  def list_tool_step_activity_by_message([]), do: %{}
637
638
  def list_tool_step_activity_by_message(assistant_message_ids)
639
      when is_list(assistant_message_ids) do
640
    from(step in ToolStep,
641
      join: receipt in ResponseReceipt,
642
      on: receipt.id == step.voice_response_receipt_id,
643
      where: receipt.assistant_message_id in ^assistant_message_ids,
644
      order_by: [asc: step.sequence],
645
      select:
646
        {receipt.assistant_message_id,
647
         %{
648
           id: step.id,
649
           sequence: step.sequence,
650
           tool_name: step.tool_name,
651
           status: step.status,
652
           raw_arguments: step.raw_arguments,
653
           result: step.result,
654
           error: step.error,
655
           executor_id: step.executor_id,
656
           executor_disclosure: step.executor_disclosure,
657
           requested_at: step.requested_at,
658
           started_at: step.started_at,
659
           completed_at: step.completed_at
660
         }}
661
    )
662
    |> Repo.all()
663
    |> Enum.group_by(&elem(&1, 0), &elem(&1, 1))
664
  end
665
622 666
  @spec record_client_event(Session.t(), String.t(), {String.t(), integer() | nil}) ::
623 667
          {:ok, ClientEvent.t()} | {:error, term()}
624 668
  def record_client_event(%Session{} = session, kind, {browser_family, browser_major}) do

@@ -852,6 +896,35 @@ defmodule OpenAgents.Voice do

852 896
  def complete_tool_step(%Session{}, %ToolStep{}, _outcome),
853 897
    do: {:error, :invalid_tool_outcome}
854 898
899
  @doc """
900
  Refuses a requested tool step with a typed host reason.
901
902
  A host limit that only answers the provider is invisible: the caller hears a
903
  turn that stops using tools and no one can tell whether the tool ran, failed,
904
  or was never allowed. Writing the refusal as a terminal step puts the reason
905
  in the same ordered activity stream a text refusal appears in.
906
  """
907
  @spec refuse_tool_step(Session.t(), ToolStep.t(), String.t(), String.t()) ::
908
          {:ok, ToolStep.t()} | {:error, term()}
909
  def refuse_tool_step(%Session{} = session, %ToolStep{} = step, code, message)
910
      when is_binary(code) and is_binary(message) do
911
    complete_tool_step(session, step, %{
912
      "schema" => "sarah.tool_outcome.v1",
913
      "call_id" => step.provider_call_id,
914
      "module_ref" => %{
915
        "module_id" => step.module_id,
916
        "tool_name" => step.tool_name,
917
        "version" => step.tool_version
918
      },
919
      "executor_ref" => %{"id" => "sarah.host", "disclosure" => "Sarah host execution limits"},
920
      "status" => "refused",
921
      "result" => nil,
922
      "error" => %{"code" => code, "message" => message},
923
      "target_receipt_refs" => [],
924
      "attribution_refs" => []
925
    })
926
  end
927
855 928
  @spec tool_continuation_output(ToolStep.t()) :: {:ok, map()} | {:error, term()}
856 929
  def tool_continuation_output(%ToolStep{id: step_id}) do
857 930
    step = Repo.get!(ToolStep, step_id)
lib/openagents/voice_sessions/session_server.ex modified +19 -9

@@ -977,19 +977,29 @@ defmodule OpenAgents.VoiceSessions.SessionServer do

977 977
  # Hitting the per-turn tool budget refuses the call instead of failing the
978 978
  # session; once the active response finishes, one tool-free response is
979 979
  # driven so the model reports what it already has instead of going silent.
980
  defp refuse_tool_call_limit(%ProviderEvent{payload: payload}, state) do
981
    _ =
982
      send_tool_refusal(
983
        state,
984
        payload["call_id"],
985
        "tool_call_limit_reached",
986
        "This turn reached the host limit of #{@maximum_tool_calls} tool calls. " <>
987
          "Do not call tools again this turn; report what you already have."
988
      )
980
  #
981
  # The refusal is also written as a terminal step, so the budget is visible in
982
  # the transcript with its typed reason. A limit that only answers the provider
983
  # lets a truncated run look like a complete one.
984
  defp refuse_tool_call_limit(%ProviderEvent{payload: payload} = event, state) do
985
    message =
986
      "This turn reached the host limit of #{@maximum_tool_calls} tool calls. " <>
987
        "Do not call tools again this turn; report what you already have."
988
989
    _ = record_refused_tool_step(event, state, "tool_call_limit_reached", message)
990
    _ = send_tool_refusal(state, payload["call_id"], "tool_call_limit_reached", message)
989 991
990 992
    {:noreply, %{state | limit_refused?: true}}
991 993
  end
992 994
995
  defp record_refused_tool_step(event, state, code, message) do
996
    with {:ok, step, _disposition} <-
997
           Voice.request_tool_step(state.session, event, state.tool_snapshot),
998
         {:ok, refused_step} <- Voice.refuse_tool_step(state.session, step, code, message) do
999
      {:ok, refused_step}
1000
    end
1001
  end
1002
993 1003
  defp send_tool_refusal(state, call_id, code, message) do
994 1004
    refusal = %{
995 1005
      "schema" => "sarah.tool_continuation.v1",
lib/openagents_web/live/chat_live.ex modified +15 -5

@@ -543,12 +543,22 @@ defmodule OpenAgentsWeb.ChatLive do

543 543
544 544
  defp composer_form, do: to_form(%{"message" => ""}, as: :chat)
545 545
  # Keyed by assistant message so the transcript can show what Sarah did next to
546
  # what she said, and so a reload rebuilds it from PostgreSQL.
546
  # what she said, and so a reload rebuilds it from PostgreSQL. Voice steps are
547
  # merged in from their response receipts: a spoken tool call carries the same
548
  # authority as a typed one, so it stays in the ordered stream after the call
549
  # ends instead of vanishing with the live panel.
547 550
  defp message_activity(messages) do
548
    messages
549
    |> Enum.filter(&(&1.role == "assistant"))
550
    |> Enum.map(& &1.id)
551
    |> Conversations.list_tool_step_activity_by_message()
551
    assistant_message_ids =
552
      messages
553
      |> Enum.filter(&(&1.role == "assistant"))
554
      |> Enum.map(& &1.id)
555
556
    text = Conversations.list_tool_step_activity_by_message(assistant_message_ids)
557
    voice = Voice.list_tool_step_activity_by_message(assistant_message_ids)
558
559
    Map.merge(text, voice, fn _message_id, text_steps, voice_steps ->
560
      Enum.sort_by(text_steps ++ voice_steps, & &1.sequence)
561
    end)
552 562
  end
553 563
554 564
  defp first_id([message | _messages]), do: message.id
lib/openagents_web/live/memory_live.ex modified +70 -2

@@ -19,6 +19,7 @@ defmodule OpenAgentsWeb.MemoryLive do

19 19
20 20
  use OpenAgentsWeb, :live_view
21 21
22
  alias OpenAgents.Accounts
22 23
  alias OpenAgents.Analytics
23 24
  alias OpenAgents.Conversations
24 25
  alias OpenAgents.DataRights

@@ -44,6 +45,7 @@ defmodule OpenAgentsWeb.MemoryLive do

44 45
     |> assign(:memory_status, nil)
45 46
     |> assign(:pending_memory_action, nil)
46 47
     |> assign(:reset_enabled?, DataRights.reset_enabled?())
48
     |> assign(:leaderboard_opted_out?, current_user.public_leaderboard_opted_out)
47 49
     |> assign(:recording_config, Recordings.config())
48 50
     |> assign(:privacy_delete_form, to_form(%{"confirmation" => ""}, as: :privacy))
49 51
     |> reload_memory()}

@@ -65,6 +67,7 @@ defmodule OpenAgentsWeb.MemoryLive do

65 67
        pending_memory_action={@pending_memory_action}
66 68
        privacy_delete_form={@privacy_delete_form}
67 69
        recording_config={@recording_config}
70
        leaderboard_opted_out?={@leaderboard_opted_out?}
68 71
      />
69 72
    </Layouts.app>
70 73
    """

@@ -111,6 +114,37 @@ defmodule OpenAgentsWeb.MemoryLive do

111 114
    end
112 115
  end
113 116
117
  # The public board is the one projection that leaves the account boundary, so
118
  # the account it is about decides whether it appears there. The control states
119
  # the current publication state rather than a bare toggle, since "leaderboard"
120
  # on its own does not say which way it is set.
121
  def handle_event("set_leaderboard_visibility", %{"opted_out" => opted_out}, socket)
122
      when opted_out in ["true", "false"] do
123
    opted_out? = opted_out == "true"
124
125
    case Accounts.set_public_leaderboard_opt_out(socket.assigns.current_user, opted_out?) do
126
      {:ok, user} ->
127
        message =
128
          if opted_out?,
129
            do: "Your account no longer appears on the public leaderboard.",
130
            else: "Your account appears on the public leaderboard again."
131
132
        {:noreply,
133
         socket
134
         |> assign(:current_user, user)
135
         |> assign(:leaderboard_opted_out?, user.public_leaderboard_opted_out)
136
         |> assign(:memory_status, {:ok, message})}
137
138
      {:error, _changeset} ->
139
        {:noreply,
140
         assign(
141
           socket,
142
           :memory_status,
143
           {:error, "Sarah could not change your leaderboard preference. Try again."}
144
         )}
145
    end
146
  end
147
114 148
  def handle_event("cancel_memory_action", _params, socket) do
115 149
    {:noreply, assign(socket, :pending_memory_action, nil)}
116 150
  end

@@ -236,6 +270,7 @@ defmodule OpenAgentsWeb.MemoryLive do

236 270
  attr :pending_memory_action, :any, default: nil
237 271
  attr :privacy_delete_form, :any, required: true
238 272
  attr :recording_config, :map, required: true
273
  attr :leaderboard_opted_out?, :boolean, required: true
239 274
240 275
  defp memory_manager(assigns) do
241 276
    ~H"""

@@ -252,10 +287,11 @@ defmodule OpenAgentsWeb.MemoryLive do

252 287
        <div class="memory-header__actions">
253 288
          <%!-- The way out of a panel belongs in the panel. This used to be a
254 289
          sidebar row, which meant leaving depended on chrome outside the thing
255
          you were leaving. --%>
290
          you were leaving. Memory is a place with its own URL, so the way back
291
          navigates to the conversation rather than toggling a panel. --%>
256 292
          <.text_button
257 293
            id="toggle-memory"
258
            phx-click="toggle_memory"
294
            navigate={~p"/sarah"}
259 295
            aria-label="Return to conversation"
260 296
          >
261 297
            <.icon name="arrow-left" /> Return to conversation

@@ -327,6 +363,38 @@ defmodule OpenAgentsWeb.MemoryLive do

327 363
        <.memory_record :for={record <- @memory_records} record={record} />
328 364
      </div>
329 365
366
      <.card id="leaderboard-preference" aria-labelledby="leaderboard-preference-heading">
367
        <header>
368
          <h2 id="leaderboard-preference-heading">Public leaderboard</h2>
369
          <p>
370
            The public leaderboard publishes a rank, your GitHub login, name, avatar, and one
371
            token total. It publishes nothing else about your account.
372
          </p>
373
          <p id="leaderboard-preference-state">
374
            <%= if @leaderboard_opted_out? do %>
375
              Your account is withheld from the board.
376
            <% else %>
377
              Your account appears on the board when its token total is above zero.
378
            <% end %>
379
          </p>
380
        </header>
381
        <footer>
382
          <.button
383
            id="toggle-leaderboard-preference"
384
            size={:sm}
385
            variant={:secondary}
386
            phx-click="set_leaderboard_visibility"
387
            phx-value-opted_out={to_string(not @leaderboard_opted_out?)}
388
          >
389
            <%= if @leaderboard_opted_out? do %>
390
              SHOW ME ON THE LEADERBOARD
391
            <% else %>
392
              HIDE ME FROM THE LEADERBOARD
393
            <% end %>
394
          </.button>
395
        </footer>
396
      </.card>
397
330 398
      <.card
331 399
        id="privacy-controls"
332 400
        variant={:danger}
ops/ci/allowed-sarah-references.txt modified +1

@@ -11,6 +11,7 @@

11 11
^docs/2026-08-21-hot-deploy-gap-audit\.md:
12 12
^docs/2026-08-21-repository-storage-architecture-audit\.md:
13 13
^docs/2026-08-21-sarah-computers-and-scv-architecture-audit\.md:
14
^docs/2026-08-23-sarah-memory-voice-session-control-audit\.md:
14 15
^docs/chat-inference-plan\.md:
15 16
^docs/component-library\.md:
16 17
^docs/episode-triage\.md:
test/openagents/leaderboard_test.exs modified +22

@@ -157,6 +157,28 @@ defmodule OpenAgents.LeaderboardTest do

157 157
    assert Leaderboard.compute_entries() == []
158 158
  end
159 159
160
  test "an account can withhold itself from the board and publish itself again" do
161
    user = account("self-withheld")
162
163
    complete_typed_turn(user, "Publish me first.", %{
164
      "input_tokens" => 60,
165
      "output_tokens" => 40,
166
      "total_tokens" => 100
167
    })
168
169
    assert [_entry] = Leaderboard.compute_entries()
170
    assert Enum.map(Leaderboard.refresh(), & &1.github_login) == [user.github_login]
171
172
    assert {:ok, withheld} = Accounts.set_public_leaderboard_opt_out(user, true)
173
    assert withheld.public_leaderboard_opted_out
174
    assert Leaderboard.compute_entries() == []
175
    assert Leaderboard.refresh() == []
176
177
    assert {:ok, published} = Accounts.set_public_leaderboard_opt_out(withheld, false)
178
    refute published.public_leaderboard_opted_out
179
    assert Enum.map(Leaderboard.refresh(), & &1.github_login) == [user.github_login]
180
  end
181
160 182
  test "drops an account from the board when it deletes its data" do
161 183
    user = account("erased")
162 184
test/openagents/profile_memory_test.exs modified +71

@@ -288,6 +288,77 @@ defmodule OpenAgents.ProfileMemoryTest do

288 288
             )
289 289
  end
290 290
291
  test "deletion, correction, and purge refuse another owner's record and leak no existence" do
292
    {first_owner, first_conversation} = owner("profile-delete-first-browser")
293
    {second_owner, second_conversation} = owner("profile-delete-second-browser")
294
    first_source = source_message(first_conversation, "Remember my private project is Atlas.")
295
    second_source = source_message(second_conversation, "Remember my private project is Beacon.")
296
297
    {:ok, candidate} =
298
      ProfileMemory.create_candidate(
299
        first_owner,
300
        attributes(first_source, category: "project", claim: "My project is Atlas.")
301
      )
302
303
    {:ok, active} =
304
      ProfileMemory.transition(first_owner, candidate.id, candidate.generation, "active")
305
306
    foreign_selector = %{
307
      "mode" => "record",
308
      "record_id" => active.id,
309
      "expected_generation" => active.generation
310
    }
311
312
    absent_selector = %{
313
      "mode" => "record",
314
      "record_id" => Ecto.UUID.generate(),
315
      "expected_generation" => active.generation
316
    }
317
318
    # Identical answers: the second owner cannot tell a record it may not touch
319
    # from one that was never stored.
320
    assert {:ok, %{disposition: "already_absent", records: []}} =
321
             ProfileMemory.forget_active(second_owner, foreign_selector)
322
323
    assert {:ok, %{disposition: "already_absent", records: []}} =
324
             ProfileMemory.forget_active(second_owner, absent_selector)
325
326
    assert {:error, :not_found} =
327
             ProfileMemory.correct(
328
               second_owner,
329
               active.id,
330
               active.generation,
331
               attributes(second_source, category: "project", claim: "My project is Beacon.")
332
             )
333
334
    assert {:error, :not_found} =
335
             ProfileMemory.purge(second_owner, active.id, active.generation)
336
337
    # A stale generation is no shortcut around ownership, and forgetting in one
338
    # scope leaves the other scope's record active.
339
    assert {:error, :not_found} =
340
             ProfileMemory.transition(second_owner, active.id, active.generation + 1, "forgotten")
341
342
    assert {:ok, [listed]} = ProfileMemory.list_current(first_owner)
343
    assert listed.id == active.id
344
    assert listed.status == "active"
345
346
    assert {:ok, %{disposition: "forgotten", records: [forgotten]}} =
347
             ProfileMemory.forget_active(first_owner, foreign_selector)
348
349
    assert forgotten.id == active.id
350
    assert {:ok, []} = ProfileMemory.list_current(first_owner)
351
352
    # Purge is the only irreversible path, so it stays owner-scoped and terminal.
353
    assert {:error, :not_found} =
354
             ProfileMemory.purge(second_owner, forgotten.id, forgotten.generation)
355
356
    assert {:ok, :purged} =
357
             ProfileMemory.purge(first_owner, forgotten.id, forgotten.generation)
358
359
    assert {:error, :not_found} = ProfileMemory.get(first_owner, forgotten.id)
360
  end
361
291 362
  test "database backstops reject orphan activation, source deletion, and in-place claim edits" do
292 363
    {owner, conversation} = owner("profile-database-constraints")
293 364
    {_foreign_owner, foreign_conversation} = owner("profile-database-foreign-source")
test/openagents/voice_sessions_test.exs modified +8

@@ -1085,6 +1085,14 @@ defmodule OpenAgents.VoiceSessionsTest do

1085 1085
                    %{"type" => "response.create", "response" => %{"tool_choice" => "none"}}},
1086 1086
                   1_000
1087 1087
1088
    # The refusal the provider hears is also evidence the person can read: a
1089
    # truncated run must not read back as a complete one.
1090
    assert refused =
1091
             Enum.find(Voice.list_tool_step_activity(session), &(&1.status == "refused"))
1092
1093
    assert refused.error["code"] == "tool_call_limit_reached"
1094
    assert refused.error["message"] =~ "host limit of 8 tool calls"
1095
1088 1096
    alive = Voice.get_session!(session.id)
1089 1097
    assert alive.status in ~w(listening responding)
1090 1098
    assert is_pid(VoiceSessions.whereis(session.id))
test/openagents/voice_test.exs modified +91

@@ -474,6 +474,97 @@ defmodule OpenAgents.VoiceTest do

474 474
           ]
475 475
  end
476 476
477
  test "a refused host limit is durable and keyed to the assistant message it belongs to" do
478
    {:ok, conversation} = Conversations.ensure_conversation("voice-durable-activity-browser")
479
    snapshot = Registry.current!()
480
    {:ok, session} = Voice.admit_session(conversation, enabled_config())
481
482
    {:ok, session, _event, :created} =
483
      Voice.record_provider_event(
484
        session,
485
        session.generation,
486
        event(:user_transcript_final, "evt-user-activity", %{
487
          "item_id" => "item-user-activity",
488
          "response_id" => nil,
489
          "content" => "Keep looking things up."
490
        })
491
      )
492
493
    assert {:ok, context} =
494
             Voice.capture_response_context(session, "item-user-activity", snapshot)
495
496
    {:ok, session, _event, :created} =
497
      Voice.record_provider_event(
498
        session,
499
        session.generation,
500
        event(:response_started, "evt-start-activity", %{"response_id" => "response-activity"}),
501
        response_context: context
502
      )
503
504
    request =
505
      event(:tool_call_requested, "evt-tool-activity", %{
506
        "response_id" => "response-activity",
507
        "item_id" => "item-call-activity",
508
        "call_id" => "call-activity",
509
        "tool_name" => "memory_list",
510
        "raw_arguments" => ~s({"category":"","first":1})
511
      })
512
513
    {:ok, session, _event, :created} =
514
      Voice.record_provider_event(session, session.generation, request)
515
516
    assert {:ok, requested, :created} = Voice.request_tool_step(session, request, snapshot)
517
518
    assert {:ok, refused} =
519
             Voice.refuse_tool_step(
520
               session,
521
               requested,
522
               "tool_call_limit_reached",
523
               "This turn reached the host limit of 8 tool calls."
524
             )
525
526
    assert refused.status == "refused"
527
    assert refused.error["code"] == "tool_call_limit_reached"
528
    assert refused.executor_id == "sarah.host"
529
530
    {:ok, session, _event, :created} =
531
      Voice.record_provider_event(
532
        session,
533
        session.generation,
534
        event(:assistant_transcript_final, "evt-assistant-activity", %{
535
          "item_id" => "item-assistant-activity",
536
          "response_id" => "response-activity",
537
          "content" => "I stopped short of finishing."
538
        })
539
      )
540
541
    {:ok, session, _event, :created} =
542
      Voice.record_provider_event(
543
        session,
544
        session.generation,
545
        event(:response_completed, "evt-done-activity", %{
546
          "response_id" => "response-activity",
547
          "status" => "completed",
548
          "usage" => %{}
549
        })
550
      )
551
552
    assert [receipt] = Voice.list_response_receipts(session)
553
    assistant_message_id = receipt.assistant_message_id
554
    assert is_binary(assistant_message_id)
555
556
    assert %{^assistant_message_id => [activity]} =
557
             Voice.list_tool_step_activity_by_message([assistant_message_id])
558
559
    assert activity.tool_name == "memory_list"
560
    assert activity.status == "refused"
561
    assert activity.error["code"] == "tool_call_limit_reached"
562
    refute Map.has_key?(activity, :provider_call_id)
563
564
    assert Voice.list_tool_step_activity_by_message([]) == %{}
565
    assert Voice.list_tool_step_activity_by_message([Ecto.UUID.generate()]) == %{}
566
  end
567
477 568
  defp event(kind, event_id, payload) do
478 569
    %ProviderEvent{kind: kind, provider_event_id: event_id, payload: payload}
479 570
  end
test/openagents_web/live/chat_live_test.exs modified +105

@@ -394,6 +394,103 @@ defmodule OpenAgentsWeb.ChatLiveTest do

394 394
    refute has_element?(view, "#messages-voice-live-item-live-1")
395 395
  end
396 396
397
  test "voice tool activity stays in the ordered transcript after the call ends", %{conn: conn} do
398
    key = "voice-durable-activity-credential-0000000000000"
399
    user = github_user(key)
400
    conn = log_in_github_user(conn, key)
401
    assert {:ok, opening, _html} = live(conn, ~p"/sarah")
402
    GenServer.stop(opening.pid)
403
    conversation = Conversations.get_conversation_for_user(user)
404
    snapshot = OpenAgents.Tools.Registry.current!()
405
    config = OpenAgents.Voice.Config.build!(enabled_voice())
406
    assert {:ok, session} = Voice.admit_session(conversation, config)
407
408
    {:ok, session, _event, :created} =
409
      Voice.record_provider_event(
410
        session,
411
        session.generation,
412
        voice_event(:user_transcript_final, "evt-chat-user", %{
413
          "item_id" => "item-chat-user",
414
          "response_id" => nil,
415
          "content" => "Look through what you remember."
416
        })
417
      )
418
419
    assert {:ok, context} = Voice.capture_response_context(session, "item-chat-user", snapshot)
420
421
    {:ok, session, _event, :created} =
422
      Voice.record_provider_event(
423
        session,
424
        session.generation,
425
        voice_event(:response_started, "evt-chat-start", %{"response_id" => "response-chat"}),
426
        response_context: context
427
      )
428
429
    request =
430
      voice_event(:tool_call_requested, "evt-chat-tool", %{
431
        "response_id" => "response-chat",
432
        "item_id" => "item-chat-call",
433
        "call_id" => "call-chat",
434
        "tool_name" => "memory_list",
435
        "raw_arguments" => ~s({"category":"","first":1})
436
      })
437
438
    {:ok, session, _event, :created} =
439
      Voice.record_provider_event(session, session.generation, request)
440
441
    assert {:ok, requested, :created} = Voice.request_tool_step(session, request, snapshot)
442
443
    assert {:ok, _refused} =
444
             Voice.refuse_tool_step(
445
               session,
446
               requested,
447
               "tool_call_limit_reached",
448
               "This turn reached the host limit of 8 tool calls."
449
             )
450
451
    {:ok, session, _event, :created} =
452
      Voice.record_provider_event(
453
        session,
454
        session.generation,
455
        voice_event(:assistant_transcript_final, "evt-chat-assistant", %{
456
          "item_id" => "item-chat-assistant",
457
          "response_id" => "response-chat",
458
          "content" => "I stopped short of finishing that."
459
        })
460
      )
461
462
    {:ok, session, _event, :created} =
463
      Voice.record_provider_event(
464
        session,
465
        session.generation,
466
        voice_event(:response_completed, "evt-chat-done", %{
467
          "response_id" => "response-chat",
468
          "status" => "completed",
469
          "usage" => %{}
470
        })
471
      )
472
473
    assert {:ok, _ended} = Voice.end_session(session, session.generation, "test_end")
474
475
    assert [receipt] = Voice.list_response_receipts(session)
476
    assistant_message_id = receipt.assistant_message_id
477
478
    assert {:ok, view, html} = live(conn, ~p"/sarah")
479
480
    assert html =~ "I stopped short of finishing that."
481
482
    # The spoken tool call sits in the assistant row it belongs to, not in the
483
    # live panel, which a reload empties.
484
    assert has_element?(view, "#tool-activity-messages-#{assistant_message_id}")
485
    refute has_element?(view, "#live-tool-activity")
486
487
    row = view |> element("#tool-activity-messages-#{assistant_message_id}") |> render()
488
489
    assert row =~ "wasn&#39;t permitted"
490
    assert row =~ "Denied"
491
    assert row =~ "tool_call_limit_reached"
492
  end
493
397 494
  test "reload preserves one canonical greeting without fake recognition", %{conn: conn} do
398 495
    key = "reload-browser-credential-0000000000000000"
399 496
    user = github_user(key)

@@ -1124,6 +1221,14 @@ defmodule OpenAgentsWeb.ChatLiveTest do

1124 1221
    do_eventually(assertion, deadline)
1125 1222
  end
1126 1223
1224
  defp voice_event(kind, event_id, payload) do
1225
    %OpenAgents.Voice.ProviderEvent{
1226
      kind: kind,
1227
      provider_event_id: event_id,
1228
      payload: payload
1229
    }
1230
  end
1231
1127 1232
  defp enabled_voice do
1128 1233
    [
1129 1234
      enabled: true,
test/openagents_web/live/memory_live_test.exs modified +23

@@ -47,6 +47,29 @@ defmodule OpenAgentsWeb.MemoryLiveTest do

47 47
    refute has_element?(view, "#message-form")
48 48
  end
49 49
50
  test "the way back is a link, and the board preference is the account's to set", %{conn: conn} do
51
    token = "memory-preference-browser-credential-0000000000000"
52
    conn = log_in_github_user(conn, token)
53
    user = github_user(token)
54
    assert {:ok, view, _html} = live(conn, ~p"/memory")
55
56
    # A page is left by navigating, not by toggling a panel that is not there.
57
    assert has_element?(view, "#toggle-memory[href='/sarah']")
58
59
    assert render(view) =~ "Your account appears on the board"
60
61
    view |> element("#toggle-leaderboard-preference") |> render_click()
62
63
    assert render(view) =~ "no longer appears on the public leaderboard"
64
    assert render(view) =~ "Your account is withheld from the board."
65
    assert OpenAgents.Repo.get!(OpenAgents.Accounts.User, user.id).public_leaderboard_opted_out
66
67
    view |> element("#toggle-leaderboard-preference") |> render_click()
68
69
    assert render(view) =~ "Your account appears on the board"
70
    refute OpenAgents.Repo.get!(OpenAgents.Accounts.User, user.id).public_leaderboard_opted_out
71
  end
72
50 73
  test "correction preserves supersession and reconciles another open tab", %{conn: conn} do
51 74
    token = "memory-correction-browser-credential-0000000000000"
52 75
    %{record: record} = create_profile_memory(token, "I prefer concise answers")

This page updates live while a promote is in flight · changelog