Carry reasoning and a faithful tool loop through the inference proxy

c26c1888e2cd · AtlantisPleb · 2d ago · parent 30d71aacf968

Carry reasoning and a faithful tool loop through the inference proxy

A thread-backed coder turn showed no reasoning and could not sustain a
client-side tool loop (#164). Three gaps, all between the providers and
the proxy:

- OpenAgents.Providers.ProviderEvent had no reasoning member, so both
  stream decoders dropped what their upstreams emit. The union gains
  reasoning_delta; the OpenRouter decoder reads delta.reasoning (and the
  reasoning_content spelling), the OpenAI decoder reads the
  response.reasoning_summary_text.delta and response.reasoning_text.delta
  event families, and the proxy translates the event to delta.reasoning
  alongside delta.content in the chat-completions SSE.

- The proxy dropped an assistant history message that carried only
  tool_calls, which orphaned the tool outputs answering it. A request
  message now carries its tool calls, and each adapter replays the pair
  faithfully: OpenRouter as an assistant tool_calls message followed by
  role "tool" results, OpenAI as function_call plus function_call_output
  items in place. An output whose call the caller did not replay keeps
  the previous labelled fallback rather than being dropped.

- Without previous_response_id the OpenAI adapter replaced the whole
  input with the tool outputs, sending outputs for calls the provider
  had never seen. The serial continuation path is unchanged.

The conversation lane persists only the reply, so its turn and job
runtimes pass reasoning deltas without effect instead of failing the
turn as an unknown event. The proxy still composes no persona and lets
no provider credential or raw provider JSON cross its boundary.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GoYpb8FEmdxVErsv7ABCYi

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GoYpb8FEmdxVErsv7ABCYi

Deploy story

What this commit did to the running system — joined from the forge receipt chain, the part a commit page elsewhere cannot show.

pushed: by user · WAL seq 299 · 2026-08-24T19:50:38.384762Z

Changed files

modified INVARIANTS.md
modified lib/openagents/providers/open_ai.ex
modified lib/openagents/providers/open_ai/stream_decoder.ex
modified lib/openagents/providers/open_router.ex
modified lib/openagents/providers/open_router/stream_decoder.ex
modified lib/openagents/providers/provider_event.ex
modified lib/openagents/providers/request.ex
modified lib/openagents/turns/turn_server.ex
modified lib/openagents/work/job_server.ex
modified lib/openagents_web/controllers/inference_proxy_controller.ex
modified test/openagents/providers/open_ai/request_payload_test.exs
modified test/openagents/providers/open_ai/stream_decoder_test.exs
modified test/openagents/providers/open_router/request_payload_test.exs
modified test/openagents/providers/open_router/stream_decoder_test.exs
modified test/openagents/turn_provider_events_test.exs
modified test/openagents_web/controllers/inference_proxy_controller_test.exs
modified test/support/providers/test.ex

Diff

17 files changed, +527 -23

Status: Current

Conversation and web code depend on `OpenAgents.Providers.Provider`, not OpenAI
event shapes. Adapters emit typed OpenAgents-domain lifecycle, text, tool-call,
usage, completion, failure, and cancellation events. A response ID is persisted
event shapes. Adapters emit typed OpenAgents-domain lifecycle, text, reasoning,
tool-call, usage, completion, failure, and cancellation events. A response ID is persisted
when announced, and matching explicit completion is required; stream closure
alone cannot produce a completed turn. Provider-specific events, credentials,
and raw errors never reach the receipt or browser. Response creation is a


  @doc false
  def request_payload(request) do
    input =
      case request.tool_outputs do
        [] -> request.input
        outputs -> Enum.map(outputs, &tool_output/1)
      end

    %{
      model: request.model_id,
      instructions: request.instructions,
      input: input,
      input: input_items(request),
      tools: Enum.map(request.tool_definitions, &tool_definition/1),
      parallel_tool_calls: false,
      max_output_tokens: 4_096,

91	85		`\|> maybe_put(:previous_response_id, request.previous_response_id)`
92	86		`end`
93	87
	88	+	`# A serial continuation names the response it answers, so the outputs are`
	89	+	`# the whole input: the provider already holds the transcript.`
	90	+	`defp input_items(%Request{previous_response_id: id} = request) when is_binary(id) do`
	91	+	`case request.tool_outputs do`
	92	+	`[] -> Enum.map(request.input, &message_item/1)`
	93	+	`outputs -> Enum.map(outputs, &tool_output/1)`
	94	+	`end`
	95	+	`end`
	96	+
	97	+	`# Without a previous response the transcript travels in full, so a prior`
	98	+	# assistant tool call is replayed as its `function_call` item followed by
	99	+	# the `function_call_output` that answers it — an output without its call
	100	+	`# is an item the Responses API refuses. An output whose call the caller`
	101	+	`# did not replay is appended last rather than dropped.`
	102	+	`defp input_items(%Request{} = request) do`
	103	+	`outputs_by_call_id = Map.new(request.tool_outputs, &{&1.call_id, &1})`
	104	+
	105	+	`replayed_call_ids =`
	106	+	`request.input`
	107	+	`\|> Enum.flat_map(&Map.get(&1, :tool_calls, []))`
	108	+	`\|> MapSet.new(& &1.call_id)`
	109	+
	110	+	`orphaned = Enum.reject(request.tool_outputs, &MapSet.member?(replayed_call_ids, &1.call_id))`
	111	+
	112	+	`items = Enum.flat_map(request.input, &items_for_message(&1, outputs_by_call_id))`
	113	+	`items ++ Enum.map(orphaned, &tool_output/1)`
	114	+	`end`
	115	+
	116	+	`defp items_for_message(%{tool_calls: [_call \| _rest] = calls} = message, outputs_by_call_id) do`
	117	+	`prose = if message.content == "", do: [], else: [message_item(message)]`
	118	+
	119	+	`calls`
	120	+	`\|> Enum.flat_map(fn call ->`
	121	+	`output_items =`
	122	+	`case Map.fetch(outputs_by_call_id, call.call_id) do`
	123	+	`{:ok, output} -> [tool_output(output)]`
	124	+	`:error -> []`
	125	+	`end`
	126	+
	127	+	`[function_call_item(call) \| output_items]`
	128	+	`end)`
	129	+	`\|> then(&(prose ++ &1))`
	130	+	`end`
	131	+
	132	+	`defp items_for_message(message, _outputs_by_call_id), do: [message_item(message)]`
	133	+
	134	+	`defp message_item(message), do: %{role: message.role, content: message.content}`
	135	+
	136	+	`defp function_call_item(call) do`
	137	+	`%{`
	138	+	`type: "function_call",`
	139	+	`call_id: call.call_id,`
	140	+	`name: call.name,`
	141	+	`arguments: call.arguments`
	142	+	`}`
	143	+	`end`
	144	+
94	145		`defp tool_definition(%ToolDefinition{} = definition) do`
95	146		`%{`
96	147		`type: "function",`

    {:ok, state, [{:text_delta, delta}]}
  end

  # The Responses API streams reasoning as its own event families: the
  # summary text most models expose, and the raw reasoning text some do.
  # Both become the one neutral reasoning event.
  defp decode_json(state, {:ok, %{"type" => type, "delta" => delta}})
       when type in ["response.reasoning_summary_text.delta", "response.reasoning_text.delta"] and
              is_binary(delta) and byte_size(delta) <= @maximum_delta_bytes do
    {:ok, state, [{:reasoning_delta, delta}]}
  end

  defp decode_json(
         state,
         {:ok,

    |> maybe_put_tools(request.tool_definitions)
  end

  # The proxy hands over the system text separately from the turns, and a tool
  # output arrives without the assistant call that asked for it, because the
  # calling harness flattens its own tool loop before it sends. So an output is
  # carried as a labelled user message: it keeps the result in the transcript
  # without claiming a call OpenRouter never saw.
  # The proxy hands over the system text separately from the turns. A tool
  # output whose assistant call is in the transcript is carried faithfully as
  # a `tool` role message right after the assistant turn that called it. An
  # output without that call — a harness that flattens its own tool loop
  # before it sends — is carried as a labelled user message instead: it keeps
  # the result in the transcript without claiming a call OpenRouter never saw.
  defp messages(%Request{} = request) do
    instructions =
      case String.trim(request.instructions || "") do

        text -> [%{role: "system", content: text}]
      end

    turns = Enum.map(request.input, &%{role: role(&1.role), content: &1.content})
    instructions ++ turns ++ Enum.map(request.tool_outputs, &tool_output/1)
    declared_call_ids =
      request.input
      |> Enum.flat_map(&Map.get(&1, :tool_calls, []))
      |> MapSet.new(& &1.call_id)

    {matched, orphaned} =
      Enum.split_with(request.tool_outputs, &MapSet.member?(declared_call_ids, &1.call_id))

    outputs_by_call_id = Map.new(matched, &{&1.call_id, &1})

    turns = Enum.flat_map(request.input, &turn(&1, outputs_by_call_id))
    instructions ++ turns ++ Enum.map(orphaned, &orphaned_tool_output/1)
  end

  defp turn(%{tool_calls: [_call | _rest] = calls} = message, outputs_by_call_id) do
    assistant = %{
      role: "assistant",
      content: message.content,
      tool_calls: Enum.map(calls, &assistant_tool_call/1)
    }

    results =
      calls
      |> Enum.flat_map(fn call ->
        case Map.fetch(outputs_by_call_id, call.call_id) do
          {:ok, output} -> [tool_result(output)]
          :error -> []
        end
      end)

    [assistant | results]
  end

  defp turn(message, _outputs_by_call_id),
    do: [%{role: role(message.role), content: message.content}]

  defp role(role) when role in ["system", "user", "assistant"], do: role
  defp role(_role), do: "user"

  defp tool_output(%ToolOutput{} = output) do
  defp assistant_tool_call(call) do
    %{
      id: call.call_id,
      type: "function",
      function: %{name: call.name, arguments: call.arguments}
    }
  end

  defp tool_result(%ToolOutput{} = output) do
    %{
      role: "tool",
      tool_call_id: output.call_id,
      content: Jason.encode!(output.output)
    }
  end

  defp orphaned_tool_output(%ToolOutput{} = output) do
    %{
      role: "user",
      content: "Tool result for #{output.call_id}: #{Jason.encode!(output.output)}"

    delta = if is_map(choice["delta"]), do: choice["delta"], else: %{}
    state = if is_binary(choice["finish_reason"]), do: %{state | terminal?: true}, else: state

    with {:ok, text_events} <- text(delta["content"]),
    with {:ok, reasoning_events} <- reasoning(delta),
         {:ok, text_events} <- text(delta["content"]),
         {:ok, state} <- tool_calls(state, delta["tool_calls"]) do
      {:ok, state, text_events}
      {:ok, state, reasoning_events ++ text_events}
    end
  end



  defp text(_content), do: {:error, :invalid_provider_event}

  # OpenRouter normalizes a model's thinking stream to `delta.reasoning`;
  # some upstreams spell it `reasoning_content`. Either becomes the neutral
  # reasoning event. A non-string value is ignored rather than refused: the
  # `reasoning_details` object family is provider detail this proxy does not
  # carry, and dropping it must not kill the text alongside it.
  defp reasoning(delta) do
    case delta["reasoning"] || delta["reasoning_content"] do
      content when is_binary(content) and content != "" ->
        if byte_size(content) <= @maximum_delta_bytes,
          do: {:ok, [{:reasoning_delta, content}]},
          else: {:error, :invalid_provider_event}

      _absent_or_structured ->
        {:ok, []}
    end
  end

  # A chat-completions tool call streams as fragments keyed by index: the name
  # arrives once and the arguments arrive as a string built up over chunks, so
  # the fragments are accumulated and emitted whole at the end of the stream.

  @type t ::
          {:response_started, String.t()}
          | {:text_delta, String.t()}
          | {:reasoning_delta, String.t()}
          | {:tool_call, ToolCall.t()}
          | {:usage, map()}
          | {:response_completed, String.t()}

  @enforce_keys [:model_id, :instructions, :input]
  defstruct @enforce_keys ++ [tool_definitions: [], tool_outputs: [], previous_response_id: nil]

  @type message :: %{role: String.t(), content: String.t()}
  @typedoc """
  A prior tool call an assistant turn carried, so a continuation request can
  replay the call the outputs in `tool_outputs` answer. `arguments` is the raw
  JSON string the provider produced; it is replayed, never interpreted.
  """
  @type message_tool_call :: %{call_id: String.t(), name: String.t(), arguments: String.t()}

  @type message :: %{
          :role => String.t(),
          :content => String.t(),
          optional(:tool_calls) => [message_tool_call()]
        }
  @type t :: %__MODULE__{
          model_id: String.t(),
          instructions: String.t(),

    end
  end

  # Reasoning exists for the proxy's streaming callers; this runtime persists
  # only the reply, so the deltas pass without effect rather than failing the
  # turn as an unknown event.
  defp handle_provider_event({:reasoning_delta, delta}, state) when is_binary(delta),
    do: {:noreply, state}

  defp handle_provider_event({:usage, usage}, state) when is_map(usage) do
    if valid_usage?(usage),
      do: {:noreply, %{state | current_usage: usage}},

    end
  end

  # Reasoning exists for the proxy's streaming callers; a job persists only
  # its report, so the deltas pass without effect rather than failing the job
  # as an unknown event.
  defp handle_provider_event({:reasoning_delta, delta}, state) when is_binary(delta),
    do: {:noreply, state}

  defp handle_provider_event({:usage, usage}, state) when is_map(usage) do
    {:noreply, %{state | current_usage: usage}}
  end

  defp input_message(%{"role" => "tool"}), do: []

  defp input_message(message) do
    case content_text(message) do
      "" -> []
      text -> [%{role: role(message), content: text}]
    tool_calls = message_tool_calls(message["tool_calls"])

    case {content_text(message), tool_calls} do
      {"", []} -> []
      {text, []} -> [%{role: role(message), content: text}]
      # An assistant turn that called tools is part of the transcript even
      # when it carried no prose: dropping it would orphan the tool outputs
      # that answer it.
      {text, calls} -> [%{role: role(message), content: text, tool_calls: calls}]
    end
  end

  # The assistant tool calls a caller replays from its own history, in the
  # chat-completions shape it received them. Arguments stay the raw JSON
  # string; the proxy never interprets them.
  defp message_tool_calls(calls) when is_list(calls) do
    Enum.flat_map(calls, fn
      %{"id" => id, "function" => %{"name" => name} = function}
      when is_binary(id) and id != "" and is_binary(name) and name != "" ->
        [%{call_id: id, name: name, arguments: text(function["arguments"])}]

      _invalid ->
        []
    end)
  end

  defp message_tool_calls(_calls), do: []

  defp tool_outputs(turns) do
    turns
    |> Enum.filter(&(role(&1) == "tool"))

    [data(%{"choices" => [%{"index" => 0, "delta" => %{"content" => text}}]})]
  end

  # Reasoning rides the OpenRouter chat-completions extension field —
  # `delta.reasoning` alongside `delta.content` — the shape the CLI's
  # OpenAI-compatible parser already expects from that vendor surface.
  defp event_chunks({:reasoning_delta, text}, _index) when text != "" do
    [data(%{"choices" => [%{"index" => 0, "delta" => %{"reasoning" => text}}]})]
  end

  defp event_chunks({:tool_call, tool_call}, index) do
    [
      data(%{

59	59		`"result" => %{"matches" => []}`
60	60		`}`
61	61		`end`
	62	+
	63	+	`test "without a previous response, replays a tool call as its item pair in place" do`
	64	+	`request = %Request{`
	65	+	`model_id: "test-model",`
	66	+	`instructions: "Remain OpenAgents.",`
	67	+	`input: [`
	68	+	`%{role: "user", content: "Read the file."},`
	69	+	`%{`
	70	+	`role: "assistant",`
	71	+	`content: "",`
	72	+	`tool_calls: [`
	73	+	`%{call_id: "call_read", name: "read_file", arguments: ~s({"path":"a.txt"})}`
	74	+	`]`
	75	+	`},`
	76	+	`%{role: "user", content: "And then?"}`
	77	+	`],`
	78	+	`tool_outputs: [`
	79	+	`%ToolOutput{call_id: "call_read", output: %{"content" => "hello"}}`
	80	+	`]`
	81	+	`}`
	82	+
	83	+	`payload = OpenAI.request_payload(request)`
	84	+
	85	+	`refute Map.has_key?(payload, :previous_response_id)`
	86	+
	87	+	`assert [`
	88	+	`%{role: "user", content: "Read the file."},`
	89	+	`%{`
	90	+	`type: "function_call",`
	91	+	`call_id: "call_read",`
	92	+	`name: "read_file",`
	93	+	`arguments: ~s({"path":"a.txt"})`
	94	+	`},`
	95	+	`%{type: "function_call_output", call_id: "call_read", output: encoded_output},`
	96	+	`%{role: "user", content: "And then?"}`
	97	+	`] = payload.input`
	98	+
	99	+	`assert Jason.decode!(encoded_output) == %{"content" => "hello"}`
	100	+	`end`
	101	+
	102	+	`test "without a previous response, an output whose call was not replayed still travels" do`
	103	+	`request = %Request{`
	104	+	`model_id: "test-model",`
	105	+	`instructions: "",`
	106	+	`input: [%{role: "user", content: "Continue."}],`
	107	+	`tool_outputs: [`
	108	+	`%ToolOutput{call_id: "call_orphan", output: %{"status" => "succeeded"}}`
	109	+	`]`
	110	+	`}`
	111	+
	112	+	`assert [`
	113	+	`%{role: "user", content: "Continue."},`
	114	+	`%{type: "function_call_output", call_id: "call_orphan", output: _output}`
	115	+	`] = OpenAI.request_payload(request).input`
	116	+	`end`
62	117		`end`

           ]
  end

  test "decodes reasoning summary and reasoning text deltas as reasoning events" do
    stream =
      frame(%{"type" => "response.created", "response" => %{"id" => "resp_r"}}) <>
        frame(%{"type" => "response.reasoning_summary_text.delta", "delta" => "Weighing "}) <>
        frame(%{"type" => "response.reasoning_text.delta", "delta" => "options."}) <>
        frame(%{"type" => "response.output_text.delta", "delta" => "Done."}) <>
        frame(%{"type" => "response.completed", "response" => %{"id" => "resp_r"}})

    assert {:ok, decoder, events} = feed_all([stream])
    assert {:ok, _decoder, final_events} = StreamDecoder.finish(decoder)

    assert events ++ final_events == [
             {:response_started, "resp_r"},
             {:reasoning_delta, "Weighing "},
             {:reasoning_delta, "options."},
             {:text_delta, "Done."},
             {:response_completed, "resp_r"}
           ]
  end

  test "rejects malformed JSON and a completed response with a changed ID" do
    assert {:error, :invalid_provider_event} =
             StreamDecoder.feed(StreamDecoder.new(), "data: {not-json}\n\n")

70	70		`]`
71	71		`end`
72	72
	73	+	`test "replays an assistant tool call and its output faithfully" do`
	74	+	`request = %Request{`
	75	+	`model_id: "stealth/ox-alpha",`
	76	+	`instructions: "",`
	77	+	`input: [`
	78	+	`%{role: "user", content: "Read the file."},`
	79	+	`%{`
	80	+	`role: "assistant",`
	81	+	`content: "",`
	82	+	`tool_calls: [`
	83	+	`%{call_id: "call_read", name: "read_file", arguments: ~s({"path":"a.txt"})}`
	84	+	`]`
	85	+	`},`
	86	+	`%{role: "user", content: "And then?"}`
	87	+	`],`
	88	+	`tool_outputs: [`
	89	+	`%ToolOutput{call_id: "call_read", output: %{"content" => "hello"}}`
	90	+	`]`
	91	+	`}`
	92	+
	93	+	`assert OpenRouter.request_payload(request).messages == [`
	94	+	`%{role: "user", content: "Read the file."},`
	95	+	`%{`
	96	+	`role: "assistant",`
	97	+	`content: "",`
	98	+	`tool_calls: [`
	99	+	`%{`
	100	+	`id: "call_read",`
	101	+	`type: "function",`
	102	+	`function: %{name: "read_file", arguments: ~s({"path":"a.txt"})}`
	103	+	`}`
	104	+	`]`
	105	+	`},`
	106	+	`# The tool result lands directly after the assistant call it`
	107	+	`# answers, not at the end of the transcript.`
	108	+	`%{`
	109	+	`role: "tool",`
	110	+	`tool_call_id: "call_read",`
	111	+	`content: ~s({"content":"hello"})`
	112	+	`},`
	113	+	`%{role: "user", content: "And then?"}`
	114	+	`]`
	115	+	`end`
	116	+
73	117		`test "carries a tool output as a labelled user turn" do`
74	118		`request = %Request{`
75	119		`model_id: "stealth/ox-alpha",`

31	31		`]`
32	32		`end`
33	33
	34	+	`test "decodes reasoning deltas alongside text, in stream order" do`
	35	+	`stream =`
	36	+	`frame(%{"id" => "gen-r", "choices" => [%{"delta" => %{"reasoning" => "Consider "}}]}) <>`
	37	+	`frame(%{`
	38	+	`"id" => "gen-r",`
	39	+	`"choices" => [%{"delta" => %{"reasoning" => "the request.", "content" => "Hi"}}]`
	40	+	`}) <>`
	41	+	`frame(%{"id" => "gen-r", "choices" => [%{"delta" => %{}, "finish_reason" => "stop"}]}) <>`
	42	+	`"data: [DONE]\n\n"`
	43	+
	44	+	`assert {:ok, decoder, events} = feed_in_pieces(stream, 7)`
	45	+	`assert {:ok, _decoder, final} = StreamDecoder.finish(decoder)`
	46	+
	47	+	`assert events ++ final == [`
	48	+	`{:response_started, "gen-r"},`
	49	+	`{:reasoning_delta, "Consider "},`
	50	+	`{:reasoning_delta, "the request."},`
	51	+	`{:text_delta, "Hi"},`
	52	+	`{:response_completed, "gen-r"}`
	53	+	`]`
	54	+	`end`
	55	+
	56	+	`test "carries an upstream reasoning_content spelling and ignores structured reasoning" do`
	57	+	`stream =`
	58	+	`frame(%{`
	59	+	`"id" => "gen-rc",`
	60	+	`"choices" => [%{"delta" => %{"reasoning_content" => "Thinking."}}]`
	61	+	`}) <>`
	62	+	`frame(%{`
	63	+	`"id" => "gen-rc",`
	64	+	`"choices" => [`
	65	+	`%{"delta" => %{"reasoning" => %{"detail" => "opaque"}, "content" => "Ok"}}`
	66	+	`]`
	67	+	`}) <>`
	68	+	`frame(%{"id" => "gen-rc", "choices" => [%{"delta" => %{}, "finish_reason" => "stop"}]})`
	69	+
	70	+	`assert {:ok, decoder, events} = feed_in_pieces(stream, 9)`
	71	+	`assert {:ok, _decoder, final} = StreamDecoder.finish(decoder)`
	72	+
	73	+	`assert events ++ final == [`
	74	+	`{:response_started, "gen-rc"},`
	75	+	`{:reasoning_delta, "Thinking."},`
	76	+	`{:text_delta, "Ok"},`
	77	+	`{:response_completed, "gen-rc"}`
	78	+	`]`
	79	+	`end`
	80	+
34	81		`test "accumulates tool-call fragments and emits them whole at the end" do`
35	82		`stream =`
36	83		`frame(%{`

    assert assistant_message.content == "Partial provider output."
  end

  test "reasoning deltas pass a conversation turn without effect" do
    turn = run_turn("reasoning-events-browser", "[reasoning]")

    assert turn.status == "completed"

    assistant_message =
      OpenAgents.Repo.get!(OpenAgents.Conversations.Message, turn.assistant_message_id)

    # The reply is only the text deltas; the reasoning neither fails the turn
    # nor leaks into the persisted reply.
    assert assistant_message.content == "Here is the reply."
  end

  test "provider cancellation becomes a typed durable cancellation" do
    turn = run_turn("cancelled-events-browser", "[provider-cancelled]")


96	96		`assert metered.usage["estimated_cost_microusd"] > 0`
97	97		`end`
98	98
	99	+	`test "a reasoning stream survives translation as delta.reasoning", %{conn: conn} do`
	100	+	`%{token: token} = grant("reasoning")`
	101	+
	102	+	`conn =`
	103	+	`post_chat(conn, token, %{`
	104	+	`"messages" => [%{"role" => "user", "content" => "[reasoning]"}]`
	105	+	`})`
	106	+
	107	+	`assert conn.status == 200`
	108	+
	109	+	`decoded =`
	110	+	`conn.resp_body`
	111	+	`\|> sse_events()`
	112	+	`\|> Enum.filter(&(&1 != "[DONE]"))`
	113	+	`\|> Enum.map(&Jason.decode!/1)`
	114	+
	115	+	`reasoning =`
	116	+	`decoded`
	117	+	`\|> Enum.flat_map(fn chunk ->`
	118	+	`get_in(chunk, ["choices", Access.at(0), "delta", "reasoning"]) \|> List.wrap()`
	119	+	`end)`
	120	+	`\|> Enum.join("")`
	121	+
	122	+	`assert reasoning == "Considering the request. Deciding on a reply."`
	123	+
	124	+	`# The reply text still arrives, in its own content deltas.`
	125	+	`text =`
	126	+	`decoded`
	127	+	`\|> Enum.flat_map(fn chunk ->`
	128	+	`get_in(chunk, ["choices", Access.at(0), "delta", "content"]) \|> List.wrap()`
	129	+	`end)`
	130	+	`\|> Enum.join("")`
	131	+
	132	+	`assert text == "Here is the reply."`
	133	+	`assert Enum.any?(decoded, &(get_in(&1, ["choices", Access.at(0), "finish_reason"]) == "stop"))`
	134	+	`end`
	135	+
	136	+	`test "a provider tool call reaches the caller as a tool_calls delta", %{conn: conn} do`
	137	+	`%{token: token} = grant("tool-out")`
	138	+
	139	+	`conn =`
	140	+	`post_chat(conn, token, %{`
	141	+	`"messages" => [%{"role" => "user", "content" => "[tool-loop]"}],`
	142	+	`"tools" => [`
	143	+	`%{`
	144	+	`"type" => "function",`
	145	+	`"function" => %{`
	146	+	`"name" => "recall_messages",`
	147	+	`"description" => "Recall messages",`
	148	+	`"parameters" => %{"type" => "object"}`
	149	+	`}`
	150	+	`}`
	151	+	`]`
	152	+	`})`
	153	+
	154	+	`assert conn.status == 200`
	155	+
	156	+	`decoded =`
	157	+	`conn.resp_body`
	158	+	`\|> sse_events()`
	159	+	`\|> Enum.filter(&(&1 != "[DONE]"))`
	160	+	`\|> Enum.map(&Jason.decode!/1)`
	161	+
	162	+	`[tool_call] =`
	163	+	`Enum.flat_map(decoded, fn chunk ->`
	164	+	`get_in(chunk, ["choices", Access.at(0), "delta", "tool_calls"]) \|\| []`
	165	+	`end)`
	166	+
	167	+	`assert tool_call["id"] == "call-tool-1"`
	168	+	`assert tool_call["type"] == "function"`
	169	+	`assert tool_call["function"]["name"] == "recall_messages"`
	170	+	`assert Jason.decode!(tool_call["function"]["arguments"]) == %{"query" => "quartz"}`
	171	+
	172	+	`assert Enum.any?(`
	173	+	`decoded,`
	174	+	`&(get_in(&1, ["choices", Access.at(0), "finish_reason"]) == "tool_calls")`
	175	+	`)`
	176	+	`end`
	177	+
99	178		`test "the model is pinned by the grant, not the request body", %{conn: conn} do`
100	179		`%{grant: grant, token: token} = grant("model-pin")`
101	180

176	255		`assert request.model_id == OpenAgents.Chat.OpenRouter.default_model()`
177	256		`end`
178	257
	258	+	`test "tool declarations, a replayed call, and its output reach the provider intact",`
	259	+	`%{conn: conn} do`
	260	+	`%{token: token} = grant("tool-fidelity", model_id: "ox-alpha")`
	261	+
	262	+	`conn =`
	263	+	`post_chat(conn, token, %{`
	264	+	`"messages" => [`
	265	+	`%{"role" => "user", "content" => "Read the file."},`
	266	+	`%{`
	267	+	`"role" => "assistant",`
	268	+	`"content" => "",`
	269	+	`"tool_calls" => [`
	270	+	`%{`
	271	+	`"id" => "call_read",`
	272	+	`"type" => "function",`
	273	+	`"function" => %{`
	274	+	`"name" => "read_file",`
	275	+	`"arguments" => ~s({"path":"a.txt"})`
	276	+	`}`
	277	+	`}`
	278	+	`]`
	279	+	`},`
	280	+	`%{"role" => "tool", "tool_call_id" => "call_read", "content" => "hello"}`
	281	+	`],`
	282	+	`"tools" => [`
	283	+	`%{`
	284	+	`"type" => "function",`
	285	+	`"function" => %{`
	286	+	`"name" => "read_file",`
	287	+	`"description" => "Read a file",`
	288	+	`"parameters" => %{"type" => "object"}`
	289	+	`}`
	290	+	`}`
	291	+	`]`
	292	+	`})`
	293	+
	294	+	`assert conn.status == 200`
	295	+	`assert_received {:recorded_request, "test.recording_provider", request}`
	296	+
	297	+	`assert [definition] = request.tool_definitions`
	298	+	`assert definition.name == "read_file"`
	299	+	`assert definition.input_schema == %{"type" => "object"}`
	300	+
	301	+	`# The assistant turn that carried only a tool call is not dropped from`
	302	+	`# the transcript, and its call travels with it.`
	303	+	`assert [`
	304	+	`%{role: "user", content: "Read the file."},`
	305	+	`%{role: "assistant", content: "", tool_calls: [call]}`
	306	+	`] = request.input`
	307	+
	308	+	`assert call == %{call_id: "call_read", name: "read_file", arguments: ~s({"path":"a.txt"})}`
	309	+
	310	+	`assert [output] = request.tool_outputs`
	311	+	`assert output.call_id == "call_read"`
	312	+	`assert output.output == %{"content" => "hello"}`
	313	+	`end`
	314	+
179	315		`test "a default grant stays on the default lane", %{conn: conn} do`
180	316		`%{token: token} = grant("default-lane")`
181	317

          1_000 -> {:error, {:provider_failed, "test_observer_timeout"}}
        end

      "[reasoning]" ->
        on_event.({:response_started, response_id})
        on_event.({:reasoning_delta, "Considering the request. "})
        on_event.({:reasoning_delta, "Deciding on a reply."})
        on_event.({:text_delta, "Here is the reply."})
        on_event.({:usage, %{"input_tokens" => 4, "output_tokens" => 8}})
        on_event.({:response_completed, response_id})
        :ok

      "[tool-loop]" ->
        emit_tool_request(
          on_event,

1157	1157		`Status: Current`
1158	1158
1159	1159		Conversation and web code depend on `OpenAgents.Providers.Provider`, not OpenAI
1160		-	`event shapes. Adapters emit typed OpenAgents-domain lifecycle, text, tool-call,`
1161		-	`usage, completion, failure, and cancellation events. A response ID is persisted`
	1160	+	`event shapes. Adapters emit typed OpenAgents-domain lifecycle, text, reasoning,`
	1161	+	`tool-call, usage, completion, failure, and cancellation events. A response ID is persisted`
1162	1162		`when announced, and matching explicit completion is required; stream closure`
1163	1163		`alone cannot produce a completed turn. Provider-specific events, credentials,`
1164	1164		`and raw errors never reach the receipt or browser. Response creation is a`

233	233		`end`
234	234		`end`
235	235
	236	+	`# Reasoning exists for the proxy's streaming callers; this runtime persists`
	237	+	`# only the reply, so the deltas pass without effect rather than failing the`
	238	+	`# turn as an unknown event.`
	239	+	`defp handle_provider_event({:reasoning_delta, delta}, state) when is_binary(delta),`
	240	+	`do: {:noreply, state}`
	241	+
236	242		`defp handle_provider_event({:usage, usage}, state) when is_map(usage) do`
237	243		`if valid_usage?(usage),`
238	244		`do: {:noreply, %{state \| current_usage: usage}},`

197	197		`end`
198	198		`end`
199	199
	200	+	`# Reasoning exists for the proxy's streaming callers; a job persists only`
	201	+	`# its report, so the deltas pass without effect rather than failing the job`
	202	+	`# as an unknown event.`
	203	+	`defp handle_provider_event({:reasoning_delta, delta}, state) when is_binary(delta),`
	204	+	`do: {:noreply, state}`
	205	+
200	206		`defp handle_provider_event({:usage, usage}, state) when is_map(usage) do`
201	207		`{:noreply, %{state \| current_usage: usage}}`
202	208		`end`

187	209		`[data(%{"choices" => [%{"index" => 0, "delta" => %{"content" => text}}]})]`
188	210		`end`
189	211
	212	+	`# Reasoning rides the OpenRouter chat-completions extension field —`
	213	+	# `delta.reasoning` alongside `delta.content` — the shape the CLI's
	214	+	`# OpenAI-compatible parser already expects from that vendor surface.`
	215	+	`defp event_chunks({:reasoning_delta, text}, _index) when text != "" do`
	216	+	`[data(%{"choices" => [%{"index" => 0, "delta" => %{"reasoning" => text}}]})]`
	217	+	`end`
	218	+
190	219		`defp event_chunks({:tool_call, tool_call}, index) do`
191	220		`[`
192	221		`data(%{`

73	73
74	74		`@doc false`
75	75		`def request_payload(request) do`
76		-	`input =`
77		-	`case request.tool_outputs do`
78		-	`[] -> request.input`
79		-	`outputs -> Enum.map(outputs, &tool_output/1)`
80		-	`end`
81		-
82	76		`%{`
83	77		`model: request.model_id,`
84	78		`instructions: request.instructions,`
85		-	`input: input,`
	79	+	`input: input_items(request),`
86	80		`tools: Enum.map(request.tool_definitions, &tool_definition/1),`
87	81		`parallel_tool_calls: false,`
88	82		`max_output_tokens: 4_096,`

89	89		`{:ok, state, [{:text_delta, delta}]}`
90	90		`end`
91	91
	92	+	`# The Responses API streams reasoning as its own event families: the`
	93	+	`# summary text most models expose, and the raw reasoning text some do.`
	94	+	`# Both become the one neutral reasoning event.`
	95	+	`defp decode_json(state, {:ok, %{"type" => type, "delta" => delta}})`
	96	+	`when type in ["response.reasoning_summary_text.delta", "response.reasoning_text.delta"] and`
	97	+	`is_binary(delta) and byte_size(delta) <= @maximum_delta_bytes do`
	98	+	`{:ok, state, [{:reasoning_delta, delta}]}`
	99	+	`end`
	100	+
92	101		`defp decode_json(`
93	102		`state,`
94	103		`{:ok,`

94	94		`\|> maybe_put_tools(request.tool_definitions)`
95	95		`end`
96	96
97		-	`# The proxy hands over the system text separately from the turns, and a tool`
98		-	`# output arrives without the assistant call that asked for it, because the`
99		-	`# calling harness flattens its own tool loop before it sends. So an output is`
100		-	`# carried as a labelled user message: it keeps the result in the transcript`
101		-	`# without claiming a call OpenRouter never saw.`
	97	+	`# The proxy hands over the system text separately from the turns. A tool`
	98	+	`# output whose assistant call is in the transcript is carried faithfully as`
	99	+	# a `tool` role message right after the assistant turn that called it. An
	100	+	`# output without that call — a harness that flattens its own tool loop`
	101	+	`# before it sends — is carried as a labelled user message instead: it keeps`
	102	+	`# the result in the transcript without claiming a call OpenRouter never saw.`
102	103		`defp messages(%Request{} = request) do`
103	104		`instructions =`
104	105		`case String.trim(request.instructions \|\| "") do`

106	107		`text -> [%{role: "system", content: text}]`
107	108		`end`
108	109
109		-	`turns = Enum.map(request.input, &%{role: role(&1.role), content: &1.content})`
110		-	`instructions ++ turns ++ Enum.map(request.tool_outputs, &tool_output/1)`
	110	+	`declared_call_ids =`
	111	+	`request.input`
	112	+	`\|> Enum.flat_map(&Map.get(&1, :tool_calls, []))`
	113	+	`\|> MapSet.new(& &1.call_id)`
	114	+
	115	+	`{matched, orphaned} =`
	116	+	`Enum.split_with(request.tool_outputs, &MapSet.member?(declared_call_ids, &1.call_id))`
	117	+
	118	+	`outputs_by_call_id = Map.new(matched, &{&1.call_id, &1})`
	119	+
	120	+	`turns = Enum.flat_map(request.input, &turn(&1, outputs_by_call_id))`
	121	+	`instructions ++ turns ++ Enum.map(orphaned, &orphaned_tool_output/1)`
	122	+	`end`
	123	+
	124	+	`defp turn(%{tool_calls: [_call \| _rest] = calls} = message, outputs_by_call_id) do`
	125	+	`assistant = %{`
	126	+	`role: "assistant",`
	127	+	`content: message.content,`
	128	+	`tool_calls: Enum.map(calls, &assistant_tool_call/1)`
	129	+	`}`
	130	+
	131	+	`results =`
	132	+	`calls`
	133	+	`\|> Enum.flat_map(fn call ->`
	134	+	`case Map.fetch(outputs_by_call_id, call.call_id) do`
	135	+	`{:ok, output} -> [tool_result(output)]`
	136	+	`:error -> []`
	137	+	`end`
	138	+	`end)`
	139	+
	140	+	`[assistant \| results]`
111	141		`end`
112	142
	143	+	`defp turn(message, _outputs_by_call_id),`
	144	+	`do: [%{role: role(message.role), content: message.content}]`
	145	+
113	146		`defp role(role) when role in ["system", "user", "assistant"], do: role`
114	147		`defp role(_role), do: "user"`
115	148
116		-	`defp tool_output(%ToolOutput{} = output) do`
	149	+	`defp assistant_tool_call(call) do`
	150	+	`%{`
	151	+	`id: call.call_id,`
	152	+	`type: "function",`
	153	+	`function: %{name: call.name, arguments: call.arguments}`
	154	+	`}`
	155	+	`end`
	156	+
	157	+	`defp tool_result(%ToolOutput{} = output) do`
	158	+	`%{`
	159	+	`role: "tool",`
	160	+	`tool_call_id: output.call_id,`
	161	+	`content: Jason.encode!(output.output)`
	162	+	`}`
	163	+	`end`
	164	+
	165	+	`defp orphaned_tool_output(%ToolOutput{} = output) do`
117	166		`%{`
118	167		`role: "user",`
119	168		`content: "Tool result for #{output.call_id}: #{Jason.encode!(output.output)}"`

137	137		`delta = if is_map(choice["delta"]), do: choice["delta"], else: %{}`
138	138		`state = if is_binary(choice["finish_reason"]), do: %{state \| terminal?: true}, else: state`
139	139
140		-	`with {:ok, text_events} <- text(delta["content"]),`
	140	+	`with {:ok, reasoning_events} <- reasoning(delta),`
	141	+	`{:ok, text_events} <- text(delta["content"]),`
141	142		`{:ok, state} <- tool_calls(state, delta["tool_calls"]) do`
142		-	`{:ok, state, text_events}`
	143	+	`{:ok, state, reasoning_events ++ text_events}`
143	144		`end`
144	145		`end`
145	146

153	154
154	155		`defp text(_content), do: {:error, :invalid_provider_event}`
155	156
	157	+	# OpenRouter normalizes a model's thinking stream to `delta.reasoning`;
	158	+	# some upstreams spell it `reasoning_content`. Either becomes the neutral
	159	+	`# reasoning event. A non-string value is ignored rather than refused: the`
	160	+	# `reasoning_details` object family is provider detail this proxy does not
	161	+	`# carry, and dropping it must not kill the text alongside it.`
	162	+	`defp reasoning(delta) do`
	163	+	`case delta["reasoning"] \|\| delta["reasoning_content"] do`
	164	+	`content when is_binary(content) and content != "" ->`
	165	+	`if byte_size(content) <= @maximum_delta_bytes,`
	166	+	`do: {:ok, [{:reasoning_delta, content}]},`
	167	+	`else: {:error, :invalid_provider_event}`
	168	+
	169	+	`_absent_or_structured ->`
	170	+	`{:ok, []}`
	171	+	`end`
	172	+	`end`
	173	+
156	174		`# A chat-completions tool call streams as fragments keyed by index: the name`
157	175		`# arrives once and the arguments arrive as a string built up over chunks, so`
158	176		`# the fragments are accumulated and emitted whole at the end of the stream.`

30	30		`@type t ::`
31	31		`{:response_started, String.t()}`
32	32		`\| {:text_delta, String.t()}`
	33	+	`\| {:reasoning_delta, String.t()}`
33	34		`\| {:tool_call, ToolCall.t()}`
34	35		`\| {:usage, map()}`
35	36		`\| {:response_completed, String.t()}`

4	4		`@enforce_keys [:model_id, :instructions, :input]`
5	5		`defstruct @enforce_keys ++ [tool_definitions: [], tool_outputs: [], previous_response_id: nil]`
6	6
7		-	`@type message :: %{role: String.t(), content: String.t()}`
	7	+	`@typedoc """`
	8	+	`A prior tool call an assistant turn carried, so a continuation request can`
	9	+	replay the call the outputs in `tool_outputs` answer. `arguments` is the raw
	10	+	`JSON string the provider produced; it is replayed, never interpreted.`
	11	+	`"""`
	12	+	`@type message_tool_call :: %{call_id: String.t(), name: String.t(), arguments: String.t()}`
	13	+
	14	+	`@type message :: %{`
	15	+	`:role => String.t(),`
	16	+	`:content => String.t(),`
	17	+	`optional(:tool_calls) => [message_tool_call()]`
	18	+	`}`
8	19		`@type t :: %__MODULE__{`
9	20		`model_id: String.t(),`
10	21		`instructions: String.t(),`

75	75		`defp input_message(%{"role" => "tool"}), do: []`
76	76
77	77		`defp input_message(message) do`
78		-	`case content_text(message) do`
79		-	`"" -> []`
80		-	`text -> [%{role: role(message), content: text}]`
	78	+	`tool_calls = message_tool_calls(message["tool_calls"])`
	79	+
	80	+	`case {content_text(message), tool_calls} do`
	81	+	`{"", []} -> []`
	82	+	`{text, []} -> [%{role: role(message), content: text}]`
	83	+	`# An assistant turn that called tools is part of the transcript even`
	84	+	`# when it carried no prose: dropping it would orphan the tool outputs`
	85	+	`# that answer it.`
	86	+	`{text, calls} -> [%{role: role(message), content: text, tool_calls: calls}]`
81	87		`end`
82	88		`end`
83	89
	90	+	`# The assistant tool calls a caller replays from its own history, in the`
	91	+	`# chat-completions shape it received them. Arguments stay the raw JSON`
	92	+	`# string; the proxy never interprets them.`
	93	+	`defp message_tool_calls(calls) when is_list(calls) do`
	94	+	`Enum.flat_map(calls, fn`
	95	+	`%{"id" => id, "function" => %{"name" => name} = function}`
	96	+	`when is_binary(id) and id != "" and is_binary(name) and name != "" ->`
	97	+	`[%{call_id: id, name: name, arguments: text(function["arguments"])}]`
	98	+
	99	+	`_invalid ->`
	100	+	`[]`
	101	+	`end)`
	102	+	`end`
	103	+
	104	+	`defp message_tool_calls(_calls), do: []`
	105	+
84	106		`defp tool_outputs(turns) do`
85	107		`turns`
86	108		`\|> Enum.filter(&(role(&1) == "tool"))`

67	67		`]`
68	68		`end`
69	69
	70	+	`test "decodes reasoning summary and reasoning text deltas as reasoning events" do`
	71	+	`stream =`
	72	+	`frame(%{"type" => "response.created", "response" => %{"id" => "resp_r"}}) <>`
	73	+	`frame(%{"type" => "response.reasoning_summary_text.delta", "delta" => "Weighing "}) <>`
	74	+	`frame(%{"type" => "response.reasoning_text.delta", "delta" => "options."}) <>`
	75	+	`frame(%{"type" => "response.output_text.delta", "delta" => "Done."}) <>`
	76	+	`frame(%{"type" => "response.completed", "response" => %{"id" => "resp_r"}})`
	77	+
	78	+	`assert {:ok, decoder, events} = feed_all([stream])`
	79	+	`assert {:ok, _decoder, final_events} = StreamDecoder.finish(decoder)`
	80	+
	81	+	`assert events ++ final_events == [`
	82	+	`{:response_started, "resp_r"},`
	83	+	`{:reasoning_delta, "Weighing "},`
	84	+	`{:reasoning_delta, "options."},`
	85	+	`{:text_delta, "Done."},`
	86	+	`{:response_completed, "resp_r"}`
	87	+	`]`
	88	+	`end`
	89	+
70	90		`test "rejects malformed JSON and a completed response with a changed ID" do`
71	91		`assert {:error, :invalid_provider_event} =`
72	92		`StreamDecoder.feed(StreamDecoder.new(), "data: {not-json}\n\n")`

80	80		`1_000 -> {:error, {:provider_failed, "test_observer_timeout"}}`
81	81		`end`
82	82
	83	+	`"[reasoning]" ->`
	84	+	`on_event.({:response_started, response_id})`
	85	+	`on_event.({:reasoning_delta, "Considering the request. "})`
	86	+	`on_event.({:reasoning_delta, "Deciding on a reply."})`
	87	+	`on_event.({:text_delta, "Here is the reply."})`
	88	+	`on_event.({:usage, %{"input_tokens" => 4, "output_tokens" => 8}})`
	89	+	`on_event.({:response_completed, response_id})`
	90	+	`:ok`
	91	+
83	92		`"[tool-loop]" ->`
84	93		`emit_tool_request(`
85	94		`on_event,`

28	28		`assert assistant_message.content == "Partial provider output."`
29	29		`end`
30	30
	31	+	`test "reasoning deltas pass a conversation turn without effect" do`
	32	+	`turn = run_turn("reasoning-events-browser", "[reasoning]")`
	33	+
	34	+	`assert turn.status == "completed"`
	35	+
	36	+	`assistant_message =`
	37	+	`OpenAgents.Repo.get!(OpenAgents.Conversations.Message, turn.assistant_message_id)`
	38	+
	39	+	`# The reply is only the text deltas; the reasoning neither fails the turn`
	40	+	`# nor leaks into the persisted reply.`
	41	+	`assert assistant_message.content == "Here is the reply."`
	42	+	`end`
	43	+
31	44		`test "provider cancellation becomes a typed durable cancellation" do`
32	45		`turn = run_turn("cancelled-events-browser", "[provider-cancelled]")`
33	46