Meter inference honestly: no cost without a price, and never a zero

3445eed1c4fd · AtlantisPleb · · parent e4e6d71049aa

Meter inference honestly: no cost without a price, and never a zero

Boots the metered offering behind the coder (#200) on the half that has to
be true before any of it: usage is recorded per call, attributed to the
grant that bought it, readable on the thread and over the API — and a cost
is reported only where a price exists.

The gap that made this urgent is `gpt-5.6-luna`. It is the lane the coder
actually runs on and the catalog declares no rates for it, so every surface
that read a missing `estimated_cost_microusd` as zero was showing `$0.00`
beside the account's largest real spend, in the same typeface as a figure
somebody measured. The thread page did. `grant.remaining.cost_microusd`
subtracted that zero from the ceiling and published the whole ceiling as
headroom. `Threads.spend/1` summed it into a total that looked complete.

`OpenAgents.Inference.Pricing` is now the one authority. Every metered
record stamps `pricing_id`, naming the rate table it was priced against,
and the id resolves to one of three bases: `declared` (the operator entered
the provider's published rates — the only billable basis), `provisional`
(a working figure, or a table this deployment can no longer dereference),
`unpriced` (no rates, no cost key, `cost/1` answers nil). Every rate in
`config/config.exs` today says `source: :placeholder`, because none was
read off a price page; getting real rates is an owner action and nothing
here guesses one.

Downstream, absence survives to the reader. `GET /api/v1/models` publishes
`pricing_basis` on every entry beside `availability`, so a caller knows
what a lane costs and whether to trust the figure before it spends.
`Threads.spend/1` refuses to total a session that touched an unpriced lane:
`cost.microusd` is null, `cost.unpriced_models` names why, and
`cost.priced_microusd` still reports what was measured. The thread page
shows the word `Unpriced`, labels provisional figures as working numbers
rather than bills, and renders a null ceiling as unbounded instead of
blank. `Credit.spent/1` is documented as the floor it is, with
`unpriced_calls/1` and `balance/1` publishing what it cannot see.

No new table, route, or data family — the meter stays the grant usage
records the leaderboard already consumes, with no second accounting.

INVARIANTS.md gains METER-001.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KnhfrafYx5ZGaMbzZEJQ2d
Co-Authored-By
Claude Opus 5 (1M context) <noreply@anthropic.com>

Deploy story

What this commit did to the running system — joined from the forge receipt chain, the part a commit page elsewhere cannot show.

pushed
by user · WAL seq 380 · 2026-08-25T14:30:04.464168Z

Changed files

  • modified INVARIANTS.md
  • modified config/config.exs
  • modified docs/api-authentication.md
  • modified docs/taxonomy.md
  • modified lib/openagents/inference.ex
  • modified lib/openagents/inference/credit.ex
  • modified lib/openagents/inference/models.ex
  • added lib/openagents/inference/pricing.ex
  • modified lib/openagents/threads.ex
  • modified lib/openagents_web/controllers/api_extension_controller.ex
  • modified lib/openagents_web/controllers/thread_controller.ex
  • modified lib/openagents_web/live/thread_show_live.ex
  • modified test/openagents/inference/credit_test.exs
  • added test/openagents/inference/pricing_test.exs
  • modified test/openagents/inference_test.exs
  • modified test/openagents/threads_test.exs
  • modified test/openagents_web/controllers/model_catalog_controller_test.exs
  • modified test/openagents_web/controllers/thread_controller_test.exs
  • modified test/openagents_web/live/thread_show_live_test.exs

Diff

19 files changed, +1038 -56

INVARIANTS.md modified +56

@@ -1258,6 +1258,61 @@ Evidence: `OpenAgents.Inference.Models`, `OpenAgentsWeb.ModelCatalogController`,

1258 1258
`OpenAgentsWeb.InferenceProxyControllerTest`, and
1259 1259
`OpenAgentsWeb.ThreadControllerTest`.
1260 1260
1261
### METER-001 — A cost is reported only where a price exists, and never as zero
1262
1263
Status: Current
1264
1265
Metering that prices some lanes at zero is worse than no metering: it reports a
1266
number, and the number is wrong. `gpt-5.6-luna` is the lane the coder actually
1267
runs on and this deployment has never been given its rates, so a surface that
1268
read a missing cost as zero would have shown `$0.00` beside the account's
1269
largest real spend — in the same typeface as a figure somebody measured.
1270
1271
So every metered usage record says on whose authority it was priced.
1272
`OpenAgents.Inference.Pricing` stamps `pricing_id` on every record
1273
`OpenAgents.Inference.record_usage/2` writes, and the id resolves to one of
1274
three bases:
1275
1276
- `declared` — the operator entered the provider's published rates. This is the
1277
  only basis anything may bill from (`Pricing.billable?/1`).
1278
- `provisional` — the deployment carries rates that were written to make the
1279
  system run, or rates whose table it can no longer dereference. A cost is
1280
  computed and labelled; nothing may bill from it.
1281
- `unpriced` — no rates at all. No `estimated_cost_microusd` is written,
1282
  `Pricing.cost/1` answers `nil`, and no reader substitutes a zero.
1283
1284
Concretely:
1285
1286
- `GET /api/v1/models` publishes `pricing_basis` on every entry beside
1287
  `availability`, so a caller reads what a lane will cost and whether that
1288
  figure can be trusted **before** it spends. An unpriced model publishes no
1289
  `pricing` block at all.
1290
- `OpenAgents.Threads.spend/1` refuses to total a session that touched an
1291
  unpriced lane: `cost.microusd` is `nil`, `cost.unpriced_models` names the
1292
  lanes that made it `nil`, and `cost.priced_microusd` still reports what was
1293
  measured so nothing is discarded. `GET /api/v1/threads/{id}` publishes the
1294
  same shape, nulls included.
1295
- The thread page shows the word `Unpriced` rather than a dollar figure, and
1296
  labels a provisional figure as a working number rather than a bill.
1297
- `OpenAgents.Inference.Credit.spent/1` is a floor, not a total, while
1298
  `unpriced_calls/1` is above zero; `balance/1` publishes `complete?` so no
1299
  reader shows a balance as whole when it is not.
1300
1301
An unpriced lane is not a free lane and not an error. It is the deployment
1302
saying it does not know what a call cost, which is a different fact from the
1303
call having cost nothing, and the distinction survives to every read surface.
1304
A `max_cost_microusd` ceiling therefore cannot bound an unpriced grant — it can
1305
only stop spend it can measure — so such a grant is bounded by its call and
1306
token ceilings, by the account's admission cap, and by revocation. Turning an
1307
unpriced lane into a number is an owner action: enter the provider's real rates
1308
in `config :openagents, :model_catalog` and set `source: :declared` in the same
1309
edit. No code here may guess one.
1310
1311
Evidence: `OpenAgents.Inference.Pricing`, `OpenAgents.Inference.PricingTest`,
1312
`OpenAgents.Inference.CreditTest`, `OpenAgents.ThreadsTest`,
1313
`OpenAgentsWeb.ModelCatalogControllerTest`,
1314
`OpenAgentsWeb.ThreadControllerTest`, and `OpenAgentsWeb.ThreadShowLiveTest`.
1315
1261 1316
## Durable effects
1262 1317
1263 1318
### EFFECT-001 — An effect commits with the intent that asked for it, and is delivered under lease

@@ -5405,6 +5460,7 @@ contract; the invariant prose above defines the assertion, not the filename.

5405 5460
| PROVENANCE-001 | `test/openagents/turn_provenance_test.exs` |
5406 5461
| PROVIDER-001 | `test/openagents/providers/provider_contract_test.exs`, `test/openagents/turn_provider_events_test.exs`, `test/openagents/dependency_boundary_test.exs` |
5407 5462
| PROVIDER-002 | `test/openagents/inference/models_test.exs`, `test/openagents_web/controllers/model_catalog_controller_test.exs`, `test/openagents_web/controllers/inference_proxy_controller_test.exs`, `test/openagents_web/controllers/thread_controller_test.exs` |
5463
| METER-001 | `test/openagents/inference/pricing_test.exs`, `test/openagents/inference/credit_test.exs`, `test/openagents/threads_test.exs`, `test/openagents_web/controllers/model_catalog_controller_test.exs`, `test/openagents_web/controllers/thread_controller_test.exs`, `test/openagents_web/live/thread_show_live_test.exs` |
5408 5464
| EFFECT-001 | `test/openagents/effects_test.exs`, `test/openagents/effects/work_launch_test.exs` |
5409 5465
| EFFECT-002 | `test/openagents/effects_test.exs`, `test/openagents/effects/work_launch_test.exs` |
5410 5466
| TOOL-001 | `test/openagents/tools/registry_and_runner_test.exs` |
config/config.exs modified +31 -9

@@ -128,9 +128,22 @@ config :openagents,

128 128
  #
129 129
  # A model may declare a `pricing` map with `input_per_million_tokens`,
130 130
  # `output_per_million_tokens`, and optionally `cached_input_per_million_tokens`.
131
  # The values below are placeholders that make the existing test suite pass;
132
  # the operator must replace them with real provider rates before accepting
133
  # any spend. A model with no `pricing` key records no estimated cost.
131
  # A pricing map also names itself: `id` is the rate table every usage record
132
  # priced against it stamps as `pricing_id`, and `source` is where the rates
133
  # came from — `:declared` for a provider's published rates, anything else
134
  # (including omitting it) for a working figure. `OpenAgents.Inference.Pricing`
135
  # reads that word, and only `:declared` is billable (METER-001).
136
  #
137
  # Every rate below is a placeholder written to make the system run. None of
138
  # them was read off a provider's price page, so all of them say `:placeholder`
139
  # and nothing may bill from a cost they produced. Replacing them is an owner
140
  # action: enter the provider's real rates and set `source: :declared` in the
141
  # same edit.
142
  #
143
  # A model with no `pricing` key records no estimated cost at all — not a
144
  # zero. `gpt-5.6-luna` is that model and it is the lane the coder runs on, so
145
  # the account's metered spend is a floor rather than a total until it has
146
  # rates. Every read surface says `unpriced` rather than `$0.00`.
134 147
  #
135 148
  # Gemini 3.7 Flash leads, so it is what a caller that names none gets: fast,
136 149
  # a million tokens of context, and steady enough to hold a conversation.

@@ -154,10 +167,12 @@ config :openagents,

154 167
      provider_model: "google/gemini-3.7-flash",
155 168
      context_window: 1_048_576,
156 169
      max_output: 65_536,
157
      # Placeholder: the operator must set real provider rates before accepting
158
      # any spend. The cached-input rate is optional and should be omitted if
159
      # the provider does not offer one.
170
      # Placeholder: the operator must set real provider rates and flip `source`
171
      # to `:declared` before anything bills from this. The cached-input rate is
172
      # optional and should be omitted if the provider does not offer one.
160 173
      pricing: %{
174
        id: "placeholder.gemini-3.7-flash.v1",
175
        source: :placeholder,
161 176
        input_per_million_tokens: 1_250_000,
162 177
        output_per_million_tokens: 10_000_000,
163 178
        cached_input_per_million_tokens: 100_000

@@ -175,9 +190,12 @@ config :openagents,

175 190
      # with a real task spent the whole budget reasoning and returned an empty
176 191
      # 200 after three minutes, which read as the proxy having failed.
177 192
      max_output: 64_000,
178
      # Placeholder: the operator must set real provider rates before accepting
179
      # any spend. This entry does not declare a cached-input rate.
193
      # Placeholder: the operator must set real provider rates and flip `source`
194
      # to `:declared` before anything bills from this. This entry does not
195
      # declare a cached-input rate.
180 196
      pricing: %{
197
        id: "placeholder.ox-alpha.v1",
198
        source: :placeholder,
181 199
        input_per_million_tokens: 500_000,
182 200
        output_per_million_tokens: 2_000_000
183 201
      }

@@ -189,7 +207,11 @@ config :openagents,

189 207
      context_window: 272_000,
190 208
      max_output: 4_096
191 209
      # This entry deliberately omits `pricing`, so a grant pinned to it records
192
      # no estimated cost rather than a made-up zero.
210
      # no estimated cost rather than a made-up zero. Its usage records stamp
211
      # `pricing_id: "unpriced"`, `Threads.spend/1` refuses to total a session
212
      # that touched it, and the thread page shows the word instead of a
213
      # figure. Giving this lane real rates is the single highest-value edit in
214
      # this file: it is where the coder's spend actually goes.
193 215
    }
194 216
  ],
195 217
  gemini_api_key: nil,
docs/api-authentication.md modified +9

@@ -442,6 +442,15 @@ scoped to a conversation and optional paired computer, expires, is

442 442
generation-fenced and revocable, and has call, token, and cost ceilings. It is
443 443
not an OpenAI credential and cannot select a model outside the grant.
444 444
445
A cost ceiling only bounds spend the deployment can price. Read
446
`pricing_basis` on each entry of `GET /api/v1/models` before you spend:
447
`declared` is billable, `provisional` is a working figure, and `unpriced`
448
means no cost is reported for that lane at all. On a thread,
449
`spend.cost.microusd` and `grant.remaining.cost_microusd` are `null` whenever
450
an unpriced lane contributed, and `spend.cost.unpriced_models` names the lanes
451
that made them null. A client must handle the `null` rather than render a zero
452
it was never given (METER-001).
453
445 454
`OpenAgentsWeb.RouteAuthority.inventory/0` is the executable inventory for
446 455
HTTP routes and endpoint sockets. The test gate fails when a new route does not
447 456
resolve to one of the admitted authority classes with a principal and scope.
docs/taxonomy.md modified +17

@@ -343,6 +343,23 @@ without paying for it (THREAD-001, issue #243). The CLI that stops writing to

343 343
the conversation is still to come; see the audit in
344 344
`docs/2026-08-23-thread-primitive-audit.md`.
345 345
346
**Pricing basis** — one word saying whether a reported cost can be trusted, on
347
every catalog entry and every metered usage record (METER-001). `declared`
348
means the operator entered the provider's published rates and the figure is
349
billable. `provisional` means the deployment carries rates that were written to
350
make the system run, or rates whose table it can no longer dereference; the
351
figure is a working number and nothing bills from it. `unpriced` means there
352
are no rates at all. The authority is `OpenAgents.Inference.Pricing`, and every
353
record names its rate table in `pricing_id`.
354
355
**Unpriced** — the deployment does not know what a call cost. It is not the
356
same word as free, and it is never rendered as `$0.00`: an unpriced record
357
carries no `estimated_cost_microusd`, `OpenAgents.Threads.spend/1` reports a
358
null total and names the lanes that made it null, and the thread page shows the
359
word. `gpt-5.6-luna` is unpriced today, which is why the distinction is load
360
bearing rather than pedantic — it is the lane the coder runs on. Giving a lane
361
rates is an owner action, never an inference.
362
346 363
**Thread transcript** — the prompts, responses, tool activity, code changes,
347 364
and metadata that explain what happened in a thread. Authoritative copies live
348 365
in PostgreSQL (messages, tool steps, receipts), not on a Git branch. The
lib/openagents/inference.ex modified +10 -26

@@ -16,7 +16,7 @@ defmodule OpenAgents.Inference do

16 16
  """
17 17
18 18
  import Ecto.Query
19
  alias OpenAgents.Inference.{Grant, Models}
19
  alias OpenAgents.Inference.{Grant, Models, Pricing}
20 20
  alias OpenAgents.Machines.Machine
21 21
  alias OpenAgents.Repo
22 22

@@ -346,6 +346,14 @@ defmodule OpenAgents.Inference do

346 346
347 347
  # ── budget ──────────────────────────────────────────────────────────────
348 348
349
  # A cost ceiling can only stop spend it can measure. An unpriced model writes
350
  # no `estimated_cost_microusd`, so `max_cost_microusd` never fires for it and
351
  # such a grant is bounded by its call and token ceilings, by the account's
352
  # admission cap, and by revocation — not by money. That is a consequence of
353
  # not knowing the rates rather than a decision to let the lane run free, and
354
  # it is why `Threads.spend/1` and `Credit.unpriced_calls/1` publish the
355
  # unpriced calls instead of folding them into a total (METER-001).
356
349 357
  @doc false
350 358
  def over_budget?(%Grant{} = grant) do
351 359
    tokens = integer(grant.usage["total_tokens"])

@@ -400,7 +408,7 @@ defmodule OpenAgents.Inference do

400 408
401 409
    merged
402 410
    |> Map.put("total_tokens", derived_total(existing, merged))
403
    |> put_cost(model_id)
411
    |> Pricing.price(model_id)
404 412
    |> Map.put("schema", @usage_schema)
405 413
  end
406 414

@@ -414,30 +422,6 @@ defmodule OpenAgents.Inference do

414 422
    end
415 423
  end
416 424
417
  defp put_cost(merged, model_id) do
418
    case Models.fetch(model_id) do
419
      {:ok, %{pricing: %{input_per_million_tokens: i, output_per_million_tokens: o} = pricing}} ->
420
        input = integer(merged["input_tokens"])
421
        output = integer(merged["output_tokens"])
422
        cache_read = integer(merged["cache_read_input_tokens"])
423
        cache_write = integer(merged["cache_write_input_tokens"])
424
        cached_rate = Map.get(pricing, :cached_input_per_million_tokens, i)
425
426
        # Cached read tokens are split from the rest of the input and priced at
427
        # the cached rate where one is declared. Cache write tokens are charged
428
        # as regular input because they are not a cached read.
429
        uncached = max(0, input - cache_read) + cache_write
430
431
        cost =
432
          uncached * i + cache_read * cached_rate + output * o
433
434
        Map.put(merged, "estimated_cost_microusd", div(cost, 1_000_000))
435
436
      _ ->
437
        merged
438
    end
439
  end
440
441 425
  defp normalize_usage(usage) do
442 426
    Enum.reduce(@cost_fields, %{}, fn field, acc ->
443 427
      case raw_value(usage, field) do
lib/openagents/inference/credit.ex modified +66 -2

@@ -28,6 +28,7 @@ defmodule OpenAgents.Inference.Credit do

28 28
29 29
  alias OpenAgents.Conversations.Visitor
30 30
  alias OpenAgents.Inference.Grant
31
  alias OpenAgents.Inference.Pricing
31 32
  alias OpenAgents.Repo
32 33
33 34
  @doc """

@@ -50,7 +51,14 @@ defmodule OpenAgents.Inference.Credit do

50 51
  @spec visitor_allowance() :: non_neg_integer()
51 52
  def visitor_allowance, do: setting(:visitor_credit_microusd, 2_000_000)
52 53
53
  @doc "What every grant this account has held has metered, in microUSD."
54
  @doc """
55
  What every grant this account has held has metered, in microUSD.
56
57
  Only priced calls contribute, so this is a floor rather than a total whenever
58
  `unpriced_calls/1` is above zero. It stays a bare integer because the grant
59
  ceiling has to be a number — a ceiling of `nil` would admit unbounded
60
  spend — and the reader that wants the honest picture asks `balance/1`.
61
  """
54 62
  @spec spent(String.t()) :: non_neg_integer()
55 63
  def spent(visitor_id) when is_binary(visitor_id) do
56 64
    Repo.one(

@@ -69,12 +77,68 @@ defmodule OpenAgents.Inference.Credit do

69 77
    )
70 78
  end
71 79
72
  @doc "What is left of this account's credit, in microUSD. Never negative."
80
  @doc """
81
  What is left of this account's credit, in microUSD. Never negative.
82
83
  This is a ceiling rather than a balance, and the difference matters while any
84
  lane is unpriced. A call on a model with no declared rates writes no cost, so
85
  it draws nothing down here: the remainder is what the account may still be
86
  *ceiled* at, not what it has left to spend in the world. `unpriced_calls/1`
87
  is how much of the account's real spend this figure cannot see (METER-001).
88
  """
73 89
  @spec remaining(String.t()) :: non_neg_integer()
74 90
  def remaining(visitor_id) when is_binary(visitor_id) do
75 91
    max(allowance(visitor_id) - spent(visitor_id), 0)
76 92
  end
77 93
94
  @doc """
95
  How many of this account's metered calls carry no price.
96
97
  `spent/1` is a floor while this is above zero, and saying so is the whole
98
  reason this function exists. A surface that showed a balance without it would
99
  be reporting an account as barely touched while its coder ran all day on a
100
  lane nobody entered rates for.
101
  """
102
  @spec unpriced_calls(String.t()) :: non_neg_integer()
103
  def unpriced_calls(visitor_id) when is_binary(visitor_id) do
104
    Repo.all(
105
      from grant in Grant,
106
        where: grant.owner_visitor_id == ^visitor_id and grant.call_count > 0,
107
        select: {grant.model_id, grant.usage, grant.call_count}
108
    )
109
    |> Enum.filter(fn {_model, usage, _calls} ->
110
      Pricing.usage_basis(usage) == Pricing.unpriced()
111
    end)
112
    |> Enum.map(fn {_model, _usage, calls} -> calls end)
113
    |> Enum.sum()
114
  end
115
116
  @doc """
117
  The account's credit as one readable fact, including what it cannot see.
118
119
  `complete?` is false whenever an unpriced call has been metered, which is the
120
  signal a caller needs before it renders `spent_microusd` as though it were
121
  the account's whole spend.
122
  """
123
  @spec balance(String.t()) :: %{
124
          allowance_microusd: non_neg_integer(),
125
          spent_microusd: non_neg_integer(),
126
          remaining_microusd: non_neg_integer(),
127
          unpriced_calls: non_neg_integer(),
128
          complete?: boolean()
129
        }
130
  def balance(visitor_id) when is_binary(visitor_id) do
131
    unpriced = unpriced_calls(visitor_id)
132
133
    %{
134
      allowance_microusd: allowance(visitor_id),
135
      spent_microusd: spent(visitor_id),
136
      remaining_microusd: remaining(visitor_id),
137
      unpriced_calls: unpriced,
138
      complete?: unpriced == 0
139
    }
140
  end
141
78 142
  defp signed_in?(visitor_id) do
79 143
    Repo.exists?(from v in Visitor, where: v.id == ^visitor_id and not is_nil(v.user_id))
80 144
  end
lib/openagents/inference/models.ex modified +12

@@ -1,5 +1,6 @@

1 1
defmodule OpenAgents.Inference.Models do
2 2
  alias OpenAgents.Inference.Health
3
  alias OpenAgents.Inference.Pricing
3 4
4 5
  @moduledoc """
5 6
  The typed model catalog: every model this deployment serves, and the

@@ -173,6 +174,14 @@ defmodule OpenAgents.Inference.Models do

173 174
  about how the server is wired. Pricing is exposed only when the deployment
174 175
  has declared rates for a model; an unpriced model has no `pricing` key so it
175 176
  is not read as zero before spend.
177
178
  Absence is a weak signal, though — a client that forgets to check for the key
179
  reads a missing price as no price rather than as an unknown one. So every
180
  entry also carries `pricing_basis`, one word alongside `availability`:
181
  `declared` where the operator entered the provider's published rates,
182
  `provisional` where the rates are a working figure nothing may bill from, and
183
  `unpriced` where there are none. A caller can read what a lane will cost, and
184
  whether that figure can be trusted, before it spends anything (METER-001).
176 185
  """
177 186
  @spec catalog() :: [map()]
178 187
  def catalog do

@@ -185,6 +194,7 @@ defmodule OpenAgents.Inference.Models do

185 194
        "context_window" => model.context_window,
186 195
        "max_output" => model.max_output,
187 196
        "availability" => availability(model),
197
        "pricing_basis" => Pricing.basis_of(model.pricing),
188 198
        "default" => model.id == default_id
189 199
      }
190 200

@@ -200,6 +210,8 @@ defmodule OpenAgents.Inference.Models do

200 210
201 211
  defp public_pricing(pricing) do
202 212
    base = %{
213
      "id" => Pricing.pricing_id(pricing),
214
      "basis" => Pricing.basis_of(pricing),
203 215
      "input_per_million_tokens" => pricing.input_per_million_tokens,
204 216
      "output_per_million_tokens" => pricing.output_per_million_tokens
205 217
    }
lib/openagents/inference/pricing.ex added +206

@@ -0,0 +1,206 @@

1
defmodule OpenAgents.Inference.Pricing do
2
  @moduledoc """
3
  What a metered call cost, and on whose authority.
4
5
  Metering that prices some lanes at zero is worse than no metering: it reports
6
  a number, and the number is wrong. `gpt-5.6-luna` is the lane the coder
7
  actually runs on and this deployment has never been told its rates, so a
8
  surface that read a missing cost as zero would have shown `$0.00` beside a
9
  session that spent real money — and shown it in the same typeface as a figure
10
  that was measured.
11
12
  So a cost is never a bare integer here. Every metered usage record carries a
13
  `pricing_id` naming the rate table it was priced against, and the id resolves
14
  to one of three bases:
15
16
    * `declared` — the operator entered the provider's published rates. This is
17
      the only basis anything may bill from (`billable?/1`).
18
    * `provisional` — the deployment carries rates that were written to make
19
      the system run rather than read off a provider's price page, or rates
20
      whose table is unnamed. A cost is computed and labelled; no bill may
21
      derive from it.
22
    * `unpriced` — no rates at all. No `estimated_cost_microusd` is written,
23
      `cost/1` answers `nil`, and every reader shows the word rather than a
24
      zero.
25
26
  `unpriced` is not an error state and it is not a free lane. It is the
27
  deployment saying it does not know what a call cost, which is a different
28
  fact from the call having cost nothing, and the distinction survives all the
29
  way to the read surfaces. Turning it into a number is an owner action —
30
  entering real rates in `config :openagents, :model_catalog` — not something
31
  this module may guess at.
32
33
  The `pricing_id` convention is `OpenAgents.Voice.Usage`'s, so
34
  `OpenAgents.DataRights.AtifExport` reads inference usage and voice usage with
35
  the same rule. The one difference is deliberate: voice writes a zero cost
36
  beside `pricing_id: "unpriced"`, and inference writes no cost key at all.
37
  """
38
39
  alias OpenAgents.Inference.Models
40
41
  @unpriced "unpriced"
42
  @unattributed "unattributed"
43
44
  @doc "The `pricing_id` written for a lane with no declared rates."
45
  @spec unpriced() :: String.t()
46
  def unpriced, do: @unpriced
47
48
  @doc """
49
  The basis of a catalog pricing map: `declared`, `provisional`, or `unpriced`.
50
51
  A pricing map that does not say where its rates came from is `provisional`,
52
  not `declared`. Failing closed is the point: a rate somebody added without
53
  recording its source is exactly the rate nothing should bill from.
54
  """
55
  @spec basis_of(map() | nil) :: String.t()
56
  def basis_of(nil), do: @unpriced
57
  def basis_of(%{source: :declared}), do: "declared"
58
  def basis_of(%{}), do: "provisional"
59
60
  @doc "The basis for a model, by catalog id or resolved model."
61
  @spec basis(map() | String.t() | nil) :: String.t()
62
  def basis(%{pricing: pricing}), do: basis_of(pricing)
63
64
  def basis(model_id) do
65
    case Models.fetch(model_id) do
66
      {:ok, model} -> basis_of(model.pricing)
67
      :error -> @unpriced
68
    end
69
  end
70
71
  @doc """
72
  The id of the rate table a model is priced against.
73
74
  `unpriced` where the catalog declares no rates, `unattributed` where it
75
  declares rates without naming their table.
76
  """
77
  @spec pricing_id(map() | nil) :: String.t()
78
  def pricing_id(nil), do: @unpriced
79
  def pricing_id(%{} = pricing), do: Map.get(pricing, :id) || @unattributed
80
81
  @doc "The id of the rate table this model is priced against, by catalog id."
82
  @spec pricing_id_for(String.t() | nil) :: String.t()
83
  def pricing_id_for(model_id) do
84
    case Models.fetch(model_id) do
85
      {:ok, model} -> pricing_id(model.pricing)
86
      :error -> @unpriced
87
    end
88
  end
89
90
  @doc """
91
  Price a merged usage map against a model's declared rates.
92
93
  Always stamps `pricing_id`, so the stored record says on whose authority it
94
  was priced — or that it was not priced at all. `estimated_cost_microusd` is
95
  written only where rates exist, because a zero there would be read as a
96
  measurement.
97
98
  Cached read tokens are split out of the input and charged at the cached rate
99
  where the model declares one. Cache write tokens are charged as regular
100
  input, because writing a cache is not reading one.
101
  """
102
  @spec price(map(), String.t() | nil) :: map()
103
  def price(usage, model_id) when is_map(usage) do
104
    pricing =
105
      case Models.fetch(model_id) do
106
        {:ok, model} -> model.pricing
107
        :error -> nil
108
      end
109
110
    usage
111
    |> Map.put("pricing_id", pricing_id(pricing))
112
    |> put_cost(pricing)
113
  end
114
115
  defp put_cost(
116
         usage,
117
         %{input_per_million_tokens: input_rate, output_per_million_tokens: out_rate} = pricing
118
       ) do
119
    input = integer(usage["input_tokens"])
120
    output = integer(usage["output_tokens"])
121
    cache_read = integer(usage["cache_read_input_tokens"])
122
    cache_write = integer(usage["cache_write_input_tokens"])
123
    cached_rate = Map.get(pricing, :cached_input_per_million_tokens, input_rate)
124
125
    uncached = max(0, input - cache_read) + cache_write
126
    cost = uncached * input_rate + cache_read * cached_rate + output * out_rate
127
128
    Map.put(usage, "estimated_cost_microusd", div(cost, 1_000_000))
129
  end
130
131
  defp put_cost(usage, _no_rates), do: usage
132
133
  @doc """
134
  What a stored usage record cost, or `nil` where the deployment does not know.
135
136
  `nil` is the whole point of this function. Callers that need a number must
137
  decide what to do without one instead of being handed a zero that reads like
138
  a measurement.
139
  """
140
  @spec cost(map() | nil) :: integer() | nil
141
  def cost(%{} = usage) do
142
    case Map.get(usage, "estimated_cost_microusd") do
143
      value when is_integer(value) -> value
144
      value when is_float(value) -> trunc(value)
145
      _absent -> nil
146
    end
147
  end
148
149
  def cost(_usage), do: nil
150
151
  @doc """
152
  The basis of a stored usage record.
153
154
  A record written before `pricing_id` existed carries a cost and no table
155
  name. That is `provisional` — a figure whose rates cannot be dereferenced is
156
  precisely a figure nothing may bill from — and a record with neither is
157
  `unpriced`.
158
  """
159
  @spec usage_basis(map() | nil) :: String.t()
160
  def usage_basis(%{} = usage) do
161
    case Map.get(usage, "pricing_id") do
162
      @unpriced -> @unpriced
163
      id when is_binary(id) -> basis_of_id(id, usage)
164
      _absent -> if is_nil(cost(usage)), do: @unpriced, else: "provisional"
165
    end
166
  end
167
168
  def usage_basis(_usage), do: @unpriced
169
170
  defp basis_of_id(id, usage) do
171
    cond do
172
      is_nil(cost(usage)) -> @unpriced
173
      declared_id?(id) -> "declared"
174
      true -> "provisional"
175
    end
176
  end
177
178
  # A stored record names its table, and whether that table was declared is a
179
  # property of the catalog rather than of the string. A table that has since
180
  # been removed from the catalog is read as provisional: it was priced against
181
  # something this deployment can no longer dereference, and NO BILL WITHOUT A
182
  # DEREFERENCEABLE USAGE RECORD is the contract.
183
  defp declared_id?(id) do
184
    Enum.any?(Models.all(), fn model ->
185
      pricing_id(model.pricing) == id and basis_of(model.pricing) == "declared"
186
    end)
187
  end
188
189
  @doc """
190
  Whether a stored usage record may be billed from.
191
192
  Only a `declared` basis qualifies. A provisional cost is a working figure and
193
  an unpriced call is an unknown; billing from either is the failure this
194
  module exists to prevent.
195
  """
196
  @spec billable?(map() | nil) :: boolean()
197
  def billable?(usage), do: usage_basis(usage) == "declared"
198
199
  @doc "Whether this record carries a cost at all."
200
  @spec priced?(map() | nil) :: boolean()
201
  def priced?(usage), do: not is_nil(cost(usage))
202
203
  defp integer(value) when is_integer(value), do: value
204
  defp integer(value) when is_float(value), do: trunc(value)
205
  defp integer(_value), do: 0
206
end
lib/openagents/threads.ex modified +72 -6

@@ -75,7 +75,7 @@ defmodule OpenAgents.Threads do

75 75
  alias OpenAgents.Conversations
76 76
  alias OpenAgents.Conversations.Visitor
77 77
  alias OpenAgents.Inference
78
  alias OpenAgents.Inference.{Credit, Grant, Models}
78
  alias OpenAgents.Inference.{Credit, Grant, Models, Pricing}
79 79
  alias OpenAgents.Issues.Issue
80 80
  alias OpenAgents.Repo
81 81
  alias OpenAgents.Threads.Event

@@ -750,11 +750,26 @@ defmodule OpenAgents.Threads do

750 750
  Absent stays absent. A dimension no provider reported is not summed into
751 751
  existence as a zero, because a zero reads as a measurement and this is the
752 752
  absence of one (#220).
753
754
  `:cost` is the same rule applied to money, and it is stricter than the sum in
755
  `:usage` because money is what somebody eventually pays. A session that
756
  touched a lane this deployment has no rates for cannot be totalled honestly,
757
  so `cost.microusd` is `nil` and `cost.unpriced_models` names the lanes that
758
  made it `nil`. What was priced is still reported, as `cost.priced_microusd`,
759
  so nothing is thrown away — but a reader that wants "the cost" has to notice
760
  it does not have one (METER-001).
753 761
  """
754 762
  @spec spend(Thread.t() | String.t()) :: %{
755 763
          calls: non_neg_integer(),
756 764
          grants: non_neg_integer(),
757
          usage: map()
765
          usage: map(),
766
          cost: %{
767
            microusd: non_neg_integer() | nil,
768
            priced_microusd: non_neg_integer(),
769
            basis: String.t(),
770
            unpriced_calls: non_neg_integer(),
771
            unpriced_models: [String.t()]
772
          }
758 773
        }
759 774
  def spend(%Thread{id: thread_id}), do: spend(thread_id)
760 775

@@ -762,12 +777,12 @@ defmodule OpenAgents.Threads do

762 777
    grants =
763 778
      from(grant in Grant,
764 779
        where: grant.thread_id == ^thread_id,
765
        select: {grant.call_count, grant.usage}
780
        select: {grant.call_count, grant.usage, grant.model_id}
766 781
      )
767 782
      |> Repo.all()
768 783
769 784
    usage =
770
      Enum.reduce(grants, %{}, fn {_calls, usage}, acc ->
785
      Enum.reduce(grants, %{}, fn {_calls, usage, _model}, acc ->
771 786
        Enum.reduce(usage || %{}, acc, fn
772 787
          {key, value}, inner when is_integer(value) ->
773 788
            Map.update(inner, key, value, &(&1 + value))

@@ -778,12 +793,63 @@ defmodule OpenAgents.Threads do

778 793
      end)
779 794
780 795
    %{
781
      calls: Enum.sum(Enum.map(grants, fn {calls, _usage} -> calls || 0 end)),
796
      calls: Enum.sum(Enum.map(grants, fn {calls, _usage, _model} -> calls || 0 end)),
782 797
      grants: length(grants),
783
      usage: usage
798
      usage: usage,
799
      cost: cost_view(grants)
784 800
    }
785 801
  end
786 802
803
  # One grant contributes to the cost only if it bought something. A grant that
804
  # was minted and never called says nothing about pricing either way, so an
805
  # unpriced grant with no calls does not make a session's total unknown.
806
  defp cost_view(grants) do
807
    metered = Enum.filter(grants, fn {calls, _usage, _model} -> (calls || 0) > 0 end)
808
809
    priced_microusd =
810
      metered
811
      |> Enum.map(fn {_calls, usage, _model} -> Pricing.cost(usage) || 0 end)
812
      |> Enum.sum()
813
814
    unpriced =
815
      Enum.filter(metered, fn {_calls, usage, _model} ->
816
        Pricing.usage_basis(usage) == Pricing.unpriced()
817
      end)
818
819
    unpriced_calls = Enum.sum(Enum.map(unpriced, fn {calls, _usage, _model} -> calls end))
820
821
    unpriced_models =
822
      unpriced
823
      |> Enum.map(fn {_calls, _usage, model_id} -> model_id end)
824
      |> Enum.reject(&is_nil/1)
825
      |> Enum.uniq()
826
      |> Enum.sort()
827
828
    bases =
829
      metered
830
      |> Enum.map(fn {_calls, usage, _model} -> Pricing.usage_basis(usage) end)
831
      |> Enum.uniq()
832
833
    %{
834
      microusd: if(unpriced == [] and metered != [], do: priced_microusd),
835
      priced_microusd: priced_microusd,
836
      basis: basis_word(metered, bases, unpriced),
837
      unpriced_calls: unpriced_calls,
838
      unpriced_models: unpriced_models
839
    }
840
  end
841
842
  # `absent` is not `unpriced`: nothing was bought, so there is nothing to
843
  # price. `unpriced` wins over the rest because it is the one word that says
844
  # the figure beside it is incomplete, and `provisional` wins over `declared`
845
  # because a total is only as billable as its least trustworthy part.
846
  defp basis_word([], _bases, _unpriced), do: "absent"
847
  defp basis_word(_metered, _bases, [_ | _]), do: "unpriced"
848
849
  defp basis_word(_metered, bases, []) do
850
    if "provisional" in bases, do: "provisional", else: "declared"
851
  end
852
787 853
  @doc "How many threads one account may hold open at once, or `nil` for no limit."
788 854
  @spec maximum_open_per_account() :: pos_integer() | nil
789 855
  def maximum_open_per_account, do: setting(:maximum_open_threads_per_account, nil)
lib/openagents_web/controllers/api_extension_controller.ex modified +9 -1

@@ -589,7 +589,15 @@ defmodule OpenAgentsWeb.ApiExtensionController do

589 589
          "anonymous visitor against `visitor_microusd`, for the life of the " <>
590 590
          "account rather than per thread. A thread's grant is minted for the " <>
591 591
          "remainder, so `grant.max_cost_microusd` in the mint response is " <>
592
          "what is left rather than a fixed cap."
592
          "what is left rather than a fixed cap.",
593
      "unpriced_lanes" =>
594
        "A model this deployment has declared no rates for records no cost, " <>
595
          "so its calls draw nothing down and its spend is reported as " <>
596
          "unknown rather than as zero. Read `pricing_basis` on each entry of " <>
597
          "`GET /api/v1/models` before you spend: `declared` is billable, " <>
598
          "`provisional` is a working figure, and `unpriced` means a cost " <>
599
          "will not be reported at all. On a thread, `spend.cost.microusd` is " <>
600
          "null whenever `spend.cost.unpriced_models` is non-empty."
593 601
    }
594 602
  end
595 603
lib/openagents_web/controllers/thread_controller.ex modified +42 -4

@@ -48,7 +48,7 @@ defmodule OpenAgentsWeb.ThreadController do

48 48
49 49
  alias OpenAgents.Conversations
50 50
  alias OpenAgents.Inference
51
  alias OpenAgents.Inference.{Credit, Grant, Models}
51
  alias OpenAgents.Inference.{Credit, Grant, Models, Pricing}
52 52
  alias OpenAgents.Threads
53 53
  alias OpenAgents.Threads.Thread
54 54
  alias OpenAgentsWeb.ApiError

@@ -792,7 +792,22 @@ defmodule OpenAgentsWeb.ThreadController do

792 792
793 793
  defp spend_view(%Thread{} = thread) do
794 794
    spend = Threads.spend(thread)
795
    %{"calls" => spend.calls, "grants" => spend.grants, "usage" => spend.usage}
795
796
    %{
797
      "calls" => spend.calls,
798
      "grants" => spend.grants,
799
      "usage" => spend.usage,
800
      # `cost.microusd` is null when any lane this session used has no declared
801
      # rates. A client that renders it as a currency has to handle the null
802
      # rather than print a zero it was never given (METER-001).
803
      "cost" => %{
804
        "microusd" => spend.cost.microusd,
805
        "priced_microusd" => spend.cost.priced_microusd,
806
        "basis" => spend.cost.basis,
807
        "unpriced_calls" => spend.cost.unpriced_calls,
808
        "unpriced_models" => spend.cost.unpriced_models
809
      }
810
    }
796 811
  end
797 812
798 813
  # The plaintext token exists exactly once, here. Everything else in this map

@@ -824,15 +839,38 @@ defmodule OpenAgentsWeb.ThreadController do

824 839
      "call_count" => grant.call_count,
825 840
      "usage" => grant.usage,
826 841
      "limits" => limits(grant),
842
      # What this grant was priced against, so the figures below can be
843
      # dereferenced rather than trusted. `unpriced` means the deployment has
844
      # no rates for this model: `spent_cost_microusd` is null, the cost
845
      # remainder is null, and neither is a zero (METER-001).
846
      "pricing" => %{
847
        "id" => Pricing.pricing_id_for(grant.model_id),
848
        "basis" => Pricing.basis(grant.model_id),
849
        "billable" => Pricing.billable?(grant.usage)
850
      },
851
      "spent" => %{
852
        "calls" => grant.call_count,
853
        "total_tokens" => spent(grant, "total_tokens"),
854
        "cost_microusd" => Pricing.cost(grant.usage)
855
      },
827 856
      "remaining" => %{
828 857
        "calls" => remaining(grant.max_calls, grant.call_count),
829 858
        "total_tokens" => remaining(grant.max_total_tokens, spent(grant, "total_tokens")),
830
        "cost_microusd" =>
831
          remaining(grant.max_cost_microusd, spent(grant, "estimated_cost_microusd"))
859
        # An unpriced grant has no cost remainder to report. Subtracting a zero
860
        # from the ceiling would publish the whole ceiling as headroom on a
861
        # grant that has been spending all along.
862
        "cost_microusd" => cost_remaining(grant)
832 863
      }
833 864
    }
834 865
  end
835 866
867
  defp cost_remaining(%Grant{} = grant) do
868
    case Pricing.cost(grant.usage) do
869
      nil -> nil
870
      cost -> remaining(grant.max_cost_microusd, cost)
871
    end
872
  end
873
836 874
  defp limits(%Grant{} = grant) do
837 875
    %{
838 876
      "max_calls" => grant.max_calls,
lib/openagents_web/live/thread_show_live.ex modified +38 -6

@@ -28,6 +28,7 @@ defmodule OpenAgentsWeb.ThreadShowLive do

28 28
29 29
  use OpenAgentsWeb, :live_view
30 30
31
  alias OpenAgents.Inference.Pricing
31 32
  alias OpenAgents.Markdown
32 33
  alias OpenAgents.Threads
33 34

@@ -138,20 +139,25 @@ defmodule OpenAgentsWeb.ThreadShowLive do

138 139
            </div>
139 140
            <div>
140 141
              <div class="text-xs text-muted-foreground">Calls</div>
141
              <div class="tabular-nums">{@grant.call_count} / {@grant.max_calls}</div>
142
              <div class="tabular-nums">{@grant.call_count} / {ceiling(@grant.max_calls)}</div>
142 143
            </div>
143 144
            <div>
144 145
              <div class="text-xs text-muted-foreground">Tokens</div>
145 146
              <div class="tabular-nums">
146
                {grant_spent(@grant, "total_tokens")} / {@grant.max_total_tokens}
147
                {grant_spent(@grant, "total_tokens")} / {ceiling(@grant.max_total_tokens)}
147 148
              </div>
148 149
            </div>
149 150
            <div>
150 151
              <div class="text-xs text-muted-foreground">Cost</div>
151
              <div class="tabular-nums">
152
                {dollars(grant_spent(@grant, "estimated_cost_microusd"))} / {dollars(
153
                  @grant.max_cost_microusd
154
                )}
152
              <div id="thread-budget-cost" class="tabular-nums" data-basis={cost_basis(@grant)}>
153
                {cost_spent(@grant)} / {dollars(@grant.max_cost_microusd)}
154
              </div>
155
              <div
156
                :if={cost_basis(@grant) != "declared"}
157
                id="thread-budget-cost-note"
158
                class="text-xs text-muted-foreground"
159
              >
160
                {cost_note(cost_basis(@grant))}
155 161
              </div>
156 162
            </div>
157 163
            <div :if={@grant.expires_at}>

@@ -401,8 +407,34 @@ defmodule OpenAgentsWeb.ThreadShowLive do

401 407
    end
402 408
  end
403 409
410
  # A null ceiling is unbounded, not zero and not missing. Rendering nil left
411
  # the denominator blank, which read as a rendering fault rather than as the
412
  # thread's actual authority.
413
  defp ceiling(nil), do: "∞"
414
  defp ceiling(value), do: value
415
416
  defp dollars(nil), do: "∞"
404 417
  defp dollars(microusd), do: "$#{:erlang.float_to_binary(microusd / 1_000_000, decimals: 2)}"
405 418
419
  # What this grant has spent, or the word for not knowing. `$0.00` is what
420
  # this cell used to show for a lane with no declared rates, and it was the
421
  # most confident wrong number on the page: the coder runs on that lane
422
  # (METER-001).
423
  defp cost_spent(grant) do
424
    case Pricing.cost(grant.usage) do
425
      nil -> "Unpriced"
426
      microusd -> dollars(microusd)
427
    end
428
  end
429
430
  defp cost_basis(grant), do: Pricing.usage_basis(grant.usage)
431
432
  defp cost_note("unpriced"),
433
    do: "No rates declared for this model, so this session's cost is unknown rather than zero."
434
435
  defp cost_note(_provisional),
436
    do: "Provisional rates. This is a working figure, not a bill."
437
406 438
  # The tier word plus what it means for a reader, because "dark" alone tells
407 439
  # the owner nothing about who can see the page they are looking at.
408 440
  defp visibility_label("dark"), do: "dark · only you"
test/openagents/inference/credit_test.exs modified +53

@@ -111,4 +111,57 @@ defmodule OpenAgents.Inference.CreditTest do

111 111
112 112
    assert Threads.ceilings(visitor_id) == {:error, :credit_exhausted}
113 113
  end
114
115
  # METER-001. A cost ceiling can only stop spend it can measure, so a lane
116
  # with no declared rates draws nothing down. That is a consequence of not
117
  # knowing the rates, and the account's read has to say so rather than report
118
  # a balance that looks untouched.
119
  describe "spend the deployment has no price for" do
120
    test "an unpriced call draws nothing down, and the balance says so" do
121
      owner = account("credit-unpriced")
122
      luna = Application.fetch_env!(:openagents, :openai_model)
123
124
      {:ok, thread} = Threads.open(%Visitor{id: owner.id}, "Run the unpriced lane", model: luna)
125
      {:ok, _fenced, grant, _token} = Threads.mint_grant(thread)
126
      {:ok, _metered} = Inference.record_usage(grant, %{"output_tokens" => 500_000})
127
128
      # `spent/1` is a floor, not a total, and the floor here is zero.
129
      assert Credit.spent(owner.id) == 0
130
      assert Credit.remaining(owner.id) == Credit.allowance(owner.id)
131
132
      # What stops that reading as "this account has spent nothing".
133
      assert Credit.unpriced_calls(owner.id) == 1
134
135
      balance = Credit.balance(owner.id)
136
      assert balance.spent_microusd == 0
137
      assert balance.unpriced_calls == 1
138
      refute balance.complete?
139
    end
140
141
    test "an account whose every call was priced reports a complete balance" do
142
      owner = account("credit-complete")
143
144
      {:ok, _metered} =
145
        Inference.record_usage(minted(owner.id), %{
146
          "output_tokens" => output_tokens_costing(100_000)
147
        })
148
149
      balance = Credit.balance(owner.id)
150
151
      assert balance.spent_microusd == 100_000
152
      assert balance.unpriced_calls == 0
153
      assert balance.complete?
154
    end
155
156
    test "a grant that never bought anything is not counted as unpriced spend" do
157
      owner = account("credit-idle")
158
      luna = Application.fetch_env!(:openagents, :openai_model)
159
160
      {:ok, thread} = Threads.open(%Visitor{id: owner.id}, "Mint and stop", model: luna)
161
      {:ok, _fenced, _grant, _token} = Threads.mint_grant(thread)
162
163
      assert Credit.unpriced_calls(owner.id) == 0
164
      assert Credit.balance(owner.id).complete?
165
    end
166
  end
114 167
end
test/openagents/inference/pricing_test.exs added +165

@@ -0,0 +1,165 @@

1
defmodule OpenAgents.Inference.PricingTest do
2
  @moduledoc """
3
  The rule that a cost is reported only where a price exists (METER-001).
4
5
  The failure this guards against is not a crash. It is a number: a lane with
6
  no declared rates summing to `$0.00` on a surface that looks like a bill,
7
  in the same typeface as a figure somebody measured. `gpt-5.6-luna` is that
8
  lane and it is the one the coder runs on, so every assertion below about
9
  "unpriced" is an assertion about the deployment's largest real spend.
10
  """
11
12
  use ExUnit.Case, async: true
13
14
  alias OpenAgents.Inference.Pricing
15
16
  defp unpriced_model_id, do: Application.fetch_env!(:openagents, :openai_model)
17
18
  describe "the basis of a catalog pricing map" do
19
    test "no pricing map at all is unpriced" do
20
      assert Pricing.basis_of(nil) == "unpriced"
21
    end
22
23
    test "rates that name the provider as their source are declared" do
24
      assert Pricing.basis_of(%{source: :declared, input_per_million_tokens: 1}) == "declared"
25
    end
26
27
    test "rates that do not say where they came from are provisional, not declared" do
28
      assert Pricing.basis_of(%{input_per_million_tokens: 1}) == "provisional"
29
30
      assert Pricing.basis_of(%{source: :placeholder, input_per_million_tokens: 1}) ==
31
               "provisional"
32
    end
33
  end
34
35
  describe "the id of the rate table" do
36
    test "a lane with no rates is priced against the unpriced table" do
37
      assert Pricing.pricing_id(nil) == "unpriced"
38
      assert Pricing.pricing_id_for(unpriced_model_id()) == "unpriced"
39
    end
40
41
    test "a lane with rates publishes the table those rates came from" do
42
      assert Pricing.pricing_id_for("gemini-3.7-flash") == "placeholder.gemini-3.7-flash.v1"
43
    end
44
45
    test "rates without a table name are unattributed rather than silently accepted" do
46
      assert Pricing.pricing_id(%{input_per_million_tokens: 1}) == "unattributed"
47
    end
48
49
    test "a model outside the catalog is unpriced rather than assumed free" do
50
      assert Pricing.pricing_id_for("not-a-model-this-deployment-serves") == "unpriced"
51
      assert Pricing.basis("not-a-model-this-deployment-serves") == "unpriced"
52
    end
53
  end
54
55
  describe "pricing one usage record" do
56
    test "an unpriced model writes no cost key — not a zero" do
57
      priced =
58
        Pricing.price(%{"input_tokens" => 4_000, "output_tokens" => 900}, unpriced_model_id())
59
60
      refute Map.has_key?(priced, "estimated_cost_microusd")
61
      assert priced["pricing_id"] == "unpriced"
62
      assert Pricing.cost(priced) == nil
63
      assert Pricing.usage_basis(priced) == "unpriced"
64
      refute Pricing.priced?(priced)
65
    end
66
67
    test "a priced model writes the cost and names the table it was priced against" do
68
      priced =
69
        Pricing.price(%{"input_tokens" => 1_000_000, "output_tokens" => 0}, "gemini-3.7-flash")
70
71
      assert priced["estimated_cost_microusd"] == 1_250_000
72
      assert priced["pricing_id"] == "placeholder.gemini-3.7-flash.v1"
73
      assert Pricing.cost(priced) == 1_250_000
74
    end
75
76
    test "cached reads are charged at the cached rate and cache writes at the input rate" do
77
      priced =
78
        Pricing.price(
79
          %{
80
            "input_tokens" => 1_000_000,
81
            "cache_read_input_tokens" => 400_000,
82
            "cache_write_input_tokens" => 100_000,
83
            "output_tokens" => 0
84
          },
85
          "gemini-3.7-flash"
86
        )
87
88
      # 600k uncached input + 100k cache write at $1.25/M, 400k cached read at
89
      # $0.10/M: 700_000 * 1.25 + 400_000 * 0.10, in microUSD.
90
      assert priced["estimated_cost_microusd"] == 875_000 + 40_000
91
    end
92
  end
93
94
  describe "reading a stored record back" do
95
    test "a placeholder-priced record carries a cost but is not billable" do
96
      priced = Pricing.price(%{"input_tokens" => 1_000_000}, "gemini-3.7-flash")
97
98
      assert Pricing.priced?(priced)
99
      assert Pricing.usage_basis(priced) == "provisional"
100
      refute Pricing.billable?(priced)
101
    end
102
103
    test "an unpriced record is never billable" do
104
      priced = Pricing.price(%{"input_tokens" => 1_000_000}, unpriced_model_id())
105
      refute Pricing.billable?(priced)
106
    end
107
108
    test "a record written before pricing ids existed is provisional, not declared" do
109
      legacy = %{"input_tokens" => 10, "estimated_cost_microusd" => 42}
110
111
      assert Pricing.cost(legacy) == 42
112
      assert Pricing.usage_basis(legacy) == "provisional"
113
      refute Pricing.billable?(legacy)
114
    end
115
116
    test "a record naming a table this deployment no longer carries is not billable" do
117
      retired = %{
118
        "pricing_id" => "declared.some-retired-table.v1",
119
        "estimated_cost_microusd" => 9
120
      }
121
122
      assert Pricing.usage_basis(retired) == "provisional"
123
      refute Pricing.billable?(retired)
124
    end
125
126
    test "an empty or absent record is unpriced rather than zero" do
127
      assert Pricing.cost(%{}) == nil
128
      assert Pricing.cost(nil) == nil
129
      assert Pricing.usage_basis(%{}) == "unpriced"
130
      assert Pricing.usage_basis(nil) == "unpriced"
131
    end
132
  end
133
134
  describe "the catalog this deployment actually ships" do
135
    test "no lane claims declared rates, because none has been given any" do
136
      # The moment an operator enters real rates and sets `source: :declared`,
137
      # this test fails and whoever did it has to say so here. That is the
138
      # point: turning a lane billable is a decision, not a config typo.
139
      bases = Enum.map(OpenAgents.Inference.Models.catalog(), & &1["pricing_basis"])
140
141
      assert "declared" not in bases
142
      assert "unpriced" in bases
143
      assert "provisional" in bases
144
    end
145
146
    test "the unpriced lane publishes no pricing block, and says so in one word" do
147
      luna =
148
        OpenAgents.Inference.Models.catalog()
149
        |> Enum.find(&(&1["id"] == unpriced_model_id()))
150
151
      refute Map.has_key?(luna, "pricing")
152
      assert luna["pricing_basis"] == "unpriced"
153
    end
154
155
    test "a priced lane publishes its table and its basis beside the rates" do
156
      gemini =
157
        OpenAgents.Inference.Models.catalog()
158
        |> Enum.find(&(&1["id"] == "gemini-3.7-flash"))
159
160
      assert gemini["pricing"]["id"] == "placeholder.gemini-3.7-flash.v1"
161
      assert gemini["pricing"]["basis"] == "provisional"
162
      assert gemini["pricing_basis"] == "provisional"
163
    end
164
  end
165
end
test/openagents/inference_test.exs modified +11 -1

@@ -206,9 +206,12 @@ defmodule OpenAgents.InferenceTest do

206 206
        |> div(1_000_000)
207 207
208 208
      assert metered.usage["estimated_cost_microusd"] == expected
209
      # The record names the table it was priced against, so the figure can be
210
      # dereferenced rather than trusted (METER-001).
211
      assert metered.usage["pricing_id"] == "placeholder.gemini-3.7-flash.v1"
209 212
    end
210 213
211
    test "an unpriced model records no estimated cost" do
214
    test "an unpriced model records no estimated cost — and no zero" do
212 215
      luna_id = Application.fetch_env!(:openagents, :openai_model)
213 216
      {:ok, grant, _token} = Inference.mint(Map.put(scope("usage-unpriced"), :model_id, luna_id))
214 217

@@ -219,6 +222,13 @@ defmodule OpenAgents.InferenceTest do

219 222
        })
220 223
221 224
      refute Map.has_key?(metered.usage, "estimated_cost_microusd")
225
      refute metered.usage["estimated_cost_microusd"] == 0
226
      # The record says why there is no cost rather than leaving a reader to
227
      # infer it from a missing key.
228
      assert metered.usage["pricing_id"] == "unpriced"
229
      # Tokens are still measured. Unpriced is not unmetered: the call is
230
      # evidenced, only its price is unknown.
231
      assert metered.usage["total_tokens"] == 140
222 232
    end
223 233
  end
224 234
test/openagents/threads_test.exs modified +101 -1

@@ -540,7 +540,107 @@ defmodule OpenAgents.ThreadsTest do

540 540
      user = owner("spend-none")
541 541
      {:ok, thread} = Threads.open(user, "Never spent")
542 542
543
      assert Threads.spend(thread) == %{calls: 0, grants: 0, usage: %{}}
543
      assert %{calls: 0, grants: 0, usage: %{}, cost: cost} = Threads.spend(thread)
544
545
      # Nothing was bought, so there is nothing to price. `absent` is not
546
      # `unpriced`: one says no calls were made, the other says calls were made
547
      # and this deployment cannot say what they cost.
548
      assert cost.basis == "absent"
549
      assert cost.microusd == nil
550
      assert cost.unpriced_calls == 0
551
      assert cost.unpriced_models == []
552
    end
553
  end
554
555
  # METER-001. Every assertion here is about the same wrong number: a session
556
  # on a lane with no declared rates reporting `$0.00`, which reads as a
557
  # measurement rather than as the absence of one.
558
  describe "what a thread spent, when the deployment has no price for it" do
559
    test "a thread on an unpriced model reports an unknown cost, never a zero" do
560
      luna = Application.fetch_env!(:openagents, :openai_model)
561
      user = owner("spend-unpriced")
562
      {:ok, thread} = Threads.open(user, "Run the coder's own lane", model: luna)
563
      {:ok, _fenced, grant, _token} = Threads.mint_grant(thread)
564
565
      {:ok, metered} =
566
        Inference.record_usage(grant, %{"input_tokens" => 40_000, "output_tokens" => 9_000})
567
568
      # The grant itself refuses to invent the figure.
569
      refute Map.has_key?(metered.usage, "estimated_cost_microusd")
570
      assert metered.usage["pricing_id"] == "unpriced"
571
572
      spend = Threads.spend(thread)
573
574
      assert spend.calls == 1
575
      assert spend.usage["input_tokens"] == 40_000
576
      # The one that matters: not zero.
577
      refute spend.cost.microusd == 0
578
      assert spend.cost.microusd == nil
579
      assert spend.cost.basis == "unpriced"
580
      assert spend.cost.unpriced_calls == 1
581
      assert spend.cost.unpriced_models == [luna]
582
    end
583
584
    test "a thread on a priced model reports a total, labelled by its basis" do
585
      user = owner("spend-priced")
586
      {:ok, thread, grant, _token} = Threads.open_and_mint(user, "Priced lane")
587
      {:ok, _} = Inference.record_usage(grant, %{"input_tokens" => 1_000_000})
588
589
      spend = Threads.spend(thread)
590
591
      assert spend.cost.microusd == 1_250_000
592
      assert spend.cost.priced_microusd == 1_250_000
593
      assert spend.cost.basis == "provisional"
594
      assert spend.cost.unpriced_models == []
595
    end
596
597
    test "one unpriced grant makes the whole session's total unknown, and names why" do
598
      luna = Application.fetch_env!(:openagents, :openai_model)
599
      user = owner("spend-mixed")
600
      {:ok, thread, first, _token} = Threads.open_and_mint(user, "Start priced")
601
      {:ok, _} = Inference.record_usage(first, %{"input_tokens" => 1_000_000})
602
603
      # A thread re-mints on resume, and a grant pins its own model, so a
604
      # session whose grants ran on different lanes is exactly the case a total
605
      # has to survive honestly.
606
      {:ok, _revoked} = Inference.revoke(first)
607
      {:ok, second, _token} = unpriced_grant_for(thread, luna)
608
      {:ok, _} = Inference.record_usage(second, %{"input_tokens" => 50_000})
609
610
      spend = Threads.spend(thread)
611
612
      assert spend.cost.microusd == nil
613
      # Nothing measured is thrown away — the priced half is still reported,
614
      # just not as the answer to "what did this cost".
615
      assert spend.cost.priced_microusd == 1_250_000
616
      assert spend.cost.basis == "unpriced"
617
      assert spend.cost.unpriced_models == [luna]
618
    end
619
620
    test "a grant that was minted and never called does not make the total unknown" do
621
      luna = Application.fetch_env!(:openagents, :openai_model)
622
      user = owner("spend-idle-unpriced")
623
      {:ok, thread, first, _token} = Threads.open_and_mint(user, "Priced work")
624
      {:ok, _} = Inference.record_usage(first, %{"input_tokens" => 1_000_000})
625
626
      {:ok, _revoked} = Inference.revoke(first)
627
      {:ok, _idle, _token} = unpriced_grant_for(thread, luna)
628
629
      spend = Threads.spend(thread)
630
631
      assert spend.grants == 2
632
      assert spend.cost.microusd == 1_250_000
633
      assert spend.cost.unpriced_calls == 0
634
    end
635
636
    # A thread holds at most one active grant, so a second lane is reached the
637
    # way a resume reaches it: revoke, then mint again against the same fence.
638
    defp unpriced_grant_for(thread, model_id) do
639
      Inference.mint(%{
640
        owner_visitor_id: thread.owner_visitor_id,
641
        thread_id: thread.id,
642
        model_id: model_id
643
      })
544 644
    end
545 645
  end
546 646
end
test/openagents_web/controllers/model_catalog_controller_test.exs modified +25

@@ -119,5 +119,30 @@ defmodule OpenAgentsWeb.ModelCatalogControllerTest do

119 119
      luna = Enum.find(body["models"], &(&1["id"] == luna_id))
120 120
      refute Map.has_key?(luna, "pricing")
121 121
    end
122
123
    test "every entry says in one word whether its price can be trusted", %{conn: conn} do
124
      luna_id = Application.fetch_env!(:openagents, :openai_model)
125
126
      body =
127
        conn
128
        |> put_chat_api_token("model-catalog-basis")
129
        |> get(~p"/api/v1/models")
130
        |> json_response(200)
131
132
      # A client that forgets to check for a missing `pricing` key would read
133
      # an unknown price as no price. This word is the positive signal, and it
134
      # is present on every entry rather than only the awkward ones (METER-001).
135
      assert Enum.all?(
136
               body["models"],
137
               &(&1["pricing_basis"] in ~w(declared provisional unpriced))
138
             )
139
140
      assert Enum.find(body["models"], &(&1["id"] == luna_id))["pricing_basis"] == "unpriced"
141
142
      gemini = Enum.find(body["models"], &(&1["id"] == "gemini-3.7-flash"))
143
      assert gemini["pricing_basis"] == "provisional"
144
      assert gemini["pricing"]["basis"] == "provisional"
145
      assert gemini["pricing"]["id"] == "placeholder.gemini-3.7-flash.v1"
146
    end
122 147
  end
123 148
end
test/openagents_web/controllers/thread_controller_test.exs modified +67

@@ -1236,4 +1236,71 @@ defmodule OpenAgentsWeb.ThreadControllerTest do

1236 1236
      assert Repo.get!(OpenAgents.Threads.Thread, body["thread"]["id"]).lane == "thread"
1237 1237
    end
1238 1238
  end
1239
1240
  # METER-001. The thread read is where a client learns what a session cost,
1241
  # so it is where an unpriced lane has to stop looking like a free one.
1242
  describe "reporting cost the deployment cannot price" do
1243
    test "a thread on an unpriced model reports a null cost, not a zero", %{conn: conn} do
1244
      luna = Application.fetch_env!(:openagents, :openai_model)
1245
      authenticated = put_chat_api_token(conn, "thread-cost-unpriced")
1246
1247
      created =
1248
        authenticated
1249
        |> post(~p"/api/v1/threads", %{
1250
          "objective" => "Run the coder's own lane.",
1251
          "model" => luna
1252
        })
1253
        |> json_response(201)
1254
1255
      id = created["thread"]["id"]
1256
      grant = Repo.get_by!(Grant, thread_id: id)
1257
1258
      {:ok, _spent} =
1259
        Inference.record_usage(grant, %{"input_tokens" => 900, "output_tokens" => 80})
1260
1261
      body = authenticated |> get(~p"/api/v1/threads/#{id}") |> json_response(200)
1262
1263
      # The key exists and is null. A client that renders it has to handle the
1264
      # null rather than print a zero it was never given.
1265
      assert Map.has_key?(body["thread"]["spend"]["cost"], "microusd")
1266
      assert is_nil(body["thread"]["spend"]["cost"]["microusd"])
1267
      assert body["thread"]["spend"]["cost"]["basis"] == "unpriced"
1268
      assert body["thread"]["spend"]["cost"]["unpriced_calls"] == 1
1269
      assert body["thread"]["spend"]["cost"]["unpriced_models"] == [luna]
1270
1271
      # And the same refusal on the grant: no cost spent, no cost remainder.
1272
      assert body["grant"]["pricing"]["basis"] == "unpriced"
1273
      assert body["grant"]["pricing"]["id"] == "unpriced"
1274
      assert body["grant"]["pricing"]["billable"] == false
1275
      assert is_nil(body["grant"]["spent"]["cost_microusd"])
1276
      assert is_nil(body["grant"]["remaining"]["cost_microusd"])
1277
1278
      # Tokens were still measured, so the call is evidenced even unpriced.
1279
      assert body["grant"]["spent"]["total_tokens"] == 980
1280
    end
1281
1282
    test "a priced thread reports its cost, labelled by the basis of its rates", %{conn: conn} do
1283
      authenticated = put_chat_api_token(conn, "thread-cost-priced")
1284
1285
      created =
1286
        authenticated
1287
        |> post(~p"/api/v1/threads", %{"objective" => "A lane with rates."})
1288
        |> json_response(201)
1289
1290
      id = created["thread"]["id"]
1291
      grant = Repo.get_by!(Grant, thread_id: id)
1292
      {:ok, _spent} = Inference.record_usage(grant, %{"input_tokens" => 1_000_000})
1293
1294
      body = authenticated |> get(~p"/api/v1/threads/#{id}") |> json_response(200)
1295
1296
      assert body["thread"]["spend"]["cost"]["microusd"] == 1_250_000
1297
      assert body["thread"]["spend"]["cost"]["basis"] == "provisional"
1298
      assert body["thread"]["spend"]["cost"]["unpriced_models"] == []
1299
1300
      # Priced is not the same as billable: these are placeholder rates.
1301
      assert body["grant"]["pricing"]["basis"] == "provisional"
1302
      assert body["grant"]["pricing"]["billable"] == false
1303
      assert body["grant"]["spent"]["cost_microusd"] == 1_250_000
1304
    end
1305
  end
1239 1306
end
test/openagents_web/live/thread_show_live_test.exs modified +48

@@ -112,4 +112,52 @@ defmodule OpenAgentsWeb.ThreadShowLiveTest do

112 112
      live(signed_in(conn, viewer), ~p"/threads/not-a-uuid")
113 113
    end
114 114
  end
115
116
  # METER-001. This cell used to read "$0.00 / $100.00" for a session on the
117
  # lane the coder actually runs on, which is the most confident wrong number
118
  # the product could show.
119
  describe "the budget card's cost cell" do
120
    test "an unpriced lane shows the word, never a dollar figure", %{conn: conn} do
121
      owner = github_user("thread-show-unpriced")
122
      luna = Application.fetch_env!(:openagents, :openai_model)
123
124
      {:ok, thread} = Threads.open(owner, "Run the unpriced lane", model: luna)
125
      {:ok, _fenced, grant, _token} = Threads.mint_grant(thread)
126
127
      {:ok, _metered} =
128
        OpenAgents.Inference.record_usage(grant, %{"input_tokens" => 900, "output_tokens" => 80})
129
130
      {:ok, view, _html} = live(signed_in(conn, owner), ~p"/threads/#{thread.id}")
131
132
      cost = view |> element("#thread-budget-cost") |> render()
133
134
      assert cost =~ "Unpriced"
135
      refute cost =~ "$0.00"
136
      assert view |> element("#thread-budget-cost-note") |> render() =~ "unknown rather than zero"
137
    end
138
139
    test "a priced lane shows the figure and says the rates are provisional", %{conn: conn} do
140
      owner = github_user("thread-show-priced")
141
142
      {:ok, thread} = Threads.open(owner, "Run a lane with rates")
143
      {:ok, _fenced, grant, _token} = Threads.mint_grant(thread)
144
      {:ok, _metered} = OpenAgents.Inference.record_usage(grant, %{"input_tokens" => 1_000_000})
145
146
      {:ok, view, _html} = live(signed_in(conn, owner), ~p"/threads/#{thread.id}")
147
148
      assert view |> element("#thread-budget-cost") |> render() =~ "$1.25"
149
      assert view |> element("#thread-budget-cost-note") |> render() =~ "not a bill"
150
    end
151
152
    test "an unbounded ceiling reads as unbounded rather than blank", %{conn: conn} do
153
      owner = github_user("thread-show-unbounded")
154
155
      {:ok, thread} = Threads.open(owner, "No ceilings but the account's")
156
      {:ok, _fenced, _grant, _token} = Threads.mint_grant(thread)
157
158
      {:ok, view, _html} = live(signed_in(conn, owner), ~p"/threads/#{thread.id}")
159
160
      assert view |> element("#thread-budget") |> render() =~ "\u221e"
161
    end
162
  end
115 163
end

This page updates live while a promote is in flight · changelog