The bug
config :openagents, :vercel_gateway_fallback_models (config/config.exs) tells
the Vercel AI Gateway to answer a failed google/gemini-3.7-flash call with
zai/glm-5.3, zai/glm-5.2, or openai/gpt-5.6-luna — and to return 200 when
one of them does. OpenAgents.Providers.VercelGateway sends that list as
providerOptions.gateway.models and never reads back which model answered, so
the whole metering path treated the requested model as the serving one:
OpenAgents.Inference.record_usage/2 priced the record against
grant.model_id. Gemini carries rates and Luna carries none, so a Luna call
was billed at Gemini's rates.
OpenAgents.Inference.Health.record_success/1 was called with the requested
lane's id, so a lane that failed and was rescued by a fallback published
available at GET /api/v1/models — the same failure #238 fixed.
- The
x-openagents-model header and every SSE chunk's model field named the
requested model, so a client rendered a model that did not answer.
Evidence
OpenAgents.Providers.OpenRouter.StreamDecoder — which the gateway shares —
parses id, choices, and usage off each chat-completions chunk and drops
model entirely. Nothing else on the path could have known.
The size of the wrong number, from the shipped catalog:
merge_usage(%{}, %{"input_tokens" => 1_000_000, "output_tokens" => 100_000}, "gemini-3.7-flash")
#=> pricing_id: "placeholder.gemini-3.7-flash.v1", estimated_cost_microusd: 2_250_000
$2.25 of measured-looking cost for a call served by a lane this deployment has
no rates for at all. OpenAgents.Threads.spend/1 then totalled it as a known
figure rather than reporting cost.microusd = nil, which is exactly the
$0.00-beside-real-spend failure METER-001 exists to prevent.
The fix
The response says which model served it; read it back.
- The chat-completions decoder carries
model out as
{:model_served, name} — once, and never guessed.
OpenAgents.Providers.Provider.substitutable?/0 (optional, false where not
exported) says whether a lane's provider may answer with a different model.
VercelGateway answers true exactly while a fallback list is configured, so
silence from a lane that cannot substitute still means the model it asked for.
- Every metered record carries
served_model, and Pricing.price/2 prices
against that rather than against the grant's model. Three values name no rate
table and so price at nothing: unresolved (a substitutable lane disclosed
nothing — naming the requested model would be a claim the deployment cannot
support), mixed (one grant's calls served by more than one model; a single
total cannot be charged at two rate tables), and any model outside the
catalog.
Threads.spend/1 names the serving lane in cost.unpriced_models, so an
operator is sent to price the lane that actually ran.
- Health follows the lane that answered: a fallback-served call records a
failure against the requested lane and a success against the serving lane
where the catalog has it. An unresolved response records nothing — health that
reports what it does not know is the fault being fixed, not a smaller version
of it.
- The header and chunks attribute the serving model, or the word
unresolved.
METER-001 and PROVIDER-002 are amended in the same change.
Proof: test/openagents_web/controllers/inference_proxy_fallback_test.exs and
test/openagents/providers/open_router/stream_decoder_test.exs.
The bug
config :openagents, :vercel_gateway_fallback_models(config/config.exs) tellsthe Vercel AI Gateway to answer a failed
google/gemini-3.7-flashcall withzai/glm-5.3,zai/glm-5.2, oropenai/gpt-5.6-luna— and to return 200 whenone of them does.
OpenAgents.Providers.VercelGatewaysends that list asproviderOptions.gateway.modelsand never reads back which model answered, sothe whole metering path treated the requested model as the serving one:
OpenAgents.Inference.record_usage/2priced the record againstgrant.model_id. Gemini carries rates and Luna carries none, so a Luna callwas billed at Gemini's rates.
OpenAgents.Inference.Health.record_success/1was called with the requestedlane's id, so a lane that failed and was rescued by a fallback published
availableatGET /api/v1/models— the same failure #238 fixed.x-openagents-modelheader and every SSE chunk'smodelfield named therequested model, so a client rendered a model that did not answer.
Evidence
OpenAgents.Providers.OpenRouter.StreamDecoder— which the gateway shares —parses
id,choices, andusageoff each chat-completions chunk and dropsmodelentirely. Nothing else on the path could have known.The size of the wrong number, from the shipped catalog:
$2.25of measured-looking cost for a call served by a lane this deployment hasno rates for at all.
OpenAgents.Threads.spend/1then totalled it as a knownfigure rather than reporting
cost.microusd = nil, which is exactly the$0.00-beside-real-spend failure METER-001 exists to prevent.The fix
The response says which model served it; read it back.
modelout as{:model_served, name}— once, and never guessed.OpenAgents.Providers.Provider.substitutable?/0(optional,falsewhere notexported) says whether a lane's provider may answer with a different model.
VercelGatewayanswerstrueexactly while a fallback list is configured, sosilence from a lane that cannot substitute still means the model it asked for.
served_model, andPricing.price/2pricesagainst that rather than against the grant's model. Three values name no rate
table and so price at nothing:
unresolved(a substitutable lane disclosednothing — naming the requested model would be a claim the deployment cannot
support),
mixed(one grant's calls served by more than one model; a singletotal cannot be charged at two rate tables), and any model outside the
catalog.
Threads.spend/1names the serving lane incost.unpriced_models, so anoperator is sent to price the lane that actually ran.
failure against the requested lane and a success against the serving lane
where the catalog has it. An unresolved response records nothing — health that
reports what it does not know is the fault being fixed, not a smaller version
of it.
unresolved.METER-001 and PROVIDER-002 are amended in the same change.
Proof:
test/openagents_web/controllers/inference_proxy_fallback_test.exsandtest/openagents/providers/open_router/stream_decoder_test.exs.