Serve a typed model catalog and route coder turns across house providers #199
- AtlantisPleb opened this issue 2d ago
-
A Author 2d ago Core landed on main as 5125d3f (with the grants re-mint route following as f6723c8, WAL receipt seq 301). GET /api/v3/models serves the typed catalog (id, provider, context_window, max_output, availability, default) from one config-driven source; a credential-less provider lists as unavailable and refuses selection with model_unavailable before any provider call; the proxy now refuses an unknown body model (model_not_served, echoing the served list) and a grant mismatch (model_mismatch) instead of silently ignoring the field, and every 200 attributes the effective model (x-openagents-model header plus model on every SSE chunk). PROVIDER-002 (no silent substitution) added to INVARIANTS.md. Remaining in scope here: the Gemini and Z.ai proxy-lane adapters — the report records exactly what each needs (Provider behaviour module, credential key in RuntimeConfig, lane entry, catalog entry, boundary-test classification; Chat.Gemini is the port source for the Gemini wire shape). 251 targeted tests green across the touched surfaces.
-
A Author 2d ago Status against the four contract bullets, checked live against the dev server:
- Catalog endpoint — shipped.
GET /api/v1/modelslists each offered model with id, provider, context ceiling, output ceiling, availability, and which is default. - Per-turn selection and loud failure — shipped. #160 is closed: an unsupported model returns 422
model_not_servednaming both what was requested and what is served, so substitution is never silent. - Server-side credentials — shipped. Provider credentials live behind the proxy, and usage is attributed per grant.
- Routing as server policy — partial, and the gap has teeth. Availability is a static property of the catalog entry rather than anything derived from whether the lane answers.
gemini-3.7-flashreports"availability": "available"while every call to it fails upstream (#238), so a client that reads the catalog and trusts it picks a dead lane. Balance-aware routing across providers does not exist yet either.
The remaining work for this issue is the fourth bullet, and #238 is the concrete instance of it. Leaving open for that.
- Catalog endpoint — shipped.
-
A Author 2d ago The fourth bullet — routing as server policy rather than client guesswork — has its first half on main in c23e2eb: availability is now derived from what the lane actually did, not only from whether a credential is configured, so a client reading the catalog is no longer told a dead lane is available (that was #238, now closed).
What remains on this issue is balance-aware routing across providers: choosing a lane by credit balance as well as by health. The health signal that decision needs now exists.
-
A Author 2d ago All four contract bullets are now shipped.
- Catalog endpoint —
GET /api/v1/modelslists id, provider, context and output ceilings, availability, and the default. - Per-turn selection honoured, loud on unsupported — #160, closed: a model outside the served set returns 422
model_not_servednaming both what was asked and what is served. - Server-side credentials, usage attributed per grant — and the cached-versus-fresh split now travels with it (#220).
- Routing is server policy — landed in two halves: availability is derived from what a lane actually did rather than only from whether a credential exists (c23e2eb), and
Models.select/0now picks the lane for a caller that named none (c96a1ba), preferring a configured non-degraded lane and falling back to the catalog default when every lane is degraded.
PROVIDER-002 was amended in the same change rather than left to drift, because the boundary is what makes this honest: selection applies only where nothing named a model — neither the mint nor the call. A named model is never substituted; a caller that named a lane gets that lane or an error, degraded or not, with a test pinning it. The effective model stays attributed on every response.
Credit-balance routing is deliberately not included: balances are not modelled in this repository, and guessing at them would be worse than the fixed default this replaces. If you want balance-aware routing it wants its own issue and a source of balance truth. 4,482 tests green.
- Catalog endpoint —
- closed this as completed 2d ago
Outcome
The coder's first compute lane: server-held provider credits (Gemini, Z.ai, and peer services) behind
POST /api/inference/proxy, fronted by a typed model catalog the CLI reads instead of guessing.Contract
Source
docs/2026-08-24-coder-first-cloud-complements.mdsection 5, lane 1. Part of the Coder v1 release arc.