Boot the metered inference offering behind the coder #200

Closed AtlantisPleb opened this 2d ago 5 comments

Outcome

Sell metered model access behind the coding agent: grants meter against account credits, pricing is per-model and visible before spend, and usage truth precedes any bill.

Scope note

This deliberately reopens a bounded slice of the deferred payments lane, as a named exception like the settlement proof. It is gated inside the Coder v1 arc on the model catalog's attribution being honest (see the catalog issue): a metered offering on top of untrustworthy usage numbers is the exact failure we position against (the Cursor pricing-crisis lesson).

Contract

  • Per-model pricing surface readable by the CLI and the web before any spend.
  • Grant ceilings and running spend visible in the coder status line and on the thread page.
  • Metering derived from the same grant usage records the leaderboard consumes; no second accounting.
  • No bill without a dereferenceable usage record.

Source

docs/2026-08-24-coder-first-cloud-complements.md section 5, lane 2.

  1. AtlantisPleb opened this issue 2d ago
  2. A AtlantisPleb Author 1d ago

    First two contract bullets landed on main in f3b7314, now that the gate is met (#199 closed: attribution is honest, availability is health-derived, a named model is never substituted).

    • Per-model pricing readable before spend — a model declares input, output, and optionally cached-input rates beside its ceilings, and GET /api/v1/models publishes them, so a caller sees what a lane costs before choosing it.
    • Cost derived from that pricingestimated_cost_microusd is computed from the declared rates rather than a constant, with cached reads priced at the cached rate. That is why the split was plumbed: on agentic traffic most input is a re-sent prefix, and pricing it as fresh overstates the bill on exactly the workloads the coder produces.
    • Metering from the same records — unchanged; there is no second accounting. Threads.spend/1 already sums a session's calls and usage across its grants, so running spend is readable per thread.

    The absent-versus-zero rule holds: an unpriced model publishes no pricing and records no cost, so a lane nobody has priced never reads as free. That is what keeps 'usage truth precedes any bill' true rather than merely stated. 4,501 tests green.

    The rates in config are placeholders and say so. Set real provider rates before accepting any spend.

    Remaining: ceilings and running spend in the coder status line (CLI work), and the account-credit metering and billing surface itself — no billing, invoices, or charging were built here.

  3. A AtlantisPleb Author 1d ago

    Shipped a slice in 3445eed: usage truth and honest pricing. Leaving this open — the offering is not sold yet, and the list below says what is left.

    What landed

    OpenAgents.Inference.Pricing is now the one authority on what a call cost and on whose authority. Every metered usage record stamps pricing_id naming the rate table it was priced against, and the id resolves to one of three bases:

    • declared — the operator entered the provider's published rates. The only billable basis.
    • provisional — rates written to make the system run, or a table this deployment can no longer dereference. A figure, not a bill.
    • unpriced — no rates. No estimated_cost_microusd written, Pricing.cost/1 answers nil.

    The gap this closes is gpt-5.6-luna. It is the lane the coder actually runs on and the catalog declares no rates for it, so every surface that read a missing cost as zero was showing $0.00 beside the account's largest real spend — the thread page did, grant.remaining.cost_microusd published the whole ceiling as untouched headroom, and Threads.spend/1 summed it into a total that looked complete. Metering that prices a lane at zero is worse than no metering, because it reports a number and the number is wrong.

    Downstream, absence now survives to the reader:

    • GET /api/v1/models publishes pricing_basis on every entry beside availability, plus pricing.id and pricing.basis inside the rates. Readable before any spend.
    • Threads.spend/1 refuses to total a session that touched an unpriced lane: cost.microusd is null, cost.unpriced_models names why, cost.priced_microusd still reports what was measured. GET /api/v1/threads/{id} publishes the same shape.
    • grant.pricing (id, basis, billable) and grant.spent are new on the thread read; grant.remaining.cost_microusd is null for an unpriced grant rather than a subtraction from zero.
    • The thread page shows the word Unpriced instead of a dollar figure, labels provisional figures as working numbers rather than bills, and renders a null ceiling as instead of blank.
    • Credit.spent/1 is documented as the floor it is; Credit.unpriced_calls/1 and Credit.balance/1 publish what it cannot see, including complete?.

    Every rate in config/config.exs now carries source: :placeholder, because none was read off a provider's price page. PricingTest asserts no lane claims declared today, so turning one billable is a deliberate edit that fails a test until somebody says so.

    No migration, no new route, no new data family. The meter is still the grant usage records the leaderboard consumes — no second accounting.

    INVARIANTS.md gains METER-001. docs/taxonomy.md gains pricing basis and unpriced. mix precommit green (4593 tests).

    What remains on this issue

    1. Real rates — owner action, blocking everything below. gpt-5.6-luna has no price at all and the other two lanes carry placeholders. Getting provider rates is not something code may guess at. Until an operator enters them and flips source: :declared, nothing in this system is billable by construction, and the account balance is a floor rather than a total. This is the single highest-value edit in config/config.exs.
    2. The coder status line. The contract asks for grant ceilings and running spend there. No CLI source exists in this repository — docs/2026-08-23-openagents-coder-cli-spec.md specifies it (repository · branch · model · calls · tokens) and Stage 2 owns it. The server side it needs is now published; the client is not ours to write here yet.
    3. A web pricing surface. The catalog is readable over GET /api/v1/models behind a required bearer, and nothing renders it in the UI or in priv/docs. "Visible before spend" is half-true: a CLI can read it, a person cannot.
    4. Credits that mean money. Credit is an allowance, not a purchased balance. Grants meter against it, but nothing tops it up, and there is no payment path. That is the actual "sell metered model access" half, and it is entangled with #207.
    5. Gateway fallback attribution. vercel_gateway_fallback_models can have a gemini-3.7-flash call served by openai/gpt-5.6-luna, and the usage record would still be priced at Gemini's rates. The adapter does not read back the model that actually served. This is the same honesty failure one layer down and it should be fixed before any lane goes declared.
  4. A AtlantisPleb Author 1d ago

    Shipped item 3 of the remainder in f7b54ef: the web pricing surface. Leaving this open — three of the five items are still outstanding, and one of them is the half that actually sells anything.

    What landed

    /models renders the catalog for a person. It reads Models.catalog/0 directly, so the page, GET /api/v1/models, and thread admission cannot come to disagree about what this deployment serves or what it charges. Same principal as the endpoint: the catalog names what a grant can be minted for, so the reader of the prices is the caller who can spend them.

    The absence rule survives into the layout, which is where a zero would be easiest to slip past:

    • An unpriced lane's rate cells carry the word Unpriced, never $0.00.
    • Every row names its basis and the id of the rate table it is priced against, so a provisional figure reads as a working number rather than a price.
    • A priced lane with no cached-input rate says its cached reads are charged at the input rate — which is what Pricing.price/2 does — rather than leaving the column blank and implying a discount.
    • Where no lane is declared, the page says at the top that nothing on it is billable. That notice is derived from the catalog, so it cannot outlive the fact.
    • Availability is the same Models.availability/1 word the endpoint publishes, on a slow tick.

    priv/docs/models.md documents the surface and the three bases, and the thread page's model links to the catalog so a reader looking at what a session spent can reach what a call on that lane costs. METER-001 gains the page; mix precommit green (4,653 tests).

    No rate was invented and no lane was flipped to declared.

    What is still open

    1. Real rates — owner action, still blocking everything billable. gpt-5.6-luna has no price at all and the other two lanes carry placeholders. /models now states that fact plainly rather than implying otherwise, which makes the gap visible but does not close it. Enter the provider's rates in config/config.exs and set source: :declared in the same edit; PricingTest fails until somebody does.
    2. The coder status line. Still not completable here: this repository has no CLI source. docs/2026-08-23-openagents-coder-cli-spec.md specifies the line (repository · branch · model · calls · tokens) and Stage 2 owns it. Everything the client needs is published — GET /api/v1/threads/{id} carries grant.spent, grant.remaining, and grant.pricing, and GET /api/v1/models carries the rates.
    3. A web pricing surface. Done above.
    4. Credits that mean money. Credit is still an allowance, not a purchased balance. Nothing tops it up and there is no payment path, so "sell metered model access" is not yet true in the sense the outcome states it. Entangled with #207.
    5. Gateway fallback attribution. vercel_gateway_fallback_models can still have a call requested as one model served by another and priced at the requested model's rates. Being worked separately; not touched here.
  5. A AtlantisPleb Author 1d ago

    Sweep verdict: stays open, and two of the five remainder items are now done. Restating the list so it is not read from three stacked comments.

    Verified done since the last comment

    Item 3, the web pricing surface. lib/openagents_web/live/model_catalog_live.ex and priv/docs/models.md are on openagents/main in f7b54ef. Confirmed present in a clean worktree at b2e5ee0.

    Item 5, gateway fallback attribution. This is now fixed and its issue is closed. b2e5ee0, "Price a metered call against the model that actually served it", landed the whole chain: the chat-completions decoder carries {:model_served, name} (lib/openagents/providers/open_router/stream_decoder.ex:274), Provider.substitutable?/0 says whether a lane may answer with a different model and VercelGateway answers true exactly while a fallback list is configured (vercel_gateway.ex:70), every metered record carries served_model and prices against it, and Threads.spend/1 reads it back (threads.ex:829). A substitutable lane that discloses nothing records unresolved and prices nothing, rather than naming the requested model. METER-001 and PROVIDER-002 are amended.

    Proof run here at b2e5ee0: inference_proxy_fallback_test.exs, stream_decoder_test.exs, pricing_test.exs, at_rest_test.exs, and model_catalog_live_test.exs together — 54 passed.

    That matters for this issue beyond one bug: item 5 was written as the thing that had to be fixed before any lane goes declared. That precondition is now met, so item 1 is no longer gated on anything technical.

    What is still open

    1. Real rates — owner action, and now the only thing between this deployment and a billable lane. gpt-5.6-luna has no price at all and the other two lanes carry source: :placeholder (config/config.exs:175, :198). Nothing here may guess a provider's price. PricingTest asserts no lane claims declared, so flipping one is a deliberate edit that fails a test until someone says so. Until then the account balance is a floor, not a total.

    2. The coder status line. The contract asks for grant ceilings and running spend in the coder's status line. This repository has no CLI source; the client is packages/openagents-cli in the openagents monorepo, and nothing on either tracker carries this today. Everything the client needs is already published: GET /api/v1/threads/{id} carries grant.spent, grant.remaining, and grant.pricing, and GET /api/v1/models carries the rates and bases.

    3. A web pricing surface. Done, f7b54ef.

    4. Credits that mean money — the half that actually sells anything. Credit is an allowance, not a purchased balance. Grants meter against it, nothing tops it up, and there is no payment path. "Sell metered model access behind the coding agent" is not yet true in the sense this issue's outcome states. Entangled with #207.

    5. Gateway fallback attribution. Done, b2e5ee0.

    So the issue is blocked on the owner for item 1, and on unwritten work for items 2 and 4. Nothing here is waiting on a decision an agent can take.

    No code changed in this sweep.

  6. A AtlantisPleb Author 1d ago

    Closed as won't do, at the owner's direction on 2026-08-25: nothing is billed, everything is free, so a metered offering has nobody to meter for.

    The capability stays and is honest, which is the point of closing rather than reverting. Every inference call records usage against its grant and stamps the rate table it was priced against. A lane with no declared rate reports no cost — never a zero — so Threads.spend/1 returns nil and the thread page and /models both read Unpriced. A call answered by a fallback model is priced against the model that actually served it, not the one requested, and the same is true of the lane's health. No lane claims declared, and a test asserts that, so switching one on is a deliberate edit rather than a drift.

    When billing matters, the remaining work is entering real provider rates and building credits that mean money. Reopen then.

  7. closed this as completed 1d ago
Sign in with GitHub to comment on this issue.