OpenAgents

Expose continual-learning jobs over verified licensed datasets #86

Closed AtlantisPleb opened this 23h ago 1 comment

Outcome

Expose a bounded continual-learning job API that consumes verified licensed datasets and emits reproducible training, evaluation, and model-artifact receipts.

Current behavior

Continual learning was promised as a service, and Tassadar proved long-running verified training work, but no current API connects a buyer, admitted dataset, budget, training job, evaluation, and deliverable artifact.

Source: docs/episode-triage.md, episodes 223, 227, 237, and the batch 2 integration notes.

Contract

  • Accept a versioned objective, base model, licensed dataset references, evaluation corpus, budget, runtime class, and stopping policy.
  • Resolve exact dataset and license digests before admission.
  • Run through Work and the existing fleet; do not add a separate scheduler.
  • Checkpoint durably and make resume distinct from replay.
  • Separate trainer and evaluator roles when policy requires independence.
  • Emit usage, energy, training, evaluation, artifact, and settlement-ready receipts.
  • Refuse expired consent, stale datasets, unsupported custody, or unverifiable evaluation.
  • Start with one internal buyer and one bounded workflow before multi-tenant release.

Acceptance criteria

  • An authorized caller can start, inspect, cancel, resume, and export one job.
  • The terminal artifact binds to the exact base model, data, code, configuration, checkpoints, and evaluation.
  • A revoked license blocks new jobs and has defined behavior for an active job.
  • Failed or incomplete evaluation cannot produce a qualified artifact.
  • Usage and accepted-outcome metrics reconcile with project 9 instrumentation.

Verification

Run one small reproducible training canary and fault tests for consent expiry, checkpoint loss, evaluator failure, cancellation, resume, privacy, and budget exhaustion.

Dependencies

Depends on licensed dataset listings, fleet operations, accepted-outcome receipts, and an explicitly named internal buyer.

  1. AtlantisPleb opened this issue 23h ago
  2. A AtlantisPleb Author 18h ago

    Landed on main as a548f80.

    What shipped

    One named internal buyer runs bounded continual-learning jobs whose every input is named before a round starts:

    • Admission resolves each training and evaluation dataset through the licensed artifact catalog (#85), so a job carries the exact artifact, provenance, license, and listing digests plus the acceptance receipt that licensed the use, and it binds the versioned objective, base model and digest, pinned training-code digest, and configuration digest.
    • Rounds run in the existing work lane under Horde, not a second scheduler, and match fleet capacity (#76) for the runtime class.
    • Each round commits a durable checkpoint chained by state digest before the round counter moves. A resume continues from the surviving checkpoint under the same admission digest and receipt chain; a replay is a new job that re-resolves every dataset and starts at round zero, which is what makes a reproduced artifact digest evidence.
    • A terminal artifact exists only after an admitted, independent evaluator passes and the accepted-outcome contract (#66) grades the claim. The artifact binds the model, every dataset digest, the checkpoint chain, the evaluation corpus and result, the accepted outcome, and a settlement-ready payload (#79) that names the treasury policy and the metered amount while recording that no custody moved in this lane.
    • Refusals are typed: expired or stale license, removed listing, unadmitted buyer, unadmitted model, base-model digest mismatch, unpinned training code, unadmitted or non-independent evaluator, evaluator identity mismatch, lost or missing checkpoint, exhausted budget, cancellation, and unavailable capacity.
    • Admission, usage, energy, training, evaluation, artifact, settlement, resume, and refusal receipts export as one auditable record over the operator API at /api/operator/continual-learning/jobs.

    The lane ships disabled: an operator admits the buyer, the base models, and the training-code digest through the OPENAGENTS_CONTINUAL_LEARNING_* variables before a job can start.

    Verification

    mix precommit passes for this slice. 23 tests cover admission refusals, the two-round canary, checkpoint chaining, energy and usage accounting, artifact and settlement-ready evidence, catalog transaction history, resume, replay reproducibility, cancellation, budget exhaustion, evidence privacy, and the operator routes.

    16 suite failures are unrelated to this change and reproduce without it: OpenAgents.Forge.GitPlaneTest and OpenAgents.Stacks.RestackTest need git merge-tree --write-tree (git 2.38 or later; this machine runs 2.34.1), and OpenAgents.ChangelogTest needs a seeded commit that is not in this clone's history.

    Written by Devin

  3. closed this as completed 18h ago
Sign in with GitHub to comment on this issue.