Reconcile documentation and invariant evidence

cee36e1a321f · Christopher David · · parent 4b8adbc37fa2

Reconcile documentation and invariant evidence

Deploy story

What this commit did to the running system — joined from the forge receipt chain, the part a commit page elsewhere cannot show.

Not deployed through the forge lane

No push, promotion, build, or deploy receipt references this commit (receipts are scanned over a bounded recent window). Changes shipped by full node replacement carry their proof in the release gate receipt instead.

Changed files

  • added .dockerignore
  • modified INVARIANTS.md
  • modified README.md
  • modified docs/2026-08-19-gap-implementation-plan.md
  • modified docs/2026-08-20-integration-hardening-and-staging-readiness-recommendations.md
  • modified docs/beam-hot-deployment-plan.md
  • modified docs/chat-inference-plan.md
  • modified docs/component-library.md
  • modified docs/decisions/0004-retain-scoped-github-access-tokens.md
  • modified docs/github-api-issues-projects-assessment.md
  • modified docs/github-auth-plan.md
  • modified docs/issues-projects-ui-roadmap.md
  • modified docs/issues-projects-work-plan.md
  • modified docs/sarah-integration-plan.md
  • modified lib/openagents_web/live/admin_live.ex
  • modified mix.exs
  • modified ops/ci/allowed-sarah-references.txt
  • added ops/ci/docs-check.exs
  • modified ops/ci/reference-check.sh
  • modified test/openagents_web/live/admin_live_test.exs

Diff

20 files changed, +1298 -2044

.dockerignore added +23

@@ -0,0 +1,23 @@

1
.git/
2
.github/
3
.githooks/
4
_build/
5
deps/
6
cover/
7
doc/
8
docs/
9
test/
10
tmp/
11
assets/node_modules/
12
priv/static/assets/
13
14
.env
15
.env.*
16
*.dump
17
*.log
18
*.pem
19
*.key
20
config/*.secret.exs
21
22
.DS_Store
23
**/.DS_Store
INVARIANTS.md modified +331 -86

@@ -1,13 +1,21 @@

1 1
# OpenAgents invariant ledger
2 2
3
These contracts define the current Simply OpenAgents release. A change to a listed
4
contract must update this ledger and its named test or model in the same
5
commit.
3
These contracts define the OpenAgents application. Every entry is explicitly
4
`Current` or `Proposed`. A current contract must name an executable test or a
5
concrete manual proof whose file exists in this repository. A proposed
6
contract describes a release gate, not current behavior, and cannot be used as
7
evidence that a feature is safe or enabled.
8
9
A change to a listed contract must update this ledger and its named proof in
10
the same commit. `ops/ci/docs-check.exs` verifies unique IDs and evidence-file
11
paths.
6 12
7 13
## OpenAgents identity and canon
8 14
9 15
### CANON-001 — Persona sources are immutable and status-labeled
10 16
17
Status: Current
18
11 19
Every historical source admitted to author OpenAgents's persona is pinned by
12 20
repository revision, path, content SHA-256, source status, admitted uses, and
13 21
explicit exclusions. The complete manifest has a canonical digest admitted by

@@ -19,22 +27,26 @@ Episode numbers are not source identity. The conflicting Episode 263 file is

19 27
quarantined, and the final Omega Alpha transcript is pinned by its actual path.
20 28
21 29
Evidence: `OpenAgents.Persona.SourceManifest`,
22
`priv/openagents/persona/openagents.v1.sources.json`, and
30
`priv/sarah/persona/sarah.v1.sources.json`, and
23 31
`OpenAgents.Persona.SourceManifestTest`.
24 32
25 33
### PERSONA-001 — Each inference uses one immutable persona artifact
26 34
35
Status: Current
36
27 37
OpenAgents's core identity, voice, and first-conversation greeting come from one
28 38
versioned artifact admitted by its exact content SHA-256. The artifact and its
29 39
source-manifest identity are validated and installed before the supervision
30 40
tree starts. Every provider request receives instructions composed from that
31 41
installed artifact; provider adapters contain no independent OpenAgents persona.
32 42
33
Evidence: `OpenAgents.Persona`, `priv/openagents/persona/openagents.v1.md`,
43
Evidence: `OpenAgents.Persona`, `priv/sarah/persona/sarah.v1.md`,
34 44
`OpenAgents.Context.Composer`, `OpenAgents.Turns.TurnServer`, and `OpenAgents.PersonaTest`.
35 45
36 46
### PERSONA-002 — One core OpenAgents identity composes with an admitted role
37 47
48
Status: Current
49
38 50
Protected identity and host safety precede surface truths, the selected role,
39 51
Blueprint expression facts, captured capabilities, and recalled evidence in a
40 52
deterministic composition. The public application selects only

@@ -46,23 +58,27 @@ protected layers. Inactive coding, sales, company-operations, and public-

46 58
broadcast registers fail closed to the compatible general baseline or no role.
47 59
48 60
Evidence: `OpenAgents.Context.Composer`, `OpenAgents.Roles`, `OpenAgents.Roles.Catalog`,
49
`OpenAgents.Roles.GeneralCollaborator`, `docs/ROLE_PROGRAMS.md`,
61
`OpenAgents.Roles.GeneralCollaborator`, `test/openagents/roles_test.exs`,
50 62
`OpenAgents.Context.ComposerTest`, and `OpenAgents.RolesTest`.
51 63
52 64
### PERSONA-003 — Persona promotion requires revision-bound regression evidence
53 65
66
Status: Current
67
54 68
Every persona candidate is evaluated against the committed, source-labeled
55 69
behavior corpus. Promotion requires a complete passing report bound to the
56 70
exact persona, source manifest, corpus, and model revisions. All cases must
57 71
pass, so military, founder-voice, sales, false-recognition, or generic-assistant
58 72
containment failures cannot be averaged away.
59 73
60
Evidence: `OpenAgents.Persona.Evaluation`,
61
`priv/openagents/evals/persona/corpus.v1.json`,
74
Evidence: `OpenAgents.Persona.Evaluation.Runner`,
75
`priv/sarah/evals/persona/corpus.v1.json`,
62 76
`OpenAgents.Persona.EvaluationTest`, and `mix openagents.persona.verify_promotion`.
63 77
64 78
### BLUEPRINT-001 — Platform facts are source-linked immutable revisions
65 79
80
Status: Current
81
66 82
Every admitted OpenAgents Blueprint revision is a complete canonical snapshot of
67 83
typed platform facts. Facts carry stable IDs, source refs/status/time/digests,
68 84
compatibility, introduction/retirement revisions, and admission provenance.

@@ -78,10 +94,12 @@ authorize execution.

78 94
79 95
Evidence: `OpenAgents.Blueprint`, the OpenAgents Blueprint schemas and migration,
80 96
`OpenAgents.Context.Composer`, `OpenAgents.Conversations.begin_inference/5`,
81
`docs/SARAH_BLUEPRINT.md`, and `OpenAgents.BlueprintTest`.
97
`test/openagents/blueprint_test.exs`, and `OpenAgents.BlueprintTest`.
82 98
83 99
### PROGRAM-001 — Model programs are immutable typed data, never effect authority
84 100
101
Status: Current
102
85 103
Every admitted model-program artifact has one full canonical digest covering
86 104
its signature, compiler/model/decoding identity, compatibility, Prompt IR,
87 105
parameters, independent datasets, optimizer budget, evaluator, metrics and

@@ -96,11 +114,13 @@ Each applicable turn captures one immutable artifact identity before provider

96 114
work; an in-flight capture cannot observe another catalog.
97 115
98 116
Evidence: `OpenAgents.ProgramArtifacts`, `OpenAgents.ProgramArtifacts.Reader`,
99
`priv/openagents/programs/`, `docs/PROGRAM_ARTIFACTS.md`,
117
`priv/sarah/programs/`, `test/openagents/program_artifacts_test.exs`,
100 118
`OpenAgents.Conversations.begin_inference/5`, and `OpenAgents.ProgramArtifactsTest`.
101 119
102 120
### DEGRADE-001 — Missing program artifacts degrade explicitly to a baseline
103 121
122
Status: Current
123
104 124
If an applicable signature has no admitted compatible artifact, capture returns
105 125
the deterministic baseline with no artifact ID/digest and a bounded degraded
106 126
receipt. Invalid or unapproved artifacts never enter the boot catalog. The turn

@@ -114,6 +134,8 @@ Evidence: `OpenAgents.ProgramArtifacts.capture/1`,

114 134
115 135
### PROGRAM-002 — Shadow programs have no live effect or private report payload
116 136
137
Status: Current
138
117 139
Shadow signatures validate bounded typed input/output and run under a separate
118 140
task supervisor. The live turn never consumes their result. Memory and
119 141
collective outputs remain proposals; routing can name only a capability in its

@@ -128,11 +150,13 @@ fixture path; production conversations/profile memory are not datasets.

128 150
129 151
Evidence: `OpenAgents.ShadowPrograms`, `OpenAgents.ShadowPrograms.Signatures`,
130 152
`OpenAgents.ShadowPrograms.OpenAI`, `shadow_program_runs`,
131
`priv/openagents/evals/shadow/corpus.v1.json`, `docs/SHADOW_PROGRAMS.md`, and
153
`priv/sarah/evals/shadow/corpus.v1.json`, `test/openagents/shadow_programs_test.exs`, and
132 154
`OpenAgents.ShadowProgramsTest`.
133 155
134 156
### PROGRAM-003 — Promotion is offline, human-approved, and rollback-capable
135 157
158
Status: Current
159
136 160
A compiled candidate, pinned independent evaluation, human approval, and human
137 161
activation are separate append-only artifacts/events. Evaluation is bound to
138 162
distinct train/validation/true-holdout manifests, exact evaluator identity,

@@ -146,21 +170,25 @@ an in-flight capture remains unchanged. Rollback selects the pinned predecessor

146 170
and preserves the complete prior trail.
147 171
148 172
Evidence: `OpenAgents.ProgramLifecycle`, its artifact/event/activation schemas and
149
database guards, `docs/PROGRAM_LIFECYCLE.md`, the synthetic sample promotion
173
database guards, `test/openagents/program_lifecycle_test.exs`, the synthetic sample promotion
150 174
report, and `OpenAgents.ProgramLifecycleTest`.
151 175
152 176
## Identity and authorization
153 177
154 178
### IDENTITY-001 — GitHub-authenticated account identity
155 179
180
Status: Current
181
156 182
Every OpenAgents interaction requires an active local user established through the
157 183
GitHub OAuth authorization-code flow. The immutable external key is GitHub's
158 184
numeric user ID; login and avatar URL are refreshable projections and never
159 185
authority. OAuth start uses high-entropy state plus PKCE S256. A short-lived
160 186
PostgreSQL attempt receipt makes state one-time even if an old encrypted cookie
161 187
is replayed. Phoenix exchanges the code and rereads `/user` server-side, then
162
discards the access token. The browser session contains only OpenAgents's local user
163
ID and is encrypted, signed, HTTP-only, same-site, and secure in production.
188
stores the delegated GitHub token as encrypted server-side ciphertext for
189
GitHub repository tools. The token never enters the browser. The browser
190
session contains only OpenAgents's local user ID and is encrypted, signed,
191
HTTP-only, same-site, and secure in production.
164 192
165 193
Evidence: `OpenAgents.GitHubOAuth`, `OpenAgents.Accounts`, `OpenAgentsWeb.AuthController`,
166 194
`OpenAgentsWeb.Endpoint.session_options/0`, `OpenAgents.GitHubOAuthTest`,

@@ -168,6 +196,8 @@ Evidence: `OpenAgents.GitHubOAuth`, `OpenAgents.Accounts`, `OpenAgentsWeb.AuthCo

168 196
169 197
### IDENTITY-002 — Conversation lookup never accepts a client database ID
170 198
199
Status: Current
200
171 201
The browser supplies only its encrypted OpenAgents session. Server code loads the
172 202
active local user and resolves that user's internal storage owner and canonical
173 203
conversation. A route parameter, form value, mutable GitHub login, or LiveView

@@ -181,6 +211,8 @@ and `OpenAgentsWeb.AuthGateTest`.

181 211
182 212
### IDENTITY-003 — Account continuity supersedes browser portability
183 213
214
Status: Current
215
184 216
One GitHub-authenticated user resolves the same canonical OpenAgents owner across
185 217
browsers and devices; no export/import ceremony is required for that account.
186 218
The disabled encrypted portability adapter remains only an explicit transfer

@@ -203,12 +235,14 @@ by OpenAgents. Adapter failure leaves account-local operation unchanged.

203 235
204 236
Evidence: ADR 0002 (which supersedes ADR 0001's identity decision),
205 237
`OpenAgents.Memory.Portability`, its envelope and receipt schemas,
206
`docs/MEMORY_PORTABILITY_THREAT_MODEL.md`, and `OpenAgents.MemoryPortabilityTest`.
238
`test/openagents/memory_portability_test.exs`, and `OpenAgents.MemoryPortabilityTest`.
207 239
208 240
## Data authority and synchronization
209 241
210 242
### DATA-001 — PostgreSQL is authoritative
211 243
244
Status: Current
245
212 246
Visitors, conversations, messages, and turns are persisted before their state
213 247
is presented as accepted. PubSub and LiveView streams are projections; losing
214 248
either must not lose accepted data.

@@ -218,6 +252,8 @@ Evidence: `OpenAgents.Conversations.create_turn/2` and

218 252
219 253
### DATA-002 — One canonical conversation per authenticated user
220 254
255
Status: Current
256
221 257
Database uniqueness enforces one internal storage owner per local user and one
222 258
conversation per owner. Initial greeting creation is coupled to first
223 259
conversation creation. Distinct users remain isolated; multiple sessions for

@@ -231,6 +267,8 @@ account continuity/isolation/rate tests in `OpenAgents.AccountsTest`.

231 267
232 268
### DATA-003 — History is bounded and stable
233 269
270
Status: Current
271
234 272
The newest page and every older page contain at most the configured page size.
235 273
Ordering uses persisted timestamp plus UUID, and rendered rows use stable
236 274
message IDs.

@@ -241,6 +279,8 @@ Evidence: `OpenAgents.Conversations.list_messages/2` and bounded-history test.

241 279
242 280
### MEMORY-001 — Recall is confined to the current account conversation
243 281
282
Status: Current
283
244 284
Every recall snapshot and search is bound to one canonical conversation
245 285
resolved from the authenticated local user. A snapshot from another
246 286
conversation or user is refused, and no API offers a cross-conversation or

@@ -252,6 +292,8 @@ cross-scope tests in `OpenAgents.Memory.LexicalRecallTest`.

252 292
253 293
### MEMORY-002 — Recalled history is classified evidence, not current profile truth
254 294
295
Status: Current
296
255 297
Every materially used read source is represented by a host-built
256 298
`openagents.memory_evidence.v1` value with validated source/scope, observed and
257 299
recalled times, bounded claim and relevance, host classification, and validated

@@ -273,6 +315,8 @@ database trigger, `OpenAgents.Memory.EvidenceTest`, and

273 315
274 316
### MEMORY-003 — Profile memory is explicit, inspectable, correctable, and forgettable
275 317
318
Status: Current
319
276 320
Durable profile claims live in a separate account-owner plane. A record
277 321
is only active with a same-owner complete user-message source or a host-recorded
278 322
explicit owner assertion. Candidates never activate through repetition or

@@ -287,11 +331,13 @@ deleting the independent conversation source message.

287 331
288 332
Evidence: `OpenAgents.ProfileMemory`, the profile-memory schemas and database
289 333
constraint triggers, `OpenAgentsWeb.ChatLive`, `OpenAgentsWeb.MemoryExportController`,
290
`docs/MEMORY_CONTROLS.md`, `docs/PROFILE_MEMORY.md`, `OpenAgents.ProfileMemoryTest`,
334
`test/openagents/profile_memory_test.exs`, `test/openagents/profile_memory_test.exs`, `OpenAgents.ProfileMemoryTest`,
291 335
and memory-control journeys in `OpenAgentsWeb.ChatLiveTest`.
292 336
293 337
### MEMORY-004 — Scope and snapshot boundaries are database predicates
294 338
339
Status: Current
340
295 341
Recall scope is enforced by `messages.conversation_id` in every PostgreSQL
296 342
query, never by prompt instructions. Each inference immutably records a
297 343
`message:<uuid>` high-water ref for the last eligible message before the current

@@ -348,6 +394,8 @@ cross-browser/concurrent-snapshot tests in `OpenAgents.ProfileMemoryTest`.

348 394
349 395
### MEMORY-005 — Memory writes require exact current consent
350 396
397
Status: Current
398
351 399
A sampled tool call is a proposal, not write authority. `memory_remember.v1`
352 400
accepts only an exact bounded claim directly authorized by the current complete
353 401
user message, an exact host-recorded confirmation, or an exact first-party UI

@@ -362,11 +410,13 @@ bounded reversible receipt after PostgreSQL commits. OpenAgents may acknowledge

362 410
only the exact successful result returned in the durable tool outcome.
363 411
364 412
Evidence: `OpenAgents.Memory.Consent`, the four `OpenAgents.Tools.Memory*` tools,
365
`turn_receipts.profile_memory_snapshot_ref`, `docs/MEMORY_TOOLS.md`, and
413
`turn_receipts.profile_memory_snapshot_ref`, `test/openagents/tools/profile_memory_tools_test.exs`, and
366 414
`OpenAgents.Tools.ProfileMemoryToolsTest`.
367 415
368 416
### MEMORY-006 — Semantic recall is scoped, disposable, and lexically degradable
369 417
418
Status: Current
419
370 420
Authoritative durable conversation rows — messages, and for lexical
371 421
tool-activity recall the terminal tool steps — remain the sole recall
372 422
authority; semantic embeddings exist only for messages. Embeddings are

@@ -388,12 +438,14 @@ creating vectors; its committed comparison must preserve lexical fallback and

388 438
scope isolation while improving synonym recall.
389 439
390 440
Evidence: `OpenAgents.Memory.SemanticIndex`, `OpenAgents.Memory.HybridRecall`, semantic
391
derivative tables and triggers, `docs/HYBRID_RECALL.md`,
392
`priv/openagents/evals/recall/hybrid-comparison.v1.json`, and
441
derivative tables and triggers, `test/openagents/semantic_recall_test.exs`,
442
`priv/sarah/evals/recall/hybrid-comparison.v1.json`, and
393 443
`OpenAgents.SemanticRecallTest`.
394 444
395 445
### MEMORY-007 — Learned preferences require confirmation and never confer authority
396 446
447
Status: Current
448
397 449
Behavior preferences occupy an account-owner plane separate from conversation
398 450
evidence, profile facts, roles, tool catalogs, routing authority, and collective
399 451
artifacts. An observation or model confidence can create only a candidate. The

@@ -416,11 +468,13 @@ direct or foreign-scope receipt deletion.

416 468
417 469
Evidence: `OpenAgents.Preferences`, the preference schemas and database guards,
418 470
`OpenAgents.Context.Composer`, `turn_receipts.used_preferences`,
419
`docs/GOVERNED_PREFERENCES.md`, the committed preference comparison, and
471
`test/openagents/preferences_test.exs`, the committed preference comparison, and
420 472
`OpenAgents.PreferencesTest`.
421 473
422 474
### MEMORY-008 — Experience is private, terminal, evidenced, and advisory
423 475
476
Status: Current
477
424 478
Work experience occupies an exact account-owner and conversation-work-scope
425 479
plane separate from profile facts, learned preferences, roles, tool authority,
426 480
and collective artifacts. Requested and running cases never enter recall.

@@ -446,11 +500,13 @@ Direct and foreign-scope receipt deletion remain rejected.

446 500
447 501
Evidence: `OpenAgents.ExperienceMemory`, the experience schemas and database guards,
448 502
`OpenAgents.Context.Composer`, `turn_receipts.used_experiences`,
449
`docs/PRIVATE_EXPERIENCE_MEMORY.md`, the committed benefit comparison, and
503
`test/openagents/experience_memory_test.exs`, the committed benefit comparison, and
450 504
`OpenAgents.ExperienceMemoryTest`.
451 505
452 506
### MEMORY-009 — Graph memory is a disposable, generation-atomic projection
453 507
508
Status: Current
509
454 510
The relationship graph is never memory authority. Every node and edge belongs
455 511
to one account owner, one conversation work scope, one immutable manifest
456 512
generation, and at least one exact authoritative experience-record or pattern

@@ -478,11 +534,13 @@ memberships, mutation events, cascade plans, and operation receipts after the

478 534
visitor root is gone. Direct and foreign-scope receipt deletion remain rejected.
479 535
480 536
Evidence: `OpenAgents.GraphMemory`, graph manifests/artifacts/memberships/outbox and
481
database guards, `docs/DERIVED_GRAPH_MEMORY.md`, the committed graph comparison,
537
database guards, `test/openagents/graph_memory_test.exs`, the committed graph comparison,
482 538
and `OpenAgents.GraphMemoryTest`.
483 539
484 540
### PRIVACY-001 — Secret-bearing profile memory is rejected, never scrub-stored
485 541
542
Status: Current
543
486 544
Before candidate storage, the host applies the pinned
487 545
`openagents.memory.policy.v1` policy to the claim, provenance/artifact metadata, and
488 546
same-owner source content. Credential, API/auth token, wallet/seed/payment,

@@ -498,12 +556,14 @@ cannot silently relabel old records or rejection evidence.

498 556
499 557
Evidence: `OpenAgents.Memory.Policy`, `OpenAgents.Memory.Redaction`,
500 558
`profile_memory_policy_events`, immutable policy-version trigger,
501
`docs/MEMORY_PRIVACY_POLICY.md`, and `OpenAgents.Memory.PolicyAndRedactionTest`.
559
`test/openagents/memory/policy_and_redaction_test.exs`, and `OpenAgents.Memory.PolicyAndRedactionTest`.
502 560
503 561
## Turn and provider lifecycle
504 562
505 563
### TURN-001 — At most one active turn per conversation
506 564
565
Status: Current
566
507 567
An active turn is `queued` or `streaming`. A partial unique PostgreSQL index is
508 568
the final arbiter; the UI's disabled composer is only feedback.
509 569

@@ -511,6 +571,8 @@ Evidence: `turns_one_active_per_conversation_index` and active-turn test.

511 571
512 572
### TURN-002 — Every accepted turn has durable paired messages
513 573
574
Status: Current
575
514 576
The user message, empty streaming assistant message, and turn record are
515 577
inserted in one transaction. Completion, failure, and cancellation update both
516 578
assistant-message and turn terminal state.

@@ -519,15 +581,19 @@ Evidence: `OpenAgents.Conversations.create_turn/2`, `finish_turn/5`, and turn te

519 581
520 582
### TURN-003 — Provider work never blocks the LiveView
521 583
584
Status: Current
585
522 586
Each response executes in a temporary `TurnServer` under a dynamic supervisor;
523 587
the outbound provider call executes in a supervised task. Text deltas cross a
524 588
typed provider callback and are persisted before broadcast.
525 589
526
Evidence: `OpenAgents.Turns.TurnServer`, `OpenAgents.ProviderTaskSupervisor`, and
527
`OpenAgentsWeb.ChatLiveTest` streaming test.
590
Evidence: `OpenAgents.Turns.TurnServer`, its named supervised provider task,
591
and `OpenAgentsWeb.ChatLiveTest` streaming test.
528 592
529 593
### TURN-004 — Interrupted work becomes explicit failure
530 594
595
Status: Current
596
531 597
Active records left by a runtime restart are marked failed during application
532 598
startup. A response is never left permanently presented as in progress without
533 599
an executing turn process.

@@ -537,6 +603,8 @@ Evidence: `OpenAgents.Conversations.recover_interrupted_turns/0`, the

537 603
538 604
### TURN-005 — Tool continuations are serial, bounded, and commit-first
539 605
606
Status: Current
607
540 608
One turn may request a bounded number of tool calls and provider continuations
541 609
(sixteen of each today). Calls execute one at a time; parallel calls fail the
542 610
turn. Hitting an execution bound refuses the over-limit call with a durable

@@ -553,6 +621,8 @@ Evidence: `OpenAgents.Turns.TurnServer`, `OpenAgents.Providers.OpenAI.request_pa

553 621
554 622
### PROVENANCE-001 — Every new inference has an immutable receipt
555 623
624
Status: Current
625
556 626
Before provider work starts, OpenAgents durably captures the exact model, persona,
557 627
role, instruction digest, canonical input digest, optional runtime artifact
558 628
identities, and first provider step. Identity fields never change; terminal

@@ -565,6 +635,8 @@ triggers, `OpenAgents.Provenance.Canonical`, and `OpenAgents.TurnProvenanceTest`

565 635
566 636
### PROVIDER-001 — Model providers are replaceable
567 637
638
Status: Current
639
568 640
Conversation and web code depend on `OpenAgents.Providers.Provider`, not OpenAI
569 641
event shapes. Adapters emit typed OpenAgents-domain lifecycle, text, tool-call,
570 642
usage, completion, failure, and cancellation events. A response ID is persisted

@@ -580,6 +652,8 @@ Evidence: `OpenAgents.Providers.ProviderEvent`, `OpenAgents.Providers.OpenAI`,

580 652
581 653
### TOOL-001 — A turn uses one immutable tool catalog
582 654
655
Status: Current
656
583 657
The registry validates configured tool specifications at boot. Before provider
584 658
work, each turn captures one catalog snapshot and writes its canonical digest
585 659
to the immutable receipt. Every call must match the exact tool name and version

@@ -590,6 +664,8 @@ Evidence: `OpenAgents.Tools.Registry`, `OpenAgents.Tools.Snapshot`,

590 664
591 665
### COLLECTIVE-001 — Private material crosses scope only through exact consent
592 666
667
Status: Current
668
593 669
A collective candidate can be created only in the same transaction as an active
594 670
`collective_contribution` receipt from the owning person. The receipt binds the
595 671
exact browser-owned source refs and their content digest, source-scope digest,

@@ -614,6 +690,8 @@ the consent, isolation, raw-copy, and withdrawal cases in `OpenAgents.Collective

614 690
615 691
### COLLECTIVE-002 — Generalization is bounded, content-free, and reproducible
616 692
693
Status: Current
694
617 695
Only an authenticated privacy reviewer in the candidate owner's scope can run
618 696
generalization or inspect opaque lineage. The versioned fixed-vocabulary
619 697
generalizer recognizes a bounded supported signal and emits one admitted schema

@@ -637,6 +715,8 @@ scans in `OpenAgents.CollectiveGeneralizerTest`.

637 715
638 716
### COLLECTIVE-003 — Publication requires independent evidence and operator authority
639 717
718
Status: Current
719
640 720
A generalized candidate reaches the cross-user collective catalog only when its
641 721
contribution consent remains active, its privacy generalizer is bound to an
642 722
authenticated reviewer, and a different authenticated evaluator records a

@@ -672,6 +752,8 @@ guards, and `OpenAgents.CollectivePublicationTest`.

672 752
673 753
### COMPENSATION-001 — Attribution accounting never creates payout authority
674 754
755
Status: Current
756
675 757
Technical invocation cost, contributor attribution, compensation eligibility,
676 758
and payment are distinct facts. An event is compensation-eligible only when an
677 759
operator-admitted policy with `payout_authority: false` binds an exact immutable

@@ -696,12 +778,14 @@ results, conversation content, user identity, payment instructions, custody, or

696 778
a payout operation.
697 779
698 780
Evidence: `OpenAgents.Compensation`, its seven typed receipt schemas and database
699
constraints, `docs/COMPENSATION_ACCOUNTING.md`, and duplicate, revocation,
781
constraints, `test/openagents/compensation_test.exs`, and duplicate, revocation,
700 782
allocation, adjustment, reconciliation, privacy, and no-payout cases in
701 783
`OpenAgents.CompensationTest`.
702 784
703 785
### MODULE-001 — Every invocation pins one immutable admitted module
704 786
787
Status: Current
788
705 789
The captured turn registry contains provider-neutral module artifacts whose
706 790
canonical digest covers typed input/output, lifecycle state, side-effect and
707 791
approval classes, capability/data scopes, policy facets, executor identity,

@@ -727,6 +811,8 @@ invocation reconciliation tests in `OpenAgents.ToolStepPersistenceTest`.

727 811
728 812
### MODULE-002 — Discovery and lifecycle never grant model authority
729 813
814
Status: Current
815
730 816
The model-facing discovery tool receives the exact registry already captured by
731 817
its turn and returns at most twenty bounded public metadata references. It cannot
732 818
register a module, reveal executable/provider schemas or private configuration,

@@ -749,6 +835,8 @@ database trigger, and the discovery/lifecycle tests.

749 835
750 836
### MODULE-003 — Routing proposals cannot weaken explicit policy
751 837
838
Status: Current
839
752 840
Module routing receives a captured registry, a versioned/digested host policy,
753 841
an intent digest, required capability/effect/data scope, and application-created
754 842
authorities. Publisher, cost class and numeric budget, quality, privacy,

@@ -771,6 +859,8 @@ route-receipt trigger, `OpenAgents.Turns.TurnServer`, and router/tool-loop tests

771 859
772 860
### MODULE-004 — Every capability surface preserves the same authority boundary
773 861
862
Status: Current
863
774 864
Every route and invocation names exactly one admitted surface from `text`,
775 865
`voice`, `search`, `computer`, `repository`, `mcp`, or `agent`. The selected
776 866
artifact must admit that surface, kind, and effect; revalidation refuses a

@@ -788,11 +878,13 @@ its proposals pass ordinary revalidation. Missing executors fail honestly.

788 878
789 879
Evidence: `OpenAgents.Modules.SurfacePolicy`, the `surface` field on module route
790 880
decisions/receipts, `OpenAgents.Tools.Registry.prompt_catalog/1`,
791
`OpenAgents.Tools.Runner`, `docs/MODULE_SURFACES.md`, and the surface, catalog,
881
`OpenAgents.Tools.Runner`, `test/openagents/surface_eval_test.exs`, and the surface, catalog,
792 882
approval, receipt, voice-interruption, and degradation tests.
793 883
794 884
### TOOL-002 — Model requests never widen host authority
795 885
886
Status: Current
887
796 888
Tool name, arguments, recalled text, and prompt content grant no scope or
797 889
authority. The runner checks the application-created execution context against
798 890
the captured specification before implementation code runs. It admits read-only

@@ -806,6 +898,8 @@ authority, schema, and side-effect runner tests.

806 898
807 899
### TOOL-003 — Tool outcomes are durable before provider continuation
808 900
901
Status: Current
902
809 903
Every provider call ID maps to one ordered immutable request row. A worker must
810 904
atomically claim `requested -> running`; duplicate requests return the existing
811 905
row and duplicate claims cannot execute it again. Provider continuation output

@@ -819,6 +913,8 @@ transition trigger, `OpenAgents.Conversations.tool_continuation_output/1`, and

819 913
820 914
### TOOL-004 — Every outcome identifies the actual executor
821 915
916
Status: Current
917
822 918
Every success, failure, refusal, cancellation, timeout, or unavailable result
823 919
is a bounded `openagents.tool_outcome.v1` envelope naming the captured module version
824 920
and actual executor/disclosure. Attribution and target receipt refs are bounded

@@ -835,7 +931,9 @@ Evidence: `OpenAgents.Tools.Tool`, `OpenAgents.Tools.ExecutionResult`,

835 931
`OpenAgents.Tools.Runner`, `OpenAgents.Conversations.ToolStep`, the invocation-ledger
836 932
database constraints, normalized-outcome tests, and executor-disclosure UI tests.
837 933
838
### DEGRADE-001 — Tool degradation is explicit and deterministic
934
### DEGRADE-002 — Tool degradation is explicit and deterministic
935
936
Status: Current
839 937
840 938
Unknown versions, invalid schemas or arguments, scope/authority refusal,
841 939
unsupported effects, cancellation, timeout, crashes, and oversized/invalid

@@ -856,6 +954,8 @@ degradation tests in `OpenAgents.Tools.ConversationRecallToolsTest`.

856 954
857 955
### WORK-001 — Delegated jobs are durable, budgeted, governed, and never die silently
858 956
957
Status: Current
958
859 959
A `deep_work.v1` call is only delegation, never execution: it starts one
860 960
durable `work_jobs` row scoped to the caller's conversation and owner and
861 961
returns immediately with a job reference, so the requesting turn or voice

@@ -897,6 +997,8 @@ Evidence: `OpenAgents.Work`, `OpenAgents.Work.Job`, `OpenAgents.Work.JobStep`,

897 997
898 998
### SELF-EDIT-001 — Every behavior change is anchored to a pushed commit (2026-08-19)
899 999
1000
Status: Current
1001
900 1002
OpenAgents may edit her own source only through governed repository tools acting
901 1003
on a per-job clone of her own forge, and nothing she writes becomes running
902 1004
behavior except through the receipted pipeline. Concretely:

@@ -923,13 +1025,16 @@ behavior except through the receipted pipeline. Concretely:

923 1025
924 1026
Evidence: `OpenAgents.Tools.Repository` (clone confinement, branch discipline,
925 1027
typed refusals), `OpenAgents.Work.Coding`, `OpenAgents.Forge.Pushes` /
926
`OpenAgents.Forge.Targets` / `OpenAgents.Forge.HotLoader` receipts, ADMIN-001, and
927
`OpenAgents.Tools.RepositoryTest` / `OpenAgents.CodingJobTest`.
1028
`OpenAgents.Forge.Targets` / `OpenAgents.Forge.HotLoader` receipts, ADMIN-001,
1029
`OpenAgents.CodingJobTest`, and the repository tool tests in
1030
`test/openagents/tools/repository_mutation_tools_test.exs`.
928 1031
929 1032
## Interface and release
930 1033
931 1034
### VOICE-001 — Spoken identity is admitted before media
932 1035
1036
Status: Current
1037
933 1038
Standalone OpenAgents's first voice artifact is `openagents.voice.openai.marin.v1` using
934 1039
native OpenAI Realtime `gpt-realtime-2.1` at low reasoning effort. It is a
935 1040
deliberate repository-local revision of the earlier Leda direction, not a

@@ -939,10 +1044,12 @@ Leda cascade, or built-in replacement requires a reviewed artifact revision

939 1044
and regression evidence.
940 1045
941 1046
Evidence: `OpenAgents.Voice.Config`,
942
`docs/voice/OPENAI_REALTIME_DECISION.md`, and `OpenAgents.Voice.ConfigTest`.
1047
`test/openagents/voice/config_test.exs`, and `OpenAgents.Voice.ConfigTest`.
943 1048
944 1049
### VOICE-002 — Browser media admission cannot acquire server authority
945 1050
1051
Status: Current
1052
946 1053
Voice is default-disabled. When enabled, an active authenticated user may send
947 1054
only a bounded SDP offer through the same-origin, CSRF-protected endpoint.
948 1055
Phoenix supplies the OpenAI credential, session configuration, and stable

@@ -958,6 +1065,8 @@ Evidence: `OpenAgentsWeb.VoiceCallController`,

958 1065
959 1066
### VOICE-003 — Durable voice history is generation-fenced
960 1067
1068
Status: Current
1069
961 1070
PostgreSQL owns every admitted voice generation and permits at most one active
962 1071
generation per conversation. Provider events, response receipts, and
963 1072
transcript items repeat the admitted generation; both runtime matching and

@@ -972,11 +1081,13 @@ successful resume or transcript.

972 1081
973 1082
Evidence: `OpenAgents.Voice`, the `create_voice_runtime` migration,
974 1083
`OpenAgents.VoiceSessions.SessionServer`, `OpenAgents.VoiceRecovery`,
975
`docs/voice/VOICE_RUNTIME.md`, `OpenAgents.VoiceTest`, and
1084
`test/openagents/voice_sessions_test.exs`, `OpenAgents.VoiceTest`, and
976 1085
`OpenAgents.VoiceSessionsTest`.
977 1086
978 1087
### VOICE-004 — Only bounded provider-neutral voice evidence becomes durable
979 1088
1089
Status: Current
1090
980 1091
OpenAI wire events are decoded behind the provider adapter. Audio deltas,
981 1092
partial transcript deltas, credentials, SDP, provider error text, and unbounded
982 1093
wire payloads never enter OpenAgents's durable voice domain. Final user

@@ -996,6 +1107,8 @@ Evidence: `OpenAgents.Voice.ProviderEvent`,

996 1107
997 1108
### VOICE-005 — Microphone capture is explicit, fenced, visible, and finite
998 1109
1110
Status: Current
1111
999 1112
Only a direct `START VOICE` action may request microphone access. New tracks
1000 1113
begin disabled and may transmit only while the browser peer and control channel
1001 1114
are open, playback is usable, the user has not muted, and LiveView projects a

@@ -1015,10 +1128,12 @@ such claim anywhere.

1015 1128
Evidence: `assets/js/voice_controller.js`, `assets/js/voice_state.mjs`,
1016 1129
`assets/js/voice_recording.mjs`, their Node tests, the disclosure tests in
1017 1130
`OpenAgentsWeb.ChatLiveTest` and `OpenAgentsWeb.DataControllerTest`, and
1018
`docs/voice/BROWSER_TRANSPORT.md`.
1131
`assets/test/voice_state_test.mjs`.
1019 1132
1020 1133
### VOICE-006 — Voice controls project server truth and preserve typed OpenAgents
1021 1134
1135
Status: Current
1136
1022 1137
Browser peer events cannot claim a durable listening, responding,
1023 1138
interrupted, ended, or failed state. The visible lifecycle is derived from the
1024 1139
browser-scoped, generation-fenced PostgreSQL session projected by LiveView.

@@ -1028,10 +1143,12 @@ voice generation first, so two OpenAgents responses cannot run in parallel. Voic

1028 1143
failure leaves the typed conversation intact and available.
1029 1144
1030 1145
Evidence: `OpenAgentsWeb.ChatLive`, `OpenAgentsWeb.VoiceCallController`,
1031
`OpenAgents.VoiceSessions`, their tests, and `docs/voice/BROWSER_TRANSPORT.md`.
1146
`OpenAgents.VoiceSessions`, their tests, and `assets/test/voice_state_test.mjs`.
1032 1147
1033 1148
### VOICE-007 — Every live response freezes one governed OpenAgents context
1034 1149
1150
Status: Current
1151
1035 1152
Automatic provider response creation is disabled. A final user transcript first
1036 1153
becomes a complete conversation message, then Phoenix captures the exact
1037 1154
persona/role, Blueprint, program-or-explicit-baseline receipt, tool catalog,

@@ -1065,6 +1182,8 @@ compaction tests in `OpenAgents.VoiceSessionsTest`.

1065 1182
1066 1183
### VOICE-008 — Realtime function calls use the governed tool runner
1067 1184
1185
Status: Current
1186
1068 1187
Realtime exposes the same captured typed registry used by text turns. Model
1069 1188
function arguments remain proposals: Phoenix validates call identity, schema,
1070 1189
scope, authority, consent, generation, and module version through

@@ -1092,6 +1211,8 @@ Evidence: `OpenAgents.Tools.Registry.realtime_catalog/1`,

1092 1211
1093 1212
### VOICE-009 — Text and voice share one append-only conversation authority
1094 1213
1214
Status: Current
1215
1095 1216
Final user transcriptions and completed OpenAgents transcripts project into the
1096 1217
same ordered `messages` table as text. Provider item identity makes retries
1097 1218
idempotent. Interrupted OpenAgents speech is cancelled evidence and never enters

@@ -1107,10 +1228,12 @@ so text and voice cannot produce parallel OpenAgents answers.

1107 1228
Evidence: `messages` voice provenance and transition constraints,
1108 1229
`OpenAgents.Voice.persist_transcript/3`, `OpenAgents.Conversations.provider_messages/1`,
1109 1230
`OpenAgentsWeb.ChatLive`, cross-modal chronology tests in `OpenAgents.VoiceTest`, and
1110
`priv/openagents/evals/voice/corpus.v1.json`.
1231
`priv/sarah/evals/voice/corpus.v1.json`.
1111 1232
1112 1233
### VOICE-010 — New voice admission is live-governed, globally bounded, and attributable
1113 1234
1235
Status: Current
1236
1114 1237
The deploy flag and append-only PostgreSQL release control are independent.
1115 1238
Only the latest `open` control admits a new call; `draining` and `disabled`
1116 1239
refuse it without ending active calls or typed OpenAgents. Every session pins the

@@ -1126,10 +1249,12 @@ lets the model wrap up, and a budget-ended session records and surfaces

1126 1249
Evidence: `OpenAgents.Voice.ReleaseControl`, `OpenAgents.Voice.admit_session/2`,
1127 1250
`voice_release_controls`, `OpenAgents.Voice.Usage`,
1128 1251
`OpenAgents.Voice.ReleaseOperationsTest`, and
1129
`docs/voice/RELEASE_OPERATIONS.md`.
1252
`test/openagents/voice/release_operations_test.exs`.
1130 1253
1131 1254
### VOICE-011 — Voice operations are measurable without becoming a content sink
1132 1255
1256
Status: Current
1257
1133 1258
Operational telemetry is built from strict fields and may never contain
1134 1259
credentials, browser/conversation identity, SDP, provider call identity, raw
1135 1260
audio, transcript content, composed instructions, raw tool arguments/results,

@@ -1144,10 +1269,12 @@ health cannot substitute for media, quality, browser, load, or rollback proof.

1144 1269
Evidence: `OpenAgents.Voice.OperationalTelemetry`,
1145 1270
`OpenAgents.Voice.Operations.Report`, `OpenAgents.Voice.ClientEvent`,
1146 1271
`OpenAgentsWeb.VoiceTelemetryController`, their redaction and controller tests, and
1147
`docs/voice/RELEASE_OPERATIONS.md`.
1272
`test/openagents/voice/release_operations_test.exs`.
1148 1273
1149 1274
### VOICE-012 — Call audio is bounded, sealed, fenced evidence — never authority
1150 1275
1276
Status: Current
1277
1151 1278
Voice media never reaches OpenAgents: it flows browser-to-OpenAI over WebRTC while
1152 1279
the server holds only a lifecycle sideband. A recording is therefore what one
1153 1280
browser uploaded, not what was said. It can be withheld, truncated, or stopped

@@ -1178,20 +1305,21 @@ conversation rather than a failed one.

1178 1305
Recording is a property of the voice surface, not a per-account setting. While
1179 1306
it is enabled there is no opt-out flag and none may be added without changing
1180 1307
this contract; the disclosure states the situation rather than offering a
1181
choice, and typed chat remains available and is never recorded. The account
1182
cannot play back its own audio either — the operator surface is the only place a
1183
recording is audible, and the account's export carries the recording's metadata
1184
rather than its sound.
1308
choice, and typed chat remains available and is never recorded. No current
1309
account or operator route returns stored audio; account export carries
1310
recording metadata rather than its sound.
1185 1311
1186 1312
Evidence: `OpenAgents.Voice.Recordings`, `OpenAgents.Voice.Recording`,
1187 1313
`OpenAgents.Voice.RecordingChunk`, `OpenAgents.Voice.RecordingVault`, the
1188 1314
`create_voice_recordings` migration, `OpenAgentsWeb.VoiceRecordingController`,
1189 1315
`assets/js/voice_recording.mjs`, `OpenAgents.Voice.RecordingsTest`,
1190 1316
`OpenAgentsWeb.VoiceRecordingControllerTest`, `assets/test/voice_recording_test.mjs`,
1191
and `docs/voice/RECORDINGS.md`.
1317
and `test/openagents/voice/recordings_test.exs`.
1192 1318
1193 1319
### ADMIN-001 — One operator reads across accounts, and only reads
1194 1320
1321
Status: Current
1322
1195 1323
IDENTITY-002 confines every ordinary server path to the active user's own data.
1196 1324
`/admin` is the second deliberate exception after LEADERBOARD-001, and it is
1197 1325
the opposite kind: the leaderboard publishes a narrow projection to the

@@ -1219,21 +1347,23 @@ in the WAL-backed repository are promotable, so the surface cannot introduce

1219 1347
code — it can only approve code that already survived the push path. It
1220 1348
still cannot touch any account, conversation, message, ban, or product
1221 1349
configuration; ADMIN-001's read-only rule continues to bind everything else
1222
on the operator surface, including the original `/admin` panel unchanged. What it may show is the fields of `OpenAgents.Admin.Call` and the
1223
audio itself: account display identity, call lifecycle, model, token total, and
1224
recording completeness. Transcript *content*, composed instructions, tool
1225
catalogs, provider call identity, and recall material stay out — cross-account
1226
listening was the decision, and cross-account reading of what was said is a
1227
separate one. Calls with no audio are listed with the reason rather than
1228
hidden, so the panel cannot present an incomplete history as a complete one.
1229
1230
Evidence: `OpenAgents.Accounts.admin?/1`, `OpenAgentsWeb.UserAuth.require_admin_user/2`,
1231
the `:ensure_admin` mount hook, `OpenAgents.Admin`, `OpenAgents.Admin.Call`,
1232
`OpenAgentsWeb.AdminLive`, `OpenAgentsWeb.AdminRecordingController`, `OpenAgents.AdminTest`,
1233
`OpenAgentsWeb.AdminLiveTest`, and `OpenAgentsWeb.AdminRecordingControllerTest`.
1350
on the operator surface, including the original `/admin` panel unchanged. It
1351
may show the bounded fields of `OpenAgents.Admin.Call`: account display
1352
identity, call lifecycle, model, token total, transcript-item count, and
1353
recording completeness metadata. No routed controller returns recording audio,
1354
transcript content, composed instructions, tool catalogs, provider call
1355
identity, or recall material. Calls with no uploaded recording are listed with
1356
the reason rather than hidden, so the panel cannot present an incomplete
1357
history as a complete one.
1358
1359
Evidence: `OpenAgents.Accounts.admin?/1`, `OpenAgents.Admin`,
1360
`OpenAgents.Admin.Call`, `OpenAgentsWeb.AdminLive`, `OpenAgents.AdminTest`, and
1361
`OpenAgentsWeb.AdminLiveTest`.
1234 1362
1235 1363
### DATA-004 — The authenticated user can export and delete OpenAgents product data
1236 1364
1365
Status: Current
1366
1237 1367
The server resolves export and deletion only from the active local user in the
1238 1368
encrypted session and verifies that user owns the internal storage root.
1239 1369
Export provides canonical messages, profile memory, voice summaries, and

@@ -1244,8 +1374,7 @@ audio included, through `voice_recordings`' cascade to the session. The export

1244 1374
names each call's recording — status, container, size, duration claim, digest,
1245 1375
and that it is encrypted at rest — without embedding the audio, because a JSON
1246 1376
export is the wrong container for Opus and base64 in a text field would be
1247
worse. The account cannot play back its own audio: the operator surface is the
1248
only place a recording is audible. That is a decision rather than an omission —
1377
worse. No current product route returns stored audio to an account or operator;
1249 1378
what exists is disclosed and exported as metadata, and deletion removes it.
1250 1379
Detailed terminal voice
1251 1380
operations purge automatically after 90 days while the minimal provenance stub

@@ -1262,10 +1391,13 @@ memberships, outbox events, cascade plans, and operation receipts are likewise

1262 1391
removed in that transaction only after deletion of the visitor root.
1263 1392
1264 1393
The minimal local account record (GitHub numeric ID, current login/avatar,
1265
access status, and authentication timestamps) is retained so deletion cannot
1266
erase a ban or bypass authorization. No GitHub access token is retained or
1267
exported. A later account-erasure contract must separately define moderation
1268
retention and re-enrollment behavior.
1394
access status, authentication timestamps, and currently the encrypted GitHub
1395
token ciphertext) is retained so deletion cannot erase a ban or bypass
1396
authorization. The token is never exported. Gate 6 must add and prove explicit
1397
disconnect/revocation and token-removal behavior; product-data deletion must
1398
not be described as removing the token until that implementation lands. A
1399
later account-erasure contract must separately define moderation retention and
1400
re-enrollment behavior.
1269 1401
1270 1402
Evidence: `OpenAgents.DataRights`, `OpenAgentsWeb.DataController`,
1271 1403
`OpenAgents.Voice.Retention`, database foreign keys and purge trigger,

@@ -1273,6 +1405,8 @@ Evidence: `OpenAgents.DataRights`, `OpenAgentsWeb.DataController`,

1273 1405
1274 1406
### UI-001 — Authentication gates the one-conversation interface
1275 1407
1408
Status: Current
1409
1276 1410
The public default route is an authentication boundary and cannot invoke
1277 1411
OpenAgents. It exposes one GitHub login action and only bounded authentication error
1278 1412
copy. The protected `/chat` route exposes transcript, contextual turn state,

@@ -1298,15 +1432,17 @@ only by the allowlisted operator under ADMIN-001, is read only, cannot mount or

1298 1432
invoke OpenAgents, and holds no conversation. It adds nothing to the conversation
1299 1433
interface: no link, no affordance, and no chrome, for operators and
1300 1434
non-operators alike. Being an operator tool is not license for product
1301
chrome — the anti-references in `PRODUCT.md` still describe what the product
1435
chrome — the anti-references in `docs/architecture.md` still describe what the product
1302 1436
does not become.
1303 1437
1304 1438
Evidence: `OpenAgentsWeb.HomeControllerTest`, the `OpenAgentsWeb.ChatLiveTest` surface
1305 1439
test, `OpenAgentsWeb.LeaderboardLiveTest`, `OpenAgentsWeb.AdminLiveTest`,
1306
`OpenAgentsWeb.Router` browser policy, `PRODUCT.md`, and `DESIGN.md`.
1440
`OpenAgentsWeb.Router` browser policy, `docs/architecture.md`, and `docs/component-library.md`.
1307 1441
1308 1442
### UI-002 — Tool activity is a bounded projection of PostgreSQL truth
1309 1443
1444
Status: Current
1445
1310 1446
The interface renders tool activity only from the durable, already-scrubbed
1311 1447
step row: stable step ID, sequence, public capability label, status, the
1312 1448
step's durable `raw_arguments`, the bounded durable result/error, executor id

@@ -1346,6 +1482,8 @@ Evidence: `OpenAgents.Conversations.list_tool_step_activity/1`,

1346 1482
1347 1483
### UI-003 — Product surfaces render only through the sanctioned component library
1348 1484
1485
Status: Proposed
1486
1349 1487
OpenAgents's interface is built from `OpenAgentsWeb.UI` components over Basecoat
1350 1488
primitives vendored at a pinned tag and styled by the OpenAgents pack. Product
1351 1489
surfaces do not author component-level CSS classes; hand-authored CSS is

@@ -1363,15 +1501,23 @@ identically announced players.

1363 1501
The shared corner radius, the self-hosted Geist faces, the single dark theme,
1364 1502
and the reserved semantic color meanings hold across every component. Depth is
1365 1503
limited to the sanctioned lift, halo, and state-ring tokens.
1366
Adopting an additional Basecoat component requires a `DESIGN.md` change and an
1504
Adopting an additional Basecoat component requires a `docs/component-library.md` change and an
1367 1505
explicit per-component import.
1368 1506
1369
Evidence: `assets/vendor/basecoat/README.md`, `assets/css/style-openagents.css`,
1507
This is the Gate 4 target. `OpenAgentsWeb.UI` and the OpenAgents style pack are
1508
implemented, but generated `OpenAgentsWeb.CoreComponents`, its legacy icon
1509
entry point, and a nonfunctional theme control still have catalogued callers.
1510
Until those callers are migrated or narrowly justified, this invariant cannot
1511
be used as proof that the product has one fully enforced component path.
1512
1513
Evidence: `assets/vendor/basecoat/README.md`, `assets/css/openagents.css`,
1370 1514
`priv/static/fonts`, `OpenAgentsWeb.UI`, `OpenAgentsWeb.UITest`,
1371
`OpenAgentsWeb.UIGalleryLiveTest`, and `priv/scripts/check_css_contract.exs`.
1515
`OpenAgentsWeb.UIGalleryLiveTest`, and `test/openagents_web/ui_test.exs`.
1372 1516
1373 1517
### LEADERBOARD-001 — The public board publishes one bounded projection
1374 1518
1519
Status: Current
1520
1375 1521
IDENTITY-002 confines every other server path to the active user's own data.
1376 1522
The leaderboard is the single deliberate exception, and it is an exception to
1377 1523
publication rather than merely to cross-account reads: unauthenticated

@@ -1401,11 +1547,13 @@ unbounded anonymous viewers and a per-socket reread would turn one busy voice

1401 1547
call into a database amplifier. A lost cache costs a recompute, never data.
1402 1548
1403 1549
Evidence: `OpenAgents.Leaderboard`, `OpenAgents.Leaderboard.Entry`,
1404
`OpenAgents.Leaderboard.Server`, `docs/LEADERBOARD.md`, `OpenAgents.LeaderboardTest`,
1550
`OpenAgents.Leaderboard.Server`, `test/openagents/leaderboard_test.exs`, `OpenAgents.LeaderboardTest`,
1405 1551
and `OpenAgentsWeb.LeaderboardLiveTest`.
1406 1552
1407 1553
### OBSERVABILITY-001 — Telemetry is bounded, content-free, and never authoritative
1408 1554
1555
Status: Current
1556
1409 1557
Immutable domain receipts remain the authority; operational telemetry is only a
1410 1558
lossy health projection, and versioned evaluation reports remain separate
1411 1559
release evidence. OpenAgents telemetry accepts only finite plane/status/surface

@@ -1421,10 +1569,12 @@ checks block release; stuck work is an explicit warning requiring review.

1421 1569
1422 1570
Evidence: `OpenAgents.Observability`, `OpenAgents.Observability.Readback`,
1423 1571
`OpenAgents.Observability.ReleaseGate`, `OpenAgentsWeb.Telemetry`,
1424
`docs/OBSERVABILITY.md`, and `OpenAgents.ObservabilityTest`.
1572
`test/openagents/observability_test.exs`, and `OpenAgents.ObservabilityTest`.
1425 1573
1426 1574
### RELEASE-001 — Schema precedes traffic
1427 1575
1576
Status: Current
1577
1428 1578
The production image runs all pending Ecto migrations before starting the HTTP
1429 1579
server. Health is successful only when PostgreSQL answers.
1430 1580

@@ -1433,25 +1583,29 @@ Evidence: Docker `CMD`, `OpenAgents.Release`, the `/status` route, and

1433 1583
1434 1584
### RELEASE-002 — Secrets remain runtime-only
1435 1585
1586
Status: Current
1587
1436 1588
Session, database, provider, and GitHub OAuth credentials enter through ignored
1437 1589
local runtime configuration or Secret Manager and are absent from source, the
1438 1590
Docker build context, and image build arguments. Staging mounts only staging
1439 1591
GitHub secret names through a dedicated runtime identity; production values and
1440 1592
its prepared identity are distinct and remain unmounted until production
1441 1593
cutover. Missing or environment-mismatched GitHub configuration fails startup
1442
without printing any credential value. Reserved GitHub token-encryption keys
1443
are not mounted while the runtime discards tokens after identity projection. The Cloud Logging
1444
default sink excludes only OpenAgents OAuth callback request entries so the platform
1445
cannot persist authorization-code or state query values; application and audit
1446
logging remain enabled.
1594
without printing any credential value. The GitHub token-encryption key is
1595
runtime-only because the application retains delegated tokens as encrypted
1596
server-side ciphertext. The Cloud Logging default sink excludes only OpenAgents
1597
OAuth callback request entries so the platform cannot persist authorization-code
1598
or state query values; application and audit logging remain enabled.
1447 1599
1448 1600
Evidence: `OpenAgents.GitHubOAuth.RuntimeConfig`,
1449 1601
`OpenAgents.GitHubOAuth.RuntimeConfigTest`, `config/runtime.exs`, `.gitignore`,
1450 1602
`.dockerignore`, the `openagents-oauth-callback-requests` logging exclusion, and
1451
`docs/DEPLOY.md`.
1603
`ops/ci/release-smoke.sh`.
1452 1604
1453 1605
### RELEASE-003 — Every published hostname can establish LiveView
1454 1606
1607
Status: Current
1608
1455 1609
Production accepts the primary `PHX_HOST` plus explicitly configured HTTPS
1456 1610
aliases for Phoenix origin checks. Invalid, insecure, or path-bearing origins
1457 1611
fail startup rather than silently weakening socket validation.

@@ -1461,22 +1615,29 @@ production WebSocket read-back.

1461 1615
1462 1616
### RELEASE-004 — CI runs on owned infrastructure only, and gates every release
1463 1617
1464
No hosted CI, ever: no GitHub Actions workflows (`.github/workflows/`), no
1465
GitHub-hosted or third-party runners, no repo automation, secrets, or
1466
scheduling handed to external CI compute (owner restatement 2026-07-25; same
1467
invariant as `openagents/INVARIANTS.md` "No GitHub-Hosted CI / Cloud Actions"
1468
and `AGENTS.md` "No hosted CI"). All checks run on owned machines: manually
1469
(`mix precommit`, `ops/ci/gate.sh`), through standard git hooks
1470
(`.githooks/pre-push`), or inside owned deploy tooling. The full matrix — unit
1471
suite, distributed cluster-chaos suite, relup drill, version-chain drill — must
1472
PASS for the exact commit being shipped before a fleet release;
1473
`ops/fleet/rolling-deploy.sh` refuses to roll without that receipt.
1618
Status: Proposed
1619
1620
The target release gate permits no hosted CI: no GitHub Actions workflows, no
1621
GitHub-hosted or third-party runners, and no repository automation, secrets, or
1622
scheduling handed to external CI compute. All checks run on owned machines.
1623
The full matrix must bind unit, browser, distributed cluster, coverage,
1624
release, relup, version-chain, failure, and staging evidence to the exact
1625
candidate SHA, and every deploy command must refuse a stale or absent receipt.
1474 1626
1475
Evidence: `ops/ci/gate.sh`, `.githooks/pre-push`, the receipt check in
1476
`ops/fleet/rolling-deploy.sh`, and the absence of `.github/workflows/`.
1627
This complete deploy refusal is proposed, not implemented. The repository
1628
currently has an exact-SHA baseline receipt, merged coverage, release smoke,
1629
and relup proof primitives, but Gate 12 must compose them into the final owned
1630
release gate and bind every deployment entry point to its receipt.
1631
1632
Current progress evidence: `ops/ci/baseline.sh`, `ops/ci/coverage.sh`,
1633
`ops/ci/release-smoke.sh`, `ops/relup-proof/run.sh`,
1634
`ops/relup-proof/version-chain.sh`, and
1635
`ops/relup-proof/kill-during-install.sh`.
1477 1636
1478 1637
### STATUS-001 — The status page publishes one bounded, content-free projection
1479 1638
1639
Status: Current
1640
1480 1641
The public `/status` page and `/api/status` publish exactly one projection
1481 1642
(`OpenAgents.NetworkStatus`, schema-versioned): cluster membership and quorum,
1482 1643
Raft membership, per-node release/hot-load versions, uptimes, and counts.

@@ -1500,6 +1661,8 @@ Evidence: `OpenAgents.NetworkStatus`, `OpenAgentsWeb.NetworkStatusLive`,

1500 1661
1501 1662
### TRANSPARENCY-001 — Public transparency surfaces publish per-repo leveled projections
1502 1663
1664
Status: Current
1665
1503 1666
The public transparency surfaces — `/changelog`, `/api/changelog`, and the
1504 1667
forge web UI (`/<owner>/<repo>`, `/<owner>/<repo>/commit/:sha`,
1505 1668
`/<owner>/<repo>/blob/:ref/*path` — addressed exactly like the GitHub URLs

@@ -1524,7 +1687,7 @@ a 404, because publishing one document must never become a window into

1524 1687
every past revision of that file, or into the repository's history. Adding
1525 1688
a path to that allowlist is a deliberate publication decision, and
1526 1689
operator documentation (runtime configuration, deployment mechanics,
1527
operator identifiers) stays off it — `docs/OPERATIONS.md` is never
1690
operator identifiers) stays off it — `docs/2026-08-20-integration-hardening-and-staging-readiness-recommendations.md` is never
1528 1691
published.
1529 1692
1530 1693
Bounds that hold at every level: no secrets or credentials beyond what the

@@ -1553,3 +1716,85 @@ Evidence: `OpenAgents.Forge.Visibility`, `OpenAgents.Forge.Browse`, `OpenAgents.

1553 1716
`OpenAgents.Changelog.Entry`, `OpenAgentsWeb.ChangelogLive`, `OpenAgentsWeb.CodeRepoLive`,
1554 1717
`OpenAgentsWeb.CodeCommitLive`, `OpenAgentsWeb.CodeBlobLive`,
1555 1718
`OpenAgentsWeb.ChangelogController`, and their tests.
1719
1720
## Executable proof index
1721
1722
This index is part of the ledger. Every `Current` invariant has at least one
1723
repository-owned executable proof. The documentation check requires the ID set
1724
and every file path below to remain valid. A shared test can prove more than one
1725
contract; the invariant prose above defines the assertion, not the filename.
1726
1727
| Invariant | Executable proof |
1728
| --- | --- |
1729
| CANON-001 | `test/openagents/persona/source_manifest_test.exs` |
1730
| PERSONA-001 | `test/openagents/persona_test.exs` |
1731
| PERSONA-002 | `test/openagents/context/composer_test.exs`, `test/openagents/roles_test.exs` |
1732
| PERSONA-003 | `test/openagents/persona/evaluation_test.exs` |
1733
| BLUEPRINT-001 | `test/openagents/blueprint_test.exs` |
1734
| PROGRAM-001 | `test/openagents/program_artifacts_test.exs` |
1735
| DEGRADE-001 | `test/openagents/program_artifacts_test.exs`, `test/openagents/turn_provenance_test.exs` |
1736
| PROGRAM-002 | `test/openagents/shadow_programs_test.exs` |
1737
| PROGRAM-003 | `test/openagents/program_lifecycle_test.exs` |
1738
| IDENTITY-001 | `test/openagents/github_oauth_test.exs`, `test/openagents_web/auth_controller_test.exs` |
1739
| IDENTITY-002 | `test/openagents_web/auth_gate_test.exs` |
1740
| IDENTITY-003 | `test/openagents/memory_portability_test.exs` |
1741
| DATA-001 | `test/openagents/conversations_test.exs` |
1742
| DATA-002 | `test/openagents/accounts_test.exs`, `test/openagents/conversations_test.exs` |
1743
| DATA-003 | `test/openagents/conversations_test.exs` |
1744
| MEMORY-001 | `test/openagents/memory/lexical_recall_test.exs` |
1745
| MEMORY-002 | `test/openagents/memory/evidence_test.exs`, `test/openagents/turn_memory_evidence_journeys_test.exs` |
1746
| MEMORY-003 | `test/openagents/profile_memory_test.exs` |
1747
| MEMORY-004 | `test/openagents/memory/lexical_recall_test.exs`, `test/openagents/tools/conversation_recall_tools_test.exs` |
1748
| MEMORY-005 | `test/openagents/tools/profile_memory_tools_test.exs` |
1749
| MEMORY-006 | `test/openagents/semantic_recall_test.exs` |
1750
| MEMORY-007 | `test/openagents/preferences_test.exs` |
1751
| MEMORY-008 | `test/openagents/experience_memory_test.exs` |
1752
| MEMORY-009 | `test/openagents/graph_memory_test.exs` |
1753
| PRIVACY-001 | `test/openagents/memory/policy_and_redaction_test.exs` |
1754
| TURN-001 | `test/openagents/conversations_test.exs` |
1755
| TURN-002 | `test/openagents/conversations_test.exs` |
1756
| TURN-003 | `test/openagents_web/live/chat_live_test.exs` |
1757
| TURN-004 | `test/openagents/conversations_test.exs`, `test/openagents/tool_step_persistence_test.exs` |
1758
| TURN-005 | `test/openagents/turn_tool_loop_test.exs` |
1759
| PROVENANCE-001 | `test/openagents/turn_provenance_test.exs` |
1760
| PROVIDER-001 | `test/openagents/providers/provider_contract_test.exs`, `test/openagents/turn_provider_events_test.exs` |
1761
| TOOL-001 | `test/openagents/tools/registry_and_runner_test.exs` |
1762
| COLLECTIVE-001 | `test/openagents/collective_test.exs` |
1763
| COLLECTIVE-002 | `test/openagents/collective_generalizer_test.exs` |
1764
| COLLECTIVE-003 | `test/openagents/collective_publication_test.exs` |
1765
| COMPENSATION-001 | `test/openagents/compensation_test.exs` |
1766
| MODULE-001 | `test/openagents/modules/registry_test.exs`, `test/openagents/tool_step_persistence_test.exs` |
1767
| MODULE-002 | `test/openagents/modules/discovery_test.exs`, `test/openagents/modules/lifecycle_test.exs` |
1768
| MODULE-003 | `test/openagents/modules/router_test.exs`, `test/openagents/turn_tool_loop_test.exs` |
1769
| MODULE-004 | `test/openagents/surface_eval_test.exs` |
1770
| TOOL-002 | `test/openagents/tools/registry_and_runner_test.exs` |
1771
| TOOL-003 | `test/openagents/tool_step_persistence_test.exs` |
1772
| TOOL-004 | `test/openagents/tools/registry_and_runner_test.exs`, `test/openagents_web/tool_activity_test.exs` |
1773
| DEGRADE-002 | `test/openagents/tools/registry_and_runner_test.exs`, `test/openagents/tools/conversation_recall_tools_test.exs` |
1774
| WORK-001 | `test/openagents/work_job_test.exs`, `test/openagents/deep_work_tool_loop_test.exs` |
1775
| SELF-EDIT-001 | `test/openagents/tools/repository_mutation_tools_test.exs`, `test/openagents/coding_job_test.exs` |
1776
| VOICE-001 | `test/openagents/voice/config_test.exs` |
1777
| VOICE-002 | `test/openagents_web/controllers/voice_call_controller_test.exs` |
1778
| VOICE-003 | `test/openagents/voice_test.exs`, `test/openagents/voice_sessions_test.exs` |
1779
| VOICE-004 | `test/openagents/voice/open_ai/event_decoder_test.exs`, `test/openagents/voice_test.exs` |
1780
| VOICE-005 | `assets/test/voice_state_test.mjs`, `assets/test/voice_recording_test.mjs` |
1781
| VOICE-006 | `test/openagents/voice_sessions_test.exs`, `test/openagents_web/live/chat_live_test.exs` |
1782
| VOICE-007 | `test/openagents/voice_sessions_test.exs` |
1783
| VOICE-008 | `test/openagents/voice_sessions_test.exs`, `test/openagents/voice_test.exs` |
1784
| VOICE-009 | `test/openagents/voice_test.exs` |
1785
| VOICE-010 | `test/openagents/voice/release_operations_test.exs`, `test/openagents/voice/usage_test.exs` |
1786
| VOICE-011 | `test/openagents/voice/release_operations_test.exs`, `test/openagents_web/controllers/voice_telemetry_controller_test.exs` |
1787
| VOICE-012 | `test/openagents/voice/recordings_test.exs`, `test/openagents_web/controllers/voice_recording_controller_test.exs` |
1788
| ADMIN-001 | `test/openagents/admin_test.exs`, `test/openagents_web/live/admin_live_test.exs`, `test/openagents_web/live/admin_forge_live_test.exs` |
1789
| DATA-004 | `test/openagents_web/controllers/data_controller_test.exs`, `test/openagents/data_rights/atif_export_test.exs` |
1790
| UI-001 | `test/openagents_web/auth_gate_test.exs`, `test/openagents_web/live/chat_live_test.exs` |
1791
| UI-002 | `test/openagents_web/tool_activity_test.exs`, `test/openagents_web/live/chat_live_test.exs` |
1792
| UI-003 | `test/openagents_web/ui_test.exs`, `test/openagents_web/component_catalog_test.exs` |
1793
| LEADERBOARD-001 | `test/openagents/leaderboard_test.exs`, `test/openagents_web/live/leaderboard_live_test.exs` |
1794
| OBSERVABILITY-001 | `test/openagents/observability_test.exs` |
1795
| RELEASE-001 | `ops/ci/release-smoke.sh`, `test/openagents_web/controllers/health_controller_test.exs` |
1796
| RELEASE-002 | `test/openagents/github_oauth/runtime_config_test.exs`, `ops/ci/reference-check.sh` |
1797
| RELEASE-003 | `test/openagents_web/allowed_origins_test.exs`, `ops/ci/release-smoke.sh` |
1798
| RELEASE-004 | Proposed; current primitives are listed in its entry above. |
1799
| STATUS-001 | `test/openagents/network_status_test.exs`, `test/openagents_web/live/network_status_live_test.exs` |
1800
| TRANSPARENCY-001 | `test/openagents/forge/visibility_test.exs`, `test/openagents/forge/browse_test.exs`, `test/openagents_web/live/code_live_test.exs` |
README.md modified +88 -42

@@ -1,67 +1,113 @@

1
# OpenAgents - The Agent Forge
1
# OpenAgents
2 2
3
OpenAgents is a source-code forge. We are building it to replace GitHub for our own projects, and then for customers. This repository is the public home of the project.
3
OpenAgents is an AGPL-3.0 Phoenix application for building and operating
4
agent-backed software in public. This repository contains the complete product:
5
Sarah chat and voice, governed memory and tools, delegated work and connected
6
computers, issues and projects, the Git forge, and the deployment control plane.
4 7
5
## What we are building
8
The current architecture and trust boundaries are documented in
9
[docs/architecture.md](docs/architecture.md). The staged hardening work is
10
tracked in the
11
[integration hardening plan](docs/2026-08-20-integration-hardening-and-staging-readiness-recommendations.md).
6 12
7
We are building the Agent Forge from scratch. This is a clean-room, independent implementation. It does not use code or design from any earlier internal or external forge.
13
## Capability status
8 14
9
The first public surface is an issue and project tracker. We will use the Agent Forge to build OpenAgents.com itself. Every change ships through the forge with live hot reload.
15
No part of this repository is approved for production deployment yet.
10 16
11
## Why start from scratch
17
### Implemented and locally gated
12 18
13
Starting fresh lets us build a forge that is open, contributor-friendly, and defined by its own runtime behavior rather than by compatibility with an existing platform. The BEAM runtime, Phoenix LiveView, and hot reload are the implementation choices.
19
- GitHub OAuth, encrypted server-side GitHub token storage, sessions, and
20
  account-scoped data rights.
21
- Authenticated text chat with durable turns, provider receipts, tools, memory,
22
  delegated work, and connected-computer orchestration.
23
- Voice session, transcript, usage, recording, and operator-projection domains.
24
- Issues, comments, labels, milestones, projects, public status, changelog, and
25
  bounded source-browsing surfaces.
26
- Git HTTP, push receipts, promotion targets, build receipts, and local BEAM
27
  deployment primitives.
28
- An owned local test gate covering browser JavaScript, the Phoenix application,
29
  distributed cluster cases, merged coverage, and a disposable production
30
  release smoke test.
14 31
15
## What works now
32
"Implemented" means the code and local tests exist. It does not mean the
33
feature has passed staging, security review, failure injection, or a soak.
16 34
17
- **Hot reload:** code changes reach the live cluster in seconds without a rolling restart.
18
- **Release upgrades (relups):** deploys use Erlang/OTP release upgrade patterns for zero-downtime updates.
19
- **Live surfaces:** pages update for every connected viewer at the same time through Phoenix PubSub.
35
### Disabled by default or staging-only
20 36
21
## Tech stack
37
- Voice, recording, semantic recall, experimental program paths, and deployment
38
  workers remain controlled by runtime configuration.
39
- Direct BEAM loading, relup installation, rolling replacement, and boot
40
  convergence require isolated three-node staging proof before they can be
41
  enabled outside a disposable environment.
42
- The self-hosted forge is being hardened, but GitHub remains the canonical Git
43
  remote until the proof-gated cutover in ADR 0007.
22 44
23
- **Elixir on the BEAM** — the runtime for hot reload, release upgrades, and live, concurrent page updates.
24
- **Phoenix and Phoenix LiveView** — web framework and live UI layer.
25
- **PostgreSQL** — primary database.
26
- **Google Cloud** — hosting and infrastructure.
27
- **Tailwind CSS** — styling.
28
- **DaisyUI** — UI component library.
45
### Planned or blocked on hardening
29 46
30
## First deliverable: Issues and projects
47
- Repository-backed tenant isolation for every issue and project record.
48
- Complete route-authority, token-lifecycle, recovery, build-isolation, and
49
  transactional fleet-deployment gates.
50
- Separate web and distributed staging lanes, a full regression matrix,
51
  failure-injection drills, and a 48-hour soak.
52
- Any production rollout. Production remains explicitly out of scope until the
53
  staging plan is complete and separately approved.
31 54
32
The first public surface is an issue and project tracker. We will use it to run OpenAgents.com's own development. You can:
55
## Architecture
33 56
34
- Open issues.
35
- Create and manage projects.
36
- Watch updates appear live for everyone else on the page.
37
- See the deploy receipts for each change.
57
The browser connects to one Phoenix and LiveView application. PostgreSQL is the
58
durable authority for product, authorization, work, and deployment state.
59
Provider APIs and GitHub are server-side dependencies behind explicit adapters;
60
their credentials never belong in browser state. BEAM processes, PubSub, and
61
LiveView assigns are recoverable projections of durable records.
38 62
39
## For contributors
63
Sarah is a persona and behavior package in the OpenAgents application, not a
64
private service boundary. Generic infrastructure uses the `OpenAgents`
65
namespace, while persona artifacts retain Sarah-specific identities where that
66
history is part of the contract.
40 67
41
We want contributing to feel good. The first contributor features are:
68
## Interface system
42 69
43
- Log in with GitHub.
44
- See a leaderboard of contributions.
45
- Get credit for code that trains our agents, when we build that part.
70
The product uses Tailwind CSS, pinned vendored Basecoat component styles, the
71
OpenAgents style pack in `assets/css/openagents.css`, and reusable HEEx
72
components in `OpenAgentsWeb.UI`. The live component inventory is available at
73
`/components`; [docs/component-library.md](docs/component-library.md) records
74
the current transition and extension rules.
46 75
47
## What's next
76
Fonts and icons are self-hosted. Do not add a remote font, icon font, second
77
component library, or unreviewed browser script.
48 78
49
After issues and projects, we will make it easy to import repositories into OpenAgents.com. The goal is to let you bring an existing project onto the forge and get the same live, receipted, hot-reload experience.
79
## Local development
50 80
51
## Roadmap
81
The application requires Elixir/OTP, PostgreSQL with pgvector, and Node.js for
82
the browser-side tests.
52 83
53
| Phase | Work | Outcome |
54
| --- | --- | --- |
55
| 1 | Issues and projects | Public tracker for OpenAgents.com |
56
| 2 | GitHub login and leaderboard | Contributor accounts and recognition |
57
| 3 | Repository import | Move projects onto the forge |
58
| 4 | Pull requests and reviews | Agent-native review flow |
59
| 5 | Transparency tiers | Paid and public access levels |
84
```sh
85
mix setup
86
mix phx.server
87
```
60 88
61
## Contributing
89
Before committing, run the repository-owned gate:
62 90
63
We are in the early phase. If you want to help, open an issue or watch for `good first issue` labels.
91
```sh
92
mix precommit
93
```
94
95
Distributed, merged-coverage, relup, and release-smoke checks live under
96
`ops/` and are composed by the exact-SHA baseline gate while hardening is in
97
progress. This repository deliberately has no hosted CI configuration.
98
99
## Contributing and source control
100
101
Read `AGENTS.md` before changing the application. GitHub is temporarily the
102
canonical remote during staging hardening. The forge becomes canonical only
103
after the durability, mirror, restore, and deployment proofs in ADR 0007 pass.
64 104
65 105
## License
66 106
67
This project is licensed under the GNU Affero General Public License v3.0. See `LICENSE`.
107
OpenAgents is licensed under the GNU Affero General Public License v3.0. See
108
`LICENSE`.
109
110
Vendored third-party material keeps its own license and notices. In particular,
111
Basecoat is under `assets/vendor/basecoat/`, the Apps SDK icon set is under
112
`priv/icons/`, and self-hosted font notices are under `priv/static/fonts/`.
113
Review those notices when redistributing the application.
docs/2026-08-19-gap-implementation-plan.md modified +52 -108

@@ -1,108 +1,52 @@

1
# Gap implementation plan: full port of Sarah into openagents.com
2
3
**Date:** 2026-08-19  
4
**Source:** `~/work/sarah/docs/audits/2026-08-19-openagents-com-full-port-gap-analysis.md`  
5
**Status:** in progress — this document is updated as work lands.
6
7
## Verdict
8
9
The code is across. What remains is commissioning: the test signal, runtime
10
configuration, clustering, routing and LiveViews, schema completeness, CI, and
11
the production cutover. The first job is to make `mix test` honest again.
12
13
## Phases
14
15
### Phase A — Restore the test signal (serial, blocks all else)
16
17
1. Fix the partial `visitors` unique index so `Conversations.ensure_visitor/1`
18
   can use `[:browser_key_hash]` as an `ON CONFLICT` arbiter.
19
2. Re-run the full test suite, triage the survivors, and delete the
20
   `exclude: [:skip]` plus `@moduletag :skip` scaffolding as tests pass.
21
22
### Phase B — Commission the runtime (parallel lanes)
23
24
3. Port the missing `config/` keys: the tool catalog, inference `provider`,
25
   `openai_model`, shadow programs, semantic index, recall backends, voice
26
   providers, experience/graph memory, GitHub API, and forge flags. Replace the
27
   soft `Application.get_env(:openagents, :tools, [])` default with
28
   `fetch_env!`.
29
4. Restore `OpenAgents.Application` startup: `Release.migrate/0`,
30
   `Persona.SourceManifest.load!/install!`, `ProgramArtifacts.install!/0`,
31
   `Voice.Config.validate_boot!/0`, `Tools.Embeddings.warm/0`,
32
   `Changelog.Backfill.boot/0`, and real Horde/Ra children. Replace the
33
   `cluster/registry.ex` and `cluster/dynamic_supervisor.ex` shims with real
34
   `Horde.Registry` and `Horde.DynamicSupervisor`.
35
5. Close the schema gaps: add `voice_tool_steps` and `voice_response_contexts`
36
   tables; fix `forge_pushes` / `forge_push_receipts`; add `forge_deploys.nodes`;
37
   verify the six `allow_*_privacy_deletion` migrations are present.
38
6. Port `INVARIANTS.md` from Sarah, rewrite `RELEASE-004` to dogfood the forge,
39
   rewrite `UI-003` to record the basecoat transitional exception, add the
40
   layering boundary from transcript 270, disambiguate the duplicate
41
   `DEGRADE-001`, and revert the `forge/targets.ex` `commit_store` test-only
42
   relaxation.
43
44
### Phase C — Surfaces (depends on B)
45
46
7. Port the 12 missing LiveViews verbatim, carrying `assets/vendor/basecoat/`
47
   across with the narrow import discipline. Reconcile `ui_gallery_live` with the
48
   existing `/components` gallery.
49
8. Mount `ControllerSocket` in `endpoint.ex`; route the 7 orphaned controllers.
50
9. Route `forward "/git", OpenAgents.Forge.GitHTTP`; the literal-owner code
51
   views; `/changelog`, `/status`, `/leaderboard`, `/admin/*`, data-rights
52
   exports; and `/api/inference/proxy`.
53
54
### Phase D — Dogfood the forge as CI
55
56
10. Port `ops/ci/gate.sh` and `.githooks/pre-push`. Move openagents.com to
57
    forge-canonical; demote GitHub to a MirrorWatch mirror. Run the gate on the
58
    forge's build lane.
59
60
### Phase E — Cutover
61
62
11. Data migration rehearsal against a restored Sarah snapshot; keep
63
    `changelog_entries` and `forge_*` receipt identifiers intact.
64
12. Fleet cutover, WAL continuity, DNS/MirrorWatch repoint, and Sarah
65
    deprecation.
66
67
## Progress
68
69
- `visitors` unique index fixed and migration order corrected so the test
70
  database can build cleanly.
71
- `priv/sarah/` persona and program artifacts copied; `Application.get_env`
72
  / `fetch_env!` references re-pointed from `:sarah` to `:openagents`.
73
- `config/config.exs` now carries the tool catalog, provider, model, voice,
74
  memory, and GitHub API configuration.
75
- `OpenAgents.Application` installs the persona and program-artifact catalogs at
76
  boot; the tool catalog uses `fetch_env!` instead of a silent empty default.
77
- `OpenAgentsWeb.ChatToolActivity` renamed to `OpenAgentsWeb.ToolActivity` to
78
  match the test suite; `conversation_page_size` and `maximum_message_bytes`
79
  corrected to the Sarah values.
80
- `OpenAgentsWeb.AllowedOrigins` added.
81
- Orphaned controller routes added: `/api/changelog`, `/api/status`,
82
  `/memory/export`, `/data/export`, `/data/export/atif`, `/data`, `/data/reset`,
83
  `/api/computers`, `/api/computers/pairings/:id/approve`,
84
  `/api/computers/:machine_id/agent-jobs`, `/api/computer-agent-jobs/:id`,
85
  and `forward "/git"`.
86
- `OpenAgentsWeb.ControllerSocket` mounted at `/controller`.
87
- Admin recording audio route and controller removed; audio recordings are not
88
  saved.
89
- `/status`, `/computers`, and `/machines` routed; the admin recording test
90
  deleted.
91
92
## Current test signal
93
94
`mix precommit` is green (54 passing, 893 skipped).  
95
`mix test --include skip` is now **601/947 passed**, down from the starting
96
301/947. 346 failures remain, dominated by missing LiveViews (`/admin`,
97
`/computers`, `/changelog`, `/status`, `/leaderboard`), the
98
`computer_controller_enabled` flag being `false` in tests, the `/status` JSON
99
endpoint, and the `/controller/pairings` creation route.
100
101
## Current cycle
102
103
Phase A is complete and Phase B is largely done. Phase C is now active: the next
104
commits are the missing LiveViews (`/admin`, `/admin/forge`, `/computers`,
105
`/changelog`, `/status`, `/leaderboard`), the `/status` JSON endpoint, and the
106
`/controller/pairings` creation route, or we can skip to the faster wins in
107
Phase B that are still failing (e.g. `computer_controller_enabled` in tests and
108
`voice_call_provider` wiring).
1
# Historical integration gap plan
2
3
Date: 2026-08-19
4
5
Status: Closed on 2026-08-20
6
7
Current tracker: [integration hardening and staging readiness](2026-08-20-integration-hardening-and-staging-readiness-recommendations.md)
8
9
## Purpose of this record
10
11
This plan was opened immediately after the source integration to restore an
12
honest test signal and commission the imported runtime. Its intermediate test
13
counts, stubs, missing routes, and phase labels no longer describe the tree.
14
They are not release evidence.
15
16
## Completed outcomes
17
18
- Removed the temporary skipped-test scaffold and restored the full local test
19
  suite.
20
- Completed runtime configuration for providers, tools, memory, voice, and
21
  clustering.
22
- Restored the application supervision tree, Horde/Ra integration, and boot
23
  catalog validation.
24
- Reconciled schema gaps for voice, provenance, work, forge builds, and forge
25
  deploys.
26
- Mounted the chat, computer, controller, data-rights, status, changelog,
27
  leaderboard, operator, code, issue, and project surfaces.
28
- Added the Git HTTP route, forge build/deploy records, and local release proof
29
  harnesses.
30
- Ported and enabled the integrated test suite, then added broad issue/project
31
  controller, domain, and LiveView coverage.
32
- Added an exact-SHA owned baseline gate, merged local/cluster coverage, and a
33
  disposable production-release smoke test.
34
35
## Unresolved work moved forward
36
37
Nothing remains owned by this plan. Its unresolved topics moved to explicit
38
gates in the current hardening plan:
39
40
- Documentation and invariant reconciliation: Gate 3.
41
- Dependency, Markdown, asset, icon, and component consolidation: Gate 4.
42
- Runtime configuration and startup validation: Gate 5.
43
- Identity, route authority, token lifecycle, and secret handling: Gate 6.
44
- Repository and tenant scoping for issues/projects: Gate 7.
45
- Chat, recovery, memory, work, machine, and voice hardening: Gate 8.
46
- Build isolation, artifact durability, transactional deployment, relup, and
47
  rolling replacement: Gates 9–11.
48
- Owned gates, isolated staging, regression evidence, failure injection, and
49
  soak: Gates 12–16.
50
51
Do not reopen this file as a parallel tracker. Add new findings to the current
52
hardening plan so readiness has one ordered source of truth.
docs/2026-08-20-integration-hardening-and-staging-readiness-recommendations.md modified +34 -6

@@ -2,7 +2,7 @@

2 2
3 3
Date: 2026-08-20
4 4
5
Status: In progress; Gate 0 complete, amended with measured findings
5
Status: In progress; Gates 0–3 complete, amended with measured findings
6 6
7 7
## Outcome
8 8

@@ -298,8 +298,8 @@ accident.

298 298
- Close or archive `docs/2026-08-19-gap-implementation-plan.md` after moving each
299 299
  unresolved item into the current hardening plan.
300 300
- Update `docs/component-library.md` and
301
  `docs/issues-projects-ui-roadmap.md` for Basecoat, `SarahUI`, and the actual
302
  component catalog.
301
  `docs/issues-projects-ui-roadmap.md` for Basecoat, `OpenAgentsWeb.UI`, and the
302
  actual component catalog.
303 303
- Update `docs/github-auth-plan.md` after deciding whether GitHub access tokens
304 304
  remain stored.
305 305
- Keep the test coverage audit as a dated measurement. Add a later audit instead

@@ -311,8 +311,9 @@ Review every invariant in `INVARIANTS.md` against code, schema, configuration,

311 311
tests, and documentation.
312 312
313 313
- Give every invariant a unique ID. Resolve the duplicate `DEGRADE-001` entries.
314
- Correct nonexistent paths such as `priv/openagents`, `OpenAgentsWeb.UI`, and
315
  `style-openagents.css`, or rename the implementation first.
314
- Correct inherited or nonexistent paths such as `priv/openagents` and
315
  `style-openagents.css`; use the generic `OpenAgentsWeb.UI` module established
316
  by Gate 2.
316 317
- Resolve the contradiction between discarding GitHub tokens and the current
317 318
  encrypted-token storage path.
318 319
- Distinguish implemented invariants from proposed invariants. A proposed

@@ -338,6 +339,33 @@ Add an owned local check that:

338 339
components, authentication, deployment maturity, and staging status, and all
339 340
local references resolve.
340 341
342
**Gate 3 status (2026-08-20): complete.**
343
344
- Rewrote `README.md` around the integrated AGPL application and separated
345
  locally implemented, disabled/staging-only, planned, and production-prohibited
346
  capabilities. It now names the Basecoat/OpenAgents component system and
347
  vendored license locations.
348
- Converted the chat service split, source integration, and original gap plan
349
  into closed historical records. Updated the component inventory, issue/project
350
  UI roadmap, API assessment, API work record, GitHub token contract, and BEAM
351
  deployment maturity narrative without rewriting the dated coverage audit.
352
- Reconciled `INVARIANTS.md`: all 72 IDs are unique, every entry is explicitly
353
  current or proposed, all 70 current entries map to executable repository-owned
354
  proofs, artifact/style paths match the tree, retained encrypted GitHub tokens
355
  are documented consistently, and missing admin recording routes are no longer
356
  claimed.
357
- Removed the dead operator recording playback affordance whose URL had no
358
  controller or route; the current operator surface exposes bounded recording
359
  metadata only. Added `.dockerignore` because RELEASE-002 named a build-context
360
  safeguard that was absent.
361
- Added `ops/ci/docs-check.exs` to `mix precommit`. It verifies relative Markdown
362
  links, current-document terminology, developer-local paths, invariant IDs and
363
  statuses, proof-index coverage, evidence paths, and referenced modules.
364
- Verified 21 Markdown files with the documentation gate, 14 focused operator
365
  LiveView tests, 15 browser tests, and 1,222 default Elixir tests with zero
366
  failures. The nine distributed tests remain isolated for the owned baseline
367
  gate and were unchanged by this documentation gate.
368
341 369
## Gate 4: Harden dependencies, assets, and the component system
342 370
343 371
### Complete the Markdown parser migration

@@ -1087,7 +1115,7 @@ each handoff.

1087 1115
1088 1116
- [x] The repository has one accurate architecture narrative.
1089 1117
- [x] Every remaining Sarah reference is intentional and specific.
1090
- [ ] All documentation links and invariant evidence resolve.
1118
- [x] All documentation links and invariant evidence resolve.
1091 1119
- [ ] The application has one Markdown parser, component system, and documented
1092 1120
      two-tier icon policy.
1093 1121
- [ ] The dark-only palette has no nonfunctional theme control.
docs/beam-hot-deployment-plan.md modified +24 -7

@@ -2,11 +2,12 @@

2 2
3 3
Date: 2026-08-19
4 4
5
Status: Phase 3 complete
5
Status: Design and partial local implementation; staging proof incomplete;
6
production use prohibited
6 7
7 8
## Outcome
8 9
9
OpenAgents will support three production deployment classes:
10
OpenAgents is designed to support three deployment classes:
10 11
11 12
1. Direct BEAM loading for allowlisted, code-only changes.
12 13
2. OTP release upgrades, or relups, for versioned code and state migrations.

@@ -14,9 +15,19 @@ OpenAgents will support three production deployment classes:

14 15
15 16
The deployment system will select the narrowest class that completely covers a change. It will never apply part of a candidate and call the candidate live. Every promotion, build, deployment, refusal, rollback, and boot-convergence attempt will produce an auditable receipt.
16 17
17
This plan makes the production deployment capabilities described in `README.md` concrete. Phoenix development code reloading is unrelated to this plan.
18
This document describes the target safety model and preserves the detailed
19
implementation sequence. It is not a readiness receipt. Persistence, build,
20
hot-load, relup, cluster, and proof-harness pieces exist locally, but the full
21
classifier, transactional fleet path, reverse upgrade, rolling replacement,
22
isolated staging matrix, failure injection, and soak have not passed on one
23
candidate. Phoenix development code reloading is unrelated to this plan.
18 24
19
## Success criteria
25
The current authority is
26
[`docs/architecture.md`](architecture.md) together with Gates 9–16 of the
27
[integration hardening plan](2026-08-20-integration-hardening-and-staging-readiness-recommendations.md).
28
Nothing in this historical phase sequence authorizes production deployment.
29
30
## Target success criteria
20 31
21 32
The work is complete when the system meets all of these conditions:
22 33

@@ -54,7 +65,7 @@ operator promotion

54 65
target state machine
55 66
    |
56 67
    v
57
isolated production build -----> immutable artifact and build receipt
68
isolated release build --------> immutable artifact and build receipt
58 69
    |
59 70
    v
60 71
deployment classifier

@@ -364,7 +375,10 @@ Keep the new processes disabled by default until the preceding phases pass.

364 375
8. Run a router and LiveView change after adding `OpenAgentsWeb.` to the allowlist.
365 376
9. Run a stateful relup, reverse relup, and kill-during-install recovery.
366 377
10. Run a structural change and prove that the direct lane refuses it before rolling replacement succeeds.
367
11. Enable the production workers and retain an immediate runtime kill switch.
378
11. After every preceding staging proof passes, prepare an explicitly approved
379
    candidate for a later production-readiness decision; retain an immediate
380
    runtime kill switch. This plan does not authorize enabling production
381
    workers.
368 382
369 383
Do not broaden the allowlist based only on module naming. Add each namespace after its state, side effects, on-load behavior, and smoke checks have a completed drill.
370 384

@@ -468,4 +482,7 @@ Run the final drill on a three-node staging fleet:

468 482
9. Confirm that every attempt has a complete receipt and that the public status surface contains no sensitive details.
469 483
10. Run `mix precommit` and `ops/ci/gate.sh` at the final SHA.
470 484
471
After this drill passes, direct BEAM deployment becomes the default for proven allowlisted changes, relup becomes the stateful code-upgrade path, and rolling replacement remains the universal fallback.
485
After this drill, the evidence can be reviewed as a production-readiness
486
candidate. Direct BEAM loading, relup, and rolling replacement remain disabled
487
until a separate operator decision approves a specific configuration and
488
candidate.
docs/chat-inference-plan.md modified +52 -243

@@ -1,246 +1,55 @@

1
# Chat and inference plan
1
# Historical chat and inference service-split plan
2 2
3 3
Date: 2026-08-19
4 4
5
Status: Planned
6
7
## Outcome
8
9
OpenAgents serves the `/chat` user interface and the core chat and inference logic from this public repository. The repository owns the presentation layer, the conversation and turn lifecycle, inference orchestration, memory, tools, and delegated work. The `pro.openagents.com` service owns only the Sarah persona, voice orchestration, and provider integrations that are specific to the Sarah product.
10
11
This repository includes:
12
13
- The `/chat` LiveView route and responsive shell.
14
- `OpenAgents.Conversations`, `OpenAgents.Turns`, `OpenAgents.Messages`, `OpenAgents.Receipts`, `OpenAgents.Tools`, `OpenAgents.Memory`, `OpenAgents.Agents`, and `OpenAgents.Delegation`.
15
- Server-side inference orchestration with the existing `Req` library.
16
- Public projections of tool activity, delegated work, memory, privacy controls, and voice state.
17
- Same-origin proxy endpoints where browser media or downloads require an HTTP endpoint.
18
- Tests, fixtures, accessibility behavior, and operational status.
19
20
The `pro.openagents.com` service includes:
21
22
- Sarah prompts, persona instructions, reasoning policy, and model selection.
23
- Voice-provider session orchestration, usage limits, and recording storage.
24
- Data-rights exports and deletions that cross the Sarah service boundary.
25
26
## Architecture
27
28
```text
29
browser
30
  |
31
  | LiveView WebSocket and same-origin HTTP
32
  v
33
openagents.com
34
  - OpenAgentsWeb.ChatLive
35
  - chat components and browser hooks
36
  - OpenAgents.Conversations, OpenAgents.Turns, OpenAgents.Messages
37
  - OpenAgents.Inference, OpenAgents.Tools, OpenAgents.Memory
38
  - OpenAgents.Delegation, OpenAgents.Agents
39
  |
40
  | HTTPS, service authentication, user assertion
41
  v
42
pro.openagents.com
43
  - Sarah persona and prompts
44
  - voice orchestration and recordings
45
  - data-rights back-end
46
```
47
48
The browser does not receive provider credentials. `openagents.com` makes server-side calls to inference providers and to `pro.openagents.com` for Sarah-specific and voice-specific behavior. Public projections are sent to the browser through LiveView.
49
50
## Ownership boundary
51
52
| Concern | `openagents.com` application | `pro.openagents.com` service |
53
| --- | --- | --- |
54
| User authentication | Establishes the browser session and stable user subject | Verifies the signed user assertion and enforces resource ownership |
55
| Agent identity | Renders returned display metadata such as `Sarah` | Selects persona, prompts, models, and behavior |
56
| Conversation state | Stores conversations, turns, messages, receipts, and cursors | Receives a bounded context from `openagents.com` for Sarah turns |
57
| Turn ordering | Accepts, validates, orders, rate-limits, and executes turns | Receives ordered turn context from `openagents.com` |
58
| Streaming | Resumes and projects versioned events to the browser | Returns ordered, replayable events for Sarah and voice |
59
| Markdown | Parses and sanitizes assistant text for display | Returns plain Markdown, never trusted HTML |
60
| Tool activity | Executes tools and renders bounded display-safe activity | Receives tool results for Sarah turns |
61
| Delegated work | Starts agents, controls machines, stores outcomes, and authorizes cancellation | Receives work updates that affect Sarah responses |
62
| Memory | Extracts, stores, searches, corrects, forgets, and audits memory | Receives memory context for Sarah turns |
63
| Voice | Manages browser media controls and same-origin requests | Admits sessions, connects providers, enforces limits, and stores recordings |
64
| Data rights | Presents confirmation and streams downloads | Exports or deletes the authenticated user’s durable data |
65
| Observability | Records content-free client and transport metrics | Records model, tool, policy, and durable execution metrics |
66
67
## Security invariants
68
69
Implement these rules before enabling `/chat` in production:
70
71
- **The browser never receives service or provider credentials.** Store them in runtime configuration and use them only in server-side adapters.
72
- **Every resource is user-scoped.** The back-end derives ownership from a verified assertion and does not trust a conversation or message ID by itself.
73
- **Service assertions are short-lived and signed.** Send them in `Authorization: Bearer <token>` over TLS to `pro.openagents.com`.
74
- **Model output is untrusted.** Parse Markdown through an allowlist, escape every text node, reject unsafe URL schemes, and never render untrusted HTML.
75
- **Events are ordered and resumable.** Every session event carries a monotonically increasing sequence and opaque cursor.
76
- **Mutations are idempotent.** Turn creation, cancellation, memory actions, recording chunks, and deletion requests use idempotency keys.
77
- **Logs are content-free by default.** Log request IDs, status codes, durations, byte counts, and error codes without message content or memory claims.
78
- **Destructive actions require explicit confirmation.** The UI states the scope, and the back-end revalidates the confirmation.
79
- **Voice requires a secure context and explicit microphone access.** A voice failure leaves typed chat available.
80
- **Feature capabilities come from the service.** The UI does not infer that memory, tools, delegation, recording, or voice is available.
81
82
## Authentication contract
83
84
Reuse the authenticated browser pipeline and `live_session` from `docs/github-auth-plan.md` for `/chat`.
85
86
1. Pass `current_scope` to `<Layouts.app>` and derive a stable, non-email user subject from it.
87
2. Create a short-lived service assertion for every request to `pro.openagents.com` or for a voice session. Use a signed JWT or workload-identity token with these claims:
88
   - `iss`: `openagents.com`
89
   - `aud`: `pro.openagents.com`
90
   - `sub`: the stable OpenAgents user ID
91
   - `sid`: the browser-session identifier
92
   - `scope`: the minimum scopes required for the request
93
   - `iat` and `exp`: a short validity window
94
   - `jti`: a unique token identifier
95
3. Send the assertion in `Authorization: Bearer <token>` over TLS.
96
4. Send a content-free `X-Request-ID` for correlation.
97
5. Rotate signing keys without a deploy. Keep the active key ID in the token header and publish verification keys through operator-managed configuration.
98
99
`pro.openagents.com` verifies signature, issuer, audience, expiration, scope, and resource ownership on every request. It must not accept user identity from an unsigned forwarding header.
100
101
Tests use a fake signer and fake `pro.openagents.com` adapter. They never require a production credential.
102
103
## Versioned API contract with `pro.openagents.com`
104
105
Use `/v1` for the cross-service contract. Every JSON object includes a `schema` field. Treat unknown fields as additive, but reject unknown schema major versions.
106
107
### Turn run
108
109
`POST /v1/sarah/turns/{turn_id}/run` starts a Sarah turn from the context that `openagents.com` has already built.
110
111
Request:
112
113
```json
114
{
115
  "schema": "openagents.sarah.turn_request.v1",
116
  "user_id": "user_opaque_id",
117
  "session_id": "session_opaque_id",
118
  "turn_id": "turn_opaque_id",
119
  "messages": [
120
    {
121
      "schema": "openagents.chat.message.v1",
122
      "id": "msg_opaque_id",
123
      "turn_id": "turn_opaque_id",
124
      "role": "user",
125
      "content": "Help me plan this change.",
126
      "created_at": "2026-08-19T21:00:00Z"
127
    }
128
  ],
129
  "memory": [...],
130
  "tools": [...],
131
  "capabilities": {
132
    "text": true,
133
    "voice": false
134
  }
135
}
136
```
137
138
Response is a server-sent event stream. The `pro.openagents.com` service returns:
139
140
- `message.upsert` for the assistant reply.
141
- `turn.upsert` for active or queued turn state.
142
- `activity.upsert` for display-safe tool activity.
143
- `voice.updated` when voice state changes.
144
145
When the stream returns `410 Gone` for an expired cursor, `openagents.com` fetches a fresh context and restarts from the last known state.
146
147
### Voice session
148
149
`POST /v1/sarah/voice/sessions` starts a voice session. `openagents.com` relays the browser SDP and receives a provider session. The browser may establish the resulting WebRTC media path directly, but it never receives a provider API key.
150
151
## Public module layout
152
153
Use one module per file. Keep HTTP transport outside the LiveView.
154
155
```text
156
lib/openagents/conversations.ex
157
lib/openagents/conversations/conversation.ex
158
lib/openagents/conversations/message.ex
159
lib/openagents/turns.ex
160
lib/openagents/turns/turn.ex
161
lib/openagents/turns/receipt.ex
162
lib/openagents/inference.ex
163
lib/openagents/tools.ex
164
lib/openagents/tools/tool.ex
165
lib/openagents/tools/tool_step.ex
166
lib/openagents/memory.ex
167
lib/openagents/memory/record.ex
168
lib/openagents/agents.ex
169
lib/openagents/delegation.ex
170
lib/openagents/sarah.ex
171
lib/openagents/sarah/req_client.ex
172
lib/openagents/sarah/turn.ex
173
lib/openagents/sarah/voice.ex
174
lib/openagents_web/live/chat_live.ex
175
lib/openagents_web/components/chat_shell.ex
176
lib/openagents_web/components/chat_message.ex
177
lib/openagents_web/components/chat_composer.ex
178
lib/openagents_web/components/chat_activity.ex
179
lib/openagents_web/components/chat_memory.ex
180
lib/openagents_web/components/chat_voice.ex
181
lib/openagents_web/controllers/chat_voice_controller.ex
182
lib/openagents_web/controllers/chat_export_controller.ex
183
lib/openagents_web/markdown.ex
184
assets/js/chat_voice_controller.js
185
assets/js/paced_transcript.js
186
```
187
188
`OpenAgents.Sarah` defines a behaviour for Sarah turn and voice operations. `OpenAgents.Sarah.ReqClient` implements it with the existing `Req` dependency. Tests select `OpenAgents.Sarah.Fake` through application configuration.
189
190
Do not put HTTP calls, token creation, response decoding, or retry rules directly in `OpenAgentsWeb.ChatLive`.
191
192
## Session bridge
193
194
Use one supervised bridge per `{user_subject, session_id}` to avoid one back-end event stream per browser tab.
195
196
1. Start `OpenAgents.Sarah.SessionRegistry` and `OpenAgents.Sarah.SessionSupervisor` in `OpenAgents.Application` after `OpenAgents.PubSub`.
197
2. Start or find a `SessionBridge` when an authenticated LiveView connects.
198
3. Let the bridge open the `pro.openagents.com` event stream from the session cursor.
199
4. Decode bounded frames and broadcast public DTO events on a PubSub topic derived from the opaque session ID.
200
5. Track the last confirmed cursor in bridge state.
201
6. Reconnect with exponential backoff and jitter after transport failure.
202
7. Request a fresh snapshot after `410 Gone`, invalid ordering, or an unrecoverable parse error.
203
8. Stop the bridge after the last local subscriber leaves and a short grace period expires.
204
205
The bridge holds only bounded public projections and cursor state. It does not persist message content in the public database; `OpenAgents.Conversations` owns persistence.
206
207
## Phase 1: Chat shell and authentication
208
209
1. Reuse the authenticated browser pipeline and `live_session` from `docs/github-auth-plan.md` for `/chat`.
210
2. Add `OpenAgents.Conversations` and `OpenAgents.Turns` with Ecto schemas.
211
3. Add `OpenAgents.Sarah` and `OpenAgents.Sarah.ReqClient` with a configured `pro.openagents.com` base URL.
212
4. Add `OpenAgentsWeb.ChatLive` with a composer, transcript scroller, and connection state.
213
5. Add service assertion creation and credential redaction.
214
6. Configure connection, first-byte, inactivity, and total-request timeouts by operation.
215
7. Add a fake Sarah implementation and representative JSON and event fixtures under `test/support`.
216
217
**Exit criteria:** tests cover authenticated access, session creation, message send, turn state, timeout, malformed JSON, unknown schema version, and retry.
218
219
## Phase 2: Streaming and tool activity
220
221
1. Implement the session bridge and SSE decoder.
222
2. Apply events idempotently by session ID, resource ID, and event sequence.
223
3. Ignore events for another session.
224
4. Reconcile optimistic form state with authoritative `message.upsert` and `turn.upsert` events.
225
5. Reset streams from `session.snapshot` after cursor expiry.
226
6. Add `OpenAgents.Tools` for tool execution and `OpenAgents.Tools.ToolStep` for activity projection.
227
228
**Exit criteria:** LiveView tests prove streaming, reconnect, tool activity, and cancellation.
229
230
## Phase 3: Memory and delegation
231
232
1. Add `OpenAgents.Memory` for record extraction, retrieval, ranking, correction, and deletion.
233
2. Add `OpenAgents.Delegation` for starting and tracking agent work.
234
3. Add `OpenAgents.Agents` for agent selection and authorization.
235
4. Surface memory and delegated work in the chat UI.
236
237
**Exit criteria:** users can save, search, correct, delete memory records, and start, monitor, and cancel delegated work.
238
239
## Phase 4: Voice
240
241
1. Add `OpenAgents.Sarah.Voice` for voice session creation.
242
2. Add `OpenAgentsWeb.ChatVoiceController` for same-origin voice setup.
243
3. Add `OpenAgentsWeb.Components.ChatVoice` for media controls.
244
4. Add the browser voice JavaScript.
245
246
**Exit criteria:** authenticated users can start a voice session, send audio, and receive assistant audio without ever seeing a provider credential.
5
Status: Superseded on 2026-08-20 by
6
[the integrated architecture](architecture.md) and
7
[ADR 0001](decisions/0001-integrate-the-complete-public-application.md)
8
9
## Historical decision
10
11
This plan originally proposed that the public application own the `/chat`
12
interface while a private Sarah service owned persona, voice, and provider
13
logic. The implemented repository instead owns the complete product contract:
14
presentation, durable conversations and turns, persona artifacts, memory,
15
tools, delegated work, voice, data rights, and provider orchestration.
16
17
There is no required private Sarah API in the current architecture. Do not add
18
one based on this document.
19
20
## Requirements preserved from the plan
21
22
The rejected service boundary does not invalidate these provider-boundary
23
requirements:
24
25
- Provider credentials and internal-service credentials remain server-side.
26
- LiveView owns presentation, not outbound HTTP, token construction, retries,
27
  or provider-wire decoding.
28
- Provider adapters use bounded connect, response, inactivity, and total
29
  timeouts and normalize errors before they reach receipts or users.
30
- Every accepted turn is durable before provider work begins. Streaming,
31
  PubSub, and browser state are projections rather than authority.
32
- Resource ownership is derived from an authenticated server-side principal;
33
  a client-supplied conversation, turn, message, or session ID grants nothing.
34
- Model output and Markdown are untrusted and must be bounded and sanitized.
35
- Mutations and replayable provider events use durable identities and
36
  idempotency rules.
37
- Logs and operational telemetry exclude credentials, message content,
38
  transcripts, memory claims, raw tool payloads, and provider error prose.
39
- Voice permission is explicit, provider failure leaves typed chat available,
40
  and no provider key is sent to the browser.
41
- Network providers are replaced by explicit fakes in tests.
42
43
These requirements now apply to the in-process adapters and durable contexts
44
described by [docs/architecture.md](architecture.md). If an infrastructure
45
service is introduced later, it must remain behind one of those adapters and
46
must not take ownership of public product policy or data-rights behavior.
47
48
## Historical closeout
49
50
The proposed cross-service schemas, session bridge, signed user assertions,
51
and staged HTTP phases were never adopted as the product boundary. Their valid
52
security properties are covered by the provider, identity, turn, voice, and
53
release invariants in [INVARIANTS.md](../INVARIANTS.md). Current implementation
54
and staging work is tracked only in the
55
[integration hardening plan](2026-08-20-integration-hardening-and-staging-readiness-recommendations.md).
docs/component-library.md modified +84 -140

@@ -1,142 +1,86 @@

1 1
# Component library
2 2
3
Date: 2026-08-19
4
5
This is the master list of reusable HEEx components for the Agent Forge UI.
6
7
- **Shipped** components appear on `/components`.
8
- **Planned** components come from the issues and projects UI roadmap
9
  (`docs/issues-projects-ui-roadmap.md`) and the GitHub-shaped harvest
10
  work in `sarah` (`docs/audits/2026-08-19-github-forge-atomic-components.md`,
11
  `docs/audits/2026-08-19-github-clone-harvest-candidates.md`).
12
13
Build new forge-only molecules and organisms in a dedicated module such as
14
`OpenAgentsWeb.Code` or `OpenAgentsWeb.Issues`. Keep generic atoms in
15
`OpenAgentsWeb.CoreComponents`. Add every new component to `/components`
16
when it ships.
17
18
Clone root: `~/work/projects/repos/githubclones/`.
19
20
## What exists today
21
22
These are the reusable function components on `origin/main`. The catalog
23
at `/components` renders each one.
24
25
| Layer | Component | Module | Notes |
26
| --- | --- | --- | --- |
27
| Atom | `button/1` | `CoreComponents` | Soft default and `variant="primary"`; supports `navigate` |
28
| Atom | `input/1` | `CoreComponents` | Text, checkbox, select, textarea, hidden |
29
| Atom | `icon/1` | `CoreComponents` | Heroicons via `hero-*` class names |
30
| Atom | `flash/1` | `CoreComponents` | Info and error toasts |
31
| Molecule | `header/1` | `CoreComponents` | Title, subtitle, actions |
32
| Molecule | `list/1` | `CoreComponents` | Titled description rows |
33
| Organism | `table/1` | `CoreComponents` | Zebra table; supports LiveView streams |
34
| Organism | `flash_group/1` | `Layouts` | Wraps page flashes; do not call outside layouts |
35
| Organism | `app/1` | `Layouts` | Page chrome |
36
| Atom | `theme_toggle/1` | `Layouts` | System, light, dark |
37
| Organism | `repo_header/1` | `Components` | Owner lockup and repository subnavigation tabs |
38
39
Phoenix also provides `<.form>`, `<.link>`, and `<.inputs_for>`. Use those
40
instead of hand-rolled forms.
41
42
## Planned atoms
43
44
Generic or tiny forge-only spans. Reuse `button`, `input`, and `icon`
45
where they already fit.
46
47
| Build | Role | Harvest |
48
| --- | --- | --- |
49
| `badge/1` | Visibility, issue state, label color | DaisyUI `badge`; Primer `primer-view_components/app/components/primer/beta/label.rb` |
50
| `avatar/1` | Owner and author faces | DaisyUI `avatar`; `gh-next/src/components/avatar.tsx` |
51
| `sha/1` | Short commit id that links to `/commit/:sha` | `git.limo/apps/gitgud_web/lib/gitgud_web/live/commit_diff_live.html.heex` |
52
| `ref_name/1` | Branch or tag pill | `primer-css/src/branch-name/branch-name.scss` |
53
| `counter/1` | Count beside a nav word | `primer-view_components/app/components/primer/beta/counter.rb` |
54
| `relative_time/1` | "3 hours ago" with a `datetime` | `primer-view_components/app/components/primer/beta/relative_time.rb` |
55
| `clipboard_copy/1` | Copy a SHA or clone URL | `primer-view_components/app/components/primer/beta/clipboard_copy.rb` |
56
| `truncate/1` | Single-line commit or issue title | `primer-view_components/app/components/primer/beta/truncate.rb` |
57
| `empty/1` | Empty issues, compare, or repo | DaisyUI empty pattern; `CoreComponents` has no empty yet |
58
| `kbd/1` | Keyboard hints | DaisyUI `kbd` |
59
60
## Planned molecules
61
62
| Build | Role | Harvest |
63
| --- | --- | --- |
64
| `owner_lockup/1` | Avatar + `owner/repo` links | `gh-next/src/app/(app)/[user]/[repository]/page.tsx`; `gitea/templates/repo/header.tmpl` |
65
| `path_breadcrumb/1` | Path prefixes that `patch` | `git.limo/apps/gitgud_web/lib/gitgud_web/live/tree_browser_live.html.heex`; `primer-view_components/app/components/primer/beta/breadcrumbs.html.erb` |
66
| `clone_field/1` | Readonly clone URL + copy | `git.limo/.../tree_browser_live.html.heex` (`#clone-repo`); `gitea/templates/repo/clone_panel.tmpl` |
67
| `branch_picker/1` | Current ref + menu of refs | `git.limo/.../branch_select_live.ex`; `gitea/templates/repo/branch_dropdown.tmpl` |
68
| `file_row/1` | Name, last commit, age (commit may be nil) | `gitea/templates/repo/view_list.tmpl`; `git.limo/.../tree_browser_live.html.heex` |
69
| `commit_row/1` | SHA, subject, author, time | `gitea/templates/repo/commits_table.tmpl` |
70
| `issue_row/1` | State, title, labels, author, comments | `gh-next/src/components/issues/issue-row.tsx`; `gitea/templates/repo/issue/list.tmpl` |
71
| `ref_row/1` | Branch or tag + SHA | `gitea/templates/repo/branch/list.tmpl` |
72
| `underline_nav/1` | Repo tabs with an active underline | `primer-view_components/app/components/primer/alpha/underline_nav.html.erb`; `gitea/templates/repo/navbar.tmpl` |
73
| `label_badge/1` | Colored issue label | `gh-next/src/components/label-badge.tsx`; `gitea/templates/repo/issue/labels/label_list.tmpl` |
74
| `assignee_stack/1` | Avatar group with overflow | `gh-next/src/components/issues/issue-row-avatar-stack.tsx` |
75
| `compare_ends/1` | Base picker + `...` + head picker | `gitea/templates/repo/diff/compare.tmpl` |
76
| `diff_stat/1` | File count and `+n` / `−n` | `gitea/templates/repo/diff/stats.tmpl` |
77
| `comment_form/1` | Markdown body + submit | `git.limo/.../comment_form_live.html.heex`; `gitea/templates/repo/issue/comment_tab.tmpl` |
78
79
## Planned organisms
80
81
| Build | Role | Harvest |
82
| --- | --- | --- |
83
| `repo_subnav/1` | Code, Issues, Pull requests, Projects, Settings | `gitea/templates/repo/navbar.tmpl`; `gitea/templates/repo/issue/navbar.tmpl` |
84
| `issue_list/1` | Open/closed tabs, filters, streamed rows | `gh-next/src/components/issues/issue-list.tsx`; `gitea/templates/repo/issue/list.tmpl` |
85
| `issue_detail/1` | Title, state, body, sidebar metadata | `gh-next/src/app/(app)/[user]/[repository]/issues/[number]/page.tsx`; `gitea/templates/repo/issue/view.tmpl` |
86
| `comment_thread/1` | Chronological comments | `gitea/templates/repo/issue/view_content.tmpl`; `git.limo/.../issue_live.html.heex` |
87
| `issue_form/1` | New and edit issue | `gh-next/src/components/issues/new-issue-form.tsx`; `gitea/templates/repo/issue/new_form.tmpl` |
88
| `label_manager/1` | Create, edit, delete labels | `gitea/templates/repo/issue/labels/label_list.tmpl` |
89
| `milestone_list/1` | Progress cards | `gitea/templates/repo/issue/milestones.tmpl` |
90
| `project_board/1` | Columns of items (no drag-and-drop at first) | GitHub Projects V2 REST in `docs/github-api-issues-projects-assessment.md` |
91
| `file_table/1` | Directory listing | `gitea/templates/repo/view_list.tmpl`; `git.limo/.../tree_browser_live.ex` |
92
| `blob_panel/1` | File view + actions | `gitea/templates/repo/view_file.tmpl`; `git.limo/.../blob_viewer_live.ex` |
93
| `readme_panel/1` | Rendered README | `git.limo/.../tree_browser_live.html.heex` README card |
94
| `commit_list/1` | Bounded log | `gitea/templates/repo/commits.tmpl` |
95
| `diff_viewer/1` | Unified hunks | `gitea/templates/repo/diff/box.tmpl`; `section_unified.tmpl` |
96
| `compare_panel/1` | Compare ends + diff | `gitea/templates/repo/diff/compare.tmpl` |
97
98
## Planned pages
99
100
Mount these on GitHub-shaped paths. See
101
`docs/issues-projects-ui-roadmap.md` and the sarah harvest audit.
102
103
| Page | Path | First organisms |
104
| --- | --- | --- |
105
| Owner | `/:owner` | Repo cards |
106
| Repo home | `/:owner/:repo` | `repo_header`, `file_table`, `readme_panel` |
107
| Issues | `/:owner/:repo/issues` | `repo_header`, `issue_list` |
108
| Issue | `/:owner/:repo/issues/:number` | `issue_detail`, `comment_thread` |
109
| New issue | `/:owner/:repo/issues/new` | `issue_form` |
110
| Labels | `/:owner/:repo/labels` | `label_manager` |
111
| Milestones | `/:owner/:repo/milestones` | `milestone_list` |
112
| Projects | `/:owner/:repo/projects` | `project_board` |
113
| Tree | `/:owner/:repo/tree/:ref/*path` | `file_table` |
114
| Blob | `/:owner/:repo/blob/:ref/*path` | `blob_panel` |
115
| Commit | `/:owner/:repo/commit/:sha` | `diff_viewer` |
116
| Compare | `/:owner/:repo/compare/:base...:head` | `compare_panel` |
117
| Catalog | `/components` | This page |
118
119
## Build order
120
121
1. Atoms that issues need first: `badge`, `avatar`, `relative_time`,
122
   `empty`, `label_badge`.
123
2. Molecules: `owner_lockup`, `issue_row`, `underline_nav`,
124
   `comment_form`.
125
3. Organisms: `repo_header`, `repo_subnav`, `issue_list`,
126
   `issue_detail`, `comment_thread`.
127
4. Pages on `/:owner/:repo/issues` as in the UI roadmap.
128
5. Code surfaces (`file_table`, `blob_panel`, `diff_viewer`) after the
129
   tracker dogfoods.
130
6. `/api/v3` stays the machine API from
131
   `docs/github-api-issues-projects-assessment.md`. It is not a
132
   component.
133
134
Do not add Octicons, Primer CSS as a runtime, or a second button
135
system. DaisyUI plus `CoreComponents` is the kit.
136
137
## See also
138
139
- `/components` — live catalog of shipped components
140
- `docs/issues-projects-ui-roadmap.md` — page-level UI plan
141
- `docs/issues-projects-work-plan.md` — API epics
142
- `docs/github-api-issues-projects-assessment.md` — GitHub REST subset
3
Date: 2026-08-20
4
5
Status: Current inventory; consolidation continues in hardening Gate 4
6
7
The public catalog at `/components` is the executable inventory of reusable
8
HEEx components. `OpenAgentsWeb.ComponentCatalog` supplies its navigation,
9
`OpenAgentsWeb.ComponentsLive` renders every demo, and
10
`test/openagents_web/component_catalog_test.exs` fails when a public component
11
is absent from the catalog.
12
13
## Sanctioned system
14
15
New product interface work starts in `OpenAgentsWeb.UI`. Its components wrap
16
the pinned Basecoat styles imported by `assets/css/app.css` and receive product
17
identity from `assets/css/openagents.css`.
18
19
The current `OpenAgentsWeb.UI` inventory is:
20
21
| Component | Purpose |
22
| --- | --- |
23
| `button/1`, `text_button/1` | Boxed, link, chip, destructive, and primary actions |
24
| `input/1`, `textarea/1`, `label/1`, `field/1` | Form controls and labelled groups |
25
| `alert/1`, `badge/1`, `status_indicator/1` | Explicit feedback and semantic state |
26
| `card/1`, `frame/1` | Bounded content and decorative framing |
27
| `avatar/1`, `item/1`, `event_header/1` | Identity and activity rows |
28
| `empty/1`, `kbd/1`, `menu/1` | Empty states, key hints, and native-popover menus |
29
| `audio_player/1` | Accessible native audio control in the product frame |
30
| `icon/1` | Glyphs from the vendored Apps SDK icon set |
31
32
`OpenAgentsWeb.Layouts` owns `app/1`, `flash_group/1`, `command_bar/1`,
33
`account_control/1`, and the currently catalogued theme control. Product
34
templates begin with `Layouts.app` and never render `flash_group/1` directly.
35
36
## Transitional components
37
38
`OpenAgentsWeb.CoreComponents` still contains the Phoenix-generated `button`,
39
`input`, `header`, `table`, `list`, `icon`, and `flash` components. The catalog
40
shows these separately because their names overlap the OpenAgents primitives.
41
They are compatibility surface, not a second design system. Gate 4 must either
42
migrate each remaining caller to `OpenAgentsWeb.UI` or document a narrow reason
43
to retain the generated helper.
44
45
`OpenAgentsWeb.Components.RepoHeader.repo_header/1` is the one catalogued
46
forge-specific component. Surface-specific components can live in a focused
47
module when they encode real domain composition rather than a generic control.
48
49
## Extension rules
50
51
1. Search `OpenAgentsWeb.UI` and `/components` before creating a component.
52
2. Extend `OpenAgentsWeb.UI` for a reusable primitive; keep feature composition
53
   in a feature module.
54
3. Use an individually imported Basecoat component stylesheet only when the
55
   component needs it. Never import the aggregate Basecoat bundles.
56
4. Put OpenAgents-owned component styles in `assets/css/openagents.css`; do not
57
   patch `assets/vendor/basecoat/`.
58
5. Use `OpenAgentsWeb.UI.icon/1` and the vendored icon set. Do not add an icon
59
   font, another glyph library, or handwritten SVG in a template.
60
6. Give every icon-only action an accessible name. Decorative icons beside text
61
   remain hidden from accessibility APIs.
62
7. Add the component to `OpenAgentsWeb.ComponentCatalog`, add its demo to
63
   `OpenAgentsWeb.ComponentsLive`, and add behavior/accessibility tests in the
64
   same change.
65
8. Keep forms on `Phoenix.Component.to_form/2`; use LiveView streams for
66
   collections and stable DOM IDs for testable controls.
67
68
## Current catalog gaps
69
70
Issue, project, and code surfaces currently compose generic controls directly.
71
Create new domain components only when repeated behavior justifies them. Likely
72
candidates are issue rows, comment threads, label controls, project columns,
73
repository breadcrumbs, file rows, commit rows, and bounded diff panels.
74
75
The Gate 4 consolidation also owns these known transitional issues:
76
77
- Remove the generated component/icon path after its callers are migrated.
78
- Enforce the repository's final two-tier icon policy in code and tests.
79
- Remove the nonfunctional light/system theme choice if the shipped palette
80
  remains dark-only.
81
- Prove component variants and semantic colors in compiled CSS rather than by
82
  class-name assertions alone.
83
84
See [the UI roadmap](issues-projects-ui-roadmap.md),
85
[ADR 0005](decisions/0005-use-basecoat-and-one-component-system.md), and the
86
[hardening plan](2026-08-20-integration-hardening-and-staging-readiness-recommendations.md).
docs/decisions/0004-retain-scoped-github-access-tokens.md modified +7 -5

@@ -2,7 +2,7 @@

2 2
3 3
Date: 2026-08-20
4 4
5
Status: Accepted; hardening required before staging
5
Status: Accepted model; lifecycle hardening required before staging
6 6
7 7
## Context
8 8

@@ -19,10 +19,12 @@ operator-managed key, associate it with one active user, and never expose it to

19 19
LiveView assigns, browser payloads, logs, receipts, or telemetry.
20 20
21 21
Keep OpenAgents issue and project data in PostgreSQL; do not use a retained
22
GitHub token as authority for OpenAgents-owned records. Revoke or delete the
23
token when the user disconnects GitHub, deletes their data, or loses account
24
access. Gate 6 must verify scopes, rotation, failure behavior, redaction, and
25
the disclosures shown to users.
22
GitHub token as authority for OpenAgents-owned records. The current callback
23
stores encrypted ciphertext and logout clears only the browser session. Gate 6
24
must add explicit disconnect/revocation behavior, define token removal during
25
product-data deletion or account restriction, and verify scopes, rotation,
26
failure behavior, redaction, and user disclosures. Those lifecycle actions are
27
requirements, not claims about the current implementation.
26 28
27 29
## Consequences
28 30
docs/github-api-issues-projects-assessment.md modified +79 -113

@@ -1,113 +1,79 @@

1
# GitHub Issues and Projects API — implementation assessment
2
3
Date: 2026-08-19
4
Source: `rest-api-description/descriptions/api.github.com/api.github.com.2026-03-10.yaml`
5
6
For the concrete work breakdown, parallelization map, and subagent dispatch notes, see `docs/issues-projects-work-plan.md`.
7
8
## Goal
9
10
Dogfood OpenAgents by using it to track this repo's own issues and projects. We want enough GitHub REST API parity that `gh`, Octokit, and the GitHub CLI can talk to OpenAgents without changes, but we will only build the subset we actually use.
11
12
## Summary
13
14
The 2026-03-10 OpenAPI spec has 84 endpoints tagged with `issue` or `project`. Most are for advanced features. For day-one use we need the core create/read/update/comment/label/assign/milestone flow, plus the ability to add issues to a project board and update board fields.
15
16
## Issues API — what to build
17
18
### Phase 1 — must have for dogfooding
19
20
These endpoints are enough to open, discuss, label, assign, and close issues in public:
21
22
| Method | Path | Why it matters |
23
| --- | --- | --- |
24
| GET | `/repos/{owner}/{repo}/issues` | List repository issues |
25
| GET | `/repos/{owner}/{repo}/issues/{issue_number}` | Get a single issue |
26
| POST | `/repos/{owner}/{repo}/issues` | Create an issue |
27
| PATCH | `/repos/{owner}/{repo}/issues/{issue_number}` | Edit, close, or reopen an issue |
28
| GET | `/repos/{owner}/{repo}/issues/{issue_number}/comments` | List comments on an issue |
29
| POST | `/repos/{owner}/{repo}/issues/{issue_number}/comments` | Add a comment |
30
| GET | `/repos/{owner}/{repo}/issues/comments/{comment_id}` | Get a single comment |
31
| PATCH | `/repos/{owner}/{repo}/issues/comments/{comment_id}` | Edit a comment |
32
| DELETE | `/repos/{owner}/{repo}/issues/comments/{comment_id}` | Delete a comment |
33
| GET | `/repos/{owner}/{repo}/labels` | List repository labels |
34
| GET | `/repos/{owner}/{repo}/labels/{name}` | Get a label |
35
| POST | `/repos/{owner}/{repo}/labels` | Create a label |
36
| PATCH | `/repos/{owner}/{repo}/labels/{name}` | Update a label |
37
| DELETE | `/repos/{owner}/{repo}/labels/{name}` | Delete a label |
38
| GET | `/repos/{owner}/{repo}/issues/{issue_number}/labels` | List labels on an issue |
39
| POST | `/repos/{owner}/{repo}/issues/{issue_number}/labels` | Add labels to an issue |
40
| DELETE | `/repos/{owner}/{repo}/issues/{issue_number}/labels/{name}` | Remove a label from an issue |
41
| GET | `/repos/{owner}/{repo}/assignees` | List who can be assigned |
42
| GET | `/repos/{owner}/{repo}/assignees/{assignee}` | Check if a user can be assigned |
43
| POST | `/repos/{owner}/{repo}/issues/{issue_number}/assignees` | Add assignees |
44
| DELETE | `/repos/{owner}/{repo}/issues/{issue_number}/assignees` | Remove assignees |
45
| GET | `/repos/{owner}/{repo}/milestones` | List milestones |
46
| GET | `/repos/{owner}/{repo}/milestones/{milestone_number}` | Get a milestone |
47
| POST | `/repos/{owner}/{repo}/milestones` | Create a milestone |
48
| PATCH | `/repos/{owner}/{repo}/milestones/{milestone_number}` | Update a milestone |
49
| DELETE | `/repos/{owner}/{repo}/milestones/{milestone_number}` | Delete a milestone |
50
51
### Phase 2 — nice to have
52
53
| Method | Path | Why it matters |
54
| --- | --- | --- |
55
| GET | `/repos/{owner}/{repo}/issues/events` | Activity feed |
56
| GET | `/repos/{owner}/{repo}/issues/events/{event_id}` | Single event |
57
| GET | `/repos/{owner}/{repo}/issues/{issue_number}/events` | Issue-specific events |
58
| GET | `/repos/{owner}/{repo}/issues/{issue_number}/timeline` | Full timeline |
59
| PUT | `/repos/{owner}/{repo}/issues/{issue_number}/lock` | Lock an issue |
60
| DELETE | `/repos/{owner}/{repo}/issues/{issue_number}/lock` | Unlock an issue |
61
62
### Out for now
63
64
- `/issues` and `/user/issues` — cross-repo lists. We can start repo-scoped.
65
- `/orgs/{org}/issues` — org-level issue list. Not needed for one repo.
66
- Issue dependencies and sub-issues — useful later, not required for a public tracker.
67
- Issue suggestions and issue-field-values — tied to newer GitHub custom fields and AI features. Skip until we need them.
68
69
## Projects API — what to build
70
71
### Important caveat
72
73
The 2026-03-10 GitHub REST spec has only a limited Projects V2 surface. It lists projects and items, but it does **not** include a REST endpoint to create or update a project. Board creation and field/schema changes are GraphQL in the official API.
74
75
### Phase 1 — implement the existing REST subset
76
77
| Method | Path | Why it matters |
78
| --- | --- | --- |
79
| GET | `/users/{username}/projectsV2` | List user projects |
80
| GET | `/users/{username}/projectsV2/{project_number}` | Get a user project |
81
| GET | `/users/{username}/projectsV2/{project_number}/items` | List project items |
82
| GET | `/users/{username}/projectsV2/{project_number}/items/{item_id}` | Get a project item |
83
| POST | `/users/{username}/projectsV2/{project_number}/items` | Add an issue to a project |
84
| PATCH | `/users/{username}/projectsV2/{project_number}/items/{item_id}` | Update a project item (status, field values) |
85
| DELETE | `/users/{username}/projectsV2/{project_number}/items/{item_id}` | Remove an item from a project |
86
| GET | `/users/{username}/projectsV2/{project_number}/fields` | List project fields |
87
| GET | `/users/{username}/projectsV2/{project_number}/fields/{field_id}` | Get a project field |
88
| GET | `/users/{username}/projectsV2/{project_number}/views` | Create a view for a user-owned project |
89
90
Also implement the same set under `/orgs/{org}/projectsV2` when we are ready for org-scoped projects.
91
92
### Phase 1 — add the missing write endpoints
93
94
Because the official REST spec is incomplete, add the following non-GitHub-standard OpenAgents-specific project endpoints to unblock day-one board creation and editing:
95
96
| Method | Path | Why it matters |
97
| --- | --- | --- |
98
| POST | `/{owner}/projectsV2` | Create a new project |
99
| PATCH | `/{owner}/projectsV2/{project_number}` | Update project title and settings |
100
| DELETE | `/{owner}/projectsV2/{project_number}` | Delete a project |
101
| POST | `/{owner}/projectsV2/{project_number}/fields` | Add a custom field |
102
| POST | `/{owner}/projectsV2/{project_number}/views` | Create a board view |
103
104
These are not in the GitHub REST spec, but they are required for a usable project tracker. We can make them GitHub-compatible where it makes sense and document the gap.
105
106
### Out for now
107
108
- Draft items, project views in detail, and advanced project view item ordering — add once the core board works.
109
- Organization-level projects — implement after user projects work.
110
111
## Recommendation
112
113
Build the Issues Phase 1 list first. It is enough for the public to open, discuss, and manage issues on OpenAgents.com. Then add the Projects V2 read endpoints and the non-standard project write endpoints so we can organize those issues into a public board.
1
# GitHub-shaped Issues and Projects API assessment
2
3
Date: 2026-08-20
4
5
Status: Implemented subset with known compatibility and authorization gaps
6
7
## Intent
8
9
OpenAgents exposes a bounded `/api/v3` subset so familiar GitHub-shaped clients
10
can interact with issues and project boards. The shape is a compatibility aid,
11
not a claim that the application implements the complete GitHub API.
12
13
The router and controller tests are the executable source of truth. This
14
document records the intended subset and known gaps; it must not be used to
15
infer authorization that the server does not enforce.
16
17
## Implemented issue subset
18
19
| Concern | Methods and paths |
20
| --- | --- |
21
| Issues | `GET, POST /repos/{owner}/{repo}/issues`; `GET, PATCH /repos/{owner}/{repo}/issues/{number}` |
22
| Comments | `GET, POST /repos/{owner}/{repo}/issues/{number}/comments`; `GET, PATCH, DELETE /repos/{owner}/{repo}/issues/comments/{id}` |
23
| Labels | `GET, POST /repos/{owner}/{repo}/labels`; `GET, PATCH, DELETE /repos/{owner}/{repo}/labels/{name}` |
24
| Issue labels | `GET, POST /repos/{owner}/{repo}/issues/{number}/labels`; `DELETE /repos/{owner}/{repo}/issues/{number}/labels/{name}` |
25
| Assignees | `GET /repos/{owner}/{repo}/assignees`; `GET /repos/{owner}/{repo}/assignees/{login}`; issue-assignee list/add/remove paths |
26
| Milestones | `GET, POST /repos/{owner}/{repo}/milestones`; `GET, PATCH, DELETE /repos/{owner}/{repo}/milestones/{number}` |
27
28
Cross-repository issue lists, organization issue lists, event/timeline APIs,
29
locks, dependencies, sub-issues, and suggestion APIs are not implemented.
30
31
## Implemented Projects V2 subset
32
33
| Method | Path |
34
| --- | --- |
35
| `GET` | `/users/{username}/projectsV2` |
36
| `POST` | `/{owner}/projectsV2` |
37
| `GET` | `/users/{username}/projectsV2/{project_number}` |
38
| `GET, POST` | `/users/{username}/projectsV2/{project_number}/items` |
39
| `PATCH` | `/users/{username}/projectsV2/{project_number}/items/{item_id}` |
40
| `GET` | `/users/{username}/projectsV2/{project_number}/fields` |
41
42
The project-creation endpoint is an OpenAgents extension because the comparable
43
GitHub Projects V2 creation workflow is not supplied by the assessed REST
44
surface. Project update/delete, item delete/read, field mutation, views,
45
ordering, draft items, and organization projects remain unimplemented.
46
47
## Known blockers
48
49
These are current measured behaviors, not hypothetical future concerns:
50
51
- `/api/v3` uses the generic API pipeline and does not yet have a deliberate
52
  bearer/PAT authorization model for CLI clients.
53
- The route's owner/repository values are not backed by a canonical repository
54
  foreign key across the issue/project schema.
55
- Project show, item, update-item, and field actions do not consistently enforce
56
  the username in the route.
57
- The assignee read endpoints are placeholders while issue writes accept
58
  arbitrary login strings.
59
- Label rename does not implement GitHub's `new_name` behavior.
60
- Adding a nonexistent label, removing an absent label, and some label URL
61
  encoding cases differ from GitHub behavior.
62
- Error envelopes and pagination/link headers are a bounded local contract,
63
  not complete Octokit or `gh` parity.
64
65
Gate 6 must assign an explicit API principal and mutation policy. Gate 7 must
66
add repository entities, foreign keys, scoped uniqueness, one-query ownership
67
checks, and cross-repository isolation tests. Compatibility fixes follow those
68
authority boundaries rather than preceding them.
69
70
## Evidence
71
72
- `test/openagents_web/controllers/issue_controller_test.exs`
73
- `test/openagents_web/controllers/comment_controller_test.exs`
74
- `test/openagents_web/controllers/label_controller_test.exs`
75
- `test/openagents_web/controllers/issue_label_controller_test.exs`
76
- `test/openagents_web/controllers/assignee_controller_test.exs`
77
- `test/openagents_web/controllers/issue_assignee_controller_test.exs`
78
- `test/openagents_web/controllers/milestone_controller_test.exs`
79
- `test/openagents_web/controllers/project_controller_test.exs`
docs/github-auth-plan.md modified +77 -104

@@ -1,119 +1,92 @@

1
# GitHub authentication plan
1
# GitHub authentication and token lifecycle
2 2
3
Date: 2026-08-19
3
Date: 2026-08-20
4 4
5
Source: `~/work/sarah` GitHub OAuth implementation and the current OpenAgents issue tracker UI.
5
Status: Authentication implemented; token-lifecycle hardening pending Gate 6
6 6
7
OpenAgents needs a real, server-side GitHub OAuth flow so that issue and project pages can authenticate the visitor as a real GitHub user. This plan ports the proven auth stack from `sarah`, adapts it to the OpenAgents namespace, and drives every step out with a failing test first.
7
Decision: [ADR 0004](decisions/0004-retain-scoped-github-access-tokens.md)
8 8
9
## Goal
9
## Current contract
10 10
11
A visitor can sign in with GitHub and then access the issue tracker at `/:owner/:repo/issues` and `/:owner/:repo/issues/new`. There are no seeded users, no placeholder owners, and no fake tokens. The issue tracker keeps its own data; GitHub is only used for login and identity.
11
GitHub serves two distinct roles:
12 12
13
## What we are copying from `sarah`
13
1. OAuth establishes the local OpenAgents account identity from GitHub's
14
   immutable numeric user ID.
15
2. A retained access token authorizes server-side GitHub repository tools with
16
   the user's delegated rights.
14 17
15
These `sarah` modules are the reference implementation:
18
The application currently implements the second model. The callback stores the
19
access token as AES-256-GCM ciphertext in the local user row. It does not
20
discard the token after reading the GitHub profile. Documentation and data
21
rights must not claim otherwise.
16 22
17
- `lib/sarah/accounts.ex`
18
- `lib/sarah/accounts/oauth_attempt.ex`
19
- `lib/sarah/accounts/token_vault.ex`
20
- `lib/sarah/accounts/user.ex`
21
- `lib/sarah/github.ex`
22
- `lib/sarah/github_oauth.ex`
23
- `lib/sarah/github_oauth/runtime_config.ex`
24
- `lib/sarah_web/controllers/auth_controller.ex`
25
- `lib/sarah_web/router.ex` auth routes and `UserAuth` hooks
23
## Implemented flow
26 24
27
## New modules and changes
25
1. `POST /auth/github` creates a high-entropy state value, a PKCE S256
26
   challenge, and a short-lived PostgreSQL OAuth-attempt row.
27
2. The encrypted browser session carries only the attempt reference and PKCE
28
   verifier while GitHub handles authorization.
29
3. `GET /auth/github/callback` consumes the attempt exactly once, exchanges the
30
   code server-side, and reads the GitHub `/user` projection server-side.
31
4. `OpenAgents.Accounts` upserts the local account by numeric GitHub ID and
32
   refreshes the mutable login, name, and avatar projection.
33
5. `OpenAgents.Accounts.TokenVault` encrypts the access token with the configured
34
   key before `github_token_ciphertext` is stored.
35
6. The authenticated session contains only the local user ID. Repository tools
36
   unseal the token server-side when an explicit GitHub operation needs it.
37
7. `DELETE /logout` clears the browser session but intentionally does not
38
   revoke the retained GitHub grant.
28 39
29
### Domain and persistence
40
The token must never enter LiveView assigns, HTML, JSON responses, logs,
41
telemetry, receipts, exception messages, build output, or exported account
42
data.
30 43
31
- `OpenAgents.Accounts` — upsert user, fetch active user, store and retrieve the encrypted GitHub token.
32
- `OpenAgents.Accounts.User` — Ecto schema for `users` with `github_id`, `github_login`, `github_name`, `github_avatar_url`, `status`, `last_authenticated_at`, and `github_token_ciphertext`.
33
- `OpenAgents.Accounts.OAuthAttempt` — Ecto schema for `github_oauth_attempts` with `state_digest`, `expires_at`, and `consumed_at`.
34
- `OpenAgents.Accounts.TokenVault` — AES-256-GCM seal and unseal for the GitHub access token.
35
- `OpenAgents.GitHub` — read-only GitHub REST wrapper that uses the user's token for repository operations such as listing owned repos. The issue tracker does not pull issue, label, milestone, or project data from GitHub.
36
- `OpenAgents.GitHubOAuth` — PKCE authorize URL, state/attempt handling, code exchange, and profile fetch.
37
- `OpenAgents.GitHubOAuth.RuntimeConfig` — validate `GITHUB_CLIENT_ID`, `GITHUB_CLIENT_SECRET`, and `GITHUB_REDIRECT_URI`.
44
## Current configuration
38 45
39
### Web interface
40
41
- `OpenAgentsWeb.AuthController` — `POST /auth/github`, `GET /auth/github/callback`, and `DELETE /logout`.
42
- `OpenAgentsWeb.UserAuth` — `fetch_current_user/2`, `require_authenticated/2`, `mount_current_user/4`, and `ensure_authenticated/4`.
43
- `OpenAgentsWeb.Router` — add auth routes, a `:browser` pipeline `fetch_current_user` plug, an `:authenticated` pipeline, and a `live_session :authenticated` `on_mount` hook around the `/:owner/:repo` issue and project routes.
44
- `lib/openagents_web/components/layouts.ex` — show a **Sign in with GitHub** button or the signed-in user's avatar and a **Log out** link in the navbar.
45
- `lib/openagents_web/live/home_live.ex` — add a sign-in CTA when the visitor is not authenticated.
46
- `lib/openagents_web/live/issue_new_live.ex` and other `/:owner/:repo` LiveViews — move inside the authenticated live session.
47
48
### Configuration and migrations
49
50
- `config/dev.exs`, `config/test.exs`, and `config/runtime.exs` — add `github_oauth` and `github_token_encryption_key` config.
51
- `priv/repo/migrations/..._create_users.exs` and `..._create_github_oauth_attempts.exs`.
52
53
## TDD steps and test files
54
55
For each step, write the test first, run `mix test <file>` to confirm it fails, then make it pass. Run `mix precommit` before moving on.
56
57
1. **User persistence**
58
   - `test/openagents/accounts_test.exs`
59
   - Test `upsert_github_user/1` inserts and updates a user by `github_id`.
60
   - Test `get_active_user/1` returns `{:ok, _}` for active and `{:error, :banned}` for banned.
61
   - Test `store_github_token/2` seals a token and `github_token/1` unseals it.
62
63
2. **OAuth attempt table**
64
   - `test/openagents/accounts/oauth_attempt_test.exs`
65
   - Test `create_oauth_attempt/2` and `consume_oauth_attempt/2` succeed only for valid, unexpired, unconsumed attempts.
66
67
3. **Token vault**
68
   - `test/openagents/accounts/token_vault_test.exs`
69
   - Test `seal/1` and `open/1` round-trip; test tampered ciphertext fails.
70
71
4. **GitHubOAuth flow**
72
   - `test/openagents/github_oauth_test.exs`
73
   - Test `begin_authorization/0` returns a URL with `client_id`, `code_challenge`, and `scope=read:user repo`.
74
   - Test `consume_attempt/2` validates state, verifier, and expiry.
75
   - Test `exchange_and_fetch/2` using a mocked `Req.post/2` and `Req.get/2` to return an access token and a profile.
76
77
5. **Auth controller**
78
   - `test/openagents_web/auth_controller_test.exs`
79
   - Test `POST /auth/github` redirects to `github.com/login/oauth/authorize` and sets a session attempt.
80
   - Test a valid `GET /auth/github/callback` creates the user, stores the token, sets `user_id` in the session, and redirects to `/`.
81
   - Test an invalid callback clears the session and redirects with an `auth_error`.
82
   - Test `DELETE /logout` clears the session and redirects.
83
84
6. **User auth plug and live hooks**
85
   - `test/openagents_web/plugs/user_auth_test.exs`
86
   - Test `fetch_current_user` assigns `current_user` from `user_id`.
87
   - Test `require_authenticated` redirects when there is no user.
88
   - Test `OpenAgentsWeb.UserAuth.ensure_authenticated/4` halts an unauthenticated live view.
89
90
7. **Protected issue pages**
91
   - `test/openagents_web/live/issue_new_live_test.exs`
92
   - Test an unauthenticated `GET /OpenAgents/openagents/issues/new` redirects to `/auth/github`.
93
   - Test an authenticated user with a stored token can visit `/:owner/:repo/issues/new`.
94
95
## Implementation order
96
97
1. Add `users` and `github_oauth_attempts` migrations and schemas.
98
2. Port `OpenAgents.Accounts` and `OpenAgents.Accounts.TokenVault` with tests.
99
3. Port `OpenAgents.GitHubOAuth` and `OpenAgents.GitHubOAuth.RuntimeConfig` with tests.
100
4. Add `OpenAgentsWeb.AuthController` and the `/auth/github`, `/auth/github/callback`, and `/logout` routes.
101
5. Add `OpenAgentsWeb.UserAuth` and the `:authenticated` pipeline, then wrap the `/:owner/:repo` LiveViews in an authenticated `live_session`.
102
6. Update `HomeLive` and `Layouts.app` to show the sign-in or user state.
103
104
## Configuration
105
106
Add these environment variables before running the app:
46
Runtime configuration requires:
107 47
108 48
- `GITHUB_CLIENT_ID`
109 49
- `GITHUB_CLIENT_SECRET`
110
- `GITHUB_REDIRECT_URI` — must be `http://localhost:4000/auth/github/callback` in development or an HTTPS callback in production.
111
- `GITHUB_TOKEN_ENCRYPTION_KEY` — a Base64-encoded 32-byte AES key.
112
113
## Acceptance
114
115
- `/` shows **Sign in with GitHub** when no session exists.
116
- After signing in, the navbar shows the GitHub login and a **Log out** link.
117
- Visiting `/:owner/:repo/issues/new` without a session redirects to `/auth/github`.
118
- Visiting `/:owner/:repo/issues/new` with a session renders the new issue form.
119
- `mix test` and `mix precommit` pass after each step.
50
- `GITHUB_REDIRECT_URI`
51
- `GITHUB_TOKEN_ENCRYPTION_KEY`, a Base64-encoded 32-byte key
52
53
Production-mode validation requires an HTTPS callback with the configured
54
environment host. Tests use deterministic local configuration and fake Req
55
responses; they do not require a live GitHub credential.
56
57
## Hardening required before staging
58
59
Gate 6 owns the remaining lifecycle and disclosure work:
60
61
- Request and document the minimum scopes needed by the enabled GitHub tools.
62
- Show a clear user disclosure that delegated repository access is retained
63
  encrypted after sign-in.
64
- Add an explicit disconnect operation that deletes the local ciphertext and,
65
  where GitHub supports it for this OAuth application, revokes the grant.
66
- Define what account data deletion does to the retained token. The current
67
  product-data deletion keeps the minimal local account row, so token removal
68
  must be implemented and tested rather than inferred.
69
- Support encryption-key rotation with a versioned envelope and a rehearsed
70
  rewrap path.
71
- Fail closed when the key is missing, malformed, or belongs to the wrong
72
  environment, without printing token or key material.
73
- Normalize revoked/expired token failures and require reauthorization without
74
  exposing GitHub response bodies.
75
- Add log and telemetry scans for token, code, state, verifier, and callback
76
  query leakage.
77
- Document the token's presence as metadata in export/delete disclosures
78
  without exporting the credential itself.
79
80
## Executable evidence
81
82
- `test/openagents/github_oauth_test.exs`
83
- `test/openagents/github_oauth/runtime_config_test.exs`
84
- `test/openagents/accounts_test.exs`
85
- `test/openagents/accounts/token_vault_test.exs`
86
- `test/openagents/github_test.exs`
87
- `test/openagents/tools/github_repo_tools_test.exs`
88
- `test/openagents_web/auth_controller_test.exs`
89
- `test/openagents_web/auth_gate_test.exs`
90
91
The complete route-authority and secret-handling acceptance criteria remain in
92
[the hardening plan](2026-08-20-integration-hardening-and-staging-readiness-recommendations.md).
docs/issues-projects-ui-roadmap.md modified +86 -416

@@ -1,416 +1,86 @@

1
# Issues and Projects UI roadmap
2
3
Date: 2026-08-19
4
5
Source: `docs/issues-projects-work-plan.md`, `docs/github-api-issues-projects-assessment.md`, and the GitHub clone harvest under `~/work/projects/repos/githubclones/`.
6
7
This roadmap outlines a simple browser UI for the existing OpenAgents issues, labels, assignees, milestones, comments, and Projects V2 API. The implementation uses Phoenix 1.8 LiveView, `OpenAgentsWeb.CoreComponents`, and DaisyUI component classes. The goal is a GitHub-shaped surface that is usable, not a pixel-perfect clone.
8
9
The reusable-component inventory, live catalog, and the list of components still to build are in `docs/component-library.md`. Shipped components render at `/components`.
10
11
## Scope and assumptions
12
13
- The JSON API is in place and follows the paths in `docs/github-api-issues-projects-assessment.md`.
14
- The UI lives on `/:owner/:repo` paths, starting with the surfaces in this document.
15
- `OpenAgentsWeb.CoreComponents` and DaisyUI are the building blocks.
16
- `Phoenix.Component.to_form/2` drives forms, and LiveView streams handle issue and comment lists.
17
- Markdown bodies are rendered as HTML with `OpenAgentsWeb` markdown helpers.
18
19
## What is out of scope
20
21
- Drag-and-drop project boards.
22
- Real-time presence and live updates beyond standard PubSub.
23
- Full-text issue search indexing.
24
- React-style hovercards and preview cards.
25
- File browser, code review, and pull request surfaces.
26
- Pixel-perfect GitHub Primer styling.
27
28
## Layout foundation
29
30
Build the repo header and subnavigation before any issue page.
31
32
### Global navigation and repo header
33
34
What the surface shows:
35
36
- A top **navbar** with the OpenAgents logo, a search field, and the current user avatar.
37
- A **repo header** with the owner avatar, owner name, repo name, and a visibility badge.
38
- A **subnav** row with **Code**, **Issues**, **Pull requests**, **Projects**, and **Settings** tabs. The active tab gets a highlighted underline.
39
40
DaisyUI parts:
41
42
- `navbar` for the top bar.
43
- `tabs` for the subnav.
44
- `avatar` and `badge` for the owner and visibility indicators.
45
- `btn` for star, fork, and watch actions.
46
- `input` for the search field.
47
48
Clone harvest:
49
50
- `gh-next/src/app/(app)/[user]/[repository]/page.tsx`
51
- `gitea/templates/repo/header.tmpl`
52
- `gitea/templates/repo/issue/navbar.tmpl`
53
54
Acceptance:
55
56
- The same `Layouts.app` wrapper is on every page with `current_scope` assigned.
57
- The subnav links to `/:owner/:repo/issues`, `/:owner/:repo/projects`, and so on.
58
59
## Phase 0: GitHub authentication and user identity
60
61
All issue, project, and label surfaces must use the real, signed-in GitHub user. No seeded or placeholder users are allowed. This phase ports the OAuth flow from `~/work/sarah` and wires it into the OpenAgents router, controllers, and LiveViews. See `docs/github-auth-plan.md` for the test-driven implementation plan.
62
63
### What this phase delivers
64
65
- GitHub OAuth sign-in and sign-out at `/auth/github`, `/auth/github/callback`, and `/logout`.
66
- A `users` table that stores the GitHub id, login, name, avatar, and an encrypted token.
67
- A `github_oauth_attempts` table for PKCE/state validation.
68
- `OpenAgentsWeb.UserAuth` with session management and `on_mount` hooks.
69
- Authentication required for all `/:owner/:repo` issue and project pages.
70
- A homepage sign-in button and a navbar avatar/logout control.
71
72
### Acceptance
73
74
- `/` shows a **Sign in with GitHub** button when the visitor is not authenticated.
75
- A logged-in user can open `/:owner/:repo/issues/new` without hitting a placeholder owner.
76
- `/:owner/:repo/issues` and related paths redirect an unauthenticated visitor to `/auth/github`.
77
- `mix test` covers user upsert, token vault, OAuth flow, and the protected LiveView routes.
78
79
## Phase 1: Placeholder homepage
80
81
What the surface shows:
82
83
- A **hero** section with the OpenAgents value proposition.
84
- A list of owned repositories as cards.
85
- Primary actions: **Create new repository** and **Create new issue**.
86
- Optional: a placeholder contribution activity block.
87
88
DaisyUI parts:
89
90
- `hero` for the welcome section.
91
- `card` for repository cards.
92
- `btn btn-primary` for the main call to action.
93
- `badge` for public or private status.
94
- `stat` for star and fork counts.
95
- `avatar` for the owner avatar.
96
97
Clone harvest:
98
99
- `leoronne-github-ui-clone/src/pages/Profile/index.tsx`
100
- `TiagoDiass-github-ui-clone/src/pages/Profile/Profile.tsx`
101
- `gh-next/src/app/(app)/[user]/[repository]/page.tsx`
102
103
Acceptance:
104
105
- `/` renders without a `current_scope` error.
106
- The page lists at least one owned repo.
107
- Each repo card links to `/:owner/:repo`.
108
109
## Phase 2: Issues list
110
111
What the surface shows:
112
113
- A search and filter bar.
114
- **Open** and **Closed** tabs with counts.
115
- Issue rows with state icon, title, labels, author, relative time, and comment count.
116
- Pagination.
117
- An empty state when no issues match.
118
119
DaisyUI parts:
120
121
- `tabs` for **Open** and **Closed**.
122
- `input` for the search field.
123
- `btn` and `dropdown` for filters and sort.
124
- `table` or custom flex rows for the issue list.
125
- `badge` for state and labels.
126
- `avatar` for assignees.
127
- `join` for pagination.
128
129
Clone harvest:
130
131
- `gh-next/src/components/issues/issue-row.tsx`
132
- `gh-next/src/components/issues/issue-list.tsx`
133
- `gh-next/src/components/issues/issues-list-header-form.tsx`
134
- `gitea/templates/repo/issue/list.tmpl`
135
136
API to call:
137
138
- `GET /api/v3/repos/:owner/:repo/issues`
139
140
Acceptance:
141
142
- `/:owner/:repo/issues` lists open issues by default.
143
- Clicking **Closed** lists closed issues.
144
- Each title links to `/:owner/:repo/issues/:number`.
145
- The page uses `stream` for the issue list.
146
147
## Phase 3: Issue detail and comments
148
149
What the surface shows:
150
151
- A header with the issue number, title, and state badge.
152
- Author, avatar, and relative time.
153
- A markdown-rendered body.
154
- Label, assignee, and milestone sections.
155
- A chronological comment thread with author avatars and markdown bodies.
156
- A comment form.
157
158
DaisyUI parts:
159
160
- `badge` for the open or closed state.
161
- `avatar` and `card` for comments.
162
- `textarea` for the comment form.
163
- `btn` for submit, close, and reopen actions.
164
- `timeline` for the comment thread.
165
- `collapse` or `drawer` for the metadata sidebar.
166
167
Clone harvest:
168
169
- `gh-next/src/app/(app)/[user]/[repository]/issues/[number]/page.tsx`
170
- `gitea/templates/repo/issue/view.tmpl`
171
- `gitea/templates/repo/issue/view_content.tmpl`
172
- `git.limo/apps/gitgud_web/lib/gitgud_web/live/issue_live.html.heex`
173
174
API to call:
175
176
- `GET /api/v3/repos/:owner/:repo/issues/:issue_number`
177
- `GET /api/v3/repos/:owner/:repo/issues/:issue_number/comments`
178
- `POST /api/v3/repos/:owner/:repo/issues/:issue_number/comments`
179
- `PATCH /api/v3/repos/:owner/:repo/issues/:issue_number/comments/:comment_id`
180
- `DELETE /api/v3/repos/:owner/:repo/issues/:issue_number/comments/:comment_id`
181
182
Acceptance:
183
184
- The detail page renders at `/:owner/:repo/issues/:number`.
185
- Comments appear newest first or oldest first consistently.
186
- Submitting a comment adds it without a full page reload.
187
188
## Phase 4: New and edit issue
189
190
### New issue
191
192
What the surface shows:
193
194
- A title input.
195
- A body textarea with a live markdown preview.
196
- Multi-select fields for labels and assignees.
197
- A single-select milestone field.
198
- Submit and cancel actions.
199
200
DaisyUI parts:
201
202
- `input` for the title.
203
- `textarea` for the body.
204
- `select` for the milestone.
205
- `checkbox` for labels and assignees.
206
- `btn` for submit and cancel.
207
- `card` to frame the form.
208
209
Clone harvest:
210
211
- `gh-next/src/components/issues/new-issue-form.tsx`
212
- `gitea/templates/repo/issue/new.tmpl`
213
- `gitea/templates/repo/issue/new_form.tmpl`
214
- `git.limo/apps/gitgud_web/lib/gitgud_web/live/issue_form_live.html.heex`
215
216
API to call:
217
218
- `POST /api/v3/repos/:owner/:repo/issues`
219
- `GET /api/v3/repos/:owner/:repo/labels`
220
- `GET /api/v3/repos/:owner/:repo/assignees`
221
- `GET /api/v3/repos/:owner/:repo/milestones`
222
223
Acceptance:
224
225
- `/:owner/:repo/issues/new` renders the form.
226
- Submitting a valid issue redirects to the detail page.
227
- Validation errors appear next to the title or body fields.
228
229
### Edit issue
230
231
What the surface shows:
232
233
- Inline editing of the title and body on the detail page.
234
- A state toggle, label multi-select, assignee multi-select, and milestone select.
235
- Save and cancel actions.
236
237
DaisyUI parts:
238
239
- `input` and `textarea` for editable fields.
240
- `btn` for save, cancel, close, and reopen.
241
- `modal` only if a separate edit view is preferred.
242
- `collapse` for compact edit sections.
243
244
Clone harvest:
245
246
- `git.limo/apps/gitgud_web/lib/gitgud_web/live/issue_live.html.heex`
247
- `gitea/templates/repo/issue/view_content.tmpl`
248
249
API to call:
250
251
- `PATCH /api/v3/repos/:owner/:repo/issues/:issue_number`
252
- `POST /api/v3/repos/:owner/:repo/issues/:issue_number/labels`
253
- `DELETE /api/v3/repos/:owner/:repo/issues/:issue_number/labels/:name`
254
- `POST /api/v3/repos/:owner/:repo/issues/:issue_number/assignees`
255
- `DELETE /api/v3/repos/:owner/:repo/issues/:issue_number/assignees`
256
257
Acceptance:
258
259
- The detail page allows in-place editing of title and body.
260
- State changes update the state badge immediately.
261
- Saving updates the issue and re-renders the page.
262
263
## Phase 5: Labels, milestones, and assignees
264
265
### Labels
266
267
What the surface shows:
268
269
- A count header and a sort dropdown.
270
- Label rows with colored badge, name, description, issue count, and edit and delete actions.
271
- A create button and an inline or modal create form.
272
273
DaisyUI parts:
274
275
- `badge` for label colors.
276
- `table` or `card` for label rows.
277
- `btn` for edit, delete, and create.
278
- `input`, `textarea`, and a color picker for the create and edit forms.
279
- `dropdown` for sorting.
280
- `modal` for the create and edit forms.
281
282
Clone harvest:
283
284
- `gitea/templates/repo/issue/labels/label_list.tmpl`
285
- `primer-view_components/app/components/primer/beta/label.rb`
286
287
API to call:
288
289
- `GET /api/v3/repos/:owner/:repo/labels`
290
- `POST /api/v3/repos/:owner/:repo/labels`
291
- `PATCH /api/v3/repos/:owner/:repo/labels/:name`
292
- `DELETE /api/v3/repos/:owner/:repo/labels/:name`
293
294
Acceptance:
295
296
- `/:owner/:repo/labels` lists all labels.
297
- Creating or editing a label updates the list.
298
299
### Milestones
300
301
What the surface shows:
302
303
- A count header and open or closed filter tabs.
304
- Milestone cards with title, due date, progress bar, open count, and closed count.
305
- Edit, close, and delete actions.
306
- A create form.
307
308
DaisyUI parts:
309
310
- `progress` for the milestone progress bar.
311
- `stat` for open and closed counts.
312
- `card` for milestone cards.
313
- `badge` for the open or closed state.
314
- `input` for the create and edit forms.
315
316
Clone harvest:
317
318
- `gitea/templates/repo/issue/milestones.tmpl`
319
- `primer-view_components/app/components/primer/beta/counter.rb`
320
321
API to call:
322
323
- `GET /api/v3/repos/:owner/:repo/milestones`
324
- `POST /api/v3/repos/:owner/:repo/milestones`
325
- `PATCH /api/v3/repos/:owner/:repo/milestones/:milestone_number`
326
- `DELETE /api/v3/repos/:owner/:repo/milestones/:milestone_number`
327
328
Acceptance:
329
330
- `/:owner/:repo/milestones` renders milestones with progress bars.
331
- Closing a milestone updates its state badge.
332
333
### Assignees
334
335
What the surface shows:
336
337
- Assignees appear as an avatar stack on issue rows and in the detail view.
338
- The issue detail page shows an assignee selection dropdown.
339
- An optional `/:owner/:repo/assignees` page lists users and their assigned issue counts.
340
341
DaisyUI parts:
342
343
- `avatar` and `avatar-group` for the stack.
344
- `badge` for overflow counts like `+2`.
345
- `menu` for the assignee dropdown.
346
347
Clone harvest:
348
349
- `gh-next/src/components/issues/issue-row.tsx`
350
- `gh-next/src/components/issues/issue-assignee-filter-action-list.tsx`
351
- `gitea/templates/repo/issue/sidebar/assignee_list.tmpl`
352
353
API to call:
354
355
- `GET /api/v3/repos/:owner/:repo/assignees`
356
357
## Phase 6: Projects V2 board
358
359
What the surface shows:
360
361
- A project list page with project cards.
362
- A simple board view with columns such as **To Do**, **In Progress**, and **Done**.
363
- Cards inside columns show issue title, labels, and assignees.
364
- Buttons to create a project and add a column.
365
366
DaisyUI parts:
367
368
- `card` for project cards and board cards.
369
- `collapse` for collapsible columns.
370
- `badge` for labels and status.
371
- `avatar` for assignees.
372
- `progress` for project completion.
373
- `btn` for create actions.
374
375
Clone harvest:
376
377
- `gitea/templates/repo/projects/list.tmpl`
378
- `gitea/templates/repo/projects/view.tmpl`
379
- `gitea/templates/repo/issue/sidebar/project_list.tmpl`
380
381
API to call:
382
383
- `GET /api/v3/users/:username/projectsV2`
384
- `POST /:owner/projectsV2`
385
- `GET /api/v3/users/:username/projectsV2/:project_number`
386
- `GET /api/v3/users/:username/projectsV2/:project_number/items`
387
- `POST /api/v3/users/:username/projectsV2/:project_number/items`
388
- `PATCH /api/v3/users/:username/projectsV2/:project_number/items/:item_id`
389
390
Acceptance:
391
392
- The project list renders at `/:owner/projects`.
393
- A project board renders at `/:owner/projects/:project_number`.
394
- Cards display issue title, labels, and assignees.
395
- Drag-and-drop is not required for the first pass.
396
397
## Component conventions
398
399
- Begin every LiveView template with `<Layouts.app flash={@flash} ...>`.
400
- Use `OpenAgentsWeb.CoreComponents` for `input`, `textarea`, `button`, `table`, and `icon`.
401
- Use DaisyUI classes only where they make the final layout simpler than the core component.
402
- Use `Phoenix.Component.to_form/2` for forms.
403
- Use LiveView `stream` for issue lists, comment lists, and board cards.
404
- Use `<.icon name="..." />` for icons and avoid hand-written `svg` tags.
405
406
## Implementation order
407
408
0. GitHub authentication and user identity. See `docs/github-auth-plan.md`.
409
1. Layout foundation and placeholder homepage.
410
2. Issues list with open and closed tabs.
411
3. Issue detail and comment thread.
412
4. New issue and inline edit issue forms.
413
5. Labels, milestones, and assignee selection.
414
6. Projects V2 list and simple board view.
415
416
Each phase can become its own `mix test` and `mix precommit` cycle. Keep the UI simple, build from the API already in place, and add polish once the core flows are usable.
1
# Issues and projects UI roadmap
2
3
Date: 2026-08-20
4
5
Status: Core surfaces implemented; tenant and design-system hardening pending
6
7
## Current surface
8
9
The Phoenix LiveView application currently ships authenticated pages for:
10
11
| Surface | Route | LiveView |
12
| --- | --- | --- |
13
| Issue list | `/:owner/:repo/issues` | `OpenAgentsWeb.IssueIndexLive` |
14
| New issue | `/:owner/:repo/issues/new` | `OpenAgentsWeb.IssueNewLive` |
15
| Issue detail and comments | `/:owner/:repo/issues/:number` | `OpenAgentsWeb.IssueShowLive` |
16
| Labels | `/:owner/:repo/labels` | `OpenAgentsWeb.LabelIndexLive` |
17
| Milestones | `/:owner/:repo/milestones` | `OpenAgentsWeb.MilestoneIndexLive` |
18
| Assignees | `/:owner/:repo/assignees` | `OpenAgentsWeb.AssigneeIndexLive` |
19
| Project list | `/:owner/:repo/projects` | `OpenAgentsWeb.ProjectIndexLive` |
20
| Project board | `/:owner/:repo/projects/:number` | `OpenAgentsWeb.ProjectShowLive` |
21
22
The matching `/api/v3` issue, comment, label, assignee, milestone, and Projects
23
V2 subset is implemented and covered. The dated
24
[coverage audit](2026-08-20-test-coverage-audit.md) records the original gaps
25
and the coverage added to close them.
26
27
## Interface rules
28
29
- Every page uses `Layouts.app` and the authenticated LiveView session.
30
- New reusable primitives come from `OpenAgentsWeb.UI`; domain compositions can
31
  live in a focused issue, project, or forge component module.
32
- Basecoat supplies pinned structural CSS and `assets/css/openagents.css`
33
  supplies product identity. Do not add a second component system.
34
- Forms use `Phoenix.Component.to_form/2` and stable DOM IDs.
35
- Issue, comment, and project-item collections use LiveView streams where the
36
  collection changes in place.
37
- Model-authored Markdown remains untrusted and passes through the one bounded,
38
  sanitized Markdown path selected by Gate 4.
39
- Icons come from the vendored set through `OpenAgentsWeb.UI.icon/1`; icon-only
40
  controls have accessible names.
41
42
The current component inventory and transitional generated helpers are
43
documented in [docs/component-library.md](component-library.md).
44
45
## Blocking domain work
46
47
The route shape currently looks repository-scoped, but the durable issue and
48
project data model does not yet enforce that scope. Gate 7 must complete this
49
before the tracker is treated as a multi-repository forge:
50
51
1. Add a canonical repository entity.
52
2. Add repository foreign keys and scoped uniqueness to issues, labels,
53
   milestones, comments, assignees, and repository projects.
54
3. Resolve every resource through owner, repository, and resource identity in
55
   one authorized query.
56
4. Reject cross-repository identifiers in application code and PostgreSQL.
57
5. Replace the hardcoded assignee projection with repository membership and
58
   authorization.
59
6. Enforce project ownership instead of ignoring the username in Projects V2
60
   routes.
61
7. Separate public reads from authenticated browser and API writes.
62
8. Rehearse the backfill of existing rows into the initial repository.
63
64
Until this work passes cross-repository isolation tests, the URL is
65
presentation context rather than a proven tenancy boundary.
66
67
## Remaining interface work
68
69
After Gate 7 establishes the domain boundary:
70
71
1. Reconcile every issue/project surface onto `OpenAgentsWeb.UI` and remove
72
   transitional generated component callers.
73
2. Extract repeated issue rows, comment threads, label selectors, milestone
74
   progress, and project columns only where doing so improves behavior and test
75
   ownership.
76
3. Add bounded search, filtering, pagination, and useful empty/loading/error
77
   states.
78
4. Add PubSub invalidation and database rereads where concurrent users need
79
   live updates.
80
5. Add repository authorization-aware actions and explicit refusal states.
81
6. Run accessibility, keyboard, responsive, compiled-CSS, and browser staging
82
   checks against the same candidate SHA.
83
84
Drag-and-drop boards, advanced project views, pull requests, review workflows,
85
and pixel-level compatibility with another forge remain planned rather than
86
current promises.
docs/issues-projects-work-plan.md modified +31 -271

@@ -1,282 +1,42 @@

1
# Issues and Projects API — work plan
1
# Historical issues and projects API work plan
2 2
3 3
Date: 2026-08-19
4
Source: `docs/github-api-issues-projects-assessment.md`
5 4
6
This document is a markdown-only tracker. Each `Epic` and `Task` will later become an issue or sub-issue in the OpenAgents forge. It organizes the buildout, lists dependencies, and flags work that can be parallelized or dispatched to subagents.
5
Status: Core implementation complete; superseded by Gates 6 and 7 of the
6
[integration hardening plan](2026-08-20-integration-hardening-and-staging-readiness-recommendations.md)
7 7
8
## Approach
8
## Completed scope
9 9
10
- Drive every endpoint through tests first. Follow the TDD workflow in `AGENTS.md`.
11
- Build the API in the order shown below. Start each epic with the shared schema and context module.
12
- Run `mix test test/openagents_web/controllers/<...>` for a single file, or `mix precommit` before a final commit.
13
- Each task includes acceptance criteria and a `subagent ready` flag. Use `subagent_general` for self-contained implementation tasks and `subagent_explore` for spec research.
10
The original epics delivered Ecto contexts, schemas, migrations, controllers,
11
JSON projections, and tests for:
14 12
15
## Epic 1: Issue CRUD
13
- Issue list, read, create, update, close, and reopen.
14
- Issue comments.
15
- Repository labels and issue-label relationships.
16
- Assignee listing/checking and issue-assignee relationships.
17
- Milestone list, read, create, update, and delete.
18
- Projects V2 list/read, project creation, items, item updates, and fields.
16 19
17
Goal: implement the core repository issue endpoints.
20
Controller and domain tests now cover the implemented success and error paths;
21
the paired LiveViews are covered separately. The dated
22
[coverage audit](2026-08-20-test-coverage-audit.md) records that work.
18 23
19
### E1-T1: Define the `OpenAgents.Issues` context and `Issue` schema
20
- Create the Ecto schema and migration for repository issues.
21
- Fields must match the GitHub `issue` object shape where OpenAgents stores the data.
22
- Acceptance:
23
  - `mix ecto.migrate` succeeds.
24
  - `OpenAgents.Issues.list_issues/1` and `OpenAgents.Issues.get_issue!/1` exist.
25
- Dependencies: none.
26
- Subagent ready: no.
24
## Why this plan is closed
27 25
28
### E1-T2: GET /api/v3/repos/{owner}/{repo}/issues
29
- Return a list of issues for a repo. Support query filters (`state`, `labels`, `assignee`, `milestone`, `sort`, `direction`).
30
- Acceptance:
31
  - `GET /api/v3/repos/OpenAgents/openagents/issues` returns 200 and a JSON array.
32
  - Tests cover open issues, closed issues, and empty repositories.
33
- Dependencies: E1-T1.
34
- Subagent ready: yes.
26
The implementation tasks are no longer the readiness bottleneck. The resource
27
model must now be hardened rather than expanded from this checklist:
35 28
36
### E1-T3: GET /api/v3/repos/{owner}/{repo}/issues/{issue_number}
37
- Return a single issue.
38
- Acceptance:
39
  - Existing issue returns 200 and the issue JSON.
40
  - Missing issue returns 404.
41
- Dependencies: E1-T1.
42
- Subagent ready: yes.
29
- `/api/v3` currently lacks a deliberate CLI authentication and authorization
30
  model.
31
- Owner/repository URL parameters do not yet map to a canonical repository row
32
  enforced throughout PostgreSQL.
33
- Some project actions ignore the username in the route.
34
- The assignee read surface and write behavior do not share one authorization
35
  rule.
36
- Several GitHub-compatibility edge cases are documented in the
37
  [API assessment](github-api-issues-projects-assessment.md).
43 38
44
### E1-T4: POST /api/v3/repos/{owner}/{repo}/issues
45
- Create an issue with `title`, `body`, `labels`, `assignees`, and `milestone`.
46
- Acceptance:
47
  - Valid request returns 201 and the created issue.
48
  - Invalid request returns 422 with error details.
49
- Dependencies: E1-T1.
50
- Subagent ready: yes.
51
52
### E1-T5: PATCH /api/v3/repos/{owner}/{repo}/issues/{issue_number}
53
- Update title, body, state, labels, assignees, and milestone.
54
- Acceptance:
55
  - Closing an issue returns the issue with `state: "closed"`.
56
  - Reopening an issue returns `state: "open"`.
57
- Dependencies: E1-T4.
58
- Subagent ready: yes.
59
60
## Epic 2: Issue comments
61
62
Goal: implement issue comments.
63
64
### E2-T1: Add `OpenAgents.Issues.Comment` schema and migration
65
- Acceptance: `OpenAgents.Issues.create_comment/3` exists.
66
- Dependencies: E1-T1.
67
- Subagent ready: no.
68
69
### E2-T2: GET /api/v3/repos/{owner}/{repo}/issues/{issue_number}/comments
70
- Acceptance: returns a list of comments for an issue.
71
- Dependencies: E2-T1.
72
- Subagent ready: yes.
73
74
### E2-T3: POST /api/v3/repos/{owner}/{repo}/issues/{issue_number}/comments
75
- Acceptance: returns 201 and the created comment.
76
- Dependencies: E2-T1.
77
- Subagent ready: yes.
78
79
### E2-T4: GET /api/v3/repos/{owner}/{repo}/issues/comments/{comment_id}
80
- Acceptance: returns 200 for an existing comment and 404 for a missing one.
81
- Dependencies: E2-T1.
82
- Subagent ready: yes.
83
84
### E2-T5: PATCH /api/v3/repos/{owner}/{repo}/issues/comments/{comment_id}
85
- Acceptance: updates body text and returns 200.
86
- Dependencies: E2-T3.
87
- Subagent ready: yes.
88
89
### E2-T6: DELETE /api/v3/repos/{owner}/{repo}/issues/comments/{comment_id}
90
- Acceptance: returns 204 and removes the comment.
91
- Dependencies: E2-T3.
92
- Subagent ready: yes.
93
94
## Epic 3: Labels
95
96
Goal: implement repository and issue labels.
97
98
### E3-T1: Add `OpenAgents.Issues.Label` schema and migration
99
- Acceptance: `OpenAgents.Issues.create_label/2` and `OpenAgents.Issues.list_labels/1` exist.
100
- Dependencies: none.
101
- Subagent ready: no.
102
103
### E3-T2: GET /api/v3/repos/{owner}/{repo}/labels
104
- Acceptance: returns a list of labels.
105
- Dependencies: E3-T1.
106
- Subagent ready: yes.
107
108
### E3-T3: POST /api/v3/repos/{owner}/{repo}/labels
109
- Acceptance: returns 201 and the created label.
110
- Dependencies: E3-T1.
111
- Subagent ready: yes.
112
113
### E3-T4: GET /api/v3/repos/{owner}/{repo}/labels/{name}
114
- Acceptance: returns a single label or 404.
115
- Dependencies: E3-T1.
116
- Subagent ready: yes.
117
118
### E3-T5: PATCH /api/v3/repos/{owner}/{repo}/labels/{name}
119
- Acceptance: updates name, color, and description.
120
- Dependencies: E3-T3.
121
- Subagent ready: yes.
122
123
### E3-T6: POST /api/v3/repos/{owner}/{repo}/issues/{issue_number}/labels
124
- Acceptance: adds labels to an issue and returns the updated issue.
125
- Dependencies: E1-T1, E3-T1.
126
- Subagent ready: yes.
127
128
### E3-T7: DELETE /api/v3/repos/{owner}/{repo}/issues/{issue_number}/labels/{name}
129
- Acceptance: removes a label from an issue and returns 204.
130
- Dependencies: E3-T6.
131
- Subagent ready: yes.
132
133
## Epic 4: Assignees
134
135
Goal: implement issue assignment.
136
137
### E4-T1: Add `OpenAgents.Issues.Assignee` and user participation model
138
- Acceptance: `OpenAgents.Issues.list_possible_assignees/1` exists.
139
- Dependencies: E1-T1.
140
- Subagent ready: no.
141
142
### E4-T2: GET /api/v3/repos/{owner}/{repo}/assignees
143
- Acceptance: returns a list of users who can be assigned.
144
- Dependencies: E4-T1.
145
- Subagent ready: yes.
146
147
### E4-T3: GET /api/v3/repos/{owner}/{repo}/assignees/{assignee}
148
- Acceptance: returns 204 if the user can be assigned and 404 if not.
149
- Dependencies: E4-T1.
150
- Subagent ready: yes.
151
152
### E4-T4: POST /api/v3/repos/{owner}/{repo}/issues/{issue_number}/assignees
153
- Acceptance: adds assignees and returns the updated issue.
154
- Dependencies: E1-T1, E4-T1.
155
- Subagent ready: yes.
156
157
### E4-T5: DELETE /api/v3/repos/{owner}/{repo}/issues/{issue_number}/assignees
158
- Acceptance: removes assignees and returns the updated issue.
159
- Dependencies: E4-T4.
160
- Subagent ready: yes.
161
162
## Epic 5: Milestones
163
164
Goal: implement milestones.
165
166
### E5-T1: Add `OpenAgents.Issues.Milestone` schema and migration
167
- Acceptance: `OpenAgents.Issues.create_milestone/2` exists.
168
- Dependencies: E1-T1.
169
- Subagent ready: no.
170
171
### E5-T2: GET /api/v3/repos/{owner}/{repo}/milestones
172
- Acceptance: returns a list of milestones.
173
- Dependencies: E5-T1.
174
- Subagent ready: yes.
175
176
### E5-T3: POST /api/v3/repos/{owner}/{repo}/milestones
177
- Acceptance: returns 201 and the created milestone.
178
- Dependencies: E5-T1.
179
- Subagent ready: yes.
180
181
### E5-T4: GET /api/v3/repos/{owner}/{repo}/milestones/{milestone_number}
182
- Acceptance: returns a single milestone or 404.
183
- Dependencies: E5-T1.
184
- Subagent ready: yes.
185
186
### E5-T5: PATCH /api/v3/repos/{owner}/{repo}/milestones/{milestone_number}
187
- Acceptance: updates title, state, due date, and description.
188
- Dependencies: E5-T3.
189
- Subagent ready: yes.
190
191
### E5-T6: DELETE /api/v3/repos/{owner}/{repo}/milestones/{milestone_number}
192
- Acceptance: returns 204 and removes the milestone.
193
- Dependencies: E5-T3.
194
- Subagent ready: yes.
195
196
## Epic 6: Projects V2
197
198
Goal: implement the Projects V2 read and write surface.
199
200
### E6-T1: Design the project and item schemas
201
- Define `Project`, `ProjectField`, `ProjectView`, and `ProjectItem` schemas.
202
- Acceptance: migrations run and `OpenAgents.Projects.create_project/2` exists.
203
- Dependencies: none.
204
- Subagent ready: no.
205
206
### E6-T2: GET /api/v3/users/{username}/projectsV2
207
- Acceptance: returns a list of user projects.
208
- Dependencies: E6-T1.
209
- Subagent ready: yes.
210
211
### E6-T3: GET /api/v3/users/{username}/projectsV2/{project_number}
212
- Acceptance: returns a single project.
213
- Dependencies: E6-T1.
214
- Subagent ready: yes.
215
216
### E6-T4: POST /api/v3/{owner}/projectsV2
217
- Non-standard endpoint to create a project because the GitHub REST spec does not include it.
218
- Acceptance: returns 201 and the created project.
219
- Dependencies: E6-T1.
220
- Subagent ready: yes.
221
222
### E6-T5: GET /api/v3/users/{username}/projectsV2/{project_number}/items
223
- Acceptance: returns a list of project items.
224
- Dependencies: E6-T1.
225
- Subagent ready: yes.
226
227
### E6-T6: POST /api/v3/users/{username}/projectsV2/{project_number}/items
228
- Acceptance: adds an issue to a project and returns the item.
229
- Dependencies: E1-T4, E6-T1.
230
- Subagent ready: yes.
231
232
### E6-T7: PATCH /api/v3/users/{username}/projectsV2/{project_number}/items/{item_id}
233
- Acceptance: updates field values on a project item.
234
- Dependencies: E6-T6.
235
- Subagent ready: yes.
236
237
### E6-T8: GET /api/v3/users/{username}/projectsV2/{project_number}/fields
238
- Acceptance: returns a list of project fields.
239
- Dependencies: E6-T1.
240
- Subagent ready: yes.
241
242
## Parallelization and subagent dispatch
243
244
### Dependencies
245
246
| Before | After |
247
| --- | --- |
248
| E1-T1 (issue schema) | E1-T2 to E1-T5, E2-T1, E3-T1, E4-T1, E5-T1, E6-T1 can start. |
249
| E1-T4 (issue create) | E2-T3 to E2-T6, E3-T6, E3-T7, E4-T4, E4-T5, E6-T6. |
250
| E2-T1 (comment schema) | E2-T2 to E2-T6. |
251
| E3-T1 (label schema) | E3-T2 to E3-T7. |
252
| E4-T1 (assignee model) | E4-T2 to E4-T5. |
253
| E5-T1 (milestone schema) | E5-T2 to E5-T6. |
254
| E6-T1 (project schema) | E6-T2 to E6-T8. |
255
256
### Parallel waves
257
258
**Wave 1 — foundation (serial)**
259
- E1-T1
260
261
**Wave 2 — independent contexts and read endpoints (parallel)**
262
- E1-T2, E1-T3
263
- E2-T1
264
- E3-T1, E3-T2, E3-T4
265
- E4-T1, E4-T2, E4-T3
266
- E5-T1, E5-T2, E5-T4
267
- E6-T1, E6-T2, E6-T3, E6-T5, E6-T8
268
269
**Wave 3 — write endpoints (parallel after Wave 1 + Wave 2)**
270
- E1-T4, E1-T5
271
- E2-T3 to E2-T6
272
- E3-T3, E3-T5, E3-T6, E3-T7
273
- E4-T4, E4-T5
274
- E5-T3, E5-T5, E5-T6
275
- E6-T4, E6-T6, E6-T7
276
277
### Subagent guidance
278
279
- Dispatch one `subagent_general` per task marked `subagent ready: yes`.
280
- Give each subagent the exact `AGENTS.md` rules, the endpoint path, and the expected JSON shape from `docs/github-api-issues-projects-assessment.md`.
281
- Keep schema and migration work (subagent ready: no) in the main session. Schema is the contract that all other work depends on.
282
- Before merging parallel work, run `mix precommit` to catch cross-module conflicts and compile warnings.
39
Gate 6 owns API principals and route authority. Gate 7 owns repository
40
entities, tenant-scoped queries and constraints, backfill, and isolation tests.
41
New issue/project work should be added there until those gates pass rather than
42
reopening the original endpoint epics.
docs/sarah-integration-plan.md modified +65 -483

@@ -1,490 +1,72 @@

1
# Sarah integration plan
1
# Historical Sarah integration record
2 2
3 3
Date: 2026-08-19
4 4
5
Status: Phase 9 done; Sarah tests lifted and tagged :skip while the runtime remains stubbed; `mix precommit` green
5
Status: Migration complete; this is a historical record, not an implementation
6
plan
6 7
7
Last updated: 2026-08-19
8
9
Supersedes: `docs/chat-inference-plan.md` (the previous `pro.openagents.com` split is no longer the target; the goal now is to merge the entire Sarah product into `openagents.com` as a single application).
10
11
## Progress
12
13
- **Phase 1: Foundation and application wiring** — done.
14
  - Added `{:mdex, "~> 0.3"}` and `{:websockex, "~> 0.5.1"}` to `mix.exs`.
15
  - Created `OpenAgents.Sarah.Supervisor` as a placeholder supervisor under `lib/openagents/sarah/supervisor.ex`.
16
  - Wired the supervisor into `OpenAgents.Application`.
17
  - `mix precommit` passes. `mdex` replaces the retired and CVE-flagged `earmark` package.
18
- **Phase 2: Accounts and authentication** — done.
19
  - Merged Sarah `User` fields and `OpenAgents.Accounts` functions (`admin?/1`, `admin_github_ids/0`, `ban_user/2`) into the existing `OpenAgents.Accounts`.
20
  - Added `public_leaderboard_opted_out`, `browser_key_hash`, and the `OpenAgents.Conversations.Visitor` association.
21
  - Added `admin_github_ids`, `conversation_page_size`, `maximum_message_bytes`, and `turn_rate_limit` to `config/config.exs`.
22
  - Generated `add_sarah_user_fields` migration.
23
- **Phase 3: Core chat schema and contexts** — done.
24
  - Lifted `lib/sarah/conversations/` into `lib/openagents/conversations/` and re-namespaced to `OpenAgents.Conversations`.
25
  - Created the `Conversation`, `Message`, `Turn`, `TurnReceipt`, `Visitor`, `ProviderStep`, and `ToolStep` schemas.
26
  - Generated the `create_sarah_conversations` migration for `visitors`, `conversations`, `messages`, `turns`, `turn_receipts`, `turn_provider_steps`, and `turn_tool_steps`.
27
  - Added minimal stub modules so the new contexts compile without the full memory, voice, and work subsystems.
28
  - `OpenAgents.Chat` and `OpenAgentsWeb.ChatLive` remain untouched so the existing `/chat` UI still works.
29
- **Phase 4: Memory systems** — partially done with stubs.
30
  - Lifted `lib/sarah/profile_memory/`, `lib/sarah/experience_memory/`, `lib/sarah/graph_memory/`, and `lib/sarah/memory/` to `lib/openagents/` and re-namespaced to `OpenAgents`.
31
  - Added the top-level `OpenAgents.ProfileMemory`, `OpenAgents.ProgramArtifacts`, `OpenAgents.Roles`, `OpenAgents.Observability`, and `OpenAgents.Markdown` wrappers needed for compilation.
32
  - Added most corresponding migrations, removing duplicates that collide with the pre-existing `create_sarah_conversations` migration.
33
  - Clustered workers and the `OpenAgents.Sarah.Supervisor` remain local-only (no `horde` or `ra` dependencies) until cluster wiring is finished.
34
- **Phase 5: Work, delegation, and computer activity** — partially done with stubs.
35
  - Lifted `lib/sarah/work/`, `lib/sarah/computer/`, `lib/sarah/computer_activity.ex`, `lib/sarah/machines/`, and `lib/sarah/channels/` to `lib/openagents/` and `lib/openagents_web/`.
36
  - Lifted `lib/sarah/plugs/` to `lib/openagents_web/plugs/`.
37
  - Created local-only `OpenAgents.Cluster.Registry` and `OpenAgents.Cluster.DynamicSupervisor` to stand in for `Horde.Registry` and `Horde.DynamicSupervisor`.
38
  - `OpenAgents.Work.JobServer`, `OpenAgents.Work.DelegationServer`, and `OpenAgents.Work.Coding` are stubs or removed to keep the build green.
39
- **Phase 6: Voice** — partially done with stubs.
40
  - Lifted `lib/sarah/voice/`, `lib/sarah/voice_sessions/`, `lib/sarah/voice_sessions.ex`, and `lib/sarah/voice_recovery.ex` to `lib/openagents/`.
41
  - Lifted voice controllers to `lib/openagents_web/controllers/`.
42
  - Added `OpenAgents.Voice` configuration keys to `config/config.exs`.
43
  - `OpenAgents.Voice` workers are not started in `OpenAgents.Sarah.Supervisor` yet.
44
- **Phase 7: Context, inference, admin, and supporting systems** — partially done with stubs.
45
  - Lifted `lib/sarah/context/`, `lib/sarah/changelog/`, `lib/sarah/blueprint/`, `lib/sarah/data_rights/`, `lib/sarah/incidents/`, `lib/sarah/collective/`, `lib/sarah/compensation/`, `lib/sarah/admin/`, `lib/sarah/leaderboard/`, `lib/sarah/preferences/`, `lib/sarah/persona/`, `lib/sarah/providers/`, `lib/sarah/provenance/`, `lib/sarah/modules/`, and `lib/sarah/tools/` to `lib/openagents/`.
46
  - Created `OpenAgents.Inference` from `lib/sarah/inference.ex` (re-namespaced), including `OpenAgents.Inference.Grant` and the existing `create_inference_grants` migration. `OpenAgents.NetworkStatus` and `OpenAgents.Forge.*` remain stubs.
47
  - `OpenAgents.Cluster` and `OpenAgents.NetworkStatus` are single-node stubs.
48
- **Phase 8: UI and assets** — done.
49
  - Replaced `OpenAgentsWeb.ChatLive` with the Sarah `chat_live.ex` re-namespaced to `OpenAgentsWeb`.
50
  - Lifted `lib/sarah_web/tool_activity.ex` to `OpenAgentsWeb.ChatToolActivity` and `lib/sarah_web/ui.ex` to `OpenAgentsWeb.SarahUI`.
51
  - Copied voice JavaScript (`voice_controller.js`, `voice_recording.mjs`, `voice_state.mjs`, `paced_transcript.js`) to `assets/js/` and wired them in `assets/js/app.js`.
52
  - Copied `style-sarah.css` to `assets/css/sarah.css` and `@import`ed it from `assets/css/app.css`.
53
  - Added `/voice/*` and `/healthz` routes to `OpenAgentsWeb.Router`.
54
  - Lifted `OpenAgents.Turns` and `OpenAgents.Turns.TurnServer` from `lib/sarah/turns` (re-namespaced), started `OpenAgents.TurnRegistry` and `OpenAgents.TurnSupervisor` in `OpenAgents.Sarah.Supervisor`, and filled `OpenAgentsWeb.Layouts.account_control/1` with the current user's GitHub avatar and sign-out link.
55
- **Phase 9: Tests and cutover** — done.
56
  - Lifted Sarah `test/sarah/` and `test/sarah_web/` files to `test/openagents/` and `test/openagents_web/` and re-namespaced them to `OpenAgents.` and `OpenAgentsWeb.`.
57
  - Added `@moduletag :skip` to the lifted tests and `exclude: [:skip]` to `test/test_helper.exs` so the existing `mix test` suite stays green while the Sarah runtime is still stubbed.
58
59
## Data migration
60
61
The pre-Sarah `openagents.com` database has only one table that overlaps with the new chat system: `users`. The old `OpenAgents.Chat` mock kept messages in memory; there are no durable messages, conversations, or visitors to port.
62
63
### Stock
64
65
- `users`: one row for every GitHub-authenticated account. These must each get a `visitors` row and a `conversations` row.
66
- `visitors`/`conversations`/`messages`: empty before the new Sarah migrations run.
67
- `oauth_attempts`, `token_vaults`, `forge_*`, issues, and projects: not in scope for the chat migration.
68
69
### Backfill plan
70
71
1. Run `mix ecto.migrate` to create the `visitors`, `conversations`, and `messages` tables.
72
2. Run `mix openagents.backfill_visitors` to create a `Visitor` and `Conversation` for every existing `User`.
73
3. The task is idempotent: users that already have a visitor are skipped and no messages are duplicated.
74
4. After the backfill, every authenticated user has a conversation and a single greeting message.
75
76
### Rollback
77
78
The backfill is not reversible. If a rollback is required, restore from a database snapshot taken before the task runs.
8
Superseded by: [docs/architecture.md](architecture.md) and the
9
[integration hardening plan](2026-08-20-integration-hardening-and-staging-readiness-recommendations.md)
79 10
80 11
## Outcome
81 12
82
Move the complete Sarah product from `~/work/sarah` into the `openagents.com` repository and run it from a single Elixir/Phoenix application. The final application keeps the `OpenAgents` namespace but owns the full Sarah chat, voice, memory, work, machine-delegation, and operator tooling systems.
83
84
After the integration, `openagents.com` serves:
85
86
- The existing issues, projects, labels, milestones, and docs surfaces.
87
- The full Sarah `/chat` experience, including typed turns, voice, memory, tool activity, deep work, and computer delegation.
88
- Sarah's operator and admin surfaces (admin, forge, computers, incidents, network status, leaderboard, changelog).
89
- A single authentication, database, asset pipeline, and deployment artifact.
90
91
## Scope
92
93
This plan covers the complete port of `~/work/sarah` into `/Users/christopherdavid/work/openagents.com`. It is intentionally broad: the objective is to fold the two codebases together rather than to build a remote-service bridge.
94
95
The work is divided by subsystem, not by one-shot file copy. Each phase produces a working build and a passing test suite before the next phase begins.
96
97
## Terminology
98
99
- **Sarah source** — the `sarah` application in `~/work/sarah`.
100
- **OpenAgents target** — the `openagents.com` application in `~/work/openagents.com`.
101
- **Lift** — copy a module, schema, migration, controller, live view, component, or asset file and update its namespace and references.
102
- **Merge** — combine two modules that exist in both repositories because they cover the same concern.
103
- **Re-namespace** — change `Sarah.` to `OpenAgents.` and `SarahWeb.` to `OpenAgentsWeb.`.
104
105
## Current state of the Sarah source
106
107
### Directory tree under `lib/sarah`
108
109
```text
110
lib/sarah/
111
├── accounts/                    # User, OAuth, token vault
112
├── admin/                       # Admin calls
113
├── blueprint/                   # Blueprint facts and revisions
114
├── changelog/                   # Changelog
115
├── cluster/                     # Horde/RA clustering
116
├── collective/                  # Collective learning
117
├── compensation/                 # Compensation accounting
118
├── computer/                    # Computer agent jobs
119
├── computer_activity.ex         # Live activity projection
120
├── conversations/               # Core chat: conversation, message, turn, receipt
121
├── context/                     # Context composition for inference
122
├── data_rights/                 # Exports and deletion
123
├── experience_memory/           # Experience/pattern memory
124
├── forge/                       # Git forge (builds, targets, deploys)
125
├── github/                      # GitHub OAuth
126
├── graph_memory/                # Derived graph memory
127
├── incidents/                   # Incident management
128
├── memory/                      # Semantic/lexical memory, embeddings, redaction
129
├── profile_memory/              # User profile memory
130
├── tools/                       # Memory tools
131
├── voice/                       # Voice sessions, recordings, transcripts
132
├── voice_sessions.ex
133
├── voice_sessions/              # Voice session server
134
├── voice_recovery.ex
135
├── work/                        # Deep work jobs and delegation
136
├── tools/deep_work.ex
137
└── network_status.ex
138
```
139
140
### Directory tree under `lib/sarah_web`
141
142
```text
143
lib/sarah_web/
144
├── live/
145
│   ├── chat_live.ex             # Main chat (2478 lines)
146
│   ├── admin_live.ex
147
│   ├── admin_forge_live.ex
148
│   ├── changelog_live.ex
149
│   ├── code_blob_live.ex
150
│   ├── code_commit_live.ex
151
│   ├── code_repo_live.ex
152
│   ├── computers_live.ex
153
│   ├── leaderboard_live.ex
154
│   ├── network_status_live.ex
155
│   ├── ui_gallery_live.ex
156
│   └── voice_spike_live.ex
157
├── controllers/
158
│   ├── auth_controller.ex
159
│   ├── admin_recording_controller.ex
160
│   ├── changelog_controller.ex
161
│   ├── computer_agent_jobs_controller.ex
162
│   ├── computers_controller.ex
163
│   ├── controller_pairing_controller.ex
164
│   ├── data_controller.ex
165
│   ├── health_controller.ex
166
│   ├── inference_proxy_controller.ex
167
│   ├── memory_export_controller.ex
168
│   ├── network_status_controller.ex
169
│   ├── voice_call_controller.ex
170
│   ├── voice_recording_controller.ex
171
│   └── voice_telemetry_controller.ex
172
├── channels/
173
│   ├── computer_channel.ex
174
│   └── controller_socket.ex
175
├── components/
176
│   ├── core_components.ex
177
│   ├── layouts.ex
178
│   ├── tool_activity.ex
179
│   ├── ui.ex                    # Basecoat wrapper (592 lines)
180
│   └── icons.ex
181
├── plugs/
182
│   ├── forge_git_auth.ex
183
│   └── status_probe_compat.ex
184
└── router.ex, endpoint.ex, telemetry.ex, user_auth.ex
185
```
186
187
### Migrations present in Sarah
188
189
| Timestamp range | Feature area |
190
| --- | --- |
191
| 20260816170746 | Conversations, messages, turns, visitors |
192
| 20260816203029 - 20260816224054 | Turn receipts, tool steps, provenance, lexical recall |
193
| 20260816214500 - 20260816220000 | Profile memory, snapshots, sources, policy events |
194
| 20260816214735 - 20260818150000 | Voice sessions, events, recordings, transcripts |
195
| 20260818003358 - 20260818234500 | Work jobs, job steps, inference grants, attribution |
196
| 20260817140345 | Machines, machine pairings |
197
| 20260817030000 - 20260817040000 | Experience memory, graph memory |
198
| 20260817002000 - 20260817005000 | Collective learning |
199
| 20260817010500 | Compensation |
200
| 20260817070000 - 20260817125732 | GitHub users, token vault |
201
| 20260819010000 - 20260819013000 | Forge pushes, builds, deploys |
202
| 20260819170000 | Changelog |
203
| 20260818150100 | Incidents |
204
| 20260816220000 | Blueprints |
205
| 20260817050000 | Data rights / ATIF export |
206
207
### Dependencies Sarah has that OpenAgents does not
208
209
| Dependency | Purpose | Required? |
210
| --- | --- | --- |
211
| `mdex` | Markdown parsing in chat messages | Yes for chat; replaces retired `earmark` |
212
| `websockex` | WebSocket client for voice/computer | Yes for voice and machines |
213
| `horde` | Distributed process registry | Only if cluster is kept |
214
| `ra` | Raft consensus | Only if cluster is kept |
215
| `dns_cluster` | DNS cluster discovery | Only if cluster is kept |
216
217
### Shared dependencies
218
219
Both projects already use Phoenix, Phoenix Ecto, Phoenix LiveView, Ecto SQL, Postgrex, Req, Jason, Bandit, Tailwind, esbuild, Castle/Forecastle, and the same hot-upgrade release pattern.
220
221
### Major merge conflicts
222
223
| Conflict | Resolution |
224
| --- | --- |
225
| Both have `Chat` modules | Replace `OpenAgents.Chat` mock with `Sarah.Conversations` re-namespaced to `OpenAgents.Conversations`. |
226
| Both have `Accounts.User` schemas | Merge schemas; `users` table wins. OpenAgents keeps `github_oauth_*`, Sarah adds `browser_key_hash` and token-ciphertext fields. |
227
| Both have `Forge` modules | `OpenAgents.Forge` (fleet targets/builds/deploys) and `Sarah.Forge` (git forge) need separate namespaces or a single merged forge domain. |
228
| Both have `messages` tables | OpenAgents mock chat messages are dropped; Sarah `messages` table wins. |
229
| `conversations`, `turns`, `visitors` are new | Add Sarah's migrations. |
230
| CSS token and component model | Sarah uses Basecoat (`style-sarah.css`) and oklch tokens; OpenAgents uses DaisyUI. Pick one and migrate the other incrementally. |
231
| Icon set | Sarah uses vendored Apps SDK icons; OpenAgents uses Heroicons. Converge on one or keep both with adapter component. |
232
| Router | Sarah has `/chat`, voice, admin, and computer routes. Merge into `OpenAgentsWeb.Router`. |
233
234
## Strategy
235
236
Do not lift the entire tree at once. Work in subsystem phases. After each phase:
237
238
1. Re-namespace the lifted modules.
239
2. Re-namespace or rename schemas and migrations to `openagents_*` when needed to avoid table collisions.
240
3. Resolve compile errors and warnings.
241
4. Run `mix ecto.migrate` and `mix precommit`.
242
5. Merge the new router, layout, and component paths only when the phase is stable.
243
244
The last phase is the UI cutover: replace `OpenAgentsWeb.ChatLive` and the command bar with Sarah's `chat_live.ex` and shell, then remove the mock `OpenAgents.Chat` module.
245
246
## Phase 1: Foundation and application wiring
247
248
### Goal
249
250
A re-namespaced Sarah `Application`, `Repo`, `PubSub`, `Telemetry`, and cluster supervision starts inside `OpenAgents.Application`, and the new dependencies compile.
251
252
### Tasks
253
254
1. Add missing dependencies to `mix.exs` and run `mix deps.get`:
255
   - `{:mdex, "~> 0.3"}` (replaces retired `earmark`)
256
   - `{:websockex, "~> 0.5.1"}`
257
   - `{:horde, "~> 0.9.0"}` (optional, behind a feature flag if not needed)
258
   - `{:ra, "~> 2.16"}` (optional, behind a feature flag if not needed)
259
2. Create `OpenAgents.Sarah.Supervisor` as a placeholder supervisor that will hold the re-namespaced Sarah children as they are lifted.
260
3. Wire `OpenAgents.Sarah.Supervisor` into `OpenAgents.Application` before `OpenAgentsWeb.Endpoint`.
261
4. Add the voice `/controller/socket` and any additional sockets to `OpenAgentsWeb.Endpoint` when the rest of the web layer is ported.
262
5. Lift `lib/sarah/telemetry.ex` and merge telemetry metrics with `OpenAgentsWeb.Telemetry` in a later phase.
263
264
### Exit criteria
265
266
- `mix compile` succeeds.
267
- New dependencies are in `mix.lock`.
268
- `OpenAgents.Application` starts the Sarah placeholder supervisor without runtime errors.
269
- `mix precommit` passes.
270
271
## Phase 2: Accounts and authentication
272
273
### Goal
274
275
A single `users` table and a single `Accounts` context serve both GitHub sign-in and Sarah visitor tracking.
276
277
### Tasks
278
279
1. Lift `lib/sarah/accounts/`.
280
2. Merge with `lib/openagents/accounts/`:
281
   - Keep `OpenAgents.Accounts.User`.
282
   - Add `browser_key_hash`, `github_token_ciphertext`, and `github_name` fields from Sarah.
283
   - Merge `token_vault.ex` and `oauth_attempt.ex` if they differ.
284
3. Lift the GitHub OAuth migrations and merge with OpenAgents' existing GitHub auth tables.
285
4. Update `OpenAgentsWeb.UserAuth` and `OpenAgentsWeb.Router` to use the merged user schema.
286
5. Lift `lib/sarah/accounts/accounts.ex` functions (or merge into `OpenAgents.Accounts`).
287
288
### Exit criteria
289
290
- `mix ecto.migrate` succeeds on a fresh test database.
291
- Existing GitHub auth tests still pass.
292
- Sarah `accounts_test.exs` equivalents compile and pass after re-naming.
293
294
## Phase 3: Core chat schema and contexts
295
296
### Goal
297
298
The conversation, message, turn, visitor, and receipt schemas exist under `OpenAgents.Conversations` and `OpenAgents.Turns`, and the existing `/chat` route is served by a real (still local-mock) backend.
299
300
### Tasks
301
302
1. Lift `lib/sarah/conversations/` to `lib/openagents/conversations/`.
303
2. Re-namespace `Sarah.Conversations` to `OpenAgents.Conversations`, `Sarah.Conversations.Message` to `OpenAgents.Conversations.Message`, and so on.
304
3. Drop the mock `OpenAgents.Chat` module and `OpenAgents.Chat.Message` struct.
305
4. Lift the visitor/conversation/message/turn migrations.
306
5. Merge with any existing `conversations` or `messages` migrations in OpenAgents (drop OpenAgents mock tables).
307
6. Add `OpenAgents.Turns` for turn lifecycle (if not already inside `Conversations`).
308
7. Update `OpenAgentsWeb.ChatLive` to use `OpenAgents.Conversations` with the Sarah `chat_live.ex` UI (see Phase 8).
309
310
### Exit criteria
311
312
- `OpenAgents.Conversations` can create a conversation, add user and assistant messages, and list turns.
313
- `mix precommit` passes.
314
315
## Phase 4: Memory systems
316
317
### Goal
318
319
Profile, experience, and graph memory contexts work and the `OpenAgents.Memory` namespace is stable.
320
321
### Tasks
322
323
1. Lift `lib/sarah/profile_memory/` to `lib/openagents/profile_memory/`.
324
2. Lift `lib/sarah/experience_memory/` to `lib/openagents/experience_memory/`.
325
3. Lift `lib/sarah/graph_memory/` to `lib/openagents/graph_memory/`.
326
4. Lift `lib/sarah/memory/` to `lib/openagents/memory/`.
327
5. Lift `lib/sarah/tools/memory_*.ex` to `lib/openagents/tools/`.
328
6. Lift the corresponding migrations.
329
7. Add PostgreSQL trigger and function migrations to the OpenAgents migration tree (watch for function-name collisions).
330
8. Update `OpenAgentsWeb.ChatLive` memory panel and hooks.
331
332
### Exit criteria
333
334
- `OpenAgents.ProfileMemory` can remember, correct, and forget records.
335
- `OpenAgents.ExperienceMemory` and `OpenAgents.GraphMemory` compile and pass basic tests.
336
337
## Phase 5: Work, delegation, and computer activity
338
339
### Goal
340
341
Deep work jobs, computer delegation, and live activity projections are available.
342
343
### Tasks
344
345
1. Lift `lib/sarah/work/` to `lib/openagents/work/`.
346
2. Lift `lib/sarah/computer/` and `lib/sarah/computer_activity.ex` to `lib/openagents/computer/`.
347
3. Lift `lib/sarah/channels/` to `lib/openagents_web/channels/`.
348
4. Lift `lib/sarah/plugs/` to `lib/openagents_web/plugs/`.
349
5. Add `work_jobs`, `work_job_steps`, `machines`, and `machine_pairings` migrations.
350
6. Merge `Sarah.Forge` with `OpenAgents.Forge` or keep them as `OpenAgents.Forge.Git` and `OpenAgents.Forge.Fleet`.
351
352
### Exit criteria
353
354
- `OpenAgents.Work` can start and monitor a job.
355
- `OpenAgents.ComputerActivity` broadcasts and receives PubSub activity.
356
- Forge and work tests pass after namespace changes.
357
358
## Phase 6: Voice
359
360
### Goal
361
362
Voice sessions, recordings, transcripts, and browser media work end to end.
363
364
### Tasks
365
366
1. Lift `lib/sarah/voice/`, `lib/sarah/voice_sessions.ex`, `lib/sarah/voice_sessions/`, and `lib/sarah/voice_recovery.ex` to `lib/openagents/`.
367
2. Lift voice controllers to `lib/openagents_web/controllers/`.
368
3. Add the `voice_sessions`, `voice_events`, `voice_recordings`, `voice_transcript_items`, and related migrations.
369
4. Add voice JavaScript and hooks to `assets/js/`.
370
5. Add `OpenAgents.Voice.Config` and wire it into `OpenAgents.Application`.
371
6. Update `OpenAgentsWeb.ChatLive` to render `composer_stack` with voice controls.
372
373
### Exit criteria
374
375
- Voice controller tests pass.
376
- `OpenAgents.Voice` can create a session and emit transcript items.
377
378
## Phase 7: Context, inference, and tools
379
380
### Goal
381
382
Turn context composition, tool execution, and the inference adapter are available.
383
384
### Tasks
385
386
1. Lift `lib/sarah/context/` to `lib/openagents/context/`.
387
2. Lift `lib/sarah/conversations/provider_step.ex`, `tool_step.ex`, and `turn_receipt.ex`.
388
3. Add an `OpenAgents.Inference` adapter or reuse the `pro.openagents.com` `ReqClient` from `docs/chat-inference-plan.md` if it already exists.
389
4. Lift `lib/sarah/changelog/`, `lib/sarah/blueprint/`, `lib/sarah/data_rights/`, `lib/sarah/incidents/`, `lib/sarah/collective/`, and `lib/sarah/compensation/` to their respective `lib/openagents/` directories.
390
5. Add remaining migrations.
391
392
### Exit criteria
393
394
- A typed turn can be created, its context can be composed, and tool steps can be executed against fake providers in tests.
395
396
## Phase 8: UI, components, and assets
397
398
### Goal
399
400
The user sees the Sarah chat shell and the existing OpenAgents pages keep working.
401
402
### Tasks
403
404
1. Replace `OpenAgentsWeb.ChatLive` with the full `SarahWeb.ChatLive` after re-namespacing.
405
2. Lift `lib/sarah_web/components/tool_activity.ex`, `ui.ex`, and `icons.ex`.
406
3. Merge `OpenAgentsWeb.CoreComponents` with Sarah's `CoreComponents` or keep Sarah components as `OpenAgentsWeb.SarahUI`.
407
4. Merge `OpenAgentsWeb.Layouts` with `SarahWeb.Layouts`. Keep a single app shell; the chat shell is the same layout with different inner content.
408
5. Merge the routers: add Sarah's `/chat`, voice, admin, computer, network status, leaderboard, changelog, and code routes to `OpenAgentsWeb.Router`.
409
6. Merge the CSS:
410
   - Decide: keep DaisyUI + OpenAgents custom tokens, or switch to Basecoat + Sarah tokens.
411
   - Minimum: add Sarah's custom chat styles to `assets/css/app.css` and keep the rest of the app on DaisyUI.
412
   - Migrate the chat components to use the shared style system.
413
7. Merge the icon set: keep both Heroicons and the vendored Apps SDK icons, and support both via `OpenAgentsWeb.UI.icon/1`.
414
8. Lift JavaScript:
415
   - `assets/js/voice_controller.js`
416
   - `assets/js/paced_transcript.js`
417
   - `assets/js/voice_recording.mjs`
418
   - `assets/js/voice_state.mjs`
419
9. Update `assets/js/app.js` to import the new hooks and modules.
420
10. Update `config/config.exs`, `dev.exs`, `test.exs`, and `runtime.exs` with Sarah keys (voice provider, memory model, cluster, and so on).
421
422
### Exit criteria
423
424
- `/chat` renders the Sarah UI.
425
- The existing `/OpenAgents/openagents/issues` and `/docs` pages still render with one navbar and one sidebar.
426
- `mix assets.deploy` succeeds.
427
428
## Phase 9: Test and cutover
429
430
### Goal
431
432
The full `mix precommit` passes on the merged tree and the chat is ready for staged rollout.
433
434
### Tasks
435
436
1. Lift and re-namespace all `test/sarah/` and `test/sarah_web/` tests to `test/openagents/` and `test/openagents_web/`.
437
2. Re-namespace `test/support/` fixtures and factories.
438
3. Add integration tests that exercise the merged authentication + chat + memory + work path.
439
4. Add feature flags to enable Sarah chat, voice, and memory per environment if a slow rollout is desired.
440
5. Run `mix ecto.reset` on a development database, then `mix ecto.migrate` in staging.
441
6. Run `mix precommit` and `mix test`.
442
443
### Exit criteria
444
445
- `mix precommit` passes.
446
- `mix test` passes with the new test suite.
447
- A manual smoke test of `/chat`, issue list, and docs succeeds.
448
- Sarah tests are present under `test/openagents/` and `test/openagents_web/` but are tagged `:skip` while the runtime is still stubbed.
449
450
## Re-naming and file mapping
451
452
Use the following convention when lifting a file:
453
454
| Sarah path | OpenAgents path |
455
| --- | --- |
456
| `lib/sarah/<context>/*.ex` | `lib/openagents/<context>/*.ex` |
457
| `lib/sarah_web/live/*.ex` | `lib/openagents_web/live/*.ex` |
458
| `lib/sarah_web/controllers/*.ex` | `lib/openagents_web/controllers/*.ex` |
459
| `lib/sarah_web/components/*.ex` | `lib/openagents_web/components/*.ex` |
460
| `lib/sarah_web/channels/*.ex` | `lib/openagents_web/channels/*.ex` |
461
| `lib/sarah_web/plugs/*.ex` | `lib/openagents_web/plugs/*.ex` |
462
| `priv/repo/migrations/*.exs` | `priv/repo/migrations/*.exs` (use fresh timestamps if needed) |
463
| `assets/js/*.js` / `*.mjs` | `assets/js/*.js` / `*.mjs` |
464
| `assets/css/style-sarah.css` | `assets/css/sarah.css` (or merge into `app.css`) |
465
466
Inside each lifted file, perform these replacements:
467
468
- `Sarah.` → `OpenAgents.`
469
- `SarahWeb.` → `OpenAgentsWeb.`
470
- `Sarah.Repo` → `OpenAgents.Repo`
471
- `Sarah.PubSub` → `OpenAgents.PubSub`
472
- `sarah_` prefix in config to `openagents_`
473
- Ecto schema `schema "sarah_*"` to `schema "openagents_*"` if the table already exists in OpenAgents
474
475
## Risk register
476
477
| Risk | Likelihood | Impact | Mitigation |
478
| --- | --- | --- | --- |
479
| User and visitor schema merge fails | High | High | Make `OpenAgents.Accounts.User` the canonical user; re-target Sarah `visitor_id` to user ID; use `browser_key_hash` as nullable. |
480
| Forge namespace collision | High | High | Rename one to `OpenAgents.Forge.Fleet` (existing) and `OpenAgents.Forge.Git` (Sarah). |
481
| DaisyUI + Basecoat CSS conflict | High | High | Keep both; scope Sarah chat to a `.sarah` root class and merge tokens only for shared primitives. |
482
| Migration order and table collisions | High | High | Add all new Sarah migrations after existing OpenAgents migrations and rename tables if needed. |
483
| Heroicons vs Apps SDK icons | Medium | Low | Keep both; keep the Heroicons component for app UI and the Sarah icon component for chat. |
484
| Cluster dependencies (Horde/RA) not needed | Medium | Medium | Gate behind config; stub or omit the cluster supervisor for single-node dev/test. |
485
| Chat `RepoHeader` is already removed | Low | Low | No collision; the chat UI gets the Sarah header. |
486
| Tests are large and brittle | Medium | High | Port tests subsystem by subsystem; stub external providers and voice clients. |
487
488
## Suggested next step
489
490
Start with **Phase 1** and **Phase 2** in a feature branch. Lift `lib/sarah/application.ex`, `lib/sarah/accounts/`, and the core chat schema. Re-namespace, add dependencies, and get `mix compile` green before moving into memory, voice, and work.
13
The complete conversational product was integrated into the public
14
`openagents.com` Phoenix application. The application now owns:
15
16
- Accounts, OAuth, durable conversations, messages, turns, and provenance.
17
- Persona, role, Blueprint, provider, tool, and program-artifact contracts.
18
- Profile, lexical, semantic, experience, graph, and portability memory paths.
19
- Delegated work, connected computers, machines, and incidents.
20
- Voice sessions, transcripts, usage, recordings, and release controls.
21
- Operator, status, leaderboard, changelog, data-rights, issues, projects, and
22
  forge surfaces.
23
- The corresponding migrations, browser assets, configuration, and tests.
24
25
The migration retained persona-specific Sarah artifact names and stable data
26
contracts where identity or historical compatibility required them. Generic
27
supervision, web helpers, runtime paths, styles, tests, and forge infrastructure
28
now use OpenAgents names. [ADR 0002](decisions/0002-model-sarah-as-an-openagents-persona.md)
29
defines that boundary.
30
31
## Migration sequence
32
33
The work landed in these broad stages:
34
35
1. Application dependencies, supervision, accounts, and authentication.
36
2. Conversation, turn, receipt, memory, work, machine, and voice schemas.
37
3. Persona, context, provider, module, tool, collective, and observability
38
   domains.
39
4. Chat, voice, operator, computer, status, changelog, and forge web surfaces.
40
5. Browser assets, the shared interface system, and runtime configuration.
41
6. Re-namespaced tests and support code, followed by removal of temporary test
42
   skips and local runtime stubs.
43
7. Horde/Ra clustering, forge build/deploy primitives, and owned local gates.
44
45
The original phase notes and intermediate pass/failure counts described a
46
temporary migration state and are intentionally not repeated as current facts.
47
The dated [coverage audit](2026-08-20-test-coverage-audit.md) preserves the
48
measurement history.
49
50
## Data lineage
51
52
The integration extended the existing `users` authority and added visitors,
53
conversations, messages, turns, memory, voice, work, machines, and forge
54
records. Historical migration filenames that contain Sarah names remain
55
immutable migration lineage; renaming an already-applied migration would make
56
schema history less trustworthy.
57
58
Persona and evaluation material remains under `priv/sarah/` because its stable
59
artifact IDs, digests, and wire schemas are compatibility and provenance
60
contracts, not generic application naming.
61
62
Before any nonempty staging database is upgraded, the complete migration chain
63
and visitor backfill must be rehearsed against a disposable restored copy. That
64
work belongs to Gates 13–16 of the current hardening plan.
65
66
## Remaining work
67
68
The source migration being complete does not make the application production
69
ready. Documentation reconciliation, dependency and component consolidation,
70
route authority, token lifecycle, tenant scoping, recovery, build isolation,
71
transactional fleet deployment, staging isolation, failure injection, and the
72
48-hour soak remain governed by the current hardening plan.
lib/openagents_web/live/admin_live.ex modified +8 -16

@@ -1,6 +1,7 @@

1 1
defmodule OpenAgentsWeb.AdminLive do
2 2
  @moduledoc """
3
  The operator surface: every voice call, newest first, with its audio.
3
  The operator surface: every voice call, newest first, with bounded recording
4
  metadata.
4 5
5 6
  Read-only by construction. `OpenAgents.Admin` exposes no write, so nothing here can
6 7
  ban an account, alter a conversation, or change configuration

@@ -8,12 +9,11 @@ defmodule OpenAgentsWeb.AdminLive do

8 9
9 10
  Two presentation rules are deliberate:
10 11
11
    * Calls with no audio are listed with the reason. A panel that hid them would
12
      look like an empty history instead of an honest one — every call from before
13
      recording shipped has none.
12
    * Calls with no uploaded recording are listed with the reason. A panel that
13
      hid them would look like an empty history instead of an honest one.
14 14
    * Transcript *content* never appears, only whether a transcript exists. The
15
      operator is here to listen, and cross-account reading of what was said is a
16
      separate decision from cross-account listening.
15
      operator has no recording-download route either; cross-account content
16
      access requires a separate decision and implementation.
17 17
  """
18 18
19 19
  use OpenAgentsWeb, :openagents_live_view

@@ -165,14 +165,7 @@ defmodule OpenAgentsWeb.AdminLive do

165 165
                    </div>
166 166
                  </dl>
167 167
168
                  <.audio_player
169
                    :if={Call.playable?(call)}
170
                    id={"admin-audio-#{call.session_id}"}
171
                    src={"/admin/recordings/#{call.recording.id}/audio"}
172
                    label={"Call with @#{call.github_login} on #{format_timestamp(call.started_at)}"}
173
                  />
174
175
                  <p :if={!Call.playable?(call)} class="admin-absence">
168
                  <p :if={is_nil(call.recording)} class="admin-absence">
176 169
                    {Call.absence_reason(call)}
177 170
                  </p>
178 171
                </div>

@@ -208,8 +201,7 @@ defmodule OpenAgentsWeb.AdminLive do

208 201
  end
209 202
210 203
  # The panel is paged rather than streamed: an operator scanning recent calls
211
  # wants a bounded page, and every row carries an audio element the browser will
212
  # ask metadata for.
204
  # wants a bounded page and the recording projection remains metadata-only.
213 205
  defp page_size, do: 25
214 206
215 207
  defp page_label(offset, calls, total) do
mix.exs modified +1

@@ -115,6 +115,7 @@ defmodule OpenAgents.MixProject do

115 115
        "deps.unlock --unused",
116 116
        "format",
117 117
        "cmd ops/ci/reference-check.sh",
118
        "cmd elixir ops/ci/docs-check.exs",
118 119
        "assets.test",
119 120
        "test --warnings-as-errors"
120 121
      ]
ops/ci/allowed-sarah-references.txt modified +1

@@ -15,6 +15,7 @@

15 15
^docs/sarah-integration-plan\.md:
16 16
17 17
# Current architecture and decisions discuss the Sarah persona boundary.
18
^README\.md:[0-9]+:.*[Ss][Aa][Rr][Aa][Hh]
18 19
^docs/architecture\.md:
19 20
^docs/decisions/0001-integrate-the-complete-public-application\.md:
20 21
^docs/decisions/0002-model-sarah-as-an-openagents-persona\.md:
ops/ci/docs-check.exs added +251

@@ -0,0 +1,251 @@

1
defmodule OpenAgents.DocsCheck do
2
  @moduledoc false
3
4
  @root File.cwd!()
5
  @markdown_files ["README.md", "INVARIANTS.md"] ++ Path.wildcard("docs/**/*.md")
6
7
  # These records must preserve the terms they measure or prohibit. Current
8
  # product narratives and closed migration records remain subject to the scan.
9
  @lexical_exceptions MapSet.new([
10
                        "docs/2026-08-20-integration-hardening-and-staging-readiness-recommendations.md",
11
                        "docs/2026-08-20-test-coverage-audit.md",
12
                        "docs/decisions/0005-use-basecoat-and-one-component-system.md"
13
                      ])
14
15
  @banned_terms [
16
    {~r/\bDaisyUI\b/i, "retired component library"},
17
    {~r/\b(?:pro|api)\.openagents\.com\b/i, "retired private-service domain"},
18
    {~r/OpenAgentsWeb\.SarahUI/, "retired generic component module"},
19
    {~r/OpenAgents\.Sarah\.Supervisor/, "retired generic supervisor"},
20
    {~r/style-sarah\.css|style-openagents\.css/, "retired style-pack path"},
21
    {~r/priv\/openagents\//, "nonexistent artifact root"}
22
  ]
23
24
  @developer_paths [
25
    {~r{~/(?:work|code)(?:/|\b)}, "home-relative developer path"},
26
    {~r{/Users/[A-Za-z0-9._-]+/(?:work|code)(?:/|\b)}, "macOS developer path"},
27
    {~r{/home/[A-Za-z0-9._-]+/(?:work|code)(?:/|\b)}, "Linux developer path"}
28
  ]
29
30
  def run do
31
    errors =
32
      []
33
      |> check_markdown_links()
34
      |> check_current_language()
35
      |> check_invariants()
36
37
    case Enum.reverse(errors) do
38
      [] ->
39
        IO.puts("Documentation check passed (#{length(@markdown_files)} Markdown files).")
40
41
      failures ->
42
        Enum.each(failures, &IO.puts(:stderr, "documentation check: #{&1}"))
43
        System.halt(1)
44
    end
45
  end
46
47
  defp check_markdown_links(errors) do
48
    Enum.reduce(@markdown_files, errors, fn file, acc ->
49
      content = File.read!(file)
50
51
      Regex.scan(~r/\[[^\]]*\]\(([^)]+)\)/, content, capture: :all_but_first)
52
      |> Enum.reduce(acc, fn [raw_target], link_errors ->
53
        target =
54
          raw_target
55
          |> String.trim()
56
          |> String.trim_leading("<")
57
          |> String.trim_trailing(">")
58
          |> String.split(~r/\s+"/, parts: 2)
59
          |> hd()
60
          |> String.split("#", parts: 2)
61
          |> hd()
62
          |> String.split("?", parts: 2)
63
          |> hd()
64
65
        if external_or_route?(target) do
66
          link_errors
67
        else
68
          resolved = Path.expand(target, Path.dirname(Path.join(@root, file)))
69
70
          if File.exists?(resolved) do
71
            link_errors
72
          else
73
            ["#{file} links to missing local target #{inspect(raw_target)}" | link_errors]
74
          end
75
        end
76
      end)
77
    end)
78
  end
79
80
  defp external_or_route?(target) do
81
    target == "" or String.starts_with?(target, ["#", "/", "http://", "https://", "mailto:"])
82
  end
83
84
  defp check_current_language(errors) do
85
    @markdown_files
86
    |> Enum.reject(&MapSet.member?(@lexical_exceptions, &1))
87
    |> Enum.reduce(errors, fn file, acc ->
88
      content = File.read!(file)
89
90
      acc
91
      |> scan_terms(file, content, @banned_terms)
92
      |> scan_terms(file, content, @developer_paths)
93
    end)
94
  end
95
96
  defp scan_terms(errors, file, content, patterns) do
97
    Enum.reduce(patterns, errors, fn {pattern, label}, acc ->
98
      case Regex.run(pattern, content, return: :index) do
99
        nil ->
100
          acc
101
102
        [{offset, _length} | _captures] ->
103
          ["#{file}:#{line_at(content, offset)} contains #{label}" | acc]
104
      end
105
    end)
106
  end
107
108
  defp check_invariants(errors) do
109
    content = File.read!("INVARIANTS.md")
110
    sections = invariant_sections(content)
111
    ids = Enum.map(sections, & &1.id)
112
113
    errors
114
    |> check_duplicate_ids(ids)
115
    |> check_statuses(sections)
116
    |> check_proof_index(content, sections)
117
    |> check_invariant_paths(content)
118
    |> check_module_references(content)
119
  end
120
121
  defp invariant_sections(content) do
122
    Regex.split(~r/(?=^### [A-Z][A-Z0-9-]+-\d{3} — )/m, content)
123
    |> Enum.flat_map(fn section ->
124
      case Regex.run(~r/^### ([A-Z][A-Z0-9-]+-\d{3}) — /, section, capture: :all_but_first) do
125
        [id] -> [%{id: id, body: section}]
126
        nil -> []
127
      end
128
    end)
129
  end
130
131
  defp check_duplicate_ids(errors, ids) do
132
    ids
133
    |> Enum.frequencies()
134
    |> Enum.reduce(errors, fn
135
      {id, count}, acc when count > 1 -> ["INVARIANTS.md contains duplicate ID #{id}" | acc]
136
      {_id, _count}, acc -> acc
137
    end)
138
  end
139
140
  defp check_statuses(errors, sections) do
141
    Enum.reduce(sections, errors, fn section, acc ->
142
      statuses =
143
        Regex.scan(~r/^Status: (Current|Proposed)$/m, section.body, capture: :all_but_first)
144
145
      case statuses do
146
        [[_status]] -> acc
147
        _other -> ["#{section.id} must declare exactly one Current or Proposed status" | acc]
148
      end
149
    end)
150
  end
151
152
  defp check_proof_index(errors, content, sections) do
153
    rows =
154
      Regex.scan(
155
        ~r/^\| ([A-Z][A-Z0-9-]+-\d{3}) \| (.+) \|$/m,
156
        content,
157
        capture: :all_but_first
158
      )
159
160
    proof_rows = Map.new(rows, fn [id, proof] -> {id, proof} end)
161
    section_ids = MapSet.new(sections, & &1.id)
162
    proof_ids = MapSet.new(Map.keys(proof_rows))
163
164
    errors =
165
      Enum.reduce(MapSet.difference(section_ids, proof_ids), errors, fn id, acc ->
166
        ["proof index is missing #{id}" | acc]
167
      end)
168
169
    errors =
170
      Enum.reduce(MapSet.difference(proof_ids, section_ids), errors, fn id, acc ->
171
        ["proof index names unknown invariant #{id}" | acc]
172
      end)
173
174
    Enum.reduce(sections, errors, fn section, acc ->
175
      current? = Regex.match?(~r/^Status: Current$/m, section.body)
176
      proof = Map.get(proof_rows, section.id, "")
177
178
      if current? and not Regex.match?(~r/`(?:test|assets\/test|ops)\/[^`]+`/, proof) do
179
        ["current invariant #{section.id} has no executable proof file" | acc]
180
      else
181
        acc
182
      end
183
    end)
184
  end
185
186
  defp check_invariant_paths(errors, content) do
187
    Regex.scan(~r/`([^`]+)`/, content, capture: :all_but_first)
188
    |> List.flatten()
189
    |> Enum.filter(&local_evidence_path?/1)
190
    |> Enum.uniq()
191
    |> Enum.reduce(errors, fn path, acc ->
192
      normalized = String.trim_trailing(path, "/")
193
194
      if File.exists?(normalized) do
195
        acc
196
      else
197
        ["INVARIANTS.md names missing evidence path #{path}" | acc]
198
      end
199
    end)
200
  end
201
202
  defp local_evidence_path?(path) do
203
    String.starts_with?(path, ["assets/", "config/", "docs/", "ops/", "priv/", "test/"]) or
204
      path in [".dockerignore", ".gitignore", "AGENTS.md", "INVARIANTS.md"]
205
  end
206
207
  defp check_module_references(errors, content) do
208
    module_files = Path.wildcard("{lib,test}/**/*.{ex,exs}")
209
210
    declared =
211
      module_files
212
      |> Enum.flat_map(fn file ->
213
        Regex.scan(
214
          ~r/defmodule\s+(OpenAgents(?:Web)?(?:\.[A-Z][A-Za-z0-9_]*)*)\s+do/,
215
          File.read!(file), capture: :all_but_first)
216
        |> List.flatten()
217
      end)
218
      |> MapSet.new()
219
220
    Regex.scan(~r/`([^`]+)`/, content, capture: :all_but_first)
221
    |> List.flatten()
222
    |> Enum.reject(&String.contains?(&1, "*"))
223
    |> Enum.flat_map(fn token ->
224
      case Regex.run(
225
             ~r/^(OpenAgents(?:Web)?(?:\.[A-Z][A-Za-z0-9_]*)+)/,
226
             token,
227
             capture: :all_but_first
228
           ) do
229
        [module] -> [module]
230
        nil -> []
231
      end
232
    end)
233
    |> Enum.uniq()
234
    |> Enum.reduce(errors, fn module, acc ->
235
      if MapSet.member?(declared, module) do
236
        acc
237
      else
238
        ["INVARIANTS.md names missing module #{module}" | acc]
239
      end
240
    end)
241
  end
242
243
  defp line_at(content, offset) do
244
    content
245
    |> binary_part(0, offset)
246
    |> String.split("\n")
247
    |> length()
248
  end
249
end
250
251
OpenAgents.DocsCheck.run()
ops/ci/reference-check.sh modified +1

@@ -31,6 +31,7 @@ awk 'NF && $1 !~ /^#/' "$allowlist" >"$patterns"

31 31
32 32
rg --line-number --no-heading --color never --ignore-case \
33 33
  --glob '!ops/ci/allowed-sarah-references.txt' \
34
  --glob '!ops/ci/docs-check.exs' \
34 35
  --glob '!ops/ci/reference-check.sh' \
35 36
  'sarah|/var/lib/sarah|/tmp/sarah|pro\.openagents\.com|api\.openagents\.com' \
36 37
  AGENTS.md INVARIANTS.md README.md assets config docs lib ops test >"$matches" || true
test/openagents_web/live/admin_live_test.exs modified +3 -4

@@ -81,18 +81,17 @@ defmodule OpenAgentsWeb.AdminLiveTest do

81 81
  end
82 82
83 83
  describe "the panel" do
84
    test "lists a recorded call with a player and the account it belongs to", %{conn: conn} do
84
    test "lists recording metadata without exposing a playback route", %{conn: conn} do
85 85
      caller = github_user("admin-recorded-caller")
86 86
      session = recorded_call(caller)
87
      recording = Recordings.for_session(session)
88 87
89 88
      conn = log_in_admin_user(conn, "admin-listener")
90 89
      {:ok, view, html} = live(conn, ~p"/admin")
91 90
92 91
      assert html =~ "@#{caller.github_login}"
93 92
      assert has_element?(view, "#admin-call-#{session.id}")
94
      assert has_element?(view, ~s(audio[src="/admin/recordings/#{recording.id}/audio"]))
95
      assert has_element?(view, "#admin-audio-#{session.id}[aria-label]")
93
      refute has_element?(view, "audio")
94
      refute html =~ "/admin/recordings/"
96 95
      assert html =~ "Complete upload"
97 96
    end
98 97

This page updates live while a promote is in flight · changelog