Improve issue and project triage

e46af8833ffc · Devin AI · · parent 954c9a4f7c7f

Improve issue and project triage

Co-Authored-By: Christopher David <chris@openagents.com>
Co-Authored-By
Christopher David <chris@openagents.com>

Deploy story

What this commit did to the running system — joined from the forge receipt chain, the part a commit page elsewhere cannot show.

Not deployed through the forge lane

No push, promotion, build, or deploy receipt references this commit (receipts are scanned over a bounded recent window). Changes shipped by full node replacement carry their proof in the release gate receipt instead.

Changed files

  • modified docs/2026-08-21-operator-posthog-analytics.md
  • modified docs/2026-08-21-posthog-integration-runbook.md
  • added lib/openagents/do_not_build_register.ex
  • modified lib/openagents/issues.ex
  • modified lib/openagents/posthog.ex
  • modified lib/openagents_web/controllers/api_contract_controller.ex
  • modified lib/openagents_web/docs_catalog.ex
  • modified lib/openagents_web/live/admin_analytics_live.ex
  • modified lib/openagents_web/route_authority.ex
  • modified lib/openagents_web/router.ex
  • added priv/api-contracts/do-not-build-v1.json
  • modified priv/docs/creating-issues.md
  • added priv/docs/do-not-build-register.md
  • added test/openagents/do_not_build_register_test.exs
  • modified test/openagents/issues_test.exs
  • modified test/openagents/posthog_test.exs
  • modified test/openagents_web/controllers/api_contract_controller_test.exs
  • modified test/openagents_web/live/admin_analytics_live_test.exs
  • modified test/openagents_web/live/project_workspace_live_test.exs

Diff

19 files changed, +1246 -39

docs/2026-08-21-operator-posthog-analytics.md modified +28 -7

@@ -4,11 +4,10 @@ Date: 2026-08-21

4 4
5 5
Status: Implemented; activates when read credentials are configured
6 6
7
`/admin/analytics` gives operators the trailing-twenty-four-hour product
8
picture without opening PostHog. It exists because the numbers an operator
9
acts on should be one navigation away, rendered in this product's own
10
components, and gated behind the same operator boundary as every other admin
11
surface.
7
`/admin/analytics` gives operators the product-usage and issue-triage picture
8
without opening PostHog. It exists because the numbers an operator acts on
9
should be one navigation away, rendered in this product's own components, and
10
gated behind the same operator boundary as every other admin surface.
12 11
13 12
## How it works
14 13

@@ -18,7 +17,7 @@ API key and shapes the rows into plain maps. The surface adds no aggregation

18 17
of its own: what it renders is exactly what the PostHog app answers for the
19 18
same window, so there is no second authority that can drift from the source.
20 19
21
One pull runs four bounded questions:
20
One pull runs six bounded questions:
22 21
23 22
1. **Activation funnel** — authorization starts, accounts created, returning
24 23
   sign-ins, first chat messages sent.

@@ -26,6 +25,14 @@ One pull runs four bounded questions:

26 25
   average and longest turn duration.
27 26
3. **Event volume** — every event name with count and distinct people.
28 27
4. **Top pages** — the eight most-viewed URLs.
28
5. **Triage health** — rolling 90-day median first-maintainer response time
29
   and the share of open issues still unlabeled after 24 hours.
30
6. **Weekly issue flow** — issues created and closed during each of the
31
   trailing eight weeks.
32
33
The issue projections rely on `issue_number`, `issue_state`,
34
`issue_state_changed`, `has_labels`, and `is_maintainer`. Historical events
35
without those properties are excluded rather than interpreted as zeros.
29 36
30 37
## Configuration
31 38

@@ -53,7 +60,7 @@ The surface designs its non-happy states as first-class UI:

53 60
| Unavailable | Danger notice with retry | PostHog did not answer; nothing stale is rendered |
54 61
| Loaded | Cards and tables | Fresh pull with a generated-at stamp |
55 62
56
Refresh re-runs all four questions and replaces the whole projection.
63
Refresh re-runs all six questions and replaces the whole projection.
57 64
58 65
## Boundaries
59 66

@@ -72,6 +79,20 @@ transport failures, and the disabled path against a stubbed transport.

72 79
`test/openagents_web/live/admin_analytics_live_test.exs` covers the access
73 80
gates and all three non-loaded states plus refresh.
74 81
82
## Weekly review
83
84
Use the pinned **Issue triage health** PostHog dashboard (dashboard `2022873`)
85
for the weekly review. Its saved insights are **Triage health snapshot**
86
(`11257107`) and **Weekly issue flow** (`11257108`).
87
88
1. Compare the median first-maintainer response with the prior week.
89
2. Compare issue intake with closure volume and investigate a sustained gap.
90
3. Label or close every eligible issue shown in the unlabeled count.
91
4. Treat missing response data as insufficient instrumented history.
92
5. Annotate policy, instrumentation, or staffing changes that can explain a
93
   trend break.
94
6. Track follow-up work in forge issues.
95
75 96
## Related
76 97
77 98
- [PostHog integration runbook](2026-08-21-posthog-integration-runbook.md) —
docs/2026-08-21-posthog-integration-runbook.md modified +54 -11

@@ -2,17 +2,16 @@

2 2
3 3
Date: 2026-08-21
4 4
5
Status: Implemented in code; staging enablement, dashboards, and live verification remain
5
Status: Implemented in code; live staging verification remains
6 6
7 7
PostHog ships an agentic installer, the [AI wizard](https://github.com/PostHog/wizard), that wires PostHog into a codebase end to end. The wizard does not support Phoenix or Elixir: the framework is registered as "coming soon" in PostHog's own documentation, so running `npx @posthog/wizard` against this repository would not produce a usable integration.
8 8
9 9
This document is the manual replacement. Part 1 records what the wizard does so we know the full surface we are replicating. Part 2 turns that into a concrete runbook for this Phoenix application: server-side capture from Elixir, browser analytics through our asset bundle, pageviews that survive LiveView navigation, user identification, and an event taxonomy covering every product surface. Session replay and self-driving are out of scope.
10 10
11
Steps 1 through 7 of part 2 are implemented. What remains is operational:
11
Steps 1 through 8 of part 2 are implemented. What remains is operational:
12 12
13 13
1. Set `OPENAGENTS_POSTHOG_PROJECT_TOKEN` (and optionally `OPENAGENTS_POSTHOG_API_HOST`) in the staging environment.
14
2. Build the starter dashboards through the PostHog MCP (step 8).
15
3. Run the verification checklist against a live staging deployment (step 9).
14
2. Run the verification checklist against a live staging deployment (step 9).
16 15
17 16
Use this document as the single checklist for that work. Nothing in it requires the wizard.
18 17

@@ -234,9 +233,9 @@ Issues, projects, and forge:

234 233
235 234
| Event | Where | Properties |
236 235
| --- | --- | --- |
237
| `issue_created` | `Issues.create_issue` | `owner`, `repo`, `has_labels`, `has_assignees` |
238
| `issue_updated` | `Issues.update_issue` | `owner`, `repo`, `state` |
239
| `issue_commented` | `Issues.create_comment` | `issue_number` |
236
| `issue_created` | `Issues.create_issue` | `owner`, `repo`, `issue_number`, `issue_state`, `has_labels`, `has_assignees` |
237
| `issue_updated` | `Issues.update_issue` | `owner`, `repo`, `issue_number`, `previous_issue_state`, `issue_state`, `issue_state_changed`, `has_labels` |
238
| `issue_commented` | `Issues.create_comment` | `owner`, `repo`, `issue_number`, `author_role`, `is_maintainer` |
240 239
| `label_created`, `milestone_created`, `project_created` | respective contexts | `owner`, `repo` |
241 240
| `project_item_added` | `Projects.create_project_item` | `project_number`, `has_issue` |
242 241
| `git_push_received` | `Forge.Pushes` live push path | `repo`, `refs_changed`, `duration_ms` |

@@ -252,7 +251,7 @@ Scoping decisions worth remembering:

252 251
- API read endpoints (`GET`) stay uninstrumented except where they represent product activation.
253 252
- Never capture message bodies, objective text, token material, ciphertexts, or raw query strings. `$current_url` contains query parameters; rely on the wrapper's redaction denylist and keep sensitive routes out of custom properties.
254 253
255
### Step 8: Build the starter dashboards through the PostHog MCP
254
### Step 8: Build the dashboards (implemented)
256 255
257 256
Build every dashboard from your coding agent through the PostHog MCP server instead of clicking in the web app. Dashboard, insight, and annotation objects are all writable through MCP tools, so this step becomes a scripted session you can rerun and review like code.
258 257

@@ -281,6 +280,39 @@ The five dashboards, mapped to their queries:

281 280
282 281
Keep the dashboard definitions in the agent session transcript or commit them as a script; because creation goes through MCP calls, recreating the set in staging or after a project reset is mechanical.
283 282
283
The operations project also has the pinned **Issue triage health** dashboard
284
(dashboard `2022873`). It contains:
285
286
- **Triage health snapshot** (insight `11257107`): the rolling 90-day median
287
  time from `issue_created` to the first `issue_commented` event whose
288
  `is_maintainer` property is true, plus the share of open issues that remain
289
  unlabeled 24 hours after creation.
290
- **Weekly issue flow** (insight `11257108`): issues created and closed for
291
  each of the trailing eight weeks. A closure is an `issue_updated` event with
292
  `issue_state_changed = true` and `issue_state = "closed"`.
293
294
The dashboard and its two HogQL insights were created through the PostHog REST
295
API because the MCP server was unavailable during setup. This does not change
296
their ownership or query semantics.
297
298
Historical issue events do not contain the new triage properties. Expect the
299
response, label, and closure metrics to populate from the deployment that
300
introduces those properties; do not infer historical zeros from missing data.
301
302
### Weekly triage review
303
304
Review the pinned dashboard once each week:
305
306
1. Record the median first-maintainer response and compare it with the prior
307
   week.
308
2. Compare issues created with issues closed. Investigate sustained intake
309
   above closure volume.
310
3. Open the unlabeled count and label or close each eligible issue.
311
4. Treat a blank response metric as insufficient instrumented data, not a zero.
312
5. Add an annotation for instrumentation, policy, or staffing changes that can
313
   explain a trend break.
314
6. Record follow-up work as forge issues and link the dashboard in the issue.
315
284 316
### Step 9: Verify
285 317
286 318
Work through this checklist in staging. Browser-side checks stay manual; every server-side or ingestion check runs through the PostHog MCP against the events table, which beats tailing the web UI:

@@ -298,7 +330,11 @@ Work through this checklist in staging. Browser-side checks stay manual; every s

298 330
299 331
## Operator analytics surface
300 332
301
`/admin/analytics` gives operators the trailing-24-hour picture without opening PostHog. It pulls computed results from the PostHog REST API at request time through `OpenAgents.PostHog` (a personal API key over HogQL), so it adds no second aggregation authority: the numbers match what the PostHog app answers for the same window.
333
`/admin/analytics` gives operators the usage and triage picture without opening
334
PostHog. It pulls computed results from the PostHog REST API at request time
335
through `OpenAgents.PostHog` (a personal API key over HogQL), so it adds no
336
second aggregation authority: the numbers match what the PostHog app answers
337
for the same window.
302 338
303 339
Settings (all optional; absent credentials disable the read path only, independently of capture):
304 340

@@ -308,14 +344,21 @@ Settings (all optional; absent credentials disable the read path only, independe

308 344
| `OPENAGENTS_POSTHOG_PROJECT_ID` | The numeric PostHog project id |
309 345
| `OPENAGENTS_POSTHOG_APP_HOST` | Optional; defaults to `https://us.posthog.com`. This is the app/API host, not the ingest host |
310 346
311
The page renders four bounded projections — activation funnel, chat turn outcomes and durations, event volume, top pages — plus its non-happy states as first-class UI: unconfigured credentials, an unanswered query (with retry, never stale numbers), and loading. It shows aggregates only; no conversation content is reachable from it. The route is operator-gated like the rest of `/admin` and classified in the route authority inventory as `analytics:read`.
347
The page renders six bounded projections: activation funnel, chat turn outcomes
348
and durations, event volume, top pages, triage health, and weekly issue flow.
349
Usage projections cover the trailing 24 hours, triage health covers 90 days,
350
and issue flow covers eight weeks. It renders unconfigured credentials, an
351
unanswered query (with retry, never stale numbers), and loading as first-class
352
states. It shows aggregates only; no conversation content is reachable from
353
it. The route is operator-gated like the rest of `/admin` and classified in the
354
route authority inventory as `analytics:read`.
312 355
313 356
### Rollout order
314 357
315 358
1. Steps 1-6 are implemented and covered by the test suite; they activate the moment a project token is configured.
316 359
2. Set `OPENAGENTS_POSTHOG_PROJECT_TOKEN` in staging and confirm boot with capture live.
317 360
3. Step 7 events flow automatically; watch volume in the first days.
318
4. Step 8: build dashboards through the MCP.
361
4. Step 8 is complete; review the pinned triage dashboard weekly.
319 362
5. Step 9 gates each stage; do not stack stages without verification.
320 363
321 364
## References
lib/openagents/do_not_build_register.ex added +185

@@ -0,0 +1,185 @@

1
defmodule OpenAgents.DoNotBuildRegister do
2
  @moduledoc """
3
  Reads and screens proposals against the durable do-not-build register.
4
5
  Matching uses explicit multi-word phrases from the register. It does not
6
  expand keywords, stem words, or infer semantic similarity, so unrelated work
7
  is not blocked because it mentions a product name.
8
  """
9
10
  alias OpenAgents.Analytics
11
12
  @register_path "priv/api-contracts/do-not-build-v1.json"
13
  @decision_states ~w(retired deferred rejected superseded)
14
15
  @doc "The decoded and validated public register."
16
  def load do
17
    with {:ok, bytes} <- File.read(register_file()),
18
         {:ok, register} <- Jason.decode(bytes),
19
         :ok <- validate(register) do
20
      {:ok, register}
21
    end
22
  end
23
24
  @doc "The register entries, raising when the committed contract is invalid."
25
  def entries do
26
    {:ok, register} = load()
27
    register["entries"]
28
  end
29
30
  @doc "The first precise register match for a proposal, or nil."
31
  def match(proposal) do
32
    text = proposal_text(proposal)
33
34
    Enum.find(entries(), fn entry ->
35
      Enum.any?(entry["match_phrases"], &String.contains?(text, normalize(&1)))
36
    end)
37
  end
38
39
  @doc """
40
  Screens a FastFollow backlog proposal.
41
42
  A match is suppressed and recorded without proposal text. New evidence and
43
  an explicit decision record move the result to manual register review; they
44
  do not silently override the committed decision.
45
  """
46
  def screen_fast_follow(proposal, opts \\ []) when is_list(opts) do
47
    case match(proposal) do
48
      nil ->
49
        :allow
50
51
      entry ->
52
        if reconsideration_ready?(opts) do
53
          {:review_required, entry}
54
        else
55
          record_suppression(entry, proposal)
56
          {:suppressed, entry}
57
        end
58
    end
59
  end
60
61
  @doc "Validates the public contract and its append-only decision history."
62
  def validate(%{
63
        "contract" => "openagents.do-not-build.v1",
64
        "version" => 1,
65
        "decision_states" => @decision_states,
66
        "entries" => entries
67
      })
68
      when is_list(entries) do
69
    with :ok <- validate_unique_ids(entries),
70
         :ok <- validate_entries(entries) do
71
      :ok
72
    end
73
  end
74
75
  def validate(_register), do: {:error, :invalid_contract}
76
77
  defp validate_unique_ids(entries) do
78
    ids = Enum.map(entries, & &1["id"])
79
80
    if length(ids) == length(Enum.uniq(ids)),
81
      do: :ok,
82
      else: {:error, :duplicate_ids}
83
  end
84
85
  defp validate_entries(entries) do
86
    if Enum.all?(entries, &valid_entry?/1),
87
      do: :ok,
88
      else: {:error, :invalid_entry}
89
  end
90
91
  defp valid_entry?(%{
92
         "id" => id,
93
         "retired_scope" => scope,
94
         "match_phrases" => phrases,
95
         "current" => current,
96
         "history" => history
97
       })
98
       when is_binary(id) and is_binary(scope) and is_list(phrases) and is_map(current) and
99
              is_list(history) and history != [] do
100
    valid_phrases?(phrases) and valid_decision?(current) and List.last(history) == current and
101
      Enum.all?(history, &valid_decision?/1)
102
  end
103
104
  defp valid_entry?(_entry), do: false
105
106
  defp valid_phrases?(phrases) do
107
    phrases != [] and
108
      Enum.all?(phrases, fn phrase ->
109
        is_binary(phrase) and String.length(phrase) >= 8 and
110
          phrase |> String.split() |> length() >= 2
111
      end)
112
  end
113
114
  defp valid_decision?(%{
115
         "state" => state,
116
         "decision_date" => date,
117
         "evidence" => evidence,
118
         "violated_principle" => principle,
119
         "replacement_path" => replacement,
120
         "owner" => owner,
121
         "reconsideration_trigger" => trigger,
122
         "decision_record" => record
123
       }) do
124
    state in @decision_states and valid_date?(date) and non_empty_list?(evidence) and
125
      Enum.all?(evidence, &valid_evidence?/1) and
126
      Enum.all?([principle, replacement, owner, trigger, record], &non_empty_string?/1)
127
  end
128
129
  defp valid_decision?(_decision), do: false
130
131
  defp valid_evidence?(%{"source" => source, "summary" => summary}),
132
    do: non_empty_string?(source) and non_empty_string?(summary)
133
134
  defp valid_evidence?(_evidence), do: false
135
136
  defp valid_date?(date) when is_binary(date), do: match?({:ok, _}, Date.from_iso8601(date))
137
  defp valid_date?(_date), do: false
138
139
  defp non_empty_list?(value), do: is_list(value) and value != []
140
  defp non_empty_string?(value), do: is_binary(value) and String.trim(value) != ""
141
142
  defp reconsideration_ready?(opts) do
143
    non_empty_string?(Keyword.get(opts, :new_evidence)) and
144
      non_empty_string?(Keyword.get(opts, :decision_record))
145
  end
146
147
  defp record_suppression(entry, proposal) do
148
    Analytics.capture(
149
      "fast_follow_proposal_suppressed",
150
      Analytics.system_distinct_id("fast_follow"),
151
      %{
152
        "register_id" => entry["id"],
153
        "decision_state" => entry["current"]["state"],
154
        "proposal_fingerprint" => fingerprint(proposal)
155
      }
156
    )
157
  end
158
159
  defp fingerprint(proposal) do
160
    proposal
161
    |> proposal_text()
162
    |> then(&:crypto.hash(:sha256, &1))
163
    |> Base.encode16(case: :lower)
164
  end
165
166
  defp proposal_text(proposal) when is_binary(proposal), do: normalize(proposal)
167
168
  defp proposal_text(proposal) when is_map(proposal) do
169
    [proposal[:title], proposal["title"], proposal[:body], proposal["body"]]
170
    |> Enum.filter(&is_binary/1)
171
    |> Enum.join(" ")
172
    |> normalize()
173
  end
174
175
  defp proposal_text(_proposal), do: ""
176
177
  defp normalize(text) do
178
    text
179
    |> String.downcase()
180
    |> String.replace(~r/[^\p{L}\p{N}]+/u, " ")
181
    |> String.trim()
182
  end
183
184
  defp register_file, do: Application.app_dir(:openagents, @register_path)
185
end
lib/openagents/issues.ex modified +15 -2

@@ -193,6 +193,8 @@ defmodule OpenAgents.Issues do

193 193
        Analytics.capture("issue_created", issue_distinct_id(normalized), %{
194 194
          "owner" => repository.owner,
195 195
          "repo" => repository.name,
196
          "issue_number" => issue.number,
197
          "issue_state" => issue.state,
196 198
          "has_labels" => has_labels?(normalized),
197 199
          "has_assignees" => has_assignees?(normalized)
198 200
        })

@@ -235,7 +237,11 @@ defmodule OpenAgents.Issues do

235 237
        Analytics.capture("issue_updated", actor_distinct_id(actor), %{
236 238
          "owner" => repository && repository.owner,
237 239
          "repo" => repository && repository.name,
238
          "state" => updated.state
240
          "issue_number" => updated.number,
241
          "previous_issue_state" => issue.state,
242
          "issue_state" => updated.state,
243
          "issue_state_changed" => issue.state != updated.state,
244
          "has_labels" => updated.labels != []
239 245
        })
240 246
241 247
        Repositories.broadcast_issues(issue.repository_id)

@@ -608,8 +614,15 @@ defmodule OpenAgents.Issues do

608 614
    end)
609 615
    |> case do
610 616
      {:ok, comment} ->
617
        repository = Repo.get(Repository, issue.repository_id)
618
        author_role = repository && author && Repositories.membership_role(repository, author)
619
611 620
        Analytics.capture("issue_commented", issue_distinct_id(normalized), %{
612
          "issue_number" => issue.number
621
          "owner" => repository && repository.owner,
622
          "repo" => repository && repository.name,
623
          "issue_number" => issue.number,
624
          "author_role" => author_role,
625
          "is_maintainer" => author_role in ~w(owner maintainer)
613 626
        })
614 627
615 628
        Repositories.broadcast_issues(issue.repository_id)
lib/openagents/posthog.ex modified +107 -4

@@ -37,9 +37,9 @@ defmodule OpenAgents.PostHog do

37 37
  end
38 38
39 39
  @doc """
40
  Everything the operator analytics page shows, for the trailing 24 hours.
40
  Everything the operator analytics page shows.
41 41
42
  Returns `{:ok, shaped}` with four bounded projections, or
42
  Returns `{:ok, shaped}` with six bounded projections, or
43 43
  `{:error, :not_configured | :unavailable}`. Each projection runs as its own
44 44
  HogQL query; a failure of any one fails the whole pull, because partial
45 45
  numbers presented next to each other read as complete.

@@ -50,14 +50,18 @@ defmodule OpenAgents.PostHog do

50 50
      with {:ok, events} <- run(event_counts_sql(), "event_counts"),
51 51
           {:ok, funnel} <- run(funnel_sql(), "funnel"),
52 52
           {:ok, chat} <- run(chat_turns_sql(), "chat_turns"),
53
           {:ok, pages} <- run(top_pages_sql(), "top_pages") do
53
           {:ok, pages} <- run(top_pages_sql(), "top_pages"),
54
           {:ok, triage} <- run(triage_health_sql(), "triage_health"),
55
           {:ok, issue_flow} <- run(weekly_issue_flow_sql(), "weekly_issue_flow") do
54 56
        {:ok,
55 57
         %{
56 58
           generated_at: DateTime.utc_now(),
57 59
           event_counts: shape_rows(events),
58 60
           funnel: shape_rows(funnel) |> List.first(%{}),
59 61
           chat_turns: shape_chat_turns(shape_rows(chat) |> List.first(%{})),
60
           top_pages: shape_rows(pages)
62
           top_pages: shape_rows(pages),
63
           triage_health: shape_triage_health(shape_rows(triage) |> List.first(%{})),
64
           weekly_issue_flow: shape_rows(issue_flow)
61 65
         }}
62 66
      end
63 67
    else

@@ -120,6 +124,89 @@ defmodule OpenAgents.PostHog do

120 124
    |> squash()
121 125
  end
122 126
127
  defp triage_health_sql do
128
    """
129
    WITH issue_events AS (
130
      SELECT
131
        properties.owner AS owner,
132
        properties.repo AS repo,
133
        properties.issue_number AS issue_number,
134
        minIf(timestamp, event = 'issue_created') AS created_at,
135
        argMax(properties.has_labels, timestamp) AS has_labels,
136
        argMax(properties.issue_state, timestamp) AS issue_state
137
      FROM events
138
      WHERE event IN ('issue_created', 'issue_updated')
139
        AND timestamp >= now() - INTERVAL 90 DAY
140
        AND notEmpty(properties.owner)
141
        AND notEmpty(properties.repo)
142
        AND properties.issue_number IS NOT NULL
143
      GROUP BY owner, repo, issue_number
144
    ),
145
    first_responses AS (
146
      SELECT
147
        properties.owner AS owner,
148
        properties.repo AS repo,
149
        properties.issue_number AS issue_number,
150
        min(timestamp) AS responded_at
151
      FROM events
152
      WHERE event = 'issue_commented'
153
        AND timestamp >= now() - INTERVAL 90 DAY
154
        AND properties.is_maintainer = true
155
        AND properties.issue_number IS NOT NULL
156
      GROUP BY owner, repo, issue_number
157
    ),
158
    response_times AS (
159
      SELECT dateDiff('second', issues.created_at, responses.responded_at) AS seconds
160
      FROM issue_events AS issues
161
      INNER JOIN first_responses AS responses
162
        ON issues.owner = responses.owner
163
        AND issues.repo = responses.repo
164
        AND issues.issue_number = responses.issue_number
165
      WHERE responses.responded_at >= issues.created_at
166
    ),
167
    label_health AS (
168
      SELECT
169
        countIf(created_at <= now() - INTERVAL 1 DAY AND issue_state = 'open') AS eligible,
170
        countIf(
171
          created_at <= now() - INTERVAL 1 DAY
172
          AND issue_state = 'open'
173
          AND has_labels = false
174
        ) AS unlabeled
175
      FROM issue_events
176
    )
177
    SELECT
178
      round(median(seconds) / 3600, 1) AS median_first_maintainer_response_hours,
179
      (SELECT eligible FROM label_health) AS eligible_issues,
180
      (SELECT unlabeled FROM label_health) AS unlabeled_issues,
181
      if(
182
        eligible_issues = 0,
183
        0,
184
        round(unlabeled_issues * 100.0 / eligible_issues, 1)
185
      ) AS unlabeled_after_24h_percent
186
    FROM response_times
187
    """
188
    |> squash()
189
  end
190
191
  defp weekly_issue_flow_sql do
192
    """
193
    SELECT
194
      formatDateTime(toStartOfWeek(timestamp), '%Y-%m-%d') AS week,
195
      countIf(event = 'issue_created') AS created,
196
      countIf(
197
        event = 'issue_updated'
198
        AND properties.issue_state_changed = true
199
        AND properties.issue_state = 'closed'
200
      ) AS closed
201
    FROM events
202
    WHERE timestamp >= now() - INTERVAL 8 WEEK
203
      AND event IN ('issue_created', 'issue_updated')
204
    GROUP BY week
205
    ORDER BY week ASC
206
    """
207
    |> squash()
208
  end
209
123 210
  # ── shaping ──────────────────────────────────────────────────────────────
124 211
125 212
  defp shape_rows(%{"results" => results, "columns" => columns}) when is_list(results) do

@@ -143,6 +230,16 @@ defmodule OpenAgents.PostHog do

143 230
    }
144 231
  end
145 232
233
  defp shape_triage_health(row) do
234
    %{
235
      "median_first_maintainer_response_hours" =>
236
        numeric_or_nil(row["median_first_maintainer_response_hours"]),
237
      "eligible_issues" => count_value(row["eligible_issues"]),
238
      "unlabeled_issues" => count_value(row["unlabeled_issues"]),
239
      "unlabeled_after_24h_percent" => numeric_value(row["unlabeled_after_24h_percent"])
240
    }
241
  end
242
146 243
  defp scalar(value) when is_integer(value) or is_float(value), do: value
147 244
  defp scalar(value) when is_binary(value), do: value
148 245
  defp scalar(nil), do: nil

@@ -159,6 +256,12 @@ defmodule OpenAgents.PostHog do

159 256
  defp duration_value(value) when is_float(value), do: round(value)
160 257
  defp duration_value(_other), do: nil
161 258
259
  defp numeric_value(value) when is_integer(value) or is_float(value), do: value
260
  defp numeric_value(_other), do: 0
261
262
  defp numeric_or_nil(value) when is_integer(value) or is_float(value), do: value
263
  defp numeric_or_nil(_other), do: nil
264
162 265
  # ── transport ────────────────────────────────────────────────────────────
163 266
164 267
  defp run(sql, label) do
lib/openagents_web/controllers/api_contract_controller.ex modified +11 -2

@@ -4,9 +4,18 @@ defmodule OpenAgentsWeb.ApiContractController do

4 4
  use OpenAgentsWeb, :controller
5 5
6 6
  @contract_path "priv/api-contracts/repositories-v1.json"
7
  @do_not_build_path "priv/api-contracts/do-not-build-v1.json"
7 8
8 9
  def repositories_v1(conn, _params) do
9
    contract = File.read!(contract_file())
10
    send_contract(conn, @contract_path)
11
  end
12
13
  def do_not_build_v1(conn, _params) do
14
    send_contract(conn, @do_not_build_path)
15
  end
16
17
  defp send_contract(conn, path) do
18
    contract = File.read!(contract_file(path))
10 19
11 20
    conn
12 21
    |> put_resp_content_type("application/json")

@@ -20,7 +29,7 @@ defmodule OpenAgentsWeb.ApiContractController do

20 29
  # release, and that path does not exist in the image that runs it -- so
21 30
  # `File.read!/1` raised and this endpoint answered 500 in every deployed
22 31
  # environment while passing everywhere it was compiled and run together.
23
  defp contract_file, do: Application.app_dir(:openagents, @contract_path)
32
  defp contract_file(path), do: Application.app_dir(:openagents, path)
24 33
25 34
  defp sha256(bytes) do
26 35
    bytes
lib/openagents_web/docs_catalog.ex modified +6

@@ -109,6 +109,12 @@ defmodule OpenAgentsWeb.DocsCatalog do

109 109
          title: "Assignees",
110 110
          icon: "user",
111 111
          route: "/:owner/:repo/assignees"
112
        },
113
        %{
114
          slug: "do-not-build-register",
115
          title: "Do-not-build register",
116
          icon: "info",
117
          route: "/api/contracts/do-not-build-v1.json"
112 118
        }
113 119
      ]
114 120
    },
lib/openagents_web/live/admin_analytics_live.ex modified +57 -5

@@ -81,12 +81,12 @@ defmodule OpenAgentsWeb.AdminAnalyticsLive do

81 81
          <header class="admin-heading">
82 82
            <h1 id="analytics-heading">Product analytics</h1>
83 83
            <p>
84
              Computed live from PostHog over the trailing twenty-four hours. These are
85
              aggregate operational facts about usage; no conversation or memory content
86
              is readable from this page.
84
              Computed live from PostHog. Usage covers the trailing twenty-four hours,
85
              and triage health uses the rolling windows shown below. These are aggregate
86
              operational facts; no conversation or memory content is readable from this page.
87 87
            </p>
88 88
            <div class="admin-totals">
89
              <.badge variant={:info}>TRAILING 24 HOURS</.badge>
89
              <.badge variant={:info}>LIVE POSTHOG</.badge>
90 90
              <.badge
91 91
                :if={@status == :loaded && @overview}
92 92
                variant={:dim}

@@ -128,10 +128,51 @@ defmodule OpenAgentsWeb.AdminAnalyticsLive do

128 128
          </.alert>
129 129
130 130
          <.alert :if={@status == :loading} id="analytics-loading" appearance={:row}>
131
            Querying PostHog for the trailing twenty-four hours…
131
            Querying PostHog…
132 132
          </.alert>
133 133
134 134
          <div :if={@status == :loaded && @overview} class="space-y-8">
135
            <section aria-labelledby="triage-health-heading">
136
              <.card id="analytics-triage-health">
137
                <div class="space-y-1">
138
                  <h2 id="triage-health-heading" class="card-title">Triage health</h2>
139
                  <p class="text-muted-foreground">
140
                    Response and label health use issues created in the trailing 90 days.
141
                    Issue flow covers the trailing eight weeks.
142
                  </p>
143
                </div>
144
                <% triage = @overview.triage_health %>
145
                <div class="grid gap-4 py-5 md:grid-cols-3">
146
                  <div class="rounded-md border border-border p-4">
147
                    <p class="text-sm text-muted-foreground">Median first maintainer response</p>
148
                    <p id="triage-median-response" class="mt-2 text-2xl font-semibold">
149
                      {format_hours(triage["median_first_maintainer_response_hours"])}
150
                    </p>
151
                  </div>
152
                  <div class="rounded-md border border-border p-4">
153
                    <p class="text-sm text-muted-foreground">Unlabeled after 24 hours</p>
154
                    <p id="triage-unlabeled-share" class="mt-2 text-2xl font-semibold">
155
                      {format_percent(triage["unlabeled_after_24h_percent"])}
156
                    </p>
157
                  </div>
158
                  <div class="rounded-md border border-border p-4">
159
                    <p class="text-sm text-muted-foreground">Eligible open issues</p>
160
                    <p id="triage-eligible-issues" class="mt-2 text-2xl font-semibold">
161
                      {triage["eligible_issues"]}
162
                    </p>
163
                    <p class="mt-1 text-sm text-muted-foreground">
164
                      {triage["unlabeled_issues"]} currently have no labels.
165
                    </p>
166
                  </div>
167
                </div>
168
                <.table id="analytics-weekly-issue-flow" rows={@overview.weekly_issue_flow}>
169
                  <:col :let={row} label="Week of">{row["week"]}</:col>
170
                  <:col :let={row} label="Created">{row["created"]}</:col>
171
                  <:col :let={row} label="Closed">{row["closed"]}</:col>
172
                </.table>
173
              </.card>
174
            </section>
175
135 176
            <section aria-labelledby="funnel-heading">
136 177
              <.card id="analytics-funnel">
137 178
                <h2 id="funnel-heading" class="card-title">Activation funnel</h2>

@@ -214,4 +255,15 @@ defmodule OpenAgentsWeb.AdminAnalyticsLive do

214 255
  end
215 256
216 257
  defp format_duration(ms) when is_integer(ms), do: "#{ms}ms"
258
259
  defp format_hours(nil), do: "No data"
260
  defp format_hours(hours) when is_integer(hours), do: "#{hours}h"
261
262
  defp format_hours(hours) when is_float(hours),
263
    do: "#{:erlang.float_to_binary(hours, decimals: 1)}h"
264
265
  defp format_percent(value) when is_integer(value), do: "#{value}%"
266
267
  defp format_percent(value) when is_float(value),
268
    do: "#{:erlang.float_to_binary(value, decimals: 1)}%"
217 269
end
lib/openagents_web/route_authority.ex modified +6 -2

@@ -177,8 +177,12 @@ defmodule OpenAgentsWeb.RouteAuthority do

177 177
  defp policy(%{path: "/api/changelog", verb: verb}) when verb in [:get, :head],
178 178
    do: declaration(:public_read, "anonymous", "published:changelog", false)
179 179
180
  defp policy(%{path: "/api/contracts/repositories-v1.json", verb: verb})
181
       when verb in [:get, :head],
180
  defp policy(%{path: path, verb: verb})
181
       when path in [
182
              "/api/contracts/repositories-v1.json",
183
              "/api/contracts/do-not-build-v1.json"
184
            ] and
185
              verb in [:get, :head],
182 186
       do: declaration(:public_read, "anonymous", "published:api-contract", false)
183 187
184 188
  defp policy(%{path: "/api/v3", verb: verb}) when verb in [:get, :head],
lib/openagents_web/router.ex modified +1

@@ -253,6 +253,7 @@ defmodule OpenAgentsWeb.Router do

253 253
    get "/api/status", NetworkStatusController, :show
254 254
    get "/api/changelog", ChangelogController, :show
255 255
    get "/api/contracts/repositories-v1.json", ApiContractController, :repositories_v1
256
    get "/api/contracts/do-not-build-v1.json", ApiContractController, :do_not_build_v1
256 257
257 258
    post "/controller/pairings", ControllerPairingController, :create
258 259
    get "/controller/pairings/:id", ControllerPairingController, :show
priv/api-contracts/do-not-build-v1.json added +399

@@ -0,0 +1,399 @@

1
{
2
  "contract": "openagents.do-not-build.v1",
3
  "version": 1,
4
  "published_at": "2026-08-23",
5
  "decision_states": [
6
    "retired",
7
    "deferred",
8
    "rejected",
9
    "superseded"
10
  ],
11
  "issue_template": {
12
    "heading": "Do-not-build review",
13
    "fields": [
14
      "Register checked: yes",
15
      "Matching entry: none or DNB-###",
16
      "New evidence: required for reconsideration",
17
      "Decision record: required for reconsideration"
18
    ]
19
  },
20
  "entries": [
21
    {
22
      "id": "DNB-001",
23
      "retired_scope": "Spark or another hosted wallet as the custody layer for agents.",
24
      "match_phrases": [
25
        "hosted spark wallet",
26
        "spark custodial wallet",
27
        "custody spark wallets",
28
        "spark custody service"
29
      ],
30
      "current": {
31
        "state": "superseded",
32
        "decision_date": "2026-08-23",
33
        "evidence": [
34
          {
35
            "source": "docs/episode-triage.md",
36
            "summary": "Episode triage records the move from Spark custody to the self-custodial MoneyDevKit and LDK stack."
37
          }
38
        ],
39
        "violated_principle": "Agents and users retain custody; hosted custody must not become a platform dependency.",
40
        "replacement_path": "Use the MoneyDevKit and LDK self-custodial wallet stack.",
41
        "owner": "OpenAgents product council",
42
        "reconsideration_trigger": "A self-custodial Spark design with production evidence and no hosted custody dependency.",
43
        "decision_record": "https://openagents.com/OpenAgentsInc/openagents.com/issues/93"
44
      },
45
      "history": [
46
        {
47
          "state": "superseded",
48
          "decision_date": "2026-08-23",
49
          "evidence": [
50
            {
51
              "source": "docs/episode-triage.md",
52
              "summary": "Episode triage records the move from Spark custody to the self-custodial MoneyDevKit and LDK stack."
53
            }
54
          ],
55
          "violated_principle": "Agents and users retain custody; hosted custody must not become a platform dependency.",
56
          "replacement_path": "Use the MoneyDevKit and LDK self-custodial wallet stack.",
57
          "owner": "OpenAgents product council",
58
          "reconsideration_trigger": "A self-custodial Spark design with production evidence and no hosted custody dependency.",
59
          "decision_record": "https://openagents.com/OpenAgentsInc/openagents.com/issues/93"
60
        }
61
      ]
62
    },
63
    {
64
      "id": "DNB-002",
65
      "retired_scope": "A NIP-90 data-vending-machine market without a named buyer and working product.",
66
      "match_phrases": [
67
        "nip-90 dvm market",
68
        "nip-90 data vending machine market",
69
        "data vending machine marketplace",
70
        "dvm data market"
71
      ],
72
      "current": {
73
        "state": "retired",
74
        "decision_date": "2026-08-23",
75
        "evidence": [
76
          {
77
            "source": "docs/episode-triage.md",
78
            "summary": "Seller-first markets and protocol-only launches did not establish durable buyer demand."
79
          }
80
        ],
81
        "violated_principle": "Name the buyer and ship usable software before creating a market protocol.",
82
        "replacement_path": "Use the accepted-outcome economy for a verified demand path with a named buyer.",
83
        "owner": "OpenAgents product council",
84
        "reconsideration_trigger": "A committed buyer, executable product flow, and paid end-to-end transaction.",
85
        "decision_record": "https://openagents.com/OpenAgentsInc/openagents.com/issues/93"
86
      },
87
      "history": [
88
        {
89
          "state": "retired",
90
          "decision_date": "2026-08-23",
91
          "evidence": [
92
            {
93
              "source": "docs/episode-triage.md",
94
              "summary": "Seller-first markets and protocol-only launches did not establish durable buyer demand."
95
            }
96
          ],
97
          "violated_principle": "Name the buyer and ship usable software before creating a market protocol.",
98
          "replacement_path": "Use the accepted-outcome economy for a verified demand path with a named buyer.",
99
          "owner": "OpenAgents product council",
100
          "reconsideration_trigger": "A committed buyer, executable product flow, and paid end-to-end transaction.",
101
          "decision_record": "https://openagents.com/OpenAgentsInc/openagents.com/issues/93"
102
        }
103
      ]
104
    },
105
    {
106
      "id": "DNB-003",
107
      "retired_scope": "Bun in the production runtime or package-manager trust path.",
108
      "match_phrases": [
109
        "bun production runtime",
110
        "bun in the production trust path",
111
        "replace npm with bun in production",
112
        "deploy openagents with bun"
113
      ],
114
      "current": {
115
        "state": "rejected",
116
        "decision_date": "2026-08-23",
117
        "evidence": [
118
          {
119
            "source": "docs/episode-triage.md",
120
            "summary": "The production stack standardized on the existing Node and npm asset path."
121
          }
122
        ],
123
        "violated_principle": "Minimize production trust changes that do not improve the product.",
124
        "replacement_path": "Keep Node.js and npm for the browser asset build.",
125
        "owner": "OpenAgents engineering",
126
        "reconsideration_trigger": "A measured production requirement that Node and npm cannot satisfy.",
127
        "decision_record": "https://openagents.com/OpenAgentsInc/openagents.com/issues/93"
128
      },
129
      "history": [
130
        {
131
          "state": "rejected",
132
          "decision_date": "2026-08-23",
133
          "evidence": [
134
            {
135
              "source": "docs/episode-triage.md",
136
              "summary": "The production stack standardized on the existing Node and npm asset path."
137
            }
138
          ],
139
          "violated_principle": "Minimize production trust changes that do not improve the product.",
140
          "replacement_path": "Keep Node.js and npm for the browser asset build.",
141
          "owner": "OpenAgents engineering",
142
          "reconsideration_trigger": "A measured production requirement that Node and npm cannot satisfy.",
143
          "decision_record": "https://openagents.com/OpenAgentsInc/openagents.com/issues/93"
144
        }
145
      ]
146
    },
147
    {
148
      "id": "DNB-004",
149
      "retired_scope": "GetAfter as a second source-control forge.",
150
      "match_phrases": [
151
        "build getafter as a forge",
152
        "getafter second forge",
153
        "revive getafter",
154
        "add another forge as getafter"
155
      ],
156
      "current": {
157
        "state": "superseded",
158
        "decision_date": "2026-08-23",
159
        "evidence": [
160
          {
161
            "source": "docs/episode-triage.md",
162
            "summary": "The OpenAgents forge replaced the shelved GetAfter alternative."
163
          }
164
        ],
165
        "violated_principle": "One canonical forge owns source control and work state.",
166
        "replacement_path": "Extend the OpenAgents forge instead of operating a second forge.",
167
        "owner": "OpenAgents engineering",
168
        "reconsideration_trigger": "A legally or operationally mandatory isolation boundary the current forge cannot provide.",
169
        "decision_record": "https://openagents.com/OpenAgentsInc/openagents.com/issues/93"
170
      },
171
      "history": [
172
        {
173
          "state": "superseded",
174
          "decision_date": "2026-08-23",
175
          "evidence": [
176
            {
177
              "source": "docs/episode-triage.md",
178
              "summary": "The OpenAgents forge replaced the shelved GetAfter alternative."
179
            }
180
          ],
181
          "violated_principle": "One canonical forge owns source control and work state.",
182
          "replacement_path": "Extend the OpenAgents forge instead of operating a second forge.",
183
          "owner": "OpenAgents engineering",
184
          "reconsideration_trigger": "A legally or operationally mandatory isolation boundary the current forge cannot provide.",
185
          "decision_record": "https://openagents.com/OpenAgentsInc/openagents.com/issues/93"
186
        }
187
      ]
188
    },
189
    {
190
      "id": "DNB-005",
191
      "retired_scope": "Paying providers for online presence, uptime, or unverified mining.",
192
      "match_phrases": [
193
        "pay for online presence",
194
        "presence based mining",
195
        "reward nodes for being online",
196
        "mining payments for uptime"
197
      ],
198
      "current": {
199
        "state": "retired",
200
        "decision_date": "2026-08-23",
201
        "evidence": [
202
          {
203
            "source": "docs/episode-triage.md",
204
            "summary": "Episode 224 retired presence-based rewards in favor of verified machine-learning work."
205
          }
206
        ],
207
        "violated_principle": "Settle verified outcomes, not claimed availability.",
208
        "replacement_path": "Pay against accepted work receipts with verification and settlement evidence.",
209
        "owner": "OpenAgents product council",
210
        "reconsideration_trigger": "A fraud-resistant proof that presence itself creates measurable buyer value.",
211
        "decision_record": "https://openagents.com/OpenAgentsInc/openagents.com/issues/93"
212
      },
213
      "history": [
214
        {
215
          "state": "retired",
216
          "decision_date": "2026-08-23",
217
          "evidence": [
218
            {
219
              "source": "docs/episode-triage.md",
220
              "summary": "Episode 224 retired presence-based rewards in favor of verified machine-learning work."
221
            }
222
          ],
223
          "violated_principle": "Settle verified outcomes, not claimed availability.",
224
          "replacement_path": "Pay against accepted work receipts with verification and settlement evidence.",
225
          "owner": "OpenAgents product council",
226
          "reconsideration_trigger": "A fraud-resistant proof that presence itself creates measurable buyer value.",
227
          "decision_record": "https://openagents.com/OpenAgentsInc/openagents.com/issues/93"
228
        }
229
      ]
230
    },
231
    {
232
      "id": "DNB-006",
233
      "retired_scope": "GitHub Copilot as an executor target for delegated work.",
234
      "match_phrases": [
235
        "copilot executor target",
236
        "execute work through github copilot",
237
        "add copilot as an executor",
238
        "copilot coding agent target"
239
      ],
240
      "current": {
241
        "state": "deferred",
242
        "decision_date": "2026-08-23",
243
        "evidence": [
244
          {
245
            "source": "docs/episode-triage.md",
246
            "summary": "The active fleet uses controllable harnesses with explicit execution and receipt contracts."
247
          }
248
        ],
249
        "violated_principle": "Executor targets must expose controllable lifecycle, evidence, and recovery contracts.",
250
        "replacement_path": "Use the Codex, OpenCode, Probe, and owned computer-execution paths.",
251
        "owner": "OpenAgents agent infrastructure",
252
        "reconsideration_trigger": "Copilot exposes the lifecycle, evidence, and recovery contracts required by the fleet.",
253
        "decision_record": "https://openagents.com/OpenAgentsInc/openagents.com/issues/93"
254
      },
255
      "history": [
256
        {
257
          "state": "deferred",
258
          "decision_date": "2026-08-23",
259
          "evidence": [
260
            {
261
              "source": "docs/episode-triage.md",
262
              "summary": "The active fleet uses controllable harnesses with explicit execution and receipt contracts."
263
            }
264
          ],
265
          "violated_principle": "Executor targets must expose controllable lifecycle, evidence, and recovery contracts.",
266
          "replacement_path": "Use the Codex, OpenCode, Probe, and owned computer-execution paths.",
267
          "owner": "OpenAgents agent infrastructure",
268
          "reconsideration_trigger": "Copilot exposes the lifecycle, evidence, and recovery contracts required by the fleet.",
269
          "decision_record": "https://openagents.com/OpenAgentsInc/openagents.com/issues/93"
270
        }
271
      ]
272
    },
273
    {
274
      "id": "DNB-007",
275
      "retired_scope": "A monolithic all-in-one business operating system.",
276
      "match_phrases": [
277
        "monolithic business operating system",
278
        "all-in-one business operating system",
279
        "single business operating system for everything"
280
      ],
281
      "current": {
282
        "state": "rejected",
283
        "decision_date": "2026-08-23",
284
        "evidence": [
285
          {
286
            "source": "docs/episode-triage.md",
287
            "summary": "Current plans organize work as composable issue, project, execution, receipt, and settlement surfaces."
288
          }
289
        ],
290
        "violated_principle": "Compose bounded products around the accepted-outcome loop instead of centralizing every business function.",
291
        "replacement_path": "Build interoperable forge, fleet, verification, receipt, and settlement capabilities.",
292
        "owner": "OpenAgents product council",
293
        "reconsideration_trigger": "Repeated user evidence that a bounded integration cannot satisfy a specific workflow.",
294
        "decision_record": "https://openagents.com/OpenAgentsInc/openagents.com/issues/93"
295
      },
296
      "history": [
297
        {
298
          "state": "rejected",
299
          "decision_date": "2026-08-23",
300
          "evidence": [
301
            {
302
              "source": "docs/episode-triage.md",
303
              "summary": "Current plans organize work as composable issue, project, execution, receipt, and settlement surfaces."
304
            }
305
          ],
306
          "violated_principle": "Compose bounded products around the accepted-outcome loop instead of centralizing every business function.",
307
          "replacement_path": "Build interoperable forge, fleet, verification, receipt, and settlement capabilities.",
308
          "owner": "OpenAgents product council",
309
          "reconsideration_trigger": "Repeated user evidence that a bounded integration cannot satisfy a specific workflow.",
310
          "decision_record": "https://openagents.com/OpenAgentsInc/openagents.com/issues/93"
311
        }
312
      ]
313
    },
314
    {
315
      "id": "DNB-008",
316
      "retired_scope": "A standalone Rust Sarah service.",
317
      "match_phrases": [
318
        "rust sarah service",
319
        "split sarah into a rust service",
320
        "standalone sarah rust service",
321
        "rewrite sarah as a rust service"
322
      ],
323
      "current": {
324
        "state": "superseded",
325
        "decision_date": "2026-08-23",
326
        "evidence": [
327
          {
328
            "source": "docs/decisions/0001-integrate-the-complete-public-application.md",
329
            "summary": "The integrated Phoenix application superseded the separate Sarah service plan."
330
          }
331
        ],
332
        "violated_principle": "Keep the public product integrated unless a measured boundary requires a service split.",
333
        "replacement_path": "Implement Sarah capabilities inside the Phoenix application and its owned execution boundaries.",
334
        "owner": "OpenAgents engineering",
335
        "reconsideration_trigger": "Measured isolation, scaling, or language requirements that cannot be met in the integrated application.",
336
        "decision_record": "https://openagents.com/OpenAgentsInc/openagents.com/issues/93"
337
      },
338
      "history": [
339
        {
340
          "state": "superseded",
341
          "decision_date": "2026-08-23",
342
          "evidence": [
343
            {
344
              "source": "docs/decisions/0001-integrate-the-complete-public-application.md",
345
              "summary": "The integrated Phoenix application superseded the separate Sarah service plan."
346
            }
347
          ],
348
          "violated_principle": "Keep the public product integrated unless a measured boundary requires a service split.",
349
          "replacement_path": "Implement Sarah capabilities inside the Phoenix application and its owned execution boundaries.",
350
          "owner": "OpenAgents engineering",
351
          "reconsideration_trigger": "Measured isolation, scaling, or language requirements that cannot be met in the integrated application.",
352
          "decision_record": "https://openagents.com/OpenAgentsInc/openagents.com/issues/93"
353
        }
354
      ]
355
    },
356
    {
357
      "id": "DNB-009",
358
      "retired_scope": "Claude Code or another rival-owned runtime as a hard platform dependency.",
359
      "match_phrases": [
360
        "claude code hard dependency",
361
        "rival owned runtime hard dependency",
362
        "require claude code runtime",
363
        "depend directly on claude code"
364
      ],
365
      "current": {
366
        "state": "rejected",
367
        "decision_date": "2026-08-23",
368
        "evidence": [
369
          {
370
            "source": "docs/episode-triage.md",
371
            "summary": "The Anthropic OAuth cutoff demonstrated the failure mode of a rival-owned runtime dependency."
372
          }
373
        ],
374
        "violated_principle": "OpenAgents must retain runtime portability and control of critical execution paths.",
375
        "replacement_path": "Keep rival runtimes behind optional adapters and maintain owned or multi-provider execution paths.",
376
        "owner": "OpenAgents agent infrastructure",
377
        "reconsideration_trigger": "A portable open contract with an interchangeable implementation and no vendor-controlled authentication dependency.",
378
        "decision_record": "https://openagents.com/OpenAgentsInc/openagents.com/issues/93"
379
      },
380
      "history": [
381
        {
382
          "state": "rejected",
383
          "decision_date": "2026-08-23",
384
          "evidence": [
385
            {
386
              "source": "docs/episode-triage.md",
387
              "summary": "The Anthropic OAuth cutoff demonstrated the failure mode of a rival-owned runtime dependency."
388
            }
389
          ],
390
          "violated_principle": "OpenAgents must retain runtime portability and control of critical execution paths.",
391
          "replacement_path": "Keep rival runtimes behind optional adapters and maintain owned or multi-provider execution paths.",
392
          "owner": "OpenAgents agent infrastructure",
393
          "reconsideration_trigger": "A portable open contract with an interchangeable implementation and no vendor-controlled authentication dependency.",
394
          "decision_record": "https://openagents.com/OpenAgentsInc/openagents.com/issues/93"
395
        }
396
      ]
397
    }
398
  ]
399
}
priv/docs/creating-issues.md modified +17

@@ -17,3 +17,20 @@ issue form would let a typo become a permanent label.

17 17
18 18
The number is assigned on save and is sequential within the repository. It is
19 19
not global, so two repositories both have an issue 1.
20
21
## Do-not-build review
22
23
Before filing roadmap or backlog work, check the
24
[do-not-build register](/docs/do-not-build-register). Add this block to the
25
issue body:
26
27
```text
28
Do-not-build review
29
- Register checked: yes
30
- Matching entry: none or DNB-###
31
- New evidence: required for reconsideration
32
- Decision record: required for reconsideration
33
```
34
35
A matching entry stays suppressed until new evidence and an explicit decision
36
record are available for manual review.
priv/docs/do-not-build-register.md added +64

@@ -0,0 +1,64 @@

1
# Do-not-build register
2
3
The do-not-build register records product and architecture directions that
4
OpenAgents retired, deferred, rejected, or superseded. It keeps agents and
5
maintainers from repeatedly proposing work whose replacement already exists.
6
7
The public, machine-readable contract is
8
[`/api/contracts/do-not-build-v1.json`](/api/contracts/do-not-build-v1.json).
9
It is the authority for the current decisions, exact matching phrases, and
10
decision history.
11
12
## What each entry records
13
14
Every entry has:
15
16
- A stable `DNB-###` ID.
17
- The scope covered by the decision.
18
- A current decision state: `retired`, `deferred`, `rejected`, or
19
  `superseded`.
20
- The decision date and evidence.
21
- The principle the proposal violated.
22
- The supported replacement path.
23
- An owner and a concrete reconsideration trigger.
24
- An append-only decision history.
25
26
## How screening works
27
28
Screening uses the explicit multi-word phrases in each entry. It does not
29
expand a product name into a broad keyword rule. A proposal that mentions Bun,
30
Spark, Copilot, Rust, or Claude Code for an unrelated reason is not blocked.
31
32
FastFollow integrations can call
33
`OpenAgents.DoNotBuildRegister.screen_fast_follow/2`. A match returns
34
`{:suppressed, entry}` and records `fast_follow_proposal_suppressed` with the
35
entry ID, decision state, and a content fingerprint. Proposal text is not sent
36
to analytics.
37
38
## Reconsider a decision
39
40
Do not reopen matching work from a new assertion alone.
41
42
1. Add evidence that did not exist when the current decision was made.
43
2. Write an explicit decision record.
44
3. Update the register by appending a complete history item and making it the
45
   current decision.
46
4. Review the change before starting implementation.
47
48
Supplying new evidence and a decision-record reference to the FastFollow
49
screen moves the result to `{:review_required, entry}`. It does not silently
50
override the committed register.
51
52
## Initial decisions
53
54
The first register covers:
55
56
- Hosted Spark custody.
57
- NIP-90 data-vending-machine markets without demonstrated demand.
58
- Bun in the production trust path.
59
- GetAfter as a second forge.
60
- Presence-based mining and pay-for-online rewards.
61
- Copilot as an executor target.
62
- A monolithic all-in-one business operating system.
63
- A standalone Rust Sarah service.
64
- Claude Code or another rival-owned runtime as a hard dependency.
test/openagents/do_not_build_register_test.exs added +103

@@ -0,0 +1,103 @@

1
defmodule OpenAgents.DoNotBuildRegisterTest do
2
  use ExUnit.Case, async: false
3
4
  alias OpenAgents.DoNotBuildRegister
5
6
  defmodule AnalyticsSink do
7
    def capture(event, distinct_id, properties) do
8
      send(Application.fetch_env!(:openagents, :do_not_build_test_pid), {
9
        :captured,
10
        event,
11
        distinct_id,
12
        properties
13
      })
14
    end
15
  end
16
17
  setup do
18
    original_token = Application.get_env(:openagents, :posthog_project_token)
19
    original_sink = Application.get_env(:openagents, :analytics_sink)
20
    original_pid = Application.get_env(:openagents, :do_not_build_test_pid)
21
22
    Application.put_env(:openagents, :posthog_project_token, "phc_test")
23
    Application.put_env(:openagents, :analytics_sink, AnalyticsSink)
24
    Application.put_env(:openagents, :do_not_build_test_pid, self())
25
26
    on_exit(fn ->
27
      restore_env(:posthog_project_token, original_token)
28
      restore_env(:analytics_sink, original_sink)
29
      restore_env(:do_not_build_test_pid, original_pid)
30
    end)
31
32
    :ok
33
  end
34
35
  test "the committed register is valid, complete, and history preserving" do
36
    assert {:ok, register} = DoNotBuildRegister.load()
37
    assert length(register["entries"]) == 9
38
39
    assert register["entries"]
40
           |> Enum.map(& &1["current"]["state"])
41
           |> Enum.uniq()
42
           |> Enum.sort() == ~w(deferred rejected retired superseded)
43
44
    assert Enum.all?(register["entries"], fn entry ->
45
             List.last(entry["history"]) == entry["current"]
46
           end)
47
48
    [entry | rest] = register["entries"]
49
50
    invalid =
51
      put_in(register, ["entries"], [put_in(entry, ["current", "state"], "retired") | rest])
52
53
    assert {:error, :invalid_entry} = DoNotBuildRegister.validate(invalid)
54
  end
55
56
  test "matching uses explicit phrases rather than broad product keywords" do
57
    assert %{"id" => "DNB-003"} =
58
             DoNotBuildRegister.match("Replace npm with Bun in production")
59
60
    assert %{"id" => "DNB-008"} =
61
             DoNotBuildRegister.match(%{
62
               title: "Split Sarah into a Rust service",
63
               body: "Prepare the runtime boundary."
64
             })
65
66
    refute DoNotBuildRegister.match("Use Bun for a disposable local benchmark")
67
    refute DoNotBuildRegister.match("Improve the Spark wallet activity table")
68
    refute DoNotBuildRegister.match("Document optional Claude Code adapters")
69
  end
70
71
  test "FastFollow suppresses a match and records only bounded metadata" do
72
    proposal = "Add Copilot as an executor for delegated work"
73
74
    assert {:suppressed, %{"id" => "DNB-006"}} =
75
             DoNotBuildRegister.screen_fast_follow(proposal)
76
77
    assert_receive {:captured, "fast_follow_proposal_suppressed", "system_fast_follow",
78
                    properties}
79
80
    assert properties["register_id"] == "DNB-006"
81
    assert properties["decision_state"] == "deferred"
82
    assert String.length(properties["proposal_fingerprint"]) == 64
83
    refute inspect(properties) =~ proposal
84
  end
85
86
  test "new evidence and a decision record require review instead of bypassing the register" do
87
    proposal = "Build a hosted Spark wallet"
88
89
    assert {:suppressed, %{"id" => "DNB-001"}} =
90
             DoNotBuildRegister.screen_fast_follow(proposal, new_evidence: "custody changed")
91
92
    assert {:review_required, %{"id" => "DNB-001"}} =
93
             DoNotBuildRegister.screen_fast_follow(proposal,
94
               new_evidence: "A self-custodial implementation shipped.",
95
               decision_record: "docs/decisions/0010-spark-wallet.md"
96
             )
97
98
    assert :allow = DoNotBuildRegister.screen_fast_follow("Add project dependency labels")
99
  end
100
101
  defp restore_env(key, nil), do: Application.delete_env(:openagents, key)
102
  defp restore_env(key, value), do: Application.put_env(:openagents, key, value)
103
end
test/openagents/issues_test.exs modified +81

@@ -5,6 +5,17 @@ defmodule OpenAgents.IssuesTest do

5 5
  alias OpenAgents.Issues.Comment
6 6
  alias OpenAgents.Issues.Issue
7 7
8
  defmodule AnalyticsSink do
9
    def capture(event, distinct_id, properties) do
10
      send(Application.fetch_env!(:openagents, :issues_analytics_test_pid), {
11
        :captured,
12
        event,
13
        distinct_id,
14
        properties
15
      })
16
    end
17
  end
18
8 19
  setup do
9 20
    repository = repository_fixture()
10 21

@@ -27,6 +38,73 @@ defmodule OpenAgents.IssuesTest do

27 38
    Issues.get_issue!(repository(), issue.id)
28 39
  end
29 40
41
  describe "triage analytics" do
42
    setup do
43
      original_token = Application.get_env(:openagents, :posthog_project_token)
44
      original_sink = Application.get_env(:openagents, :analytics_sink)
45
      original_pid = Application.get_env(:openagents, :issues_analytics_test_pid)
46
47
      Application.put_env(:openagents, :posthog_project_token, "phc_test")
48
      Application.put_env(:openagents, :analytics_sink, AnalyticsSink)
49
      Application.put_env(:openagents, :issues_analytics_test_pid, self())
50
51
      on_exit(fn ->
52
        restore_env(:posthog_project_token, original_token)
53
        restore_env(:analytics_sink, original_sink)
54
        restore_env(:issues_analytics_test_pid, original_pid)
55
      end)
56
57
      maintainer = repository_user_fixture("triage-maintainer")
58
59
      {:ok, _membership} =
60
        OpenAgents.Repositories.add_member(repository(), maintainer, "maintainer")
61
62
      %{maintainer: maintainer}
63
    end
64
65
    test "issue creation carries stable issue and label state", %{maintainer: maintainer} do
66
      assert {:ok, issue} =
67
               Issues.create_issue(repository(), %{title: "Measure triage"}, maintainer)
68
69
      assert_receive {:captured, "issue_created", _distinct_id, properties}
70
      assert properties["owner"] == repository().owner
71
      assert properties["repo"] == repository().name
72
      assert properties["issue_number"] == issue.number
73
      assert properties["issue_state"] == "open"
74
      assert properties["has_labels"] == false
75
    end
76
77
    test "issue updates identify real state transitions", %{maintainer: maintainer} do
78
      {:ok, issue} = Issues.create_issue(repository(), %{title: "Close me"}, maintainer)
79
      assert_receive {:captured, "issue_created", _, _}
80
81
      assert {:ok, _closed} =
82
               Issues.update_issue(issue, %{"state" => "closed"}, maintainer)
83
84
      assert_receive {:captured, "issue_updated", _distinct_id, properties}
85
      assert properties["issue_number"] == issue.number
86
      assert properties["previous_issue_state"] == "open"
87
      assert properties["issue_state"] == "closed"
88
      assert properties["issue_state_changed"] == true
89
      assert properties["has_labels"] == false
90
    end
91
92
    test "comments record whether the author is a maintainer", %{maintainer: maintainer} do
93
      {:ok, issue} = Issues.create_issue(repository(), %{title: "Respond to me"}, maintainer)
94
      assert_receive {:captured, "issue_created", _, _}
95
96
      assert {:ok, _comment} =
97
               Issues.create_comment(issue, %{body: "Acknowledged"}, maintainer)
98
99
      assert_receive {:captured, "issue_commented", _distinct_id, properties}
100
      assert properties["owner"] == repository().owner
101
      assert properties["repo"] == repository().name
102
      assert properties["issue_number"] == issue.number
103
      assert properties["author_role"] == "maintainer"
104
      assert properties["is_maintainer"] == true
105
    end
106
  end
107
30 108
  describe "list_issues/1" do
31 109
    test "returns only open issues by default" do
32 110
      open = issue_fixture(title: "open one")

@@ -671,4 +749,7 @@ defmodule OpenAgents.IssuesTest do

671 749
  defp milestone_fixture(attrs) do
672 750
    OpenAgents.MilestonesFixtures.milestone_fixture(repository(), attrs)
673 751
  end
752
753
  defp restore_env(key, nil), do: Application.delete_env(:openagents, key)
754
  defp restore_env(key, value), do: Application.put_env(:openagents, key, value)
674 755
end
test/openagents/posthog_test.exs modified +41 -1

@@ -5,6 +5,7 @@ defmodule OpenAgents.PostHogTest do

5 5
6 6
  setup do
7 7
    original = Application.get_env(:openagents, :posthog_analytics)
8
    Application.put_env(:openagents, :posthog_analytics, personal_api_key: nil, project_id: nil)
8 9
9 10
    on_exit(fn ->
10 11
      if original == nil,

@@ -54,7 +55,7 @@ defmodule OpenAgents.PostHogTest do

54 55
      assert {:error, :not_configured} = PostHog.overview()
55 56
    end
56 57
57
    test "shapes the four projections from one pull" do
58
    test "shapes the six projections from one pull" do
58 59
      configure()
59 60
60 61
      Req.Test.expect(__MODULE__, fn conn ->

@@ -112,6 +113,33 @@ defmodule OpenAgents.PostHogTest do

112 113
        })
113 114
      end)
114 115
116
      Req.Test.expect(__MODULE__, fn conn ->
117
        {:ok, body, conn} = Plug.Conn.read_body(conn)
118
        assert body =~ "properties.is_maintainer"
119
        assert body =~ "unlabeled_after_24h_percent"
120
121
        Req.Test.json(conn, %{
122
          "columns" => [
123
            "median_first_maintainer_response_hours",
124
            "eligible_issues",
125
            "unlabeled_issues",
126
            "unlabeled_after_24h_percent"
127
          ],
128
          "results" => [[12.5, 20, 3, 15.0]]
129
        })
130
      end)
131
132
      Req.Test.expect(__MODULE__, fn conn ->
133
        {:ok, body, conn} = Plug.Conn.read_body(conn)
134
        assert body =~ "toStartOfWeek"
135
        assert body =~ "properties.issue_state_changed"
136
137
        Req.Test.json(conn, %{
138
          "columns" => ["week", "created", "closed"],
139
          "results" => [["2026-08-10", 8, 5], ["2026-08-17", 11, 9]]
140
        })
141
      end)
142
115 143
      assert {:ok, overview} = PostHog.overview()
116 144
      assert %DateTime{} = overview.generated_at
117 145

@@ -131,6 +159,18 @@ defmodule OpenAgents.PostHogTest do

131 159
             }
132 160
133 161
      assert [%{"url" => "https://openagents.com/", "views" => 20} | _] = overview.top_pages
162
163
      assert overview.triage_health == %{
164
               "median_first_maintainer_response_hours" => 12.5,
165
               "eligible_issues" => 20,
166
               "unlabeled_issues" => 3,
167
               "unlabeled_after_24h_percent" => 15.0
168
             }
169
170
      assert overview.weekly_issue_flow == [
171
               %{"week" => "2026-08-10", "created" => 8, "closed" => 5},
172
               %{"week" => "2026-08-17", "created" => 11, "closed" => 9}
173
             ]
134 174
    end
135 175
136 176
    test "a rejected key is unavailable, never raised" do
test/openagents_web/controllers/api_contract_controller_test.exs modified +20

@@ -18,4 +18,24 @@ defmodule OpenAgentsWeb.ApiContractControllerTest do

18 18
    assert contract["contract"] == "openagents.repositories.v1"
19 19
    assert contract["version"] == 1
20 20
  end
21
22
  test "do-not-build contract is public, machine-readable, and stable", %{conn: conn} do
23
    response = get(conn, ~p"/api/contracts/do-not-build-v1.json")
24
25
    assert response.status == 200
26
    assert get_resp_header(response, "content-type") == ["application/json; charset=utf-8"]
27
    assert get_resp_header(response, "cache-control") == ["public, max-age=300"]
28
29
    digest =
30
      response.resp_body
31
      |> then(&:crypto.hash(:sha256, &1))
32
      |> Base.encode16(case: :lower)
33
34
    assert get_resp_header(response, "etag") == [~s("#{digest}")]
35
36
    contract = Jason.decode!(response.resp_body)
37
    assert contract["contract"] == "openagents.do-not-build.v1"
38
    assert contract["version"] == 1
39
    assert length(contract["entries"]) == 9
40
  end
21 41
end
test/openagents_web/live/admin_analytics_live_test.exs modified +27 -5

@@ -13,6 +13,7 @@ defmodule OpenAgentsWeb.AdminAnalyticsLiveTest do

13 13
14 14
  setup do
15 15
    original = Application.get_env(:openagents, :posthog_analytics)
16
    Application.put_env(:openagents, :posthog_analytics, personal_api_key: nil, project_id: nil)
16 17
17 18
    on_exit(fn ->
18 19
      if original == nil,

@@ -85,8 +86,8 @@ defmodule OpenAgentsWeb.AdminAnalyticsLiveTest do

85 86
        request_options: [plug: {Req.Test, __MODULE__}]
86 87
      )
87 88
88
      # One pull is four questions.
89
      Req.Test.expect(__MODULE__, 4, fn conn -> respond_by_query(conn) end)
89
      # One pull is six questions.
90
      Req.Test.expect(__MODULE__, 6, fn conn -> respond_by_query(conn) end)
90 91
91 92
      conn = log_in_admin_user(conn, "analytics-loaded")
92 93

@@ -94,16 +95,20 @@ defmodule OpenAgentsWeb.AdminAnalyticsLiveTest do

94 95
      render_async(view)
95 96
96 97
      assert has_element?(view, "#analytics-generated-at")
98
      assert has_element?(view, "#analytics-triage-health")
99
      assert has_element?(view, "#analytics-weekly-issue-flow")
97 100
      assert has_element?(view, "#analytics-funnel")
98 101
      assert has_element?(view, "#analytics-chat-turns")
99 102
      assert html = render(view)
103
      assert html =~ "12.5h"
104
      assert html =~ "15.0%"
100 105
      assert html =~ "$pageview"
101 106
      assert html =~ "https://openagents.com/"
102 107
103 108
      # A second full pull backs the refresh click.
104
      Req.Test.expect(__MODULE__, 4, fn conn -> respond_by_query(conn) end)
109
      Req.Test.expect(__MODULE__, 6, fn conn -> respond_by_query(conn) end)
105 110
106
      assert view |> element("#analytics-refresh") |> render_click() =~ "TRAILING 24 HOURS"
111
      assert view |> element("#analytics-refresh") |> render_click() =~ "LIVE POSTHOG"
107 112
      render_async(view)
108 113
      assert has_element?(view, "#analytics-generated-at")
109 114
    end

@@ -120,7 +125,7 @@ defmodule OpenAgentsWeb.AdminAnalyticsLiveTest do

120 125
    Req.Test.expect(__MODULE__, handler)
121 126
  end
122 127
123
  # The client asks four questions in a fixed order; each stub answers by
128
  # The client asks six questions in a fixed order; each stub answers by
124 129
  # matching the HogQL in the request body rather than relying on call order.
125 130
  defp respond_by_query(conn) do
126 131
    {:ok, body, conn} = Plug.Conn.read_body(conn)

@@ -157,6 +162,23 @@ defmodule OpenAgentsWeb.AdminAnalyticsLiveTest do

157 162
          "results" => [[6, 6, 0, 0, 4525.0, 7414]]
158 163
        })
159 164
165
      body =~ "unlabeled_after_24h_percent" ->
166
        Req.Test.json(conn, %{
167
          "columns" => [
168
            "median_first_maintainer_response_hours",
169
            "eligible_issues",
170
            "unlabeled_issues",
171
            "unlabeled_after_24h_percent"
172
          ],
173
          "results" => [[12.5, 20, 3, 15.0]]
174
        })
175
176
      body =~ "toStartOfWeek" ->
177
        Req.Test.json(conn, %{
178
          "columns" => ["week", "created", "closed"],
179
          "results" => [["2026-08-10", 8, 5], ["2026-08-17", 11, 9]]
180
        })
181
160 182
      true ->
161 183
        Req.Test.json(conn, %{
162 184
          "columns" => ["url", "views"],
test/openagents_web/live/project_workspace_live_test.exs modified +24

@@ -181,6 +181,30 @@ defmodule OpenAgentsWeb.ProjectWorkspaceLiveTest do

181 181
      assert render(view) =~ "Filed from the API"
182 182
    end
183 183
184
    test "updates, closes, reopens, and deletes converge through PubSub", context do
185
      {:ok, view, _html} = live(context.conn, ~p"/projects")
186
187
      {:ok, renamed} = Projects.update_project(context.secret, %{"title" => "Renamed board"})
188
      _ = :sys.get_state(view.pid)
189
      assert render(view) =~ "Renamed board"
190
191
      {:ok, closed} = Projects.update_project(renamed, %{"state" => "closed"})
192
      _ = :sys.get_state(view.pid)
193
      refute render(view) =~ "Renamed board"
194
195
      assert render_patch(view, ~p"/projects?state=closed") =~ "Renamed board"
196
197
      {:ok, reopened} = Projects.update_project(closed, %{"state" => "open"})
198
      _ = :sys.get_state(view.pid)
199
      refute render(view) =~ "Renamed board"
200
201
      assert render_patch(view, ~p"/projects?state=open") =~ "Renamed board"
202
203
      assert {:ok, _deleted} = Projects.delete_project(reopened)
204
      _ = :sys.get_state(view.pid)
205
      refute render(view) =~ "Renamed board"
206
    end
207
184 208
    test "closing out of band moves the row to the closed tab", context do
185 209
      {:ok, view, _html} = live(context.conn, ~p"/projects")
186 210

This page updates live while a promote is in flight · changelog