Make boot convergence understand active rolling replacements #25

Open AtlantisPleb opened this 7h ago

Problem\n\nA node that boots into a new immutable image during a rolling replacement still treats the previous live Forge target as the only valid identity. Boot convergence then attempts to converge the new node backward toward the old target and can keep readiness false. The 2026-08-22 production rollout required a temporary, operator-controlled boot-convergence disable while the three nodes rolled, followed by restoring the flag and restarting OpenAgents.Forge.BootConverge after Forge recorded the new target as live.\n\n## Required behavior\n\n- Publish the authorized rolling target identity before the first replacement node boots.\n- Let boot convergence accept the current live identity or the active, authority-bound rolling target for that node.\n- Keep a node out of service when it matches neither identity.\n- Settle the target to live only after every node reports the exact SHA and image digest.\n- Remove the need to toggle OPENAGENTS_FEATURE_BOOT_CONVERGENCE or restart the convergence child manually.\n- Preserve strict rollback and interrupted-roll recovery.\n\n## Acceptance criteria\n\n- A three-node rolling integration test starts with boot convergence enabled and replaces every node without an operator flag change.\n- The test proves load-balancer health, quorum, exact SHA and image digest, target settlement, and restart behavior.\n- Controlled failure before and after target settlement leaves a recoverable, auditable state.\n- The production runbook documents the automatic path and no longer requires the temporary workaround.\n\n## Evidence\n\nThe production rollout of 45f6fff3222c432f248ca831a6fbc882c0fc6206 exposed this gap. Final deployment receipt: f6883f15-2d6e-4705-ac39-01eeaf21d0a6.

  1. AtlantisPleb opened this issue 7h ago
Sign in with GitHub to comment on this issue.