multi-simadmin/docs/production-gateway-cutover.md

Production gateway cutover and rollback

The browser gateway normally runs as a canary on loopback port 8789. Port 8788 remains reserved: neither pnpm canary nor CANARY_PORT=8788 pnpm canary can claim it. The only owner is the separate pnpm gateway:production command, and it exits before listening unless:

MULTI_SIMADMIN_CUTOVER_ACK=I ACKNOWLEDGE MULTI-SIMADMIN OWNS PORT 8788

Do not place that acknowledgement in a persistent dotenv file. Supply it only through the reviewed cutover operation.

Plan and evidence

Create a mode-0600 plan outside the repository. It names evidence, not secrets:

{
  "assetDir": "/absolute/release/apps/web/dist",
  "backupEvidence": "/absolute/evidence/backup-verified.txt",
  "backupEvidenceSha256": "<lowercase SHA-256 of backup evidence>",
  "drillEvidence": "/absolute/evidence/rollback-drill-passed.txt",
  "drillEvidenceSha256": "<lowercase SHA-256 of drill evidence>",
  "legacyPid": 1234,
  "legacyCommand": "/usr/local/bin/node /absolute/legacy/server.js",
  "legacyIdentity": "<process start identity from ps -o lstart=>",
  "legacyStart": ["/usr/local/bin/node", "/absolute/legacy/server.js"],
  "stateDir": "/absolute/operator-state"
}

legacyCommand must exactly equal ps -p PID -o command= and legacyStart must reproduce that command exactly. The executable and all paths must be absolute. The evidence files must be regular files whose contents match the reviewed SHA-256 values. assetDir/index.html must be a regular file. The plan must not contain secrets or extra command fields.

Procedure

1. Build/release assets and start the API on loopback 8790 by the established production procedure.
2. Verify backup restoration and perform a rollback drill; write the two evidence files.
3. Record the legacy listener PID and its exact command. Never reuse a stale plan after a restart.
4. Export MULTI_SIMADMIN_GATEWAY_TOKEN only in the operator process environment. Run corepack pnpm cutover preflight /absolute/plan.json. This performs authenticated API /readyz on 8790, asset and evidence digest checks, and exact PID command ownership. The token is neither logged nor persisted.
5. In the approved window run corepack pnpm cutover cutover /absolute/plan.json once. Before SIGTERM it durably fsyncs mode-0600 prepared state. It waits for the verified legacy PID to exit, starts the production gateway, verifies that exact PID owns 8788 and passes authenticated readiness, then durably records active state. If gateway verification fails, it automatically stops the attempted gateway and restarts the exact legacy argv.
6. Verify http://127.0.0.1:8788/healthz, the UI, API reads, and an approved low-risk operation. Retain the state file.

Rollback

Run corepack pnpm cutover rollback /absolute/plan.json. Rollback first validates the strict state schema, mode-0600 permissions, agreement with the reviewed plan, exact recorded PID command, and 8788 ownership. It refuses to signal any other process. It sends SIGTERM, waits, and restores the exact saved legacy argv without shell interpretation.

If any identity check fails, stop and investigate manually. Never kill the PID, edit the state to fit a process, use SIGKILL, or start a second listener. The orchestrator deliberately does not access configuration or keychain material and does not modify the legacy server, public tree, or contracts.