Shadow mode

Toggle a secondary model that the firstparty gateway fans out to in the background after every chat completion. The shadow response is persisted on inference_log with shadow_of = <primary_completion_id> (migration 0015_inference_log_shadow.py) so operators can compare candidate models against the production model without affecting the user-facing latency or output.

The router lives at packages/serving/admin_routes.py (the shadow_router, defined alongside the model-replica admin routes). See also the 13 — Multi-model + shadow how-to.

Endpoints

Method	Path	Auth	Summary
GET	/admin/shadow	Bearer	Return the currently-shadowed model id (or null).
POST	/admin/shadow	Bearer	Set the shadow target; model must already be loaded.
DELETE	/admin/shadow	Bearer	Clear the shadow target — disable fan-out.

GET /admin/shadow

Returns the active shadow target stored on app.state.shadow_model_id. model_id is null when fan-out is disabled.

{ "model_id": "qwen2.5:7b" }

POST /admin/shadow

Set the shadow target. Body schema: SetShadowRequest (packages/serving/admin_routes.py:87).

{ "model_id": "qwen2.5:7b" }

Returns the same shape as GET /admin/shadow.

The target must already be loaded (visible in GET /admin/models/loaded); the route returns 422 otherwise so the operator gets a clear error rather than silent fan-out failures at request time. Load with POST /admin/models/load first.

curl -X POST -H "Authorization: Bearer $BIGWAVE_ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"model_id": "qwen2.5:7b"}' \
  http://localhost:8000/admin/shadow

DELETE /admin/shadow

Clears the shadow target. Returns {"model_id": null}. Pre-existing shadow rows in inference_log are not affected — historical comparisons remain queryable.

Operator notes

• Single shadow only. Only one shadow target is supported per process; fan-out cost is linear and the v0 design caps at one. The legacy BIGWAVE_SHADOW_MODELS env var seeds the same slot at startup but is overwritten by any subsequent POST /admin/shadow.
• Workspace scoping. The admin surface is single-tenant — the shadow toggle lives on app.state, not in the database. Multi-tenant workspaces share the same shadow target.
• Billing. Shadow rows land in billing_ledger with billable=False (migration 0019_billing_ledger.py) so the background fan-out does not appear in tenant cost rollups.
• Error codes. 422 — model is empty or not loaded; 503 — ReplicaManager not initialised on app.state.
• Comparison query. Once shadow rows accumulate, join inference_log to itself on shadow_of = id to produce the side-by-side response set powering the Web UI's compare panel.