Compliance-bound fine-tuning

Fine-tune a language model on production data without a single byte leaving your VPC, and produce the audit trail your GRC team needs to sign off on the deployment.

The problem

You have data that would dramatically improve a base model's performance for your domain — clinical notes, support transcripts, financial summaries, legal precedents. You can't:

• Send it to OpenAI / Anthropic / Bedrock for fine-tuning. The pinky-swear data-handling story doesn't pass internal review at most regulated firms.
• Build the self-host stack from scratch. That's six months of platform work before you ship anything: serving engine, training scheduler, eval harness, audit log, secret management, RBAC.

You also need an audit-defensible training run. When the regulator or your CISO asks "what data trained this model and who approved its deployment," the answer can't be "trust us."

The Bigwave shape

Components involved:

Component	What it does in this flow
proxy_headers AuthAdapter	Trusts your existing oauth2-proxy / Authelia / Keycloak SSO; principals come from the upstream X-Auth-Request-* headers
PostgresBackend	Tenant-isolated storage; survives across deploys for audit retention
DatasetRepository	Workspace-scoped dataset registry
TrainingBackend.unsloth	Single-GPU LoRA fine-tuning; subprocess isolation keeps the API process import-free of torch
EvalGate	Refuses adapter activation when the candidate's pass-rate Wilson lower bound regresses below baseline
AuditLogRepository	Append-only record of every dataset upload, training submit, gate decision, adapter activation
WorkspaceTokenBucketLimiter	Per-workspace request throttle (handy when one BU runs a long eval and another is serving prod)

End-to-end walkthrough

1. Deploy Bigwave on-premise

Single GPU host, no outbound network access. Postgres for storage so the audit log persists across container restarts.

git clone https://github.com/bigwaveml/bigwave.git
cd bigwave
./install.sh --non-interactive \
    --profile=SME tier \
    --storage=postgres \
    --auth=proxy_headers \
    --observability=stdout \
    --model=qwen2.5:7b \
    --port=8080

Required env vars (the installer asks for these or you can pre-seed ./.env):

BIGWAVE_STORAGE_URL=postgresql+asyncpg://bigwave:<pw>@db.internal:5432/bigwave
BIGWAVE_AUTH=proxy_headers
BIGWAVE_TRUSTED_PROXY_CIDRS=10.0.0.0/8
BIGWAVE_OBSERVABILITY=stdout
BIGWAVE_LOG_DIR=/var/log/bigwave
BIGWAVE_DATA_DIR=/srv/bigwave/data
BIGWAVE_TRAINING_WORKDIR=/srv/bigwave/training

Front Bigwave with your SSO. The proxy_headers auth adapter trusts identity headers from any peer in BIGWAVE_TRUSTED_PROXY_CIDRS; non-trusted peers get 401.

2. Verify the network posture

The deploy is air-gapped — verify nothing calls out:

# From inside the API container, attempt to reach an external host
docker compose exec api curl -sS -o /dev/null -w "%{http_code}\n" \
    --max-time 5 https://api.openai.com/v1/models
# Expect: timeout / connection refused

# Verify Ollama is local-only
docker compose exec api curl -sS http://ollama:11434/api/tags
# Expect: list of locally pulled models

Document this for your security review packet. The whole point of the on-premise deploy is that the egress posture is trivially verifiable.

3. Upload your training data

Datasets are workspace-scoped. Upload via the admin API; the AuditLogRepository records the action:

curl -H "Authorization: Bearer $BIGWAVE_ADMIN_TOKEN" \
     -F "file=@./private-transcripts.jsonl" \
     "http://localhost:8000/admin/datasets?name=q4-transcripts&kind=jsonl&source=case-management-export"
# {"id": "ds-...", "name": "q4-transcripts", ...}

# Verify the audit row landed:
curl -H "Authorization: Bearer $BIGWAVE_ADMIN_TOKEN" \
     "http://localhost:8000/admin/audit?action=dataset.create&limit=1"
# {"entries": [{"action": "dataset.create", "actor_principal": "alice@your-org.com",
#               "entity": "dataset:ds-...", "ts": "..."}]}

The audit row carries actor_principal (the SSO-resolved identity), workspace_id, and timestamp — the three fields most audit packets ask for.

4. Establish a baseline

Before fine-tuning, score the base model against your golden set so the eval gate has a baseline to compare against:

curl -H "Authorization: Bearer $BIGWAVE_ADMIN_TOKEN" \
     -H "Content-Type: application/json" \
     "http://localhost:8000/admin/evals/runs" \
     -d @./baseline-eval-request.json
# {"run_id": "run-baseline-...", "status": "queued"}

Sample request body:

{
  "suite": "golden_set",
  "dataset_jsonl": "{\"prompt\":\"...\",\"reference\":\"...\"}\n...",
  "target_model": "qwen2.5:7b",
  "target_adapter": null
}

Stream the lifecycle:

curl -H "Authorization: Bearer $BIGWAVE_ADMIN_TOKEN" \
     -H "Accept: text/event-stream" \
     "http://localhost:8000/admin/evals/runs/run-baseline-.../stream"

When the run reports status=succeeded, fetch the report:

curl -H "Authorization: Bearer $BIGWAVE_ADMIN_TOKEN" \
     "http://localhost:8000/admin/evals/runs/run-baseline-.../report"
# {"pass_rate": 0.78, "wilson_ci_lower": 0.71, "wilson_ci_upper": 0.84, ...}

Record the report_id — you'll reference it in the gate policy.

5. Configure the eval gate

The gate refuses adapter activations whose candidate pass-rate Wilson lower bound is below the baseline. Register a policy for the workspace:

# Run from inside the API container or via a thin admin script
from packages.eval.gate import GatePolicy

policy = GatePolicy(
    workspace_id="default",
    scope="adapter_attach",
    baseline_report_id="run-baseline-...",
    evaluation_dataset=open("./golden-set.jsonl").read(),
    scorer="exact_match",
)
# Persist via your GatePolicyRepository implementation

In production this lands behind an admin endpoint (/admin/eval-policies) which is itself audited.

6. Submit the fine-tuning job

curl -H "Authorization: Bearer $BIGWAVE_ADMIN_TOKEN" \
     -H "Content-Type: application/json" \
     "http://localhost:8000/admin/training/jobs" \
     -d '{
       "base_model": "qwen2.5:7b",
       "dataset_id": "ds-...",
       "recipe": "sft",
       "hparams": {"lr": 2e-5, "epochs": 3},
       "eval_split": 0.1
     }'
# {"job_id": "job-..."}

# Stream status
curl -H "Authorization: Bearer $BIGWAVE_ADMIN_TOKEN" \
     -H "Accept: text/event-stream" \
     "http://localhost:8000/admin/training/jobs/job-.../stream"

The training subprocess runs unsloth — never imported in the API process, so the API stays light. The subprocess writes its artefact JSON to disk; the tracker registers the resulting adapter in the AdapterRegistry automatically on success.

The audit log captures the submit and the eventual success.

7. Score the candidate against the golden set

curl -H "Authorization: Bearer $BIGWAVE_ADMIN_TOKEN" \
     -H "Content-Type: application/json" \
     "http://localhost:8000/admin/evals/runs" \
     -d '{
       "suite": "golden_set",
       "dataset_jsonl": "...",
       "target_model": "qwen2.5:7b",
       "target_adapter": "adp-..."
     }'

Wait for status=succeeded, fetch the report. Compare to the baseline. The gate will run the same comparison automatically when you activate; this is just for sanity.

8. Activate (the gate decides)

curl -H "Authorization: Bearer $BIGWAVE_ADMIN_TOKEN" \
     -H "Content-Type: application/json" \
     "http://localhost:8000/admin/models/qwen2.5:7b/adapters" \
     -d '{"adapter_id": "adp-..."}'

Two outcomes:

• Allow. Response is 200 with swap_ms + new_handle_id. The audit log records the activation. Chat from now on hits the new adapter.
• Block. Response is 422 with the gate's reason ("eval gate blocked activation for adapter_id='adp-...': candidate pass_rate Wilson upper bound (0.71) is below baseline lower bound (0.74)"). The old handle stays live. The audit log records the refusal.

This is the bit your GRC team cares about: the gate is binding. A regression cannot ship even if a misclick says "Activate."

9. Pull the audit packet

For an audit, export every row that touched this adapter:

curl -H "Authorization: Bearer $BIGWAVE_ADMIN_TOKEN" \
     "http://localhost:8000/admin/audit?entity=adapter:adp-...&limit=500"

Returns the full lifecycle: dataset upload → training submit → training success → eval runs → activation decision (allow or block). Every row carries actor_principal (SSO email) and ts (UTC).

That's the artefact you ship to your auditor.

What you'll have at the end

• A fine-tuned adapter living entirely on your GPU host, never having touched any external service.
• A baseline eval report and a candidate eval report, both workspace-scoped, with Wilson 95% CIs.
• A binding gate decision that refused or allowed the activation.
• An append-only audit log of every action, attributable to the SSO-resolved actor.
• A reproducible runbook your team can repeat for the next fine-tune.

What's not covered here (explicitly)

• Differential privacy / encrypted gradients. If your audit requires DP, layer it on top via your training subprocess — Bigwave doesn't ship DP guarantees out of the box.
• BAA / DPA / regulatory paperwork. That's between you and the regulator; Bigwave is the technical substrate, not legal cover.
• Multi-region replication. Single-Postgres deploys do not survive a region outage. For HA, add streaming replication to Postgres + a load-balanced API tier.
• Model-license review. You're responsible for verifying the base model's licence permits your use case (commercial, internal, derivative weights).

See also

• proxy_headers auth adapter
• Workspace fence concept
• Eval-as-a-gate concept
• Postgres backend