Datasets

POST/GET/DELETE /admin/datasets — workspace-scoped datasets used by the training and eval surfaces. Datasets are the input side of the chat → feedback → DPO → eval → promote loop (PRD 05): training backends consume them via the registered dataset_id; the golden_set eval suite consumes the same registry.

Datasets are content-addressed: identical bytes produce the same SHA-256 digest, so re-uploading the same file is a no-op rather than a duplicate. Format inference is automatic for JSONL, CSV, Parquet, and Argilla-exported JSONL.

Endpoints

Method	Path	Auth	Summary
POST	/admin/datasets	Bearer + workspace	Multipart upload; registers dataset.
GET	/admin/datasets	Bearer + workspace	List the caller workspace's datasets.
GET	/admin/datasets/{dataset_id}	Bearer + workspace	Single dataset; 404 if missing or cross-tenant.
GET	/admin/datasets/{dataset_id}/rows	Bearer + workspace	Stream rows (paginated).
GET	/admin/datasets/{dataset_id}/export/argilla	Bearer + workspace	Argilla-format JSONL export.
DELETE	/admin/datasets/{dataset_id}	Bearer + workspace	Delete the dataset; 204 on success.

POST /admin/datasets

Multipart upload. Form fields:

• file — the dataset bytes. JSONL / CSV / Parquet / Argilla JSONL.
• kind — sft or dpo (the dataset family the training backend consumes).
• name — operator-chosen display name.

Response: 200 OK with the registered Dataset record (id, content-hash, row count, kind, created_at). The id is the content-addressed digest, so re-uploading the same file returns the same id.

curl -X POST https://your-host/admin/datasets \
  -H "Authorization: Bearer $BIGWAVE_ADMIN_TOKEN" \
  -F "file=@./preferences.jsonl" \
  -F "kind=dpo" \
  -F "name=support-preferences-v3"

GET /admin/datasets

Lists every dataset registered in the caller's workspace. Cross-tenant datasets are not enumerated (workspace fence).

curl https://your-host/admin/datasets \
  -H "Authorization: Bearer $BIGWAVE_ADMIN_TOKEN"

GET /admin/datasets/{dataset_id}

Returns one dataset. 404 for an unknown id and for cross-tenant access (not 403 — the workspace fence treats both cases identically to avoid leaking the existence of another tenant's dataset).

GET /admin/datasets/{dataset_id}/rows

Paginated row stream. Use this to spot-check what a training run is actually consuming. Query parameters:

• offset — starting row index (default 0).
• limit — rows to return (default 100, max 1000).

GET /admin/datasets/{dataset_id}/export/argilla

Streams the dataset in Argilla's JSONL shape — useful when the operator wants to ship a copy to an Argilla labelling instance for human-in-the-loop curation before re-uploading the cleaned version.

DELETE /admin/datasets/{dataset_id}

Removes the dataset registration and its bytes. Returns 204 No Content. Training jobs that reference the dataset by id keep their historical artifact reference but cannot be re-run against the deleted bytes.

Operator notes

• Workspace fence. All endpoints filter by the caller's workspace. Cross-tenant requests get 404; lists never include another workspace's datasets.
• Content-addressed dedup. Two operators in the same workspace uploading the same bytes get the same dataset_id. Mutating an existing dataset is not supported — upload a new file, delete the old one.
• Format inference. The server peeks the first few bytes; mixed formats inside one file return 422 Unprocessable Entity.
• Errors:
  • 400 Bad Request — missing required form field, unparseable format.
  • 401 Unauthorized — missing bearer.
  • 403 Forbidden — bearer is valid but lacks workspace scope.
  • 404 Not Found — unknown dataset id (or cross-tenant).
  • 413 Payload Too Large — file exceeds operator-configured cap.
  • 422 Unprocessable Entity — file shape unsupported.

Related

• how-to/04-datasets — operator runbook for uploading and curating a dataset.
• admin/adapters — the downstream surface that consumes datasets via the training pipeline.
• admin/training — the training-job submit endpoint that reads dataset_id from this surface.
• PRD 05 — the contract this surface implements.