> ## Documentation Index
> Fetch the complete documentation index at: https://docs.datafdn.org/llms.txt
> Use this file to discover all available pages before exploring further.
# Trace Integration Guide
> How any data provider registers records, submits metadata updates, and reads public audit views through the Trace API.
# Trace Provider Data Audit — Integration & API Reference
The single source of truth for the Trace data audit API: provider onboarding, auth, the Trace Schema, write and read endpoints, search, stats, scoped groups, limits, and validation.
The API and Trace Schema are provider-agnostic — every provider integrates through the same endpoints, headers, and schema. Examples use **Kled**, a live provider; substitute your own provider identity wherever Kled appears.
## Environments
| Environment | Base URL | Auth |
| -------------- | --------------------------------------- | ----------------------------------------------------- |
| **Staging** | `https://staging-api.storyprotocol.net` | Writes gated by staging API key. Reads are public. |
| **Production** | `https://api.dataapis.io` | Writes gated by production API key. Reads are public. |
Staging is where new provider integrations are onboarded and tested. Production is deployed but may not yet have provider data — check with the DATA Foundation team before sending production traffic.
## Getting access
Write access is gated. Before integrating, contact the DATA Foundation team to be onboarded: we whitelist your provider identity, issue staging and production API keys, and assign the `X-Provider` value your write requests must use. Until that is done, write requests are rejected. Read, search, and scoped-group endpoints are public audit views and do not require a key.
## Auth and provider scope
Every write request must include:
```text theme={null}
X-API-Key:
X-Provider: kled
X-Batch-Id:
```
* The API key must belong to your provider and match `X-Provider`; use the key for the environment you're targeting.
* For backlog ingestion, add `X-Ingestion-Source: backlog`. For live ingestion, omit the header — explicit `live` is not accepted.
* Read APIs are keyed by the global `data_id`; `provider` is returned as a field and doubles as an optional filter.
Read, search, and scoped-group endpoints are **public audit views**. Do not put enterprise-only or sensitive fields into the public Trace payload. If a provider-only or partner-only read tier is ever needed, that is a separate API/product decision.
## Ingestion flow
The provider client POSTs to a webhook batch endpoint.
Accepted items are enqueued and processed asynchronously by the DATA Foundation.
Records become available through the read and search endpoints.
A `records:batch` response always includes per-item statuses:
* `202 Accepted` — at least one item was accepted for asynchronous processing and no item conflicted.
* `200 OK` — every item was a duplicate, so no new async work was enqueued.
* `409 Conflict` — at least one item conflicted; any `accepted` items in the same response were still enqueued.
Inspect item statuses, and retry transient request failures with the same payload and the same `X-Batch-Id`.
## Record lifecycle
Each record's history is an append-only event log that Trace renders as a timeline. Each step maps to exactly one value; a step appears on the timeline when its value is present.
| # | Step | Value |
| - | ----------------- | ---------------------------------------------------------------------------------- |
| 1 | Captured | `timestamps.captured_at` |
| 2 | Uploaded | `timestamps.uploaded_at` |
| 3 | Attested | `attestation.signed_at_utc` |
| 4 | Registered | `ingested_at` of the `DataRegistered` event (`seq: 0`) — DATA Foundation-generated |
| 5 | Anchored on-chain | `tx_hash` (empty string until the DATA Foundation broadcasts) |
| 6 | Updated | Root-level `occurred_at` of each `MetadataUpdated` event (`seq` 1–100) |
| 7 | Payment credited | `timestamps.payment_credited_at` — stored today, surfaced on the timeline soon |
Steps 1–3 come from the registration payload's metadata; steps 4–7 are audit events. Send each milestone as its own metadata update — one milestone, one timeline event.
The root `occurred_at` on each batch item is required, provider-supplied, and dates that specific event — registration time on the `DataRegistered` event, update time on each `MetadataUpdated` event. Trace surfaces the **Registered** lifecycle step with its own `ingested_at` — the moment the DATA Foundation accepted the record, returned on per-record reads. `timestamps.originated_at` inside the attested payload carries the record's origin moment: set it equal to the registration's root `occurred_at`, and keep it fixed on later updates (each update's root `occurred_at` dates the update itself). Records registered before this field was renamed carry the same value under the legacy name `timestamps.occurred_at` — readers should accept both. The only values the DATA Foundation adds are `ingested_at` and `tx_hash`.
## Trace Schema v1.0
Populate the standardized Trace Schema fields directly and preserve your full original public payload under `provider_payload`. The normalized fields are the portable contract the frontend and audit flows use; nothing provider-specific is lost.
* Use `schema_version: trace-v1.0` in `initial_metadata_json` and `metadata_json`. The value is not validated — a missing or unparseable `schema_version` defaults to `trace-v1.0`, and whatever value is sent is folded into the event hash.
* The DATA Foundation canonicalizes metadata JSON before computing internal event hashes, so object key order does not affect idempotency or conflict detection.
* Do not send a transaction hash in write payloads. The DATA Foundation owns `tx_hash` and returns it on read responses for registration, metadata update, and search result rows — as an empty string until the DATA Foundation fills it after broadcast.
### Canonical content hash (required)
Initial registrations and metadata updates must include one canonical content hash. Accepted fields are `asset.hash`, `content_hash`, `file.content_sha256`, or `file.hashes.sha256`; all normalize to `sha256:<64-lowercase-hex>`. If more than one alias is present, they must represent the same hash.
### Recommended top-level shape
```json theme={null}
{
"schema_version": "trace-v1.0",
"file": {
"content_sha256": "sha256:<64-hex>",
"mime_type": "video/mp4",
"media_category": "video",
"size_bytes": 123456,
"hashes": {
"phash64": "facebeef01234567",
"dhash64": "1b9072d44a8be3c1",
"ahash64": "ff7e3c1a90d5e8c2",
"keyframe_phashes": [
"0011223344556677",
"8899aabbccddeeff"
]
},
"behavior": {
"captured_at_utc": "2026-05-13",
"uploaded_at_utc": "2026-05-13",
"capture_to_upload_seconds": 421,
"capture_to_upload_bucket": "5-60min",
"upload_session_size": 3,
"upload_session_kind": "gallery_pick",
"captured_via": "ios_native_camera",
"uploaded_via": "ios_app",
"client_version": "ios-1.42.0"
}
},
"file_specific": {
"base": {
"motion": {
"compass_heading": 247.3,
"compass_heading_reference": "true_north",
"speed_bucket": "stationary"
}
},
"video": {},
"image": {},
"document": {}
},
"asset": {
"collection_id": "kled-collection-2026-05",
"customer_id": "kled-customer-001",
"task_id": "kled-task-0042"
},
"contributor": {
"anon_id": "kled-public-user-id",
"kyc_status": "verified",
"kyc_country": "US",
"geo_region": "US",
"tax_status": "submitted",
"account_verification_status": "verified",
"consent": {
"tos_version": "2026-05-20",
"tos_hash": "sha256:<64-hex-policy-hash>",
"tos_uri": "https://kled.ai/terms/2026-05-20",
"privacy_policy_version": "2026-05-20",
"privacy_policy_hash": "sha256:<64-hex-policy-hash>",
"privacy_policy_uri": "https://kled.ai/privacy/2026-05-20"
}
},
"app": {
"platform_name": "kled.ai",
"legal_entity": "Nitrility Inc. (Delaware, USA)"
},
"timestamps": {
"originated_at": "2026-05-13T00:00:00Z",
"uploaded_at": "2026-05-13T00:00:00Z",
"captured_at": "2026-05-12T23:59:00Z",
"payment_credited_at": "2026-05-20T12:00:00Z"
},
"attestation": {
"payload_hash": "sha256:",
"signature": "optional-signature",
"key_id": "optional-key-id",
"key_url": "https://kled.ai/.well-known/verification-keys.json",
"signed_at_utc": "2026-05-13T00:00:02Z"
},
"provider_payload": {
"...": "full provider public payload"
}
}
```
### Searchable fields
Current exact-match searchable fields:
| Field | Aliases |
| ------------------------- | ---------------------------------------------------------------------------------- |
| `provider` | — |
| `source_record_id` | `media_id_public` (when present in metadata) |
| `file.content_sha256` | `asset_hash`, `asset.hash`, `content_hash`, `content_sha256`, `file.hashes.sha256` |
| `file.mime_type` | `mime_type`, `mimetype`, `file.mimetype` |
| `file.media_category` | `media_category` |
| `contributor.anon_id` | — |
| `collection_id` | `asset.collection_id` |
| `customer_id` | `asset.customer_id` |
| `task_id` | `asset.task_id` |
| `contributor.kyc_status` | — |
| `contributor.kyc_country` | — |
| `contributor.geo_region` | — |
| `tos_hash` | `contributor.consent.tos_hash` |
| `privacy_policy_hash` | `contributor.consent.privacy_policy_hash` |
The only searchable hash is the canonical content SHA-256 — perceptual hashes (`phash64`, `dhash64`, `ahash64`, `keyframe_phashes`) and `md5` are stored but not indexed. The same goes for `app.platform_name`, `app.legal_entity`, `file.behavior.*`, `file_specific.base.motion.*`, and `timestamps.payment_credited_at`: send them when useful — stored fields can be indexed later without resending old data. Use `/stats` for distributions and `provider` as an optional query scope.
### Field guidance
* `source_record_id` — provider-owned stable public media ID (for example `kmf_...`). The service trims surrounding whitespace, preserves case, rejects control characters, and accepts up to 512 bytes.
* `contributor.anon_id` — provider-owned public anonymized contributor ID.
* `asset.collection_id`, `asset.customer_id`, `asset.task_id` — optional provider-assigned grouping IDs: a collection or batch of records, the customer or campaign they were produced for, and the task they belong to. Each is exact-match searchable, and matching is case-sensitive like `source_record_id`. Send them at the top level (as in the shape above) or inside `provider_payload` — both are read. Omit any that don't apply.
* `contributor.kyc_status` — recommended values: `verified`, `pending`, `failed`, `unverified`.
* `contributor.kyc_country` — ISO 3166-1 alpha-2 country code from KYC, if available. Country only; no address or GPS-derived country.
* `contributor.tax_status` — recommended values: `submitted`, `not_submitted`, `not_applicable`, `unknown`. For a boolean like `tax_form_on_file`, map `true` to `submitted` and `false` to `not_submitted`.
* `contributor.account_verification_status` — recommended values: `verified`, `pending`, `failed`, `unverified`.
* `contributor.consent.tos_*` and `contributor.consent.privacy_policy_*` — the exact policy version, hash, and URI the contributor accepted for this record. Provider-level *active* policies are set separately through `PUT /webhook/v1/data-audit/provider-policy`, never inside record payloads.
* `attestation` — signature is optional on staging. For production verification, send `payload_hash`, `signature`, `key_id`, `key_url`, and `signed_at_utc`.
* `app.legal_entity` — legal counterparty information alongside the app/platform name.
* `file.behavior` — non-PII capture/upload behavior signals.
* `file_specific.base.motion` — shared motion signals that apply across media types.
* `timestamps.payment_credited_at` — optional UTC timestamp for when the provider finalized and credited the contributor's payment. Omit when nothing was credited (or the payout was forfeited); if payment lands after registration, send it as a full-state metadata update. Stored-only.
| Version | Value | Purpose |
| --------------------------- | ------------------------ | ------------------------------------------------------------------------------------- |
| Trace schema | `trace-v1.0` | Normalized metadata shape expected in `initial_metadata_json` and `metadata_json`. |
| Event hash | `event-hash-v1` | Version of the internal audit event hash envelope. |
| Event hash canonicalization | `json-canonical-v1` | JSON is normalized before hashing so object key order does not affect the event hash. |
| Read model schema | `read-model-v1` | Version embedded in read-model partition keys. |
| Stats shard schema | `shard-v2` | Version embedded in sharded stats counter partition keys. |
| Search index schema | `postgres-read-model-v1` | Version stamped on search index rows returned by `/search`. |
Stored on durable and index rows. Future changes should dual-write/dual-read versions during transition rather than mutating history.
## Write API
### Set active provider policies
Use this endpoint when the provider publishes a new Terms of Service or Privacy Policy. Trace stores the active version, hash, and URI so the frontend can link to the policy and stats can compare records against it.
Policy rows are audit-only — not indexed or aggregated. Record payloads still carry `contributor.consent.*`; those record-level references are what `/stats`, scoped-group summaries, and policy-hash search use.
```http theme={null}
PUT /webhook/v1/data-audit/provider-policy
Content-Type: application/json
X-API-Key:
X-Provider: kled
```
```json theme={null}
{
"tos": {
"version": "2026-06-01",
"hash": "sha256:<64-hex-policy-hash>",
"uri": "https://kled.ai/terms/2026-06-01",
"effective_at": "2026-06-01T00:00:00Z"
},
"privacy_policy": {
"version": "2026-06-01",
"hash": "sha256:<64-hex-policy-hash>",
"uri": "https://kled.ai/privacy/2026-06-01",
"effective_at": "2026-06-01T00:00:00Z"
}
}
```
Both the `tos` and `privacy_policy` objects are required, each with `version`, `hash`, and `uri`; `effective_at` is optional. The hash accepts the same forms as asset hashes (`sha256:<64-hex>`, bare hex, `0x<64-hex>`, `0x1220<64-hex>`) and is stored canonically as `sha256:<64-lowercase-hex>`. The request body is capped at `1 MiB`.
Read current and historical policy documents:
```http theme={null}
GET /api/v1/data-audit/providers/kled/policy
GET /api/v1/data-audit/providers/kled/policies/tos/sha256:<64-hex-policy-hash>
GET /api/v1/data-audit/providers/kled/policies/privacy_policy/sha256:<64-hex-policy-hash>
```
### Register records
Use this endpoint for initial backlog and live registration batches. The provider sends its stable `source_record_id`; the DATA Foundation generates the `data_id` and returns the mapping.
```http theme={null}
POST /webhook/v1/data-audit/records:batch
Content-Type: application/json
X-API-Key:
X-Provider: kled
X-Batch-Id: kled-records-000001
X-Ingestion-Source: backlog
```
The request body is a JSON array. `initial_metadata_json` follows the [recommended top-level shape](#recommended-top-level-shape) above (abbreviated here):
```json theme={null}
[
{
"source_record_id": "kmf_8a9c2e7d4b1f0e23",
"initial_metadata_root": "sha256:",
"initial_metadata_json": {
"schema_version": "trace-v1.0",
"file": {
"content_sha256": "sha256:<64-hex>",
"mime_type": "video/mp4",
"media_category": "video",
"size_bytes": 123456
},
"file_specific": {
"video": { "duration_ms": 120000, "width": 1920, "height": 1080 }
},
"contributor": {
"anon_id": "kup_123",
"kyc_status": "verified",
"kyc_country": "US",
"geo_region": "US",
"tax_status": "submitted",
"account_verification_status": "verified",
"consent": {
"tos_version": "2026-05-20",
"tos_hash": "sha256:<64-hex-policy-hash>",
"tos_uri": "https://kled.ai/terms/2026-05-20",
"privacy_policy_version": "2026-05-20",
"privacy_policy_hash": "sha256:<64-hex-policy-hash>",
"privacy_policy_uri": "https://kled.ai/privacy/2026-05-20"
}
},
"app": {
"platform_name": "kled.ai",
"legal_entity": "Nitrility Inc. (Delaware, USA)"
},
"timestamps": {
"originated_at": "2026-05-13T00:00:00Z",
"uploaded_at": "2026-05-13T00:00:00Z"
},
"attestation": {
"payload_hash": "sha256:",
"signature": "optional-on-staging",
"key_id": "kled-verify-2026-q1",
"key_url": "https://kled.ai/.well-known/verification-keys.json",
"signed_at_utc": "2026-05-13T00:00:02Z"
},
"provider_payload": {
"media_id_public": "kmf_8a9c2e7d4b1f0e23"
}
},
"occurred_at": "2026-05-13T00:00:00Z"
}
]
```
`initial_metadata_root` should be the provider's deterministic non-zero hash of the canonical Trace Schema v1.0 metadata JSON. The DATA Foundation stores this value as submitted; hash verification against `initial_metadata_json` is not enforced yet.
Successful response (`202` when new items were enqueued, `200` when every item was a duplicate, `409` when any item conflicted):
```json theme={null}
{
"request_id": "story-request-uuid",
"provider": "kled",
"batch_id": "kled-records-000001",
"format": "json",
"kind": "records",
"records": 1,
"accepted": 1,
"duplicates": 0,
"conflicts": 0,
"messages": 1,
"items": [
{
"source_record_id": "kmf_8a9c2e7d4b1f0e23",
"data_id": "story-generated-uuid",
"status": "accepted"
}
]
}
```
The returned `data_id` is the canonical ID for future metadata updates and reads. Re-sending the same `X-Provider` + `source_record_id` generates the same `data_id`.
Per-item statuses: an item already persisted with the exact same payload returns `duplicate`; the same `source_record_id` with *different* initial metadata returns `conflict`. Neither is (re-)enqueued, and other valid items in the batch are unaffected. An overlapping retry while the first request is still queued may return `accepted` — downstream ingestion stays idempotent.
If a caller already has DATA Foundation-assigned UUIDs, the lower-level `POST /webhook/v1/data-audit/data-ids:batch` endpoint exists. It requires `data_id` on every record and is not the recommended provider path.
### Submit metadata updates
Use this endpoint for later corrections or mutable metadata changes. `seq` must be `1`–`100` per `data_id` (a hard cap of 100 updates per record, all environments). Changes that belong to the same provider-side revision can share one update and one `seq`.
`metadata_json` must include a canonical content hash and be the **full latest Trace metadata state** — not a diff or patch. If only KYC changed, still include the unchanged file, app, consent, and provider payload fields, so each event verifies against `metadata_root` and the latest state can be rebuilt without merge rules.
```http theme={null}
POST /webhook/v1/data-audit/metadata-updates:batch
Content-Type: application/json
X-API-Key:
X-Provider: kled
X-Batch-Id: kled-metadata-000001
X-Ingestion-Source: backlog
```
Request body is a JSON array:
```json theme={null}
[
{
"data_id": "11111111-1111-4111-8111-111111111111",
"seq": 1,
"prev_metadata_root": "sha256:",
"metadata_root": "sha256:",
"metadata_json": {
"schema_version": "trace-v1.0",
"asset": {
"hash": "sha256:<64-hex>"
},
"contributor": {
"anon_id": "kup_123",
"kyc_status": "unverified",
"consent": {
"tos_version": "2026-06-01",
"tos_hash": "sha256:<64-hex-policy-hash>",
"tos_uri": "https://kled.ai/terms/2026-06-01",
"privacy_policy_version": "2026-05-20",
"privacy_policy_hash": "sha256:<64-hex-policy-hash>",
"privacy_policy_uri": "https://kled.ai/privacy/2026-05-20"
}
},
"app": {
"platform_name": "kled.ai"
},
"provider_payload": {
"media_id_public": "kmf_8a9c2e7d4b1f0e23",
"reason": "kyc_status_changed"
}
},
"occurred_at": "2026-05-13T00:00:01Z"
}
]
```
`metadata_root` should be the provider's deterministic non-zero hash of the canonical full updated Trace Schema v1.0 metadata JSON. `prev_metadata_root` must also be a non-zero root. The DATA Foundation stores these values as submitted; hash verification against `metadata_json` is not enforced yet.
The normal integration path is `application/json` on `/records:batch` — that route accepts JSON only. For backlog tooling, these lower-level route variants also exist; data ID file routes require `data_id` in every record.
```text theme={null}
POST /webhook/v1/data-audit/data-ids:batch-ndjson
POST /webhook/v1/data-audit/metadata-updates:batch-ndjson
POST /webhook/v1/data-audit/data-ids:batch-csv
POST /webhook/v1/data-audit/metadata-updates:batch-csv
POST /webhook/v1/data-audit/data-ids:batch-txt
POST /webhook/v1/data-audit/metadata-updates:batch-txt
```
* **NDJSON** requires `Content-Type: application/x-ndjson` and `Content-Encoding: gzip`. Each decompressed line is one JSON record with the same fields as the JSON endpoints.
* **CSV** requires `Content-Type: text/csv`.
* Data ID columns: `data_id,source_record_id,initial_metadata_root,initial_metadata_json,occurred_at`
* Metadata update columns: `data_id,seq,prev_metadata_root,metadata_root,metadata_json,occurred_at`
* `initial_metadata_json` and `metadata_json` must be valid JSON in a quoted CSV field and include one canonical content hash.
* **TXT** requires `Content-Type: text/plain` and accepts line-delimited JSON, a JSON array, or header-delimited comma/tab text using the same CSV columns.
## Read API
Read endpoints are public audit views. `provider` is optional and acts as a narrowing filter.
Read model:
* `GET /data-ids/{data_id}` returns the registration profile plus the latest raw metadata event — the exact payload of the highest stored sequence, not necessarily a diff. The response also carries a top-level `ingested_at` — the DATA Foundation-generated timestamp of the registration (seq 0) ingestion — omitted when unavailable.
* `GET /data-ids/{data_id}/metadatas` returns the full append-only metadata history, including registration at `seq: 0` and later metadata updates.
* Search, asset receipt lookup, and scoped-group summaries use the DATA Foundation's normalized latest-state projection derived from those events. This projection powers fields such as MIME type, media category, KYC status, TOS/privacy versions, lifecycle status, and `tx_hash`.
### Get trace by data ID
```http theme={null}
GET /api/v1/data-audit/data-ids/11111111-1111-4111-8111-111111111111
```
```json theme={null}
{
"data_id": "11111111-1111-4111-8111-111111111111",
"provider": "kled",
"ingested_at": "2026-05-13T00:00:01.234567Z",
"profile": {
"data_id": "11111111-1111-4111-8111-111111111111",
"provider": "kled",
"tx_hash": ""
},
"latest_metadata": {
"data_id": "11111111-1111-4111-8111-111111111111",
"seq": 0,
"event_type": "DataRegistered",
"tx_hash": ""
}
}
```
### List metadata history
```http theme={null}
GET /api/v1/data-audit/data-ids/11111111-1111-4111-8111-111111111111/metadatas
```
Each metadata row includes `tx_hash`, initially as an empty string:
```json theme={null}
{
"data_id": "11111111-1111-4111-8111-111111111111",
"metadatas": [
{ "seq": 0, "event_type": "DataRegistered", "tx_hash": "" },
{ "seq": 1, "event_type": "MetadataUpdated", "tx_hash": "" }
]
}
```
### Search by indexed field
```http theme={null}
GET /api/v1/data-audit/search?field=source_record_id&value=kmf_8a9c2e7d4b1f0e23
```
Search is exact-match on the [searchable fields](#searchable-fields). Matching is case-sensitive for `source_record_id`, `contributor.anon_id`, `collection_id`, `customer_id`, and `task_id`; other fields match case-insensitively, and content-hash values are canonicalized before matching. Each match includes `tx_hash` (initially an empty string), and responses include `next_cursor` for pagination. `limit` defaults to `100` and is capped at `1000` — larger values are silently clamped, not rejected.
Typical flow: use `/search` to locate records by an exact field value, then `/data-ids/{data_id}` or `/data-ids/{data_id}/metadatas` for the canonical event payload used in audit verification. Use `/stats` for distribution counts, and `/recent` or `/feed` when the UI needs the newest records.
More examples:
```text theme={null}
GET /api/v1/data-audit/search?field=asset_hash&value=sha256:<64-hex>
GET /api/v1/data-audit/search?field=file.content_sha256&value=<64-hex-or-sha256-prefixed-hex>
GET /api/v1/data-audit/search?field=customer_id&value=
GET /api/v1/data-audit/search?field=task_id&value=
GET /api/v1/data-audit/search?field=collection_id&value=
GET /api/v1/data-audit/search?field=contributor.consent.tos_hash&value=sha256:<64-hex-policy-hash>
GET /api/v1/data-audit/search?field=contributor.consent.privacy_policy_hash&value=sha256:<64-hex-policy-hash>
GET /api/v1/data-audit/search?field=collection_id&value=&provider=kled&limit=100&cursor=
```
### Provider totals (`/stats`)
```http theme={null}
GET /api/v1/data-audit/stats
GET /api/v1/data-audit/stats?provider=kled
```
```json theme={null}
{
"total_records": 1000324,
"total_contributors": 902111,
"provider": "kled",
"provider_records": 100234,
"provider_contributors": 87234,
"kyc_status": { "verified": 91234, "pending": 9000 },
"account_verification": { "verified": 90000, "pending": 10234 },
"tax_status": { "complete": 80000, "submitted": 20234 },
"tos_versions": { "2026-05": 100234 },
"privacy_policy_versions": { "2026-05": 100234 },
"media_category_coverage": { "video": 80000, "image": 20234 },
"mime_distribution": { "video/mp4": 80000, "image/jpeg": 20234 },
"geo_distribution": { "us": 70234, "ca": 30000 },
"total_size_bytes": 1234567890,
"size_record_count": 100234,
"average_size_bytes": 12316,
"active_tos_version": "2026-05",
"active_tos_hash": "sha256:<64-hex-policy-hash>",
"active_tos_uri": "https://kled.ai/terms/2026-05",
"active_privacy_policy_version": "2026-05",
"active_privacy_policy_hash": "sha256:<64-hex-policy-hash>",
"active_privacy_policy_uri": "https://kled.ai/privacy/2026-05"
}
```
Record totals count each registered record once; contributor totals count distinct `provider + contributor_anon_id` values. Metadata updates change neither, but distributions and size totals follow the latest projection, so a full-state update can move a record between buckets. `average_size_bytes` is `total_size_bytes / size_record_count`, counting records with a positive size. `shard_count` is reserved (currently `0`). The `provider_*` fields appear only with `?provider=`, and `active_*` only when that provider has an active policy. App/platform fields are not stats scopes.
### Recent ingestion feed
```http theme={null}
GET /api/v1/data-audit/feed?limit=50
GET /api/v1/data-audit/feed?provider=kled&limit=50
GET /api/v1/data-audit/feed?provider=kled&limit=50&cursor=
```
The feed returns recent audit events — both registrations and metadata updates — newest first by ingestion time. Rows include `event_type`, `seq`, `data_id`, `source_record_id`, `asset_hash`, `occurred_at`, and `ingested_at`. Use `next_cursor` to fetch older rows.
### Recent registered records
```http theme={null}
GET /api/v1/data-audit/recent?limit=50
GET /api/v1/data-audit/recent?provider=kled&limit=50
GET /api/v1/data-audit/recent?provider=kled&limit=50&cursor=
```
Returns registered receipt rows only, newest first by ingestion time. Use `next_cursor` to fetch older rows.
### Asset receipts by content hash
```http theme={null}
GET /api/v1/data-audit/assets/sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
```
Accepted hash forms are `sha256:<64-hex>`, plain `<64-hex>`, `0x<64-hex>`, and `0x1220<64-hex>`. The response returns the canonical `sha256:<64-lowercase-hex>` asset hash and all matched receipt rows.
## Scoped groups
Scoped groups let the Trace frontend or a partner reviewer create a public snapshot over a review set. For provider review workflows, labs should submit the provider's `source_record_id` values; Trace computes the deterministic `data_id`, verifies every record exists, and only creates the group if the full set is valid. (Groups can also be created from content hashes via `hashes` / `hashes_text` — the two input kinds are mutually exclusive.) Creation is asynchronous: poll `GET /scoped-groups/{group_id}` until `status` is `complete`.
### Create a group
```http theme={null}
POST /api/v1/data-audit/scoped-groups
Content-Type: application/json
Idempotency-Key: kled-review-000001
```
```json From source record IDs theme={null}
{
"title": "Kled review set",
"description": "Kled source record IDs supplied to a lab",
"provider": "kled",
"source_record_ids": ["kmf_1", "kmf_2"]
}
```
```json From pasted text theme={null}
{
"title": "Kled pasted review set",
"provider": "kled",
"source_record_ids_text": "kmf_1\nkmf_2\nkmf_3"
}
```
```json From an upload theme={null}
{
"title": "Kled uploaded review set",
"description": "CSV uploaded through presigned S3",
"provider": "kled",
"upload_id": "up_.csv"
}
```
Source-record groups are all-or-nothing:
| Condition | Response |
| ---------------------------------------------- | -------- |
| Duplicate `provider + source_record_id` values | `400` |
| Missing records | `404` |
| Profile/source mismatches | `409` |
| Records registered without an asset projection | `422` |
### Uploads for larger review sets
For larger source-record CSV/TXT inputs, request a presigned upload URL first:
```http theme={null}
POST /api/v1/data-audit/scoped-groups/uploads
Content-Type: application/json
```
```json theme={null}
{ "format": "csv" }
```
Supported formats are `csv` and `txt`. Upload the file bytes to the returned `upload_url` with the returned `Content-Type` header, then create the group with the returned `upload_id`.
```csv One-column CSV (requires top-level provider) theme={null}
source_record_id
kmf_1
kmf_2
```
```csv Mixed-provider CSV (no top-level provider) theme={null}
provider,source_record_id
kled,kmf_1
oto,oto_1
```
```text TXT (requires top-level provider) theme={null}
kmf_1
kmf_2
```
Omit `provider` in the create request only when the uploaded CSV has a `provider` column.
### Read a group
```http theme={null}
GET /api/v1/data-audit/scoped-groups/{group_id}
GET /api/v1/data-audit/scoped-groups/{group_id}/items?limit=100
GET /api/v1/data-audit/scoped-groups/{group_id}/items?limit=100&cursor=
GET /api/v1/data-audit/scoped-groups/{group_id}/export.csv
```
`GET /scoped-groups/{group_id}` returns the group status and, once complete, aggregate metrics in `summary`. `profile.status` is one of `pending`, `processing`, `complete`, or `failed`; `summary` is only present once the group is `complete`.
```json theme={null}
{
"profile": {
"group_id": "sg_...",
"title": "Kled review set",
"description": "Kled source record IDs supplied to a lab",
"manifest_kind": "source_record",
"status": "complete",
"submitted_items": 2,
"unique_items": 2,
"computed_at": "2026-06-04T15:30:00Z"
},
"summary": {
"records_in_set": 2,
"submitted_items": 2,
"unique_items": 2,
"matched_items": 2,
"missing_items": 0,
"matched_receipts": 2,
"kyc_verified_percent": 50,
"distinct_media_categories": ["image", "video"],
"distinct_tos_versions": ["2026-05"],
"total_size_bytes": 7340032,
"average_size_bytes": 3670016,
"source_distribution": { "kled": 2 },
"media_category_coverage": { "image": 1, "video": 1 },
"mime_distribution": { "image/jpeg": 1, "video/mp4": 1 },
"tos_versions": { "2026-05": 2 },
"privacy_policy_versions": { "2026-05": 2 },
"kyc_status": { "verified": 1, "unverified": 1 },
"geo_distribution": { "us": 2 },
"lifecycle_status": { "registered": 2 },
"metadata_presence": { "custom.camera": 1, "tos_acknowledgment": 2 }
}
}
```
`/items` returns one row per submitted `source_record_id` with the resolved `data_id`, `asset_hash`, and receipt summary. It is cursor-paginated and returns `next_cursor` when more rows exist. Both `/items` and `/export.csv` require the group to be `complete` and return `409` before that. For source-record groups, the CSV export columns are `input_type,provider,source_record_id,data_id,asset_hash,status,mime_type,media_category,kyc_status,tos_version,privacy_policy_version,tx_hash`; hash-manifest groups start with `hash,status,data_id,...` instead.
## Limits and retry behavior
| Limit | Value |
| ---------------------------------------- | ----------- |
| Max webhook batch payload (decompressed) | `25 MiB` |
| Max provider-policy request body | `1 MiB` |
| Max SQS message chunk | `240 KiB` |
| Max SQS batch payload | `256 KiB` |
| Max serialized record size | `350 KiB` |
| Max metadata updates per data ID | `100` |
| Max inline scoped-group body | `5 MiB` |
| Max inline scoped-group hashes | `10,000` |
| Max inline scoped-group source records | `10,000` |
| Max source\_record\_id length | `512 bytes` |
Retry guidance:
* Retry `502`, `503`, `504`, network timeouts, and `429` with exponential backoff and jitter. (Only the public scoped-group create/upload endpoints emit `429` — 60 requests/minute per IP; webhook batch endpoints never do.)
* Do not retry validation/auth `4xx` until the request is fixed.
* Keep `data_id`, request body, and `X-Batch-Id` stable across retries.
* Use `X-Ingestion-Source: backlog` only for backlog work; omit it for live work.
* The write path is idempotent for the same `data_id`, event key, and event hash.
* If the same `data_id` and event key are retried with different metadata, the record is treated as a conflict and rejected.
## Validation rules
| Rule | Behavior |
| -------------------------------------------------------------------------- | ------------------------------------- |
| Missing `X-Provider` on write endpoints | Request is rejected. |
| Invalid provider name | Request is rejected. |
| Provider outside the configured allowlist | Write request is rejected. |
| API key not authorized for the `X-Provider` value | Write request is rejected with `403`. |
| Missing `X-Batch-Id` on write endpoints | Request is rejected. |
| `X-Ingestion-Source` present with any value other than `backlog` | Request is rejected. |
| Missing `source_record_id` on `/records:batch` | Request is rejected. |
| Duplicate `source_record_id` inside one `/records:batch` request | Request is rejected. |
| Existing `/records:batch` item with the same initial registration payload | Item returns `status: "duplicate"`. |
| Existing `/records:batch` item with different initial registration payload | Item returns `status: "conflict"`. |
| Non-UUID `data_id` on lower-level data ID and metadata update routes | Request is rejected. |
| Missing `occurred_at` | Request is rejected. |
| Invalid `occurred_at` timestamp | Request is rejected. |
| Metadata `seq` outside `1` through `100` | Request is rejected. |
| Missing, malformed, or zero required metadata root fields | Request is rejected. |
## Delivery semantics
* Delivery is at least once, so duplicate submissions may occur.
* Duplicate submissions of the same event are treated idempotently.
* Same `data_id`, same metadata sequence, and different event content is treated as a conflict.
* Metadata updates may arrive before the initial data ID registration.
* Audit data is durable and does not expire.
* Event hashes are computed from canonicalized metadata JSON plus the event/hash/schema version fields.