Exporting the audit ledger¶

Compliance teams do not read portals. This guide gets the append-only ledger — every facade fetch and every administrative action — into a SIEM continuously and without gaps.

Two shapes of the same feed:

Path	Shape	Who holds the state
Pull	`GET /api/v1/audit/export` streams NDJSON.	The collector, as one number.
Push	A registered sink has batches POSTed to it, signed and retried.	The gateway, as the sink's cursor.

Telemetry is not this path

OpenTelemetry carries operational signal that may be sampled and dropped. Compliance evidence may not be. The ledger export is a separate, lossless path by design.

What a vetting entry carries¶

A chain run lands two kinds of entry, both attributed to the system actor kind — the vetting chain is the gateway's own automated subsystem, not a person, so the portal audit table and the ledger show it as system, never as a human actor.

A vetting-verdict entry per vetter on the ingestion run. On a re-vet, only for a vetter whose verdict changed since the snapshot's previous run (a different state, finding count or worst severity, or a vetter version that was not in that run's chain); a re-vet that changed nothing writes none. Its detail leads with vetter=state and then carries the finding count, the worst severity present (none when there are none), and the id of the chain run — so the entry is auditable on its own rather than a pointer back into the vetting tables. A clean pass additionally states what the vetter examined (the files it scanned and the rules it applied), so a pass reads as positive coverage rather than as silence:

secret-scan=fail; findings=2; worst=critical; run=1841
prompt-injection=pass; findings=0; worst=none; run=1841; scanned 12 instruction file(s) (.md, .mdc, .markdown, .txt); applied 7 injection-marker rules plus invisible-character analysis

A vetting-completed entry per run, carrying the trigger, the fail-closed outcome, the vetter count, the same run= id, and the chain identity. On a re-vet it also carries changed=, the number of vetting-verdict entries the run wrote:

trigger=revet-scheduled; outcome=clear; vetters=5; changed=0; run=1902; chain=secret-scan@1,prompt-injection@1,…;mode=run-all

Counting runs

Count vetting runs by vetting-completed, not by vetting-verdict. An unchanged re-vet writes vetting-completed (changed=0) and revet-clear and no verdict entries; every verdict of every run stays in the vetting report (GET /api/v1/snapshots/{id}/vetting).

The cursor¶

Both paths are a cursor over the ledger, not a copy of it. The cursor is the id of the last ledger entry a consumer has been handed, and the ledger itself is the queue — nothing is duplicated into a per-consumer outbox. That is what makes replay a single write rather than a re-materialisation of history.

Every exported entry carries its ledger id, so it doubles as the receiver's de-duplication key.

Pull: NDJSON¶

$ curl -D- 'localhost:8080/api/v1/audit/export?after=0&limit=1000'
X-Skills-Gateway-Audit-Cursor: 842

{"id":1,"ts":"2026-08-15T09:04:11Z","source":"10.0.0.4","principal":"alice@example.com","marketplace":"acme","event":"info-refs","ref":"refs/heads/main","sha":"3f9c2ab..."}
{"id":2,"ts":"2026-08-15T09:04:11Z","source":"10.0.0.4","principal":"alice@example.com","marketplace":"acme","event":"upload-pack","ref":null,"sha":"3f9c2ab..."}

One compact JSON object per line, in ascending ledger sequence — the format Splunk, Elastic and Sentinel ingest natively. A JSON array would force the consumer to buffer the whole response; the stream is written and flushed in fixed-size chunks, so gateway memory is bounded by the chunk rather than by limit.

Parameter	Default	Bounds
`after`	`0`	The cursor to resume from. Negative values are treated as 0.
`limit`	`1000`	Clamped to `skills-gateway.audit-export.max-page-size` (`10000`).

The whole poller is a loop over one header. Send after, read X-Skills-Gateway-Audit-Cursor from the response, and use it as the next after. When a response comes back empty the consumer is caught up.

$ cursor=$(cat cursor.txt 2>/dev/null || echo 0)
$ next=$(curl -s -D headers.txt "localhost:8080/api/v1/audit/export?after=${cursor}" \
    -o batch.ndjson && sed -n 's/^X-Skills-Gateway-Audit-Cursor: //Ip' headers.txt | tr -d '\r')
$ [ -s batch.ndjson ] && ingest batch.ndjson && echo "${next}" > cursor.txt

The endpoint sits behind the OIDC session like the rest of /api/**, so a collector polls it with a session the same way the portal does; the portal's own Download ledger (NDJSON) link is this endpoint with no cursor.

Push: sinks¶

A sink is a named consumer with a stored cursor. Creating one also creates its delivery channel — an ordinary webhook subscriber filtered to the single audit.export event — so signing, retry, backoff and the delivery record are the lifecycle webhook machinery, not a second delivery engine.

PortalAPI

Integrations → Audit sinks → New sink → Sink name and Target URL → Add sink. The signing secret appears in a show-once dialog.

$ curl -X POST localhost:8080/api/v1/audit/sinks \
    -H 'Content-Type: application/json' \
    -d '{"name":"siem","url":"https://siem.example.com/ingest/skills-gateway",
         "after":0,"batchSize":500}'

{"id":1,"name":"siem","kind":"webhook",
 "url":"https://siem.example.com/ingest/skills-gateway",
 "cursorPosition":0,"batchSize":500,"secret":"whsec_...","createdAt":"..."}

Field	Rule
`name`	`^[a-z0-9][a-z0-9_-]$`. Shares a namespace with webhook subscribers, because the sink's channel is one — a collision is 409. A bad name is 422*.
`url`	Same scheme allowlist as marketplace and subscriber registration, failing closed. 400.
`after`	Ledger sequence to start after. Omit to start at the beginning.
`batchSize`	Entries per batch. Defaults to `skills-gateway.audit-export.batch-size`, clamped to `max-page-size`.

A batch arrives as one signed POST whose body wraps the entries:

{"event":"audit.export","sink":"siem","exportedAt":"2026-08-15T09:20:00Z",
 "fromCursor":800,"toCursor":842,"count":42,
 "entries":[{"id":801,"ts":"...","source":"10.0.0.4","principal":"alice@example.com",
             "marketplace":"acme","event":"info-refs","ref":"refs/heads/main","sha":"3f9c2ab..."}]}

Verify X-Skills-Gateway-Signature exactly as for a lifecycle event — HMAC-SHA256 over the raw body with the sink's whsec_ secret. The delivery attempts show up on the portal's Webhooks page alongside lifecycle deliveries.

A lifecycle subscriber never receives audit batches

audit.export is not a subscribable lifecycle event: POST /api/v1/webhooks rejects it, and a * subscriber does not match it either. The only way to receive audit batches is to be a sink.

At-least-once, and why¶

An export pass reads the entries after the cursor, enqueues one durable delivery row, and only then advances the cursor. A crash between the two costs a duplicate, never a gap. De-duplicate on each entry's ledger id; the delivery-id header remains the transport-level de-duplication key.

A permanently failing sink stalls at its cursor

If the dispatcher exhausts the attempt budget the delivery is marked failed and the exporter does not rewind — a poisoned batch would otherwise loop forever. Fix the receiver, then replay explicitly.

Replay¶

Replay is a cursor write, not a mode:

PortalAPI

The sink row's Replay button sets the position back to 0 and toasts Sink '{name}' will replay the ledger.

$ curl -X PUT localhost:8080/api/v1/audit/sinks/1/cursor \
    -H 'Content-Type: application/json' -d '{"after":800}'

Everything after that position is delivered again on the next pass. 200 with the updated sink, 404 if the sink is unknown.

Replay reaches only what the ledger still holds

If ledger retention is enabled, trimming removes only entries every enabled sink has already taken. So rewinding an existing sink can never ask for something it was not handed once — but a sink registered later, starting at position 0, receives the ledger from the oldest surviving entry rather than from the beginning of time.

Register the sinks you need before enabling the trim. A sink that does not exist yet holds no position, and so bounds nothing.

The sink list also reports how far behind the ledger head each sink is, which is the signal that a receiver is failing or that a batch size is too small for the poll interval:

$ curl localhost:8080/api/v1/audit/sinks

[{"id":1,"name":"siem","kind":"webhook","url":"https://siem.example.com/...",
  "cursorPosition":842,"ledgerHead":844,"behind":2,"batchSize":500,
  "enabled":true,"createdAt":"..."}]

DELETE /api/v1/audit/sinks/{id} removes the sink and its delivery channel — 204, or 404 if it never existed. It is the only way to remove that channel: deleting it through the webhooks API is refused with 409, and the database refuses it too.

Why exports lag a few seconds¶

The ledger's id is a BIGSERIAL, and nextval is assigned before commit. Entry 101 can therefore become visible while 100 is still in flight; a naive id > cursor reader would advance past 100 and never see it again.

Both export paths close that window the same way: entries younger than skills-gateway.audit-export.lag (default 5s) are withheld. Ledger appends are single-statement inserts that commit in milliseconds, so the settling window has orders of magnitude of headroom, and the price is bounded staleness instead of a silently skipped entry.

Do not tune the lag to zero

A shorter lag narrows the settling window; zero removes it, and the export can then step over an entry that was mid-commit. A compliance feed with a hole in it is worse than one that is five seconds behind.

The rest of the exporter tuning — poll interval, batch size, page-size caps — is in Configuration.