Audit Logs
On this page
Audit Logs give you a structured record of important actions performed in Ververica Cloud. Each entry captures who did what, when, and whether the action succeeded. They're especially useful for:
- Security investigations
- Compliance evidence collection
- Operational troubleshooting
- Change tracking and accountability
How audit logging works
Ververica Cloud captures audit events at the point where a request reaches the control plane. Each audited action can produce up to two records that share the same traceId, so you can correlate them:
ISSUED: the action was requested.EXECUTED: the action was processed, and a result is available.
From there, an event reaches you through the following path:
- The control plane captures the audit information for the action.
- The audit event is written to a dedicated log file.
- A log-forwarding sidecar (FluentBit) tails that file and uploads log chunks to object storage.
- Ververica Cloud exposes stream endpoints that let you read live audit events or request historical backfills.
- Your script, service, or SIEM integration connects to the endpoint, stores its cursor, and keeps consuming events.
In other words, audit logs use a pull model: your systems retrieve events from Ververica Cloud through API endpoints, rather than Ververica Cloud pushing events into infrastructure you manage, such as a Kafka cluster you'd otherwise have to run yourself. Delivery is near real time (typically tens of seconds, not milliseconds) and at-least-once, so your consumer might occasionally receive the same event more than once after a reconnect or retry. Use the event ID as an idempotency key when you store events downstream.
What's recorded
Each audit event contains the following fields:
If you haven't configured an encryption key, audit logs still include all the metadata fields above; request and response content is left out rather than stored unencrypted. See Audit Log Encryption Keys to set one up.
Turning on audit logging
Audit logs are available for an individual Ververica Cloud account or for an organization.
For an organization, the organization owner (an organization has exactly one) is the only user who can enable, disable, or poll audit logs, at least for now.
- For an organization, the organization owner turns audit logging on or off from the Audit Logs tab in the organization admin panel.
- For a personal account, you can turn it on or off from Audit Logs in your profile menu.
Turning on audit logging by itself starts collecting event metadata. To also capture request and response content, add a public encryption key when you turn it on, or afterward. Turning audit logging off keeps your configured key on file and keeps previously collected events available to pull; it only stops new events from being recorded.
Retention
By default, Ververica Cloud retains audit logs for 30 days. You can request a backfill for any event within that window; events older than the retention period are no longer available.
Consuming audit events
You retrieve audit events over HTTPS using an API token as a bearer credential. There's currently no UI for creating or managing that token; create and store it through the API. Document who owns the token, where it's stored, how it's rotated, and which integration depends on it, the same as you would for any other long-lived credential.
Two endpoints are available, and each comes in an organization-scoped and an individual-account-scoped form:
GET /api/v1/organizations/{organizationId}/audit/livefor an organization, orGET /api/v1/users/{userId}/audit/livefor an individual account, streams new events as a server-sent event stream, for ongoing collection.POST /api/v1/organizations/{organizationId}/audit/backfillfor an organization, orPOST /api/v1/users/{userId}/audit/backfillfor an individual account, streams events for a given start and end time, for retrieving a past window (for example, when you first set up a consumer, or to replay a recent window) until the range is exhausted.
1curl -N \
2 -H "Authorization: Bearer $TOKEN" \
3 "https://<ververica-cloud-host>/api/v1/organizations/$ORG_ID/audit/live"
4
5# Individual account:
6curl -N \
7 -H "Authorization: Bearer $TOKEN" \
8 "https://<ververica-cloud-host>/api/v1/users/$USER_ID/audit/live"Include Authorization: Bearer <token> on every request. On first connection, you can provide a starting timestamp; after that, resume from the last event ID your consumer processed. For an organization, only the organization owner can call these endpoints, the same as for turning audit logging on or off.
Example: a polling script
The following script wraps the live and backfill endpoints for either an organization or an individual account, and optionally decrypts requestBody/state fields in place if you pass it your private key. See Audit Log Encryption Keys for the encryption background.
1#!/usr/bin/env bash
2# Pull audit delivery events for an organization or a user.
3#
4# Two modes:
5# live (default) — GET /api/v1/organizations/{orgId}/audit/live
6# /api/v1/users/{userId}/audit/live
7# Streams Server-Sent Events indefinitely. Press Ctrl+C to stop.
8# backfill — POST /api/v1/organizations/{orgId}/audit/backfill
9# /api/v1/users/{userId}/audit/backfill
10# Streams SSE events for a bounded time range and exits when done.
11#
12# Requires: curl
13set -euo pipefail
14
15usage() {
16 cat <<EOF
17Usage: $(basename "$0") [OPTIONS]
18
19Pull audit delivery events for an organization or a user (Server-Sent Events stream).
20
21Required:
22 --base-url <url> Portal API base URL (e.g. https://api.cloud.ververica.com)
23 --token <token> Bearer auth token, created and stored through the platform API
24
25Subject (exactly one required):
26 --org-id <id> Organization ID → /api/v1/organizations/{id}/audit/...
27 --user-id <id> User ID → /api/v1/users/{id}/audit/...
28
29Mode (default: live):
30 --mode live Stream live audit events (runs until Ctrl+C)
31 --mode backfill Stream a bounded time range (requires --start-time and --end-time)
32
33Backfill options (required when --mode backfill):
34 --start-time <iso8601> Start of the time range (e.g. 2024-01-01T00:00:00Z)
35 --end-time <iso8601> End of the time range (e.g. 2024-01-02T00:00:00Z)
36
37Resume options (optional):
38 --from <iso8601> Live mode: only emit events after this time
39 --last-event-id <id> Resume from a previously received event ID
40
41Decryption (optional):
42 --private-key <file> Path to the PEM-encoded RSA private key that matches the audit
43 public key registered for the subject. When set, the encrypted
44 'requestBody' and 'state' fields of each event are decrypted in
45 place before being printed. Fields that are absent, null, or not
46 decryptable are left untouched.
47 Requires: python3 with the 'cryptography' package.
48
49 -h, --help Show this help
50
51Examples:
52 # Live stream for an organization (runs indefinitely):
53 $(basename "$0") \\
54 --base-url https://api.cloud.ververica.com \\
55 --org-id my-org-id \\
56 --token eyJhbGci...
57
58 # Live stream for a user:
59 $(basename "$0") \\
60 --base-url https://api.cloud.ververica.com \\
61 --user-id my-user-id \\
62 --token eyJhbGci...
63
64 # Backfill for a specific day (organization):
65 $(basename "$0") \\
66 --base-url https://api.cloud.ververica.com \\
67 --org-id my-org-id \\
68 --token eyJhbGci... \\
69 --mode backfill \\
70 --start-time 2024-06-01T00:00:00Z \\
71 --end-time 2024-06-02T00:00:00Z
72
73 # Live stream for a user, decrypting requestBody/state with a private key:
74 $(basename "$0") \\
75 --base-url https://api.cloud.ververica.com \\
76 --user-id my-user-id \\
77 --token eyJhbGci... \\
78 --private-key ./audit-private-key.pem
79EOF
80 exit 1
81}
82
83BASE_URL=""
84ORG_ID=""
85USER_ID=""
86TOKEN=""
87MODE="live"
88START_TIME=""
89END_TIME=""
90FROM=""
91LAST_EVENT_ID=""
92PRIVATE_KEY=""
93
94while [[ $# -gt 0 ]]; do
95 case "$1" in
96 --base-url) BASE_URL="$2"; shift 2 ;;
97 --org-id) ORG_ID="$2"; shift 2 ;;
98 --user-id) USER_ID="$2"; shift 2 ;;
99 --token) TOKEN="$2"; shift 2 ;;
100 --mode) MODE="$2"; shift 2 ;;
101 --start-time) START_TIME="$2"; shift 2 ;;
102 --end-time) END_TIME="$2"; shift 2 ;;
103 --from) FROM="$2"; shift 2 ;;
104 --last-event-id) LAST_EVENT_ID="$2"; shift 2 ;;
105 --private-key) PRIVATE_KEY="$2"; shift 2 ;;
106 -h|--help) usage ;;
107 *) echo "Unknown option: $1" >&2; usage ;;
108 esac
109done
110
111[[ -z "$BASE_URL" ]] && { echo "Error: --base-url is required." >&2; usage; }
112[[ -z "$TOKEN" ]] && { echo "Error: --token is required." >&2; usage; }
113
114if [[ -n "$ORG_ID" && -n "$USER_ID" ]]; then
115 echo "Error: --org-id and --user-id are mutually exclusive." >&2; usage
116fi
117if [[ -z "$ORG_ID" && -z "$USER_ID" ]]; then
118 echo "Error: one of --org-id or --user-id is required." >&2; usage
119fi
120
121if [[ "$MODE" != "live" && "$MODE" != "backfill" ]]; then
122 echo "Error: --mode must be 'live' or 'backfill'." >&2; usage
123fi
124
125if [[ "$MODE" == "backfill" ]]; then
126 [[ -z "$START_TIME" ]] && { echo "Error: --start-time is required for backfill mode." >&2; usage; }
127 [[ -z "$END_TIME" ]] && { echo "Error: --end-time is required for backfill mode." >&2; usage; }
128fi
129
130# ── Validate decryption prerequisites ───────────────────────────────────────────
131if [[ -n "$PRIVATE_KEY" ]]; then
132 [[ -f "$PRIVATE_KEY" ]] || { echo "Error: private key file not found: $PRIVATE_KEY" >&2; exit 1; }
133 command -v python3 >/dev/null 2>&1 \
134 || { echo "Error: python3 is required for --private-key decryption." >&2; exit 1; }
135 python3 -c 'import cryptography' >/dev/null 2>&1 \
136 || { echo "Error: the python 'cryptography' package is required (pip install cryptography)." >&2; exit 1; }
137fi
138
139# ── Decryption filter ───────────────────────────────────────────────────────────
140# Reads the SSE stream on stdin and, for each "data:" line carrying a JSON audit event,
141# decrypts the "requestBody" and "state" fields in place. Matches the hybrid RSA/AES-GCM
142# format described in "Audit Log Encryption Keys":
143# base64( [RSA-wrapped AES key (512B)] + [GCM nonce (12B)] + [AES-GCM ciphertext+tag] )
144# using RSA-OAEP with SHA-256 and AES-GCM (128-bit tag).
145read -r -d '' DECRYPT_PY <<'PYEOF' || true
146import base64
147import json
148import sys
149
150from cryptography.hazmat.primitives import hashes, serialization
151from cryptography.hazmat.primitives.asymmetric import padding
152from cryptography.hazmat.primitives.ciphers.aead import AESGCM
153
154RSA_WRAPPED_KEY_BYTES = 512
155NONCE_BYTES = 12
156DATA_PREFIX = "data:"
157ENCRYPTED_FIELDS = ("requestBody", "state")
158
159with open(sys.argv[1], "rb") as fh:
160 private_key = serialization.load_pem_private_key(fh.read(), password=None)
161
162
163def decrypt_field(value):
164 blob = base64.b64decode(value)
165 wrapped_key = blob[:RSA_WRAPPED_KEY_BYTES]
166 nonce = blob[RSA_WRAPPED_KEY_BYTES:RSA_WRAPPED_KEY_BYTES + NONCE_BYTES]
167 ciphertext = blob[RSA_WRAPPED_KEY_BYTES + NONCE_BYTES:]
168 aes_key = private_key.decrypt(
169 wrapped_key,
170 padding.OAEP(mgf=padding.MGF1(hashes.SHA256()), algorithm=hashes.SHA256(), label=None),
171 )
172 # cryptography's AESGCM expects ciphertext||tag, which is exactly what Java's
173 # AES/GCM/NoPadding produces, so the trailing 16-byte tag needs no special handling.
174 return AESGCM(aes_key).decrypt(nonce, ciphertext, None).decode("utf-8")
175
176
177for line in iter(sys.stdin.readline, ""):
178 line = line.rstrip("\n")
179 if line.startswith(DATA_PREFIX):
180 payload = line[len(DATA_PREFIX):]
181 try:
182 event = json.loads(payload)
183 except ValueError:
184 print(line, flush=True)
185 continue
186 for field in ENCRYPTED_FIELDS:
187 value = event.get(field)
188 if isinstance(value, str) and value:
189 try:
190 event[field] = decrypt_field(value)
191 except Exception:
192 pass # not encrypted / wrong key — leave the original value untouched
193 print(DATA_PREFIX + json.dumps(event, ensure_ascii=False), flush=True)
194 else:
195 print(line, flush=True)
196PYEOF
197
198# Applies the decryption filter when --private-key is set; otherwise passes through unchanged.
199maybe_decrypt() {
200 if [[ -n "$PRIVATE_KEY" ]]; then
201 python3 -u -c "$DECRYPT_PY" "$PRIVATE_KEY"
202 else
203 cat
204 fi
205}
206
207# ── Build subject path prefix ──────────────────────────────────────────────────
208if [[ -n "$ORG_ID" ]]; then
209 SUBJECT_PATH="/api/v1/organizations/${ORG_ID}"
210 SUBJECT_LABEL="org '${ORG_ID}'"
211else
212 SUBJECT_PATH="/api/v1/users/${USER_ID}"
213 SUBJECT_LABEL="user '${USER_ID}'"
214fi
215
216# ── Build common headers ───────────────────────────────────────────────────────
217COMMON_HEADERS=(
218 -H "Authorization: Bearer ${TOKEN}"
219 -H "Accept: text/event-stream"
220)
221[[ -n "$LAST_EVENT_ID" ]] && COMMON_HEADERS+=(-H "Last-Event-ID: ${LAST_EVENT_ID}")
222
223# ── Live mode ──────────────────────────────────────────────────────────────────
224if [[ "$MODE" == "live" ]]; then
225 URL="${BASE_URL}${SUBJECT_PATH}/audit/live"
226 if [[ -n "$FROM" ]]; then
227 ENCODED_FROM=$(python3 -c "import urllib.parse,sys; print(urllib.parse.quote(sys.argv[1]))" "$FROM" 2>/dev/null \
228 || printf '%s' "$FROM" | sed 's/:/%3A/g; s/+/%2B/g')
229 URL="${URL}?from=${ENCODED_FROM}"
230 fi
231
232 echo "Streaming live audit events for ${SUBJECT_LABEL} (press Ctrl+C to stop)..."
233 echo "URL: ${URL}"
234 echo ""
235
236 curl -N --no-buffer "${COMMON_HEADERS[@]}" "$URL" | maybe_decrypt || true
237 exit 0
238fi
239
240# ── Backfill mode ──────────────────────────────────────────────────────────────
241echo "Streaming backfill audit events for ${SUBJECT_LABEL} [${START_TIME} – ${END_TIME}]..."
242echo ""
243
244curl -N --no-buffer "${COMMON_HEADERS[@]}" \
245 -X POST \
246 -H "Content-Type: application/json" \
247 --data "{\"startTime\": \"${START_TIME}\", \"endTime\": \"${END_TIME}\"}" \
248 "${BASE_URL}${SUBJECT_PATH}/audit/backfill" | maybe_decrypt || trueBest practices
- Run a durable consumer: a script, service, or SIEM connector that reconnects automatically.
- Persist the last processed event ID outside process memory, so a restart resumes from the right place.
- Store your API token in a secret manager, not in code or plain configuration.
- Design downstream storage to tolerate duplicate events, since delivery is at-least-once.
- Monitor your consumer, and alert if it stops polling or falls behind.
- Protect your private key if you've configured one; see Audit Log Encryption Keys.