Blob Storage

Provide Credentials Using Mounted Files

Instead of embedding blob storage credentials directly in your Helm values.yaml, you can provide them as files mounted into the Ververica Platform container. Ververica Platform reads the credential files from a configured directory at startup and distributes the values to services that require them.

This approach keeps credentials out of your Helm values, which might be stored in version control or visible to operators who have access to the Helm release.

How It Works

Each credential is stored in a separate file. Ververica Platform discovers credentials by scanning the configured directory and reading every file whose name matches the pattern <provider>.<key>. Each file must contain exactly one value: the raw credential string with no additional formatting.

Example: S3 Credentials

Create one file per credential in your credentials directory:

1s3.accessKeyId

The file name determines which provider and key the value is assigned to. The file content is the credential value.

Configuration

Mount your credentials directory into the Ververica Platform pod and set the path in your values.yaml:

1global:
2  blobStorage:
3    credentialsDir: /conf/blob-creds

Replace /conf/blob-creds with the path where your credentials files are mounted inside the container.

Provide Credentials Using Kubernetes Secrets

Instead of mounting credential files, you can store blob storage credentials in a Kubernetes Secret and reference the Secret by name in your values.yaml. Ververica Platform reads the credentials from the Secret at startup and distributes them to services that require them.

This approach integrates with Kubernetes-native secret management and is compatible with tools like Sealed Secrets, External Secrets Operator, or Vault agent injection.

Create the Secret

Create a Kubernetes Secret with one key per credential. Key names must follow the <provider>.<key> pattern, using the same convention as mounted credential files.

For S3 credentials:

1kubectl create secret generic blob-storage-credentials \
2  --from-literal=s3.accessKeyId=AKIAEXAMPLEACCESSKEYID \
3  --from-literal=s3.secretAccessKey=wJalrXUtnFEMIEXAMPLEKEYsecretkey \
4  --namespace vvp-system

Reference the Secret in Your Values

Set the secret name in your values.yaml:

1global:
2  blobStorage:
3    credentialsSecret: blob-storage-credentials  

Replace blob-storage-credentials with the name of your Secret and vvp-system with the namespace where Ververica Platform is installed.

Update Blob Storage Configuration After Installation

You can change blob storage configuration after the initial Helm installation without performing a full reinstall. Run helm upgrade with your updated values.yaml:

1helm upgrade --install <RELEASE_NAME> \
2  oci://registry.ververica.cloud/platform-charts/ververica-platform \
3  --version <VERSION> \
4  --namespace vvp-system \
5  --values values.yaml

Replace <RELEASE_NAME> with your Helm release name and <VERSION> with the installed platform version.

Provide License Using a Kubernetes Secret

Instead of embedding your Ververica Platform license directly in values.yaml, you can supply it as a Kubernetes Secret mounted into the platform pods. This keeps the license out of Helm configuration files, which may be stored in version control or exposed through CI/CD pipelines.

Create the Secret

Create a Kubernetes Secret containing your license file:

1kubectl create secret generic vvp-license \
2  --from-file=license.yaml=/path/to/your/license.yaml \
3  --namespace vvp-system

Reference the Secret in Your Values

Set the secret name in your values.yaml:

1global:
2  license:
3    existingSecret: vvp-license  

Replace vvp-license with the name of your Secret and vvp-system with the namespace where Ververica Platform is installed.

Use s3a:// Warehouses for Paimon and Iceberg Catalogs

Catalogs that resolve storage through the Hadoop file system API, including Apache Paimon and Apache Iceberg with catalog-type=hadoop, address their warehouse with the s3a:// scheme. The class that implements this scheme, org.apache.hadoop.fs.s3a.S3AFileSystem, ships in flink-s3-fs-hadoop, which is loaded by the isolated plugin class loader. The Hadoop file system API loads from the main class loader, so without additional handling a catalog on s3a:// fails with a ClassNotFoundException.

Ververica Platform places the required class on the SQL Gateway class path when the gateway starts. This is controlled by the following value, which is enabled by default:

1vvp-appagent:
2  sqlService:
3    s3aCompatibility:
4      enabled: true

Set enabled: false to keep the main class path free of the unshaded Hadoop S3 file system. Catalogs on s3a:// then fail to resolve.

The class is added to the extracted engine distribution used by the SQL Gateway only. Deployment and session cluster class paths are not modified.

Create a Paimon Catalog on s3a://

Supply the storage credentials and endpoint in the catalog WITH clause:

1CREATE CATALOG paimon_cat WITH (
2  'type' = 'paimon',
3  'metastore' = 'filesystem',
4  'warehouse' = 's3a://<BUCKET>/<PATH>',
5  'fs.s3a.endpoint' = 'https://<S3_ENDPOINT>',
6  'fs.s3a.access.key' = '<ACCESS_KEY>',
7  'fs.s3a.secret.key' = '<SECRET_KEY>',
8  'fs.s3a.path.style.access' = 'true'
9);

The configuration is applied at gateway startup, so it survives a restart of the SQL Gateway without being reapplied.

Storage Behind a Private Certificate Authority

When the S3 endpoint presents a certificate signed by an internal authority, the JobManager and TaskManager must trust that authority, otherwise the connection fails with a PKIX path validation error. Deliver the trust store to Flink pods through the platform-wide deployment defaults rather than per deployment. See Platform-Wide Deployment Defaults (publishing pending review) for the private-certificate-authority walkthrough.