Ververica DocsVerverica Docs
Docs Home
Viewing docs for
Self-ManagedNot available for BYOC

Engine Artifacts in Blob Storage

Applies toSelf-Managed v3
2 min read
On this page

Starting with Ververica Platform 3.1.4, the platform keeps the VERA engine artifacts (engine archives, connectors, and engine SDKs) in the blob storage you configure in global.blobStorage.baseUri, and reads them from there. The Application Manager uploads them each time it starts, so installation and upgrade need no extra step. The in-cluster vera service is no longer deployed.

Requirements

  • Blob storage type: Amazon S3 and S3-compatible storage such as MinIO (s3://, s3i://), and Azure Blob Storage (wasb://, wasbs://, wiaz://).
  • Space: about 12 GiB for each VERA version. Ververica Platform 3.1.4 ships VERA 4.5.4 and 4.6.1, about 25 GiB in total. Artifacts of VERA versions from earlier releases aren't removed.
  • Permissions: the credentials you configure for blob storage (global.blobStorage, global.blobStorageCredentialsSecret, or Azure Workload Identity) must allow listing, reading, and writing objects under the path of baseUri. The upload never deletes objects.

Where the Artifacts Are Stored

The artifacts are stored under vera-artifacts/, below the path of global.blobStorage.baseUri:

global.blobStorage.baseUriLocation of the artifacts
s3i://my-bucketBucket my-bucket, prefix vera-artifacts/
s3i://my-bucket/vvpBucket my-bucket, prefix vvp/vera-artifacts/
wasbs://my-container@myaccount.blob.core.windows.net/vvpContainer my-container, prefix vvp/vera-artifacts/

Startup Time

The upload runs before the Application Manager starts, so the vvp-appmanager-0 pod stays in Init until the upload finishes.

  • First start after installation or upgrade: the artifacts are copied. This takes several minutes for each VERA version, depending on the bandwidth to your blob storage.
  • Every later start: objects that are already present are skipped, but each one is still compared with the copy in your blob storage. On Amazon S3, this adds about 2 minutes for each VERA version, about 4 minutes in total in Ververica Platform 3.1.4.

Check the Upload

Each VERA version is uploaded by its own init container of the Application Manager pod, named after the version. The log ends with a done in line that reports how many objects are stored:

BASH
1kubectl -n vvp-system logs vvp-appmanager-0 -c vera-artifact-4-5-4
2kubectl -n vvp-system logs vvp-appmanager-0 -c vera-artifact-4-6-1

If the upload fails, for example because the credentials don't allow writing, the init container exits with an error, its log names the cause, and the Application Manager doesn't start. Fix the cause. The next start of the pod resumes the upload.

Private Image Registry

The artifacts come from the platform-images/vera-artifact image, one tag for each VERA version. If you use a private image registry, mirror every tag listed in vvp-appmanager.veraArtifactSeed.veraArtifactVersions in the chart values. For Ververica Platform 3.1.4, these are:

  • platform-images/vera-artifact:4.5.4_s068cyizCBtR
  • platform-images/vera-artifact:4.6.1_s068cyinnUlp

Starting with Ververica Platform 3.1.4, platform-images/vera-service is no longer needed.

Upgrading from 3.1.3

  • The vera Deployment and Service are removed. The first start of the Application Manager after the upgrade uploads the artifacts, as described in Startup Time.
  • VERA 4.6 becomes available: vera-4.6-jdk11-flink-1.20 and vera-4.6-jdk17-flink-1.20. See VERA Engine Versions.
  • The SQL Service volume (vvp-appagent.persistentVolume) needs more space with four engine versions. New installations get 30 GiB. An upgraded installation keeps its existing volume, by default 10 GiB, which fills up. Set vvp-appagent.persistentVolume.size to 30Gi when you upgrade. The storage class must allow volume expansion.

Uninstalling

Uninstalling Ververica Platform leaves the artifacts in your blob storage. To free the space, delete the vera-artifacts/ prefix below the path of baseUri.

Was this helpful?