Artifact Management
Ververica Platform provides simple file storage per Ververica Platform Namespace that enables users to upload Apache Flink® job artifacts and reference them in Deployments.
To use the built-in artifact management, specify a blob storage base URI in your configuration.
This article explains how to a upload an job artifact to Ververica Platform. Please see Packaging your Application for how to package your Apache Flink® program into a JAR to upload.
Uploading Artifacts
To upload an artifact, visit the "Artifacts" section in the Ververica Platform UI. Here, you can upload files up to 50MB, which are usable in Deployments in the same Namespace.
There is currently a limit of 1000 artifacts per Namespace. If there are more than 1000 objects in the storage location for a given Namespace's artifacts, only the first 1000 lexicographically will be visible in the Ververica Platform UI.
Uploads are limited to 50MB. If your files are larger than this, you must upload them directly to your storage provider (see storage scheme below). Files placed in the appropriate location in storage will be visible in the Ververica Platform UI.
Uploading a file with the same name as an existing one will overwrite the existing file.
Using Artifacts
To use an application artifact in a Deployment you need to provide it in the Deployment template (spec.template.spec.artifact.jarUri).
Managed Artifacts
To use a managed artifact in a Deployment, use the URI of the uploaded artifact described in Artifact Storage Scheme or use the drop-down menu in the Deployment creation form and choose the desired artifact.
Choosing an artifact in this way is equivalent to providing the full URI to the artifact. A Deployment is agnostic to whether the artifact is managed by Ververica Platform or externally.
External Artifacts
For external artifacts, Ververica Platform supports http/https (with basic auth) as well as the schemes of all supported Storage providers.
Check out the Artifact Storage page if your artifact storage uses TLS and serves a certificate signed by a non-public CA.
Ververica Platform does not store external JAR artifacts permanently. If you want to be able to go back to earlier versions of your Flink job, ensure that you are versioning the JARs properly, and that they are stored for the duration you want to be able to reference them.
Bundled Artifacts
You can bundle artifacts as part of a custom Flink Docker image and reference them via the file:// scheme in the artifact section of the Deployment. During startup, these bundled dependencies will be linked to Flink's usrlib directory (/flink/usrlib). Where possible (e.g. JAR artifacts), it is allowed to mix bundled and external dependencies.
When building the custom Flink Docker image, the bundled artifact(s) must be placed under a different location than /flink/usrlib, because this directory is the mount point for dynamically fetched artifacts.
Managing Artifacts
To download and delete artifacts, visit the "Artifacts" section in the Ververica Platform UI.
Artifact Storage Scheme
Artifacts are stored in the configured blob storage backend, scoped to a Namespace. For example,
given a blob storage base URI of s3://my-bucket/vvp, the artifact my-jar.jar in Namespace
namespaces/default will be stored at the path:
s3://my-bucket/vvp/artifacts/namespaces/default/my-jar.jar
If you wish to upload files larger than 50MB or integrate with a CI/CD pipeline, you should place your artifacts directly into storage at the appropriate path.