In this guide you will setup Ververica Platform, integrate it with Minio for blob storage and run your first stateful Apache Flink Deployment.
This quickstart provides a simple development environment, but is not meant for production. For production deployment information, see the installation guide.
This quickstart requires access to
- a Kubernetes cluster,
helmto install the platform, and
- Minio for blob storage.
Check out the appendix for instructions on how to set up the required components on your local machine.
First download the Ververica Platform Helm Chart from the customer portal or the Ververica website (non-customers).
Then create a helm configuration file
values.yaml with the following keys to configure the platform to use Minio for blob storage.
vvp: blobStorage: baseUri: s3://<your_bucket> s3: endpoint: <your_minio_endpoint> blobStorageCredentials: s3: accessKeyId: <access_key_id> secretAccessKey: <secret_access_key>
The bucket already needs to exist in Minio or S3.
The appendix contains a full example configuration. Now you can install the platform using
$ helm install \ --name vvp \ --values values.yaml \ ververica-platform-2.0.2.tgz
You can then access the Ververica Platform UI by port forwarding to
$ kubectl port-forward service/vvp-ververica-platform 8080:80
Before you can create a Deployment you need to add a Deployment Target.
A Deployment Target is used to select the Kubernetes namespace, which your Apache Flink applications will be deployed into.
Deployment Targets in the left side-bar and click
Add Deployment Target.
Give it a name and use the same Kubernetes namespace as for the platform itself.
Deployment, click the
Create Deployment button on the top right corner of the page to create a new Deployment and provide it a name such as Top Speed Windowing.
Then under the
Behavior section you can specify how the application should be managed.
Deployment Target: Choose the Deployment Target that you just created to deploy the Flink application into the Kubernetes namespace it references.
Upgrade Strategy: The upgrade strategy should be
Stateful so your application state is carried across Flink jobs.
Initial State: The initial state should be
Running so that Ververica Platform will automatically deploy a Flink job when you finish creating the Deployment.
Restore Strategy: The restore strategy specifies what state should be used when starting a new Flink job. We will set this to
We can set up the configuration for your specific job by providing a URI to the Jar with your Flink program. There are two ways to do this:
either you switch to the
Artifacts screen and upload your artifact directly to the platform, or you point Ververica Platform to an externally stored artifact.
If you do not have an artifact to upload, you can use the Top Speed Windowing example which the Flink community uploads to Maven Central for each release:
Under the resources section, you specify the parallelism of your job as
1 and give jobmanager and taskmanagers
0.5 CPUs and
1G of memory each.
Finally, you press the
Create Deployment button to start your job.
After creating your job, you can try out another feature of the platform and make a change to the Deployment in a stateful and consistent manner. This could be changing the jar to release a new version of the application, or in this case, change a configuration such as the parallelism.
To make a change, you press
Modify Deployment which will either take us back to the creation screen you used earlier, or let you edit the deployment as
You can use either, but in this case use
Edit as YAML and change the parallelism of your application from
2 and press
Once the changes have been applied, Ververica Platform will take care of the rest and transition your application to use the updated configuration.
When finished, press the
Suspend button to gracefully stop all resources and stop the application.
Now that you have a running Ververica Platform instance there are multiple directions you can go into.
If you do not have a Kubernetes installation readily available you can use Minikube and Minio to run this quickstart locally.
Install Minikube by following the official instructions in the Kubernetes documentation. Once minikube has been installed you can start your local Kubernetes cluster.
$ minikube start --memory 8192 --cpus 4
Next, you will want to install Helm, a Kubernetes package manager. A full walk through of Helm can be found in the installation instructions.
$ kubectl --namespace kube-system create serviceaccount tiller $ kubectl create clusterrolebinding tiller --clusterrole cluster-admin --serviceaccount=kube-system:tiller $ helm init --service-account tiller
With the Helm install command below you can quickly install Minio, an S3 compatible object store. Note that you might need to change the clusterIP adress depending on your installation.
# Install Minio with username: admin and password: password, default bucket: vvp $ helm install \ --set accessKey=admin,secretKey=password,service.clusterIP=10.96.1.1,defaultBucket.enabled=true,defaultBucket.name=vvp \ --name my-minio stable/minio
To configure Ververica Platform in this particular setup, you can use the following
vvp: blobStorage: baseUri: s3://vvp s3: endpoint: http://10.96.1.1 blobStorageCredentials: s3: accessKeyId: admin secretAccessKey: password