Logo
2.10
  • Getting Started
    • Getting Started - Installation
    • Getting Started - Flink SQL
    • Getting Started - Flink Operations
  • Installation & Upgrades
  • Administration & Configuration
  • User Guide
  • Resources
  • Release Notes
Ververica Platform
  • Docs »
  • Getting Started »
  • Getting Started - Flink Operations

Getting Started - Flink Operations¶

See also

Please see Getting Started - Installation to install Ververica Platform before going through this guide.

Note

If you have installed the optional logging or metrics integrations, please check the logging setup and the metrics setup to ensure you have the correct services port-forwarded to access them on your local machine.

In this guide, you will create and manage a Ververica Platform Deployment based on a JAR that packages an Apache Flink® DataStream, DataSet or Table API program. Please see Getting Started - Flink SQL for information on how to use Flink SQL in Ververica Platform.

Creating Your First Deployment¶

Deployments are the core resource to manage Apache Flink® jobs within Ververica Platform. A Deployment specifies the desired state of an application and its configuration. At the same time, Ververica Platform tracks and reports each Deployment’s status and derives other resources from it. Whenever the Deployment specification is modified, Ververica Platform will ensure the running application will eventually reflect this change.

You can either use the web user interface or the REST API to manage Deployments.

Web User Interface
REST API

Click + Create Deployment in the top right of the “Deployments” page. For your first Deployment we recommend to use the Standard view.

Standard Form
  • Name: Provide a name such as Top Speed Windowing
  • Deployment Target: Create a new Deployment Target. A Deployment Target links a Deployment to a Kubernetes namespace, which your Flink applications will be deployed into. In this case you can use the vvp-jobs namespace that we created earlier.
  • Parallelism: Set the parallelism to 1
  • Jar URI: Provide a URI to the JAR containing your Flink program. If you do not have have an artifact at hand, you can use

..https://maven.ververica.com/artifactory/vvp-flink-release/com/ververica/flink/flink-examples-streaming_2.12/1.16.1-stream1/flink-examples-streaming_2.12-1.16.1-stream1-TopSpeedWindowing.jar

To provide your own JAR, click on the upload icon on the right side of the Jar URI field and upload your program.

Finally, click Create Deployment to start your Apache Flink® application.

POST vvp-resources/deployment_target.yaml to Ververica Platform’s REST API to create the Deployment Target:

$ curl localhost:8080/api/v1/namespaces/default/deployment-targets \
    -X POST \
    -H "Content-Type: application/yaml" \
    -H "Accept: application/yaml" \
    --data-binary @vvp-resources/deployment_target.yaml

Afterwards, you can POST vvp-resources/deployment.yaml to Ververica Platform REST API to create the Deployment:

$ curl localhost:8080/api/v1/namespaces/default/deployments \
    -X POST \
    -H "Content-Type: application/yaml" \
    -H "Accept: application/yaml" \
    --data-binary @vvp-resources/deployment.yaml

Ververica Platform will now create a highly available Flink cluster, which runs your Flink application in the vvp-jobs Kubernetes namespace. Checkpointing and Savepoints, as well as Flink Jobmanager failover, have automatically been configured by the platform based on our Universal Blob Storage configuration.

Once the Deployment has reached the RUNNING state you can also checkout the Flink UI.

Deployment Overview

Upgrading Your First Deployment¶

One of the core features of Ververica Platform is application lifecycle management for stateful stream processing applications. As part of this, Ververica Platform takes care of consistently migrating your distributed state when you make changes to your application. For example, you can change your Deployment by changing the parallelism, i.e. rescaling your Flink job.

Web User Interface
REST API

In the Deployment overview page, click Configure Deployment, change the parallelism to 2, and click Save Changes.

Standard Form

Change the value of spec.template.parallelism in vvp-resources/deployment.yaml to 2. Then PATCH the existing Deployment with the changed resource. For this, you need the metadata.name of your Deployment. If you did not change the default in vvp-resources/deployment.yaml, the name is top-speed-windowing.

Afterwards, you can PATCH your Deployment with the modified version of vvp-resources/deployment.yaml to scale up the Deployment.

$ curl localhost:8080/api/v1/namespaces/default/deployments/top-speed-windowing \
    -X PATCH \
    -H "Content-Type: application/yaml" \
    -H "Accept: application/yaml" \
    --data-binary @vvp-resources/deployment.yaml

Under the hood, Ververica Platform now performs an application upgrade according to the configured Upgrade and Restore Strategy. For your Deployment, these default to STATEFUL and LATEST_STATE and Ververica Platform has triggered a graceful shutdown of your Flink application while taking a consistent snapshot of its state via a savepoint. You can see a list of all past savepoints and retained checkpoints for this Deployment in the Snapshots tab. It then restarts your application from the latest snapshot, which is the one taken during shutdown.

Next Steps¶

Now that you have a Ververica Platform instance and your first Deployment is up and running, there are multiple areas you can look into to learn more about the platform.

  • Have a look at the Advanced tab for configuring Deployments and learn more about Deployments and how to manage their lifecycle
  • Take a deep-dive into administration and customization, e.g. around logging and metrics, deployment defaults or access control
  • Have a look at how to manage artifacts.
Next Previous

© Copyright 2023, Ververica GmbH.

Apache Flink, Apache Hadoop, Apache Kafka, Apache ORC, Apache Parquet, Apache Avro, Apache HCatalog, Apache HBase, Apache Cassandra, Flink®, Hadoop®, Kafka®, ORC®, Parquet®, Avro®, HCatalog®, HBase®, Cassandra®, Apache®, the squirrel logo, and the Apache feather logo and any other Apache project name or logo are either registered trademarks or trademarks of The Apache Software Foundation.

Report an issue with this documentation page | Imprint

Other Versions v: v2.10
Tags
v2.10
v2.9
v2.8
v2.7
v2.6
v2.5
v2.4
v2.3
v2.2
v2.1
v2.0
v1.4
v1.3
v1.2
v1.1
v1.0
sql-eap