Skip to main content
Version: 2.12

Ververica Platform 2.2.0

Release Date: 2020-08-04

Changelog​

  • With the addition of our distribution of Apache Flink® 1.11.1 Ververica Platform 2.2.0 adds support for Apache Flink® 1.11. Apache Flink® versions 1.10 and 1.11 are supported under SLA. Apache Flink® 1.9 images are no longer provided in this version but are still supported on a best-effort basis.

  • Please check the release notes of Flink 1.11.0 and Flink 1.11.1 for a complete list of features, improvements and fixes.

  • For Stream Edition the following Apache Flink® Docker images are available. Please check Ververica Platform Docker Images for all available Apache Flink® images and additional tags.

    • 1.11.1-stream1-scala_2.11
    • 1.11.1-stream1-scala_2.12

  • For Spring Edition the following / archives are available

Vulnerability Fixes​

The following security vulnerabilities have been fixed compared to 1.10.1-[stream|spring]2:

CVE-2019-0201, CVE-2019-12399, CVE-2019-12402

  • This update to our distribution of Apache Flink® 1.10.1 fixes a bug in the Kubernetes-based Flink Jobmanager failover. When using high-availability: vvp-kubernetes in conjunction with state.backend.incremental: true some checkpoint files were cleaned up prematurely leading to FileNotFoundExceptions during recovery.

  • For Stream Edition the following Apache Flink® Docker images are available. Please check Ververica Platform Docker Images for all available Apache Flink® images and additional tags.

    • 1.10.1-stream2-scala_2.11
    • 1.10.1-stream2-scala_2.12

  • For Spring Edition the following Apache Flink® archives are available

Vulnerability Fixes​

The following security vulnerabilities have been fixed compared to 1.10.1-[stream|spring]1:

CVE-2017-5645, CVE-2020-9488

  • This update to our distribution of Apache Flink® 1.9.3 fixes a bug in the Kubernetes-based Flink Jobmanager failover. When using high-availability: vvp-kubernetes together with with incremental checkpoints state.backend.incremental: true some checkpoint files were cleaned up prematurely leading to FileNotFoundExceptions during recovery.

  • For Stream Edition the following Apache Flink® Docker images are available. Please check Ververica Platform Docker Images for all available Apache Flink® images and additional tags.

    • 1.9.3-stream2-scala_2.11
    • 1.9.3-stream2-scala_2.12

  • For Spring Edition the following Apache Flink® archives are available

Vulnerability Fixes​

None

Autopilot​

Ververica Platform 2.2.0 introduces an autopilot for Apache Flink® streaming applications. When activated for a Deployment the autopilot will adjust the parallelism of your Deployment to varying load in order to maintain a back-pressure free configuration that matches a specified target utilization of your pipeline. Please check out Autopilot for configuration options, assumptions and future work.

Other Changes​

Universal Blob Storage with Apache Hadoop®​

You can now use HDFS 2 or HDFS 3 for Universal Blob Storage including authentication via Kerberos Keytabs. Besides HDFS, AWS S3, and Azure Blob Storage, as well as any S3-compatible object storage services like MinIO have been supported for a while.

Support for SQL Server​

You can configure Ververica Platform to store its metadata in Microsoft SQL Server as well as Azure SQL.

The full list of supported remote RDBMS now includes: MariaDB, Microsoft SQL Server, MySQL, and PostgreSQL.

Please see Persistence Configuration for more details.

Suspend with Draining​

When a Deployment is suspended or a stateful upgrade is triggered, Ververica Platform submits a stop command to Apache Flink®. This command will atomically trigger a savepoint and stop the job. You can now instruct Ververica Platform to additionally drain the pipeline. Draining will result in all future event time timers of the Flink job to fire before the stop command is executed. Checkout Draining for details.

Bootstrap Tokens​

Ververica Platform now supports a Bootstrap Token, specified during installation or upgrade, which can be used as an API token with administrator privileges. This is useful for performing certain bootstrapping tasks such as creating an initial Namespace and assigning its members.

Apache Flink® 1.10+ allows to set checkpointing configurations via the Deployment. The following entries have been added to the default Global Deployment Defaults.

  flinkConfiguration:
    execution.checkpointing.interval: 10s
    execution.checkpointing.externalized-checkpoint-retention: RETAIN_ON_CANCELLATION

In addition, execution.checkpointing.min-pause defaults to the value of execution.checkpointing.interval if not configured otherwise.

:::attention For existing Deployments that configure the checkpointing interval in application code but do not set the execution.checkpointing.min-pause, execution.checkpointing.min-pause will be configured to 10s effectively prohibiting checkpointing intervals lower than 10 seconds. In this case, explicitly setting execution.checkpointing.min-pause to the desired value on the Deployment resolves the issue. :::

In particular, Ververica Platform now warns users if their checkpointing configuration does not support the selected upgrade strategy.

UBS with AWS S3 on EKS​

For Universal Blob Storage on AWS S3, we do not require hard coded credentials anymore if the credentials are inherited from the roles attached to EC2 instances underlying your Kubernetes Cluster.

If you have instance-level authentication configured, you can skip providing additional credentials during installation.

Bundled Apache Hadoop® S3 File System​

From Flink 1.10 onwards all Flink images include both the HadoopS3Filesystem and the PrestoS3FileSystem. In our experience the PrestoS3FileSystem should be used by default unless you are using the StreamingFileSink, which requires the HadoopS3Filesystem. You can switch the file system implementation on a use case basis via the scheme.

SchemeFile System
s3PrestoS3FileSystem
s3pPrestoS3FileSystem
s3aHadoopS3Filesystem

Deployment Overview Page​

We have slightly reworked the Deployment page in the web user interface. It now displays the full Deployment configuration including logging configuration and pod templates.

Community Edition Feedback Widget​

For community edition, the web user interface now contains a widget that allows to directly submit feedback & questions from within the application.

CVE-2017-18640, CVE-2018-10237, CVE-2019-12402, CVE-2019-12418, CVE-2019-17563, CVE-2019-20444, CVE-2019-20445, CVE-2020-1935, CVE-2020-1938, CVE-2020-1938, CVE-2020-5397, CVE-2020-5398, CVE-2020-8022, CVE-2020-8022, CVE-2020-8203, CVE-2020-9484, CVE-2020-9484, CVE-2020-9488, CVE-2020-11612, CVE-2020-11996, CVE-2020-13692

Upgrade​

We recommend upgrading via Helm using the following commands:

    helm repo add ververica https://charts.ververica.com
    helm upgrade [RELEASE] ververica/ververica-platform --version 4.2.0 --values custom-values.yaml