Ververica Platform 2.5.0¶

Release Date: 2021-06-11

Ververica Platform 2.5.0 supports Apache Flink® 1.13 and Apache Flink® 1.12 under SLA. Apache Flink® 1.11 is deprecated in this version and supported on a best-effort basis. Apache Flink® 1.10 was previously deprecated and has now been dropped.

Java 11 Support ¶

Ververica Platform 2.5.0 Stream Edition adds support for Java 11, in addition to the continued Java 8 support, on new Apache Flink® Docker images.
Starting with 1.12.4, Docker image names contain the supported Java version suffix (-java8 or -java11).

Apache Flink® 1.13.1 ¶

With the addition of our distribution of Apache Flink® 1.13.1, Ververica Platform 2.5.0 adds support for Apache Flink® 1.13.
Please check the release notes of Flink 1.13.0 and Flink 1.13.1 for a complete list of features, improvements and fixes.
For Stream Edition the following Apache Flink® Docker images are available. Please check Ververica Platform Docker Images for all available Apache Flink® images and additional tags.
- 1.13.1-stream1-scala_2.11-java8
- 1.13.1-stream1-scala_2.11-java11
- 1.13.1-stream1-scala_2.12-java8
- 1.13.1-stream1-scala_2.12-java11
For Spring Edition the following / archives are available
- flink-1.13.1-spring1-scala_2.11.tgz
- flink-1.13.1-spring1-scala_2.12.tgz

Apache Flink® 1.12.4 ¶

This update to our distribution of Apache Flink® 1.12.4 includes a bugfix for FLINK-22494 that was not backported to the official Apache Flink® 1.12.4 release.
- The fix handles cases where persisting the Checkpoint metadata to High Availability systems fails after the data has been written to the underlying storage.
- Previously, the Checkpoint was discarded but the metadata was not, which could lead to issues during recovery.
- Now the Checkpoint is kept for safer recovery, though you must now manually clean up the High Availability metadata in these situations.
For Stream Edition the following Apache Flink® Docker images are available. Please check Ververica Platform Docker Images for all available Apache Flink® images and additional tags.
- 1.12.4-stream1-scala_2.11-java8
- 1.12.4-stream1-scala_2.11-java11
- 1.12.4-stream1-scala_2.12-java8
- 1.12.4-stream1-scala_2.12-java11
For Spring Edition the following / archives are available
- flink-1.12.4-spring1-scala_2.11.tgz
- flink-1.12.4-spring1-scala_2.12.tgz

Vulnerability Fixes ¶

The following security vulnerabilities have been fixed compared to 1.12.2-[stream|spring]2:

IO-675, a transitive dependency of Hadoop

Flink SQL ¶

Views are now supported in the SQL Editor and the built-in, persistent Ververica Platform catalog.
The JDBC Catalog is now packaged and can be used out of the box.
Custom catalog implementations can now be added to Ververica Platform.
Hive has been dropped as a packaged catalog and connector.
The SET statement is now supported.

Support for Views ¶

The built-in Ververica Platform Catalog now supports views. A view is defined by a query and allows reusing a query in different places without having to use connectors in-between. Apache Flink® will inline the query defining the view in queries referencing it.

In the SQL Editor, views are now listed next to the tables of a catalog in the schema explorer.

Packaged JDBC Catalog for Postgres ¶

The JDBC catalog in Flink is now packaged with the Postgres driver and can be used out of the box. Other JDBC drivers may be supported in the future.

CREATE CATALOG pg_catalog WITH (
  'type' = 'jdbc',
  'base-url' = 'jdbc:postgresql://…',
  'default-database' = '…',
  'username' = '…',
  'password' = '…'
)

See JDBC Catalog for more information.

Custom Catalogs ¶

Ververica Platform 2.5.0 supports registering custom catalogs per namespace through the REST API and the UI, similar to custom connectors and formats. These catalogs can then be used both in the SQL Editor and for SQL Deployments.

Custom catalogs can optionally also be marked as read-only catalogs. In this case, Ververica Platform wraps the catalog to prevent any write operation to the catalog such as dropping tables.

See Custom Catalogs for the full documentation.

SET Statement ¶

Ververica Platform 2.5.0 supports the SET statement to allow modifying options for a SQL script. This statement must always appear prior DDL, DML or DQL statements. Both key and value must be quoted with a single quote:

SET 'table.local-time-zone' = 'UTC';

Note that options which are set and required by Ververica Platform or affect the Deployment itself are not supported to be modified this way and an error will be thrown.

Savepoint deletion ¶

Savepoints can now be deleted via the REST API and the web user interface if universal blob storage is enabled. Both the Savepoint resource as well as the underlying data in blob storage will be removed.

Please see Deleting a Savepoint Resource for more details on deletion requests and responses as well as prerequisites and limitations.

REST API ¶

When creating a Deployment via a POST or PUT operation, invalid names will no longer be silently converted if metadata.displayName is not provided. The request will be rejected instead.

Session Clusters ¶

You can now add additional labels to the Kubernetes deployments, jobs, and services that are created for Apache Flink® session clusters. Please refer to the Kubernetes Resources page for more information.

Deployments ¶

You can now add additional labels to the Kubernetes deployments, jobs, and services that are created for Apache Flink® application clusters. Please refer to the Kubernetes Resources page for more information.

Kubernetes Garbage Collection ¶

For Application Mode Deployments, you can now specify how Kubernetes cleans up Flink JobManager and TaskManager Pods during shutdown and upgrade via annotations. This allows you to tightly control the requested resources during these scenarios.

See Kubernetes Garbage Collection for more information.

Helm Charts ¶

You can now specify a path to the license file via the licenseConfigPath parameter, which can be mounted into the Ververica Platform pod from an external source (Kubernetes secret, etc.). For more information, please check the configuration page.

volumes:
- name: "license"
  secret:
    secretName: "vvp-license"

volumeMounts:
- name: "my-volume"
  mountPath: "/etc/"

licenseConfigPath: "/etc/license.yaml"

Additional labels can now be added to all resources created by the chart via the extraLabels parameter.

extraLabels:
  team: "infra"
  env: "prod"

Vulnerability Fixes (outside of Apache Flink®)¶

The following security vulnerability in non-Flink components of Ververica Platform have been fixed compared to 2.4.2:

Other Changes ¶

The Event resources can now be automatically cleaned up, which can improve listing performance for long-running Deployments with many events. See event service configuration for more information.

Upgrade ¶

We recommend upgrading via Helm using the following commands:

$ helm repo add ververica https://charts.ververica.com
$ helm repo update
$ helm upgrade [RELEASE] ververica/ververica-platform --version 5.1.0 --values custom-values.yaml

Note

If any newly-introduced Savepoint deletion features are used after upgrade, downgrading to any previous Ververica Platform version will require manual database changes.