Ververica Platform 2.3.0
Release Date: 2020-10-21
Changelog
Helm Charts
Additional memory is allocated to the gateway container hosting the SQL service. If memory is scarce in your environment and you don't use the SQL functionality yet, feel free to limit the requested memory as follows:
gateway:
resources:
limits:
memory: 1Gi
requests:
memory: 1Gi
If on the other hand, you expect many concurrent SQL users, we recommend to increase the available memory as described in the advanced SQL configuration guide.
Apache Flink®
Ververica Platform 2.3.0 continues to support Apache Flink® 1.10 and 1.11 under SLA. We have released new Flink 1.10.2 and 1.11.2 builds that enable Ververica Platform's support for Flink SQL and fix vulnerabilities in Flink's dependencies.
Apache Flink® 1.11.2
-
For Stream Edition the following Apache Flink® Docker images are available. Please check Ververica Platform Docker Images for all available Apache Flink® images and additional tags.
1.11.2-stream2-scala_2.11
1.11.2-stream2-scala_2.12
-
For Spring Edition the following / archives are available
Vulnerability Fixes
The following security vulnerabilities have been fixed compared to 1.11.2-[stream|spring]1
:
None
Apache Flink® 1.10.2
-
For Stream Edition the following Apache Flink® Docker images are available. Please check Ververica Platform Docker Images for all available Apache Flink® images and additional tags.
1.10.2-stream2-scala_2.11
1.10.2-stream2-scala_2.12
-
For Spring Edition the following / archives are available
Vulnerability Fixes
The following security vulnerabilities have been fixed compared to 1.10.2-[stream|spring]1
:
Flink SQL on Ververica Platform
With this release Ververica Platform becomes and end-to-end platform for the development and operations of Flink SQL. Please check out the SQL Quick Start and SQL Development User Guide for how to get started. Here are some highlights:
A Web-Based SQL Editor
Ververica Platform comes with a purpose-built SQL editor as part of its web user interface. It provides auto-completion, continuous validation, a schema explorer and last but not least query previews.
Flink SQL Jupyter Extension
In addition, you can develop and submit Apache Flink® SQL queries from IPython notebooks via a publicly available Jupyter extension. The Jupyter Extension is released & versioned independently of the platform and has not yet reached feature parity with the editor. Specifically, result previews will only be added in the next release of the extension.
An Operational Framework for Flink SQL
SQL queries are configured and deployed just like regular Deployments, providing the same powerful lifecycle management for SQL queries (recovery, upgrades) as for JAR-based deployments. See Managing SQL Deployments for more.
Catalog & Connector Management
Ververica Platform comes with a built-in catalog to store tables and functions metadata, reducing the need for an external catalog service. In addition, the platform is able to connect to external catalogs like Apache Hive®’s Metastore. Please see Connectors for a list of all packaged connectors and formats.
User-Defined Functions (UDF) Management
Apache Flink® requires Java or Scala user-defined functions to be packaged as JAR files. Ververica Platform simplifies the management (registration, update, deletion) of UDF JARs and the registration of functions in the catalog.
REST API
As always, all of the above functionality is exposed via a public REST API.
Other Changes
Session Clusters
You can now use Ververica Platform to manage Apache Flink® Session Clusters. In this release, the primary use case of Session Clusters in Ververica Platform is running preview queries submitted via the SQL editor. Full support for Session Clusters is planned for Ververica Platform 2.4.0 including:
- Deployments on Session Clusters
- SSL
- Kubernetes-based Flink Jobmanager failover
- Reactive cluster scaling
maven.ververica.com
So far, we have distributed our fork of Apache Flink® via Docker images (Stream Edition) and archives (Spring Edition). These Docker images only contain the binary distribution of Apache Flink® and, hence, do not cover all of the components that we support commercially: connectors, for example, are not part of the binary distribution.
In order to support you more efficiently, we will, from now, publish all dependencies for each release of our distribution of Apache Flink in a public Maven repository hosted under maven.ververica.com. Please check Packaging your Application for how to integrate our repository into your build system.
Web User Interface
Navigation
The navigation side-bar has been reworked, specifically:
Secret Values
was moved toAdministration
Namespaces
was moved to the namespace picker and is now calledManage Namespaces
.- A top-level
SQL
section was added. - A
Session Cluster
page was added under Deployments.
Deployment Form Improvements
You can now create your first Deployment Target per namespace right in the Deployment creation form instead of the Deployment Targets
page.
Similarly, you can upload JAR artifacts directly from the Deployment creation form, not only via the Artifacts
page.
Additional Dependencies for JAR Artifacts
For JAR Deployments, you may provide additional dependencies besides your main artifact containing your Apache Flink® program.
artifact:
kind: jar
jarUri: https://artifacts/flink-job.jar
additionalDependencies:
- https://artifacts/flink-connector.jar
- https://artifacts/additional-library.jar
Vulnerability Fixes (outside of Apache Flink®)
The following security vulnerabilities have been fixed compared to 2.2.1
:
CVE-2017-18869, CVE-2019-20149, CVE-2020-7598, CVE-2020-5421, CVE-2020-8203, CVE-2020-14344, CVE-2020-14363
Upgrade
We recommend upgrading via Helm using the following commands:
helm repo add ververica https://charts.ververica.com
helm upgrade [RELEASE] ververica/ververica-platform --version 4.3.0 --values custom-values.yaml
As mentioned in the Helm Charts section, the default resource requirements of Ververica Platform has changed with this release.