Ververica Platform 2.0.0

Changelog

General

  • Ververica Platform now includes three years of support for all its components including our distribution of Apache Flink®. Previously, the support for each version was limited to one year. Please check Supported Versions & Compatibility for details.

  • We changed the high-level structure of this documentation:

    • Getting Started gives a hands-on introduction to Ververica Platform.
    • Installation & Upgrades contains all information around installation of Ververica Platform on Kubernetes.
    • Administration & Configuration is directed towards administrators and teams operating Ververica Platform. It covers more advanced configuration and customization scenarios and introduces Namespaces and Deployment Targets.
    • User Guide covers the main concepts and topics for users of the platform like Apache Flink Deployments and Artifacts.
    • Streaming Ledger gives an introduction to Ververica Platform Streaming Ledger, including a dedicated quick start.
  • For Ververica Platform 2.x the minimum Kubernetes version was raised to Kubernetes 1.11. To facilitate the upgrade, Ververica Platform 2.0 only requires Kubernetes 1.9 (as Ververica Platform 1.x). Please check Supported Versions & Compatibility for details.

Helm Charts

  • The name and labels of the generated Kubernetes resources (pods, services, etc.) have changed (primarily “daplatform” has been renamed to “ververica-platform”).
  • The “ververica-platform” pod (formerly “appmanager”) now runs three Docker containers (“appmanager”, “gateway” and “ui”).
  • The format and structure of Ververica Platform’s configuration as well as the way this configuration is passed via helm’s values.yaml has changed. Check out Configuration and Installation using Helm for the details.
  • As of this release all Ververica Platform images share the same Debian base image (openjdk:8-jre-slim-buster). The images are published under registry.platform.data-artisans.net/v2.0.

Namespaces & Access Control

  • The Namespace resource has been changed and is not compatible with previous versions.

  • API tokens are now namespaced and the resource is not compatible with previous versions.

  • The configuration of OpenID Connect has slightly changed. Please check Authentication for details.

  • In Ververica Platform 2.0 we have simplified our approach to role-based access control. (Cluster)Role and (Cluster)RoleBinding resources have been removed. Instead, users or groups can be bound to one of three preset roles per Namespace.

    • Owners have full access to the Namespace, including role and secret management.
    • Editors have write access to all the resources of the Namespace excepts API tokens and Deployment Targets.
    • Viewers have read-only access to all the resources of the Namespace except Secret Values and the Flink web user interface.

    Please see Authorization for details.

Universal Blob Storage

From this release on Ververica Platform is (optionally) configured with a blob storage baseUri. The platform uses this location for all blob storage requirements of its components:

  • Artifacts (see below)
  • Apache Flink® checkpoints, savepoints and high-availability storage. The pre-configured locations can be overriden on a Deployment basis.

Initially AWS S3, Azure Blob Storage and local filesystem are supported. Please check Universal Blob Storage for details.

Attention

Ververica Platform shares the blob storage credentials with Apache Flink® via the Flink configuration (a Kubernetes ConfigMap in the namespace of the respective Apache Flink® deployment). This is due to a current shortcoming in Apache Flink® and we plan to use Kubernetes secrets and environment variables in the future.

Artifact Management

Ververica Platform can now be used to manage application artifacts directly. The artifacts are stored via the configured blob storage service (see above). Externally stored artifacts adressable via http(s) are still supported. Please check Artifact Management for details.

Web User Interface & REST API

  • The web user interface was given a general overhaul. Most notably, all top-level resources have been pulled out of “Settings” into separate menu items.
  • We have added configuration assistants for Flink restart strategies and Flink high availability when configuring a Deployment via the web user interface.
  • The placeholders for logging & metrics link templates have changed. See Web User Interface for details.
  • Default Apache Flink® images and the Apache Flink® image tag drop-down are configured via the flinkVersionMetadata now. See Flink Version Metadata for details.
  • Except for Namespace s (see Namespaces) and ApiToken s (see API Tokens) the REST API is compatible with previous versions.

Migration

  • Ververica Platform 2.0.0 introduces breaking changes to Ververica Platform 1.x.x. Hence, an automatic upgrade is not possible. The upgrade involves a new installation of Ververica Platform 2.0.0 and semi-manual migration of existing Namespaces, Deployments, Savepoints, etc.. Please reach out to your technical account manager for assistance prior to the upgrade.
  • An upgrade to Ververica Platform 2.0.0 will introduce a short downtime for all Apache Flink® Deployments comparable to a normal Apache Flink® application upgrade or rescale operation.