Configuration
This section contains example configurations for Ververica Platform. The configuration can be passed
to Ververica Platform during the installation with helm via the
values.yaml
file under the vvp
key.
Persistence Configuration
Remote RDBMS persistence is only available in Ververica Platform Stream Edition and above.
Ververica Platform persists its metadata using JDBC, either in a remote RDBMS or locally using SQLite.
This refers to data owned by the Platform itself and accessed via its API such as Namespaces, Deployments, Jobs, and Savepoint metadata, but does not include artifacts, Apache Flink® checkpoint and savepoint data etc.
Currently, the following remote RDBMSs are supported:
- MariaDB/MySQL Persistence
- PostgreSQL Persistence
- Microsoft SQL Server Persistence
The simplest mode of operation is using the preset configuration vvp.persistence.type: local
which uses a Kubernetes PVC to store a SQLite database.
This preset also configures SQLite with some important settings. If you use SQLite with vvp.persistence.type: jdbc, we recommend using the following JDBC URL parameters: journal_mode=WAL&synchronous=FULL&busy_timeout=10000
Please do not use a network-backed filesystem for backing your SQLite database. There is no guarantee that in the case of a network partition, the data in SQLite is not corrupted. We recommend using a dedicated remote RDBMS for production workloads.
Instead, if you wish to use a remote datastore, specify vvp.persistence.type: jdbc
and provide
an appropriate Spring datasource configuration under vvp.persistence.datasource
.
MariaDB/MySQL Persistence
Ververica Platform supports MySQL persistence using the MySQL-compatible MariaDB JDBC connector.
Therefore, if you wish to use MySQL with Ververica Platform, you must use mariadb
in your
JDBC connection URL.
vvp:
persistence:
type: jdbc
datasource:
url: jdbc:mariadb://mysql.internal:3306/vvp
username: vvp
password: password
PostgreSQL Persistence
vvp:
persistence:
type: jdbc
datasource:
url: jdbc:postgresql://postgresql.internal:5432/vvp
username: vvp
password: password
Microsoft SQL Server Persistence
vvp:
persistence:
type: jdbc
datasource:
url: jdbc:sqlserver://mssql.internal;databaseName=vvp
username: vvp
password: password
Authentication Configuration
Bootstrap Token
Ververica Platform supports a "bootstrap token", specified during installation or upgrade, which can be used as an API token with administrator privileges. This is useful for performing certain bootstrapping tasks such as creating an initial Namespace and assigning its members.
The token can be any non-empty string and is set by assigning vvp.auth.bootstrapToken.token
in
a Helm values file or on the command line.
For example, if you install or upgrade Ververica Platform using Helm and include the CLI option:
--set vvp.auth.bootstrapToken.token=my-secret-token
, you (or an automated task) could create a
Namespace by running:
$ curl \
-X POST \
-H 'Authorization: Bearer my-secret-token' \
-H 'Content-Type: application/json' \
https://vvp.internal/namespaces/v1/namespaces \
-d '
{
"name": "namespaces/my-namespace",
"roleBindings": [{
"role": "owner",
"members": ["group:vvp-users"]
}]
}
'
Google Authentication Configuration Example
(Scope under vvp.auth
)
# Google auth does not support groups, so administrators must be specified manually in this
# list, or with an environment variable: vvp.admins=user:admin1@example.com,admin2@example.com
admins:
- user:admin1@example.com
- user:admin2@example.com
oidc:
#groupsClaim: # Google auth does not support groups
registrationId: google
registration:
clientId: 1009242745340-9piji4g84vkrzbp2qyp19asrk8p2ug2s.apps.googleusercontent.com
clientSecret: 4wHQZc_KHN0u8QqgpmV6TY86
provider:
userNameAttribute: email # Required to correctly identify users
Azure Authentication Configuration Example
(Scope under vvp.auth
)
admins:
- user:admin1@example.com
- user:admin2@example.com
oidc:
# NOTE: see the following Azure doc on how to enable groupsClaim at the Azure side:
# https://docs.microsoft.com/en-us/azure/active-directory/hybrid/how-to-connect-fed-group-claims
groupsClaim: groups
# NOTE: Your Azure application needs a redirect URI of <baseUrl>/login/oauth2/code/vvp
registrationId: vvp
registration:
clientId: xxxxxx-your-client-id-xxxxxx
clientSecret: xxxxxx-your-client-secret-xxxxxx
redirectUri: "{baseUrl}/{action}/oauth2/code/{registrationId}"
clientAuthenticationMethod: basic
authorizationGrantType: authorization_code
scope:
- openid
- profile
provider:
# Let Spring Boot figure out parameters itself from
# https://login.microsoftonline.com/xxxxxx-your-tenant-id-xxxxxx/v2.0/.well-known/openid-configuration
# Note: External users may not be able to sign in if you use the 'common'
# tenant ID. Instead, find your Azure AD's tenant ID and use that.
issuerUri: https://login.microsoftonline.com/xxxxxx-your-tenant-id-xxxxxx/v2.0 # No trailing slash!
# make sure, spring-boot does not fetch user info
# see https://github.com/spring-projects/spring-security/issues/7679
userInfoUri:
userNameAttribute: preferred_username # Required to correctly identify users
Amazon AWS Authentication Configuration Example
(Scope under vvp.auth
)
# This uses AWS Cognito User Pool as an identity provider. You need to create a user pool,
# create and configure an app client in the user pool, and create a Cognito domain.
admins:
- user:admin1@example.com
- user:admin2@example.com
oidc:
groupsClaim: cognito:groups
registrationId: vvp
registration:
clientId: <ClientId>
clientSecret: <ClientSecret>
redirectUri: "{baseUrl}/{action}/oauth2/code/{registrationId}"
clientAuthenticationMethod: basic
authorizationGrantType: authorization_code
scope:
- openid
- profile
provider:
issuerUri: https://cognito-idp.<AWSRegion>.amazonaws.com/<UserPoolID>
userNameAttribute: username
# LogoutURL is the logout URL of the created app client in the User Pool
endSessionEndpoint: "https://<UserPoolDomainPrefix>.auth.<AWSRegion>.amazoncognito.com/logout?client_id=<ClientId>&logout_uri=<LogoutURL>"
Full Example Configuration
This full example demonstrates most of the options needed to configure Ververica Platform for your environment.
vvp:
auth:
enabled: true
admins:
- group:vvp-admins # The OIDC-supplied group which indicates an administrator
bootstrapToken:
token: dmVydmVyaWNhLmNvbS9jYXJlZXJz
oidc:
groupsClaim: roles # The OIDC ID token claim containing a list of a user's groups
registrationId: my-oidc-provider
registration:
clientId: vvp
clientSecret: secret
redirectUri: "{baseUrl}/{action}/oauth2/code/{registrationId}"
clientAuthenticationMethod: basic
authorizationGrantType: authorization_code
scope:
- openid
provider:
authorizationUri: http://my-oidc-provider.internal/openid-connect/auth
tokenUri: http://my-oidc-provider.internal/openid-connect/token
userInfoUri: http://my-oidc-provider.internal/openid-connect/userinfo
jwkSetUri: http://my-oidc-provider.internal/openid-connect/certs
userNameAttribute: email # Required to correctly identify users
endSessionEndpoint: http://my-oidc-provider.internal/openid-connect/logout
persistence:
type: jdbc
datasource:
url: jdbc:postgresql://postgresql.internal:5432/vvp
username: vvp
password: password
blobStorage:
baseUri: s3://my-bucket/vvp
# Add additional custom Flink images to the UI, optionally setting them as the default image
# for a particular Flink minor version
flinkVersionMetadata:
- flinkVersion: 1.9.0 # The full Flink version this image supplies
imageTag: 1.9.0-custom1 # The Docker image tag for the Flink repository specified below
defaultFor:
- 1.9 # Make this the default image for deployments on Flink 1.9
flinkDeploymentDefaults:
registry: my-custom-registry.internal/vvp
repository: flink
license:
# Ververica Platform License (www.ververica.com/enterprise-trial)
data: {
"kind": "License",
"apiVersion": "v1",
"metadata": {
"id": "53b8cf22-1af2-44bd-a7ba-7420418f6572",
"createdAt": "2020-02-21T12:56:52.407899Z",
"annotations": {
"signature": "<omitted>",
"licenseSpec": "ewogICJsaWNlbnNlSWQiIDogIjUzYjhjZjIyLTFhZjItNDRiZC1hN2JhLTc0MjA0MThmNjU3MiIsCiAgImxpY2Vuc2VkVG8iIDogInRlc3QiLAogICJleHBpcmVzIiA6ICIyMDIwLTAzLTIyVDEyOjU2OjUxLjg3MzU1M1oiLAogICJwYXJhbXMiIDogewogICAgInF1b3RhLnR5cGUiIDogIlVOTElNSVRFRCIsCiAgICAidHJpYWwiIDogInRydWUiCiAgfQp9"
}
},
"spec": {
"licenseId": "53b8cf22-1af2-44bd-a7ba-7420418f6572",
"licensedTo": "My Company Inc.",
"expires": "2020-03-22T12:56:51.873553Z",
"params": {
"quota.type": "UNLIMITED",
"trial": "true"
}
}
}