Session Clusters

Applies toSelf-Managed v2

1 min read

On this page

Limitations
Specification
- Desired State
- Full Example

Session clusters are long-lived Flink clusters that can be used to execute multiple applications simultaneously or run short-lived, interactive jobs on demand. It is possible to execute deployments on session clusters by using session mode, see Deployments.

Limitations

Support for session clusters currently has some limitations compared with Deployments:

SSL/TLS: Auto-provisioned SSL/TLS for Flink intra-cluster and external communication is not supported. SSL/TLS has to be configured manually.
Autopilot: Autoscaling is not supported for session clusters and is limited to deployments running in session mode, see Deployments.

Specification

Session clusters are managed via namespaced SessionCluster resources which are configured similarly to Deployments. However, SessionClusters have fewer configurable options than Deployments since this resource only configures the Flink cluster itself and not the applications that will run on it.

Desired State

A SessionCluster resource has a desired state specified at spec.state. The desired state can be either:

RUNNING when the cluster should be provisioned and kept running, or
STOPPED when the cluster should be torn down, along with all currently running applications

Warning

All deployments running on a session cluster must be terminated before the session cluster can be stopped.

Changing a Running SessionCluster

Only the desired state and number of TaskManagers of a session cluster may be changed while the cluster is in a non-terminal state non-terminal-state. A SessionCluster is in a terminal state when its desired state is STOPPED and there are no in-progress operations on the cluster, such as when the cluster is starting, stopping, or being updated.

Note

Scaling down a running session cluster (by reducing the value of spec.numberOfTaskManagers) can cause applications running on the cluster to restart.

Full Example

The following snippet is a complete example of a SessionCluster, including optional keys:

YAML

1kind: SessionCluster
2apiVersion: v1
3metadata:
4  name:
5  labels:
6    env: testing
7spec:
8  state: RUNNING
9  deploymentTargetName: default
10  flinkVersion: "{flinkVersion}"
11  flinkImageRegistry: "registry.ververica.com/v{shortVersion}"
12  flinkImageRepository: flink
13  flinkImageTag: "{ververicaFlinkRelease}-scala_2.12"
14  numberOfTaskManagers: 5
15  resources:
16    jobmanager:
17      cpu: 2
18      memory: 1g
19    taskmanager:
20      cpu: 16
21      memory: 32g
22  flinkConfiguration:
23    taskmanager.numberOfTaskSlots: 32
24  logging:
25    loggingProfile: default
26    log4jLoggers:
27      "": INFO
28      org.apache.flink.streaming.examples: DEBUG
29  kubernetes:
30    pods:
31      envVars:
32        - name: KEY
33          value: VALUE

Was this helpful?

Yes No