Kubernetes Resources

Applies toSelf-Managed v2

4 min read

On this page

Overview
Kubernetes Pod Options
Flink Pod Templates (Recommended)

The Ververica Platform creates various Kubernetes resources to support {flink} clusters. For example, Kubernetes jobs, deployments, and services are used to run and expose the JobManagers and TaskManagers.

The Kubernetes pods that back these resources can be configured via simplified pod options or fully-fledged pod templates.

Caution

For Deployments executed in session mode, you have to configure Kubernetes pod templates in the SessionCluster resource referenced by the Deployment.

Overview

The Kubernetes resources created for a Flink cluster can be customized as part of the Deployment Template.

YAML

1kind: Deployment
2spec:
3  template:
4    spec:
5      kubernetes:
6        labels: <Map<String, String>>
7        pods: <KubernetesPodOptions>
8        jobManagerPodTemplate: <V1PodTemplateSpec>
9        taskManagerPodTemplate: <V1PodTemplateSpec>

The labels attribute allows specifying additional labels to the Kubernetes deployments, jobs, and services created for Flink clusters, with precedence given to system-defined labels (../../../platform-operations/deployment-targets.mdx#resource-labels).

The pods attribute allows customization of common Kubernetes options that apply to both JobManager and TaskManager pods.

The jobManagerPodTemplate and taskManagerPodTemplate allow you to individually customize the JobManager and/or TaskManager pods with full flexibility by exposing the Kubernetes V1PodTemplateSpec API object.

In both cases, the attributes you specify will be merged with the system configuration.

Note

Resource limits (CPU and memory) defined in pod templates override the corresponding settings in the Ververica Platform UI. The UI displays both the configured values and any active overrides.

Note

It is only possible to either specify the pods attribute or the jobManagerPodTemplate and taskManagerPodTemplate attributes.

Kubernetes Pod Options

User-provided Kubernetes pod options under pods apply to both the JobManager and TaskManager pods and expose common configuration options.

YAML

1kind: Deployment
2spec:
3  template:
4    spec:
5      kubernetes:
6        pods:
7          annotations:
8            key: value
9          labels:
10            key: value
11          nodeSelector:
12            key: value
13          affinity: <V1Affinity>
14          tolerations:
15          - <V1Toleration>
16          imagePullSecrets:
17          - name: secretName
18          volumeMounts:
19          - name: volumeAndMountName
20            volume: <V1Volume>
21            volumeMount: <V1VolumeMount>
22          envVars:
23          - name: envVarName
24            value: envVarValue
25            valueFrom: <V1EnvVarSource>
26          securityContext: <V1PodSecurityContext>
27          serviceAccountName: value

Kubernetes API objects such as V1Volume are only validated lazily at Flink cluster creation time.

Annotations

The annotations configured as part of the Kubernetes options are added to the annotations of the created JobManager and TaskManager pods.

YAML

1kind: Deployment
2spec:
3  template:
4    spec:
5      kubernetes:
6        pods:
7          annotations:
8            key: value

All provided annotations are subject to the restrictions enforced by the Kubernetes API.

Examples

Customize the Names of Kubernetes Resources

You can customize the names of Kubernetes resources (e.g., pods) created for a deployment using the deployment.resource-name-prefix annotation. This allows better alignment with organizational naming conventions or project-specific resource identification.

YAML

1kind: Deployment
2apiVersion: v1
3metadata:
4  annotations:
5    deployment.resource-name-prefix: sql-job

This annotation must be specified under the metadata.annotations section of the deployment YAML.

Note

The value for deployment.resource-name-prefix must not exceed 27 characters.

Labels

The labels configured as part of the Kubernetes options are added to the labels of the created JobManager and TaskManager pods.

YAML

1kind: Deployment
2spec:
3  template:
4    spec:
5      kubernetes:
6        pods:
7          labels:
8            key: value

All provided labels are subject to the restrictions enforced by the Kubernetes API.

Note

The labels app, component, deploymentId, deploymentName, jobId, system, sessionClusterId, sessionClusterName, and vvpNamespace are reserved by Ververica Platform.

Node Selector

The node selector configured as part of the Kubernetes options is added to the node selector of the created JobManager and TaskManager pods.

YAML

1kind: Deployment
2spec:
3  template:
4    spec:
5      kubernetes:
6        pods:
7          nodeSelector:
8            key: value

Affinity

The affinity configured as part of the Kubernetes options is added set as the affinity of the created JobManager and TaskManager pods.

YAML

1kind: Deployment
2spec:
3  template:
4    spec:
5      kubernetes:
6        pods:
7          affinity: <V1Affinity>

V1Affinityrefers to the respective Kubernetes API object. Kubernetes API objects such as V1Affinity are only validated lazily at Flink cluster creation time.

Tolerations

The tolerations configured as part of the Kubernetes options are added to the tolerations of the created JobManager and TaskManager pods.

YAML

1kind: Deployment
2spec:
3  template:
4    spec:
5      kubernetes:
6        pods:
7          tolerations:
8          - <V1Toleration>

V1Tolerationrefers to the respective Kubernetes API object. Kubernetes API objects such as V1Toleration are only validated lazily at Flink cluster creation time.

Image Pull Secrets

The image pull secrets configured as part of the Kubernetes options are added to the image pull secrets of the created JobManager and TaskManager pods.

YAML

1kind: Deployment
2spec:
3  template:
4    spec:
5      kubernetes:
6        pods:
7          imagePullSecrets:
8          - name: secretName

All provided secret names are subject to the restrictions enforced by the Kubernetes API. If a referenced secret does not exist, the Flink cluster will fail to start.

Volume Mounts

The volume and volume mount configured as part of the Kubernetes options are added to the volumes of the created JobManager and TaskManager pods and mounted in the respective Flink containers.

YAML

1kind: Deployment
2spec:
3  template:
4    spec:
5      kubernetes:
6        pods:
7          volumeMounts:
8          - name: volumeAndMountName
9            volume: <V1Volume>
10            volumeMount: <V1VolumeMount>

V1Volumerefers to the respective Kubernetes API object. V1VolumeMountrefers to the respective Kubernetes API object.

Kubernetes API objects such as V1Volume or V1VolumeMount are only validated lazily at Flink cluster creation time. The name of the configured volume and volume mount must match the provided volumeAndMountName.

Example: Mounting a Secret

YAML

1kind: Deployment
2spec:
3  template:
4    spec:
5      kubernetes:
6        pods:
7          volumeMounts:
8          - name: my-secret
9            volume:
10              name: my-secret
11              secret:
12                secretName: my-secret
13            volumeMount:
14              name: my-secret
15              mountPath: /var/run/secrets/some-secret

Environment Variables

The environment variables configured as part of the Kubernetes options are added to the environment variables of the JobManager and TaskManager containers.

YAML

1kind: Deployment
2spec:
3  template:
4    spec:
5      kubernetes:
6        pods:
7          envVars:
8          - name: envVarName
9            value: envVarValue
10            valueFrom: <V1EnvVarSource>

You can either configure value directly or a V1EnvVarSource. Kubernetes API objects such as V1EnvVarSource are only validated lazily at Flink cluster creation time. If a referenced environment variable source does not exist, the Flink cluster will fail to start.

Example: Setting Environment Variables

YAML

1kind: Deployment
2spec:
3  template:
4    spec:
5      kubernetes:
6        pods:
7          envVars:
8            - name: ENV
9              value: testing
10            - name: POD_IP
11              valueFrom:
12                fieldRef:
13                  fieldPath: status.podIP

Pod Security Context

The security context configured as part of the Kubernetes options is set as the security context of the created JobManager and TaskManager pods.

YAML

1kind: Deployment
2spec:
3  template:
4    spec:
5      kubernetes:
6        pods:
7          securityContext: <V1PodSecurityContext>

V1PodSecurityContextrefers to the respective Kubernetes API object. Kubernetes API objects such as V1PodSecurityContext are only validated lazily at Flink cluster creation time.

Flink Pod Templates (Recommended)

User-provided Kubernetes pod templates under jobManagerPodTemplate and taskManagerPodTemplateapply to the JobManager and TaskManager pods individually and provide full flexibility by exposing the complete V1PodTemplateSpec API.

YAML

1kind: Deployment
2spec:
3  template:
4    spec:
5      kubernetes:
6        jobManagerPodTemplate: <V1PodTemplateSpec>
7        taskManagerPodTemplate: <V1PodTemplateSpec>

The templates are used by Ververica Platform during deployment of Flink clusters. System-level configuration is applied on top of the user-provided templates.

Overall correctness of the resulting pod specs is only validated lazily at Flink cluster creation time, but each provided V1PodTemplateSpec is validated eagerly for structural correctness.

Container Names

Flink clusters launched by Ververica Platform for Deployments in application mode create two pods, one for the JobManager and one for the TaskManager. The following containers are executed as part of the pods.

Container Name	Container Type	Pod	Description
Container Name	Container Type	Pod	Description
flink-jobmanager	Container	JobManager	Flink JobManager process.
flink-taskmanager	Container	TaskManager	Flink TaskManager process.
artifact-fetcher	Init Container	JobManager, TaskManager	Fetch artifacts before execution.

In order to modify the configuration of these containers, you have to include them in the provided pod templates. The configuration is subject to the merge rules outlined below.

Merge Rules and System Reserved Fields

Some attributes provided as part of the pod templates cannot be overwritten by user-provided options. The following table describes how user-provided options are merged.

The rules for containers only apply to containers named flink-jobmanager or flink-taskmanager and the rules for init containers only apply to init containers named artifact-fetcher.

Field Path	Merge Strategy
Field Path	Merge Strategy
metadata.annotations	Add (system has precedence for conflicting names)
metadata.labels	Add (system has precedence for conflicting names)
spec.containers[].args	Replace with user-provided value
spec.containers[].env	Add (system has precedence for conflicting names)
spec.containers[].envFrom	Add (system has precedence for conflicting names)
spec.containers[].image	Replace with user-provided value
spec.containers[].imagePullPolicy	Replace with user-provided value
spec.containers[].ports	Add (system has precedence for conflicting names)
spec.containers[].resources	Merge with system-provided resources (user has precedence)
spec.containers[].terminationMessagePath	Ignore user-provided value
spec.containers[].terminationMessagePolicy	Ignore user-provided value
spec.containers[].volumeMounts	Add (system has precedence for conflicting names)
spec.serviceAccountName	Replace with user-provided value (see also required permissions for vvp-kubernetes high-availability)
spec.volumes	Add (system has precedence for conflicting names)

All omitted fields are fully configurable.

Note

You can set validation of non empty spec.serviceAccountName with the global setting vvp.accessControl.serviceAccountNameEnforcementEnabled=true. Deployment should have spec.serviceAccountName defined in jobManagerPodTemplate and taskManagerPodTemplate. Validation requires format deployment-${namespace}-${deploymentName}.

Fail Early on Fatal Exceptions

To ensure that the platform catches fatal Flink exceptions early and prevents unnecessary retry loops, you can configure the flinkFatalExceptionFailEarly field in the deployment specification.

YAML

1kind: Deployment
2spec:
3  template:
4    spec:
5      flinkFatalExceptionFailEarly: true

Examples

Add Pod Labels

The following example adds a env: testing label to all pods.

YAML

1kind: Deployment
2spec:
3  template:
4    spec:
5      kubernetes:
6        jobManagerPodTemplate:
7          metadata:
8            labels:
9              env: testing
10        taskManagerPodTemplate:
11          metadata:
12            labels:
13              env: testing

Set Image Pull Policy

The following example sets imagePullPolicy: IfNotPresent for all containers.

YAML

1kind: Deployment
2spec:
3  template:
4    spec:
5      kubernetes:
6        jobManagerPodTemplate:
7          spec:
8            containers:
9            - name: flink-jobmanager
10              imagePullPolicy: IfNotPresent
11            initContainers:
12            - name: artifact-fetcher
13              imagePullPolicy: IfNotPresent
14        taskManagerPodTemplate:
15          spec:
16            containers:
17            - name: flink-taskmanager
18              imagePullPolicy: IfNotPresent
19            initContainers:
20            - name: artifact-fetcher
21              imagePullPolicy: IfNotPresent

Set Image Pull Secret

The following example sets imagePullSecrets: xxx for Task and Job manager pod templates.

YAML

1kind: Deployment
2spec:
3  template:
4    spec:
5      kubernetes:
6        jobManagerPodTemplate:
7          spec:
8            imagePullSecrets:
9            - name: secret_name
10        taskManagerPodTemplate:
11          spec:
12            imagePullSecrets:
13            - name: secret_name

Add Sidecar Containers

The following example adds an additional sidecar container named my-logging-sidecar to both the JobManager and TaskManager pods.

YAML

1kind: Deployment
2spec:
3  template:
4    spec:
5      kubernetes:
6        jobManagerPodTemplate:
7          spec:
8            containers:
9              - name: my-logging-sidecar
10                image: registry.acme.org/logging:3.2.1
11        taskManagerPodTemplate:
12          spec:
13            containers:
14              - name: my-logging-sidecar
15                image: registry.acme.org/logging:3.2.1

Request Ephemeral TaskManager Storage

The following example requests ephemeral storage for TaskManager containers.

YAML

1kind: Deployment
2spec:
3  template:
4    spec:
5      kubernetes:
6        taskManagerPodTemplate:
7          spec:
8            containers:
9            - name: flink-taskmanager
10              resources:
11                requests:
12                  ephemeral-storage: "2Gi"
13                limits:
14                  ephemeral-storage: "4Gi"

Was this helpful?

Yes No