Kubernetes Pod Templates

The Kubernetes resources created for Apache Flink® clusters can be customized via Kubernetes pod options or full-fledged pod templates.

Attention

For Deployments executed in session mode, you have to configure Kubernetes pod templates in the SessionCluster resource referenced by the Deployment.

Overview

The Kubernetes pods created for a Flink cluster can be customized as part of the Deployment Template.

kind: Deployment
spec:
  template:
    spec:
      kubernetes:
        pods: <KubernetesPodOptions>
        jobManagerPodTemplate: <V1PodTemplateSpec>
        taskManagerPodTemplate: <V1PodTemplateSpec>

The pods attribute allows customization of common Kubernetes options that apply to both JobManager and TaskManager pods.

The jobManagerPodTemplate and taskManagerPodTemplate allow you to individually customize the JobManager and/or TaskManager pods with full flexibility by exposing the Kubernetes V1PodTemplateSpec API object.

In both cases, the attributes you specify will be merged with the system configuration.

Note

It is only possible to either specify the pods attribute or the jobManagerPodTemplate and taskManagerPodTemplate attributes.

Kubernetes Pod Options

User-provided Kubernetes pod options under pods apply to both the JobManager and TaskManager pods and expose common configuration options.

kind: Deployment
spec:
  template:
    spec:
      kubernetes:
        pods:
          annotations:
            key: value
          labels:
            key: value
          nodeSelector:
            key: value
          affinity: <V1Affinity>
          tolerations:
          - <V1Toleration>
          imagePullSecrets:
          - name: secretName
          volumeMounts:
          - name: volumeAndMountName
            volume: <V1Volume>
            volumeMount: <V1VolumeMount>
          envVars:
          - name: envVarName
            value: envVarValue
            valueFrom: <V1EnvVarSource>
          securityContext: <V1PodSecurityContext>

Kubernetes API objects such as V1Volume are only validated lazily at Flink cluster creation time.

Annotations

The annotations configured as part of the Kubernetes options are added to the annotations of the created JobManager and TaskManager pods.

kind: Deployment
spec:
  template:
    spec:
      kubernetes:
        pods:
          annotations:
            key: value

All provided annotations are subject to the restrictions enforced by the Kubernetes API.

Labels

The labels configured as part of the Kubernetes options are added to the labels of the created JobManager and TaskManager pods.

kind: Deployment
spec:
  template:
    spec:
      kubernetes:
        pods:
          labels:
            key: value

All provided labels are subject to the restrictions enforced by the Kubernetes API.

Note

The labels app, component, deploymentId, deploymentName, jobId, system, sessionClusterId, sessionClusterName, and vvpNamespace are reserved by Ververica Platform.

Node Selector

The node selector configured as part of the Kubernetes options is added to the node selector of the created JobManager and TaskManager pods.

kind: Deployment
spec:
  template:
    spec:
      kubernetes:
        pods:
          nodeSelector:
            key: value

Affinity

The affinity configured as part of the Kubernetes options is added set as the affinity of the created JobManager and TaskManager pods.

kind: Deployment
spec:
  template:
    spec:
      kubernetes:
        pods:
          affinity: <V1Affinity>

V1Affinity refers to the respective Kubernetes API object. Kubernetes API objects such as V1Affinity are only validated lazily at Flink cluster creation time.

Tolerations

The tolerations configured as part of the Kubernetes options are added to the tolerations of the created JobManager and TaskManager pods.

kind: Deployment
spec:
  template:
    spec:
      kubernetes:
        pods:
          tolerations:
          - <V1Toleration>

V1Toleration refers to the respective Kubernetes API object. Kubernetes API objects such as V1Toleration are only validated lazily at Flink cluster creation time.

Image Pull Secrets

The image pull secrets configured as part of the Kubernetes options are added to the image pull secrets of the created JobManager and TaskManager pods.

kind: Deployment
spec:
  template:
    spec:
      kubernetes:
        pods:
          imagePullSecrets:
          - name: secretName

All provided secret names are subject to the restrictions enforced by the Kubernetes API. If a referenced secret does not exist, the Flink cluster will fail to start.

Volume Mounts

The volume and volume mount configured as part of the Kubernetes options are added to the volumes of the created JobManager and TaskManager pods and mounted in the respective Flink containers.

kind: Deployment
spec:
  template:
    spec:
      kubernetes:
        pods:
          volumeMounts:
          - name: volumeAndMountName
            volume: <V1Volume>
            volumeMount: <V1VolumeMount>

V1Volume refers to the respective Kubernetes API object. V1VolumeMount refers to the respective Kubernetes API object.

Kubernetes API objects such as V1Volume or V1VolumeMount are only validated lazily at Flink cluster creation time. The name of the configured volume and volume mount must match the provided volumeAndMountName.

Example: Mounting a Secret

kind: Deployment
spec:
  template:
    spec:
      kubernetes:
        pods:
          volumeMounts:
          - name: my-secret
            volume:
              name: my-secret
              secret:
                secretName: my-secret
            volumeMount:
              name: my-secret
              mountPath: /var/run/secrets/some-secret

Environment Variables

The environment variables configured as part of the Kubernetes options are added to the environment variables of the JobManager and TaskManager containers.

kind: Deployment
spec:
  template:
    spec:
      kubernetes:
        pods:
          envVars:
          - name: envVarName
            value: envVarValue
            valueFrom: <V1EnvVarSource>

You can either configure value directly or a V1EnvVarSource. Kubernetes API objects such as V1EnvVarSource are only validated lazily at Flink cluster creation time. If a referenced environment variable source does not exist, the Flink cluster will fail to start.

Example: Setting Environment Variables

kind: Deployment
spec:
  template:
    spec:
      kubernetes:
        pods:
          envVars:
            - name: ENV
              value: testing
            - name: POD_IP
              valueFrom:
                fieldRef:
                  fieldPath: status.podIP

Pod Security Context

The security context configured as part of the Kubernetes options is set as the security contet of the created JobManager and TaskManager pods.

kind: Deployment
spec:
  template:
    spec:
      kubernetes:
        pods:
          securityContext: <V1PodSecurityContext>

V1PodSecurityContext refers to the respective Kubernetes API object. Kubernetes API objects such as V1PodSecurityContext are only validated lazily at Flink cluster creation time.