Configure Kubernetes¶
In this page we will cover how to configure Kubernetes as part of your Deployment.
The Deployment Template section of your Deployment provides a kubernetes
attribute that allows you to specify Kubernetes-specific options for your Deployment.
Overview¶
Currently, you can specify these options for pods created for Flink jobs.
kind: Deployment
spec:
template:
spec:
kubernetes:
pods:
annotations:
key: value
nodeSelector:
key: value
affinity:
KubernetesAffinity
tolerations:
- KubernetesToleration
volumeMounts:
- name: name
volume: KubernetesVolume
volumeMount: KubernetesVolumeMount
envVars:
- name: name
value: value
Annotations¶
You can attach annotations to pods created for Flink jobs via pods.annotations
.
kind: Deployment
spec:
template:
spec:
kubernetes:
pods:
annotations:
key: value
Provided annotations will be added to the metadata section of created pods.
Node Selector¶
You can attach a node selector to pods created for Flink jobs in order to constrain pods to run on particular nodes via pods.nodeSelector
.
kind: Deployment
spec:
template:
spec:
kubernetes:
pods:
nodeSelector:
key: value
Affinity¶
You can attach an affinity to pods created for Flink jobs in order to constrain pods to run on particular nodes via pods.affinity
.
kind: Deployment
spec:
template:
spec:
kubernetes:
pods:
affinity:
KubernetesAffinity
KubernetesAffinity
is expected to be of type Affinity. The provided affinity will only be validated when the actual job is created and not when creating/modifying the resource.
Tolerations¶
You can attach tolerations to pods created for Flink jobs in order ensure that pods are not
scheduled onto inappropriate nodes via pods.tolerations
.
kind: Deployment
spec:
template:
spec:
kubernetes:
pods:
tolerations:
- KubernetesToleration
KubernetesToleration
is expected to be of type Toleration. The provided tolerations will only be validated when the actual job is created and not when creating/modifying the resource.
Volume Mounts¶
You can attach volumes and volume mounts to pods created for Flink jobs, for instance an NFS mount for use as a state backend via pods.volumeMounts
.
kind: Deployment
spec:
template:
spec:
kubernetes:
pods:
volumeMounts:
- name: name
volume: KubernetesVolume
volumeMount: KubernetesVolumeMount
KubernetesVolume
is expected to be of type Volume and KubernetesVolumeMount
is expected to be of type VolumeMount. The provided volumes and volume mounts will only be validated when the actual job is created and not when creating/modifying the resource.
Example: Mounting an NFS and Secret¶
kind: Deployment
spec:
template:
spec:
kubernetes:
pods:
volumeMounts:
- name: my-volume
volume:
name: my-volume
nfs:
server: 10.1.2.3
path: /daplatform-foo
volumeMount:
name: my-volume
mountPath: /foo/bar
- name: my-secret
volume:
name: my-secret
secret:
secretName: my-secret
volumeMount:
name: my-secret
mountPath: /var/run/secrets/some-secret
Environment Variables¶
You can set environment variables for pods created for Flink jobs. These can be useful to dynamically configure image-specific behaviour, such as using a custom log42j.xml or overwriting Flink-specific variables.
kind: Deployment
spec:
template:
spec:
kubernetes:
pods:
envVars:
- name: name
value: value
Example: Using a Custom Log Configuration File¶
In this example, we will mount a custom log4j2.xml configuration and use it in our Deployment.
For this example, we have the following content in a file called custom-log4j2.xml
:
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<Configuration xmlns="http://logging.apache.org/log4j/2.0/config" strict="true">
<Appenders>
<Appender name="STDOUT" type="Console">
<Layout pattern="%d{yyyy-MM-dd HH:mm:ss,SSS} %-5p %-60c %x - %m%n" type="PatternLayout"/>
</Appender>
</Appenders>
<Loggers>
<Root level="INFO">
<AppenderRef ref="STDOUT"/>
</Root>
</Loggers>
</Configuration>
We create a Kubernetes ConfigMap with the contents of this file as follows:
kubectl create configmap custom-log4j-config --from-file=log4j2.xml=custom-log42j.xml
We can now mount the ConfigMap in our Flink Deployment and use use the mounted file by setting the environment variable LOG4J_CONF
.
kind: Deployment
spec:
template:
spec:
kubernetes:
pods:
volumeMounts:
- name: logconfig
volume:
name: logconfig
configMap:
name: custom-log4j-config
volumeMount:
name: logconfig
mountPath: /opt/flink/conf
envVars:
- name: LOG4J_CONF
value: /opt/flink/conf/log42j.xml