When a Deployment is suspended or a stateful upgrade is triggered, Ververica Platform will submit a Stop-with-Savepoint command to Apache Flink®. This command will atomically trigger a Savepoint and stop the job.
You can instruct Ververica Platform to additionally drain the pipeline. Draining will result in all future event time timers of the Flink job to fire before the Stop-with-Savepoint command is executed.
The following sections describehow how to enable draining and provide example usage scenarios in which you would want to enable draining.
Draining emits the maximum watermark before stopping the job. When the watermark is emitted, all event time timers will fire, allowing you to process events that depend on this timer (e.g. time windows or process functions).
This is useful when you want to fully shut down your job without leaving any unhandled events or state.
Another common scenario is triggering an incompatible job upgrade. Draining allows you to preserve important state such as Apache Kafka® offsets while you flush out incompatible state. Without draining you would typically need to start your upgraded job from a clean slate.
You can enable draining by setting the following annotation:
kind: Deployment metadata: annotations: "com.dataartisans.appmanager.controller.deployment.stop-with-drain": "true"
As long as the annotation is set to true, suspensions and stateful upgrades will drain the pipeline.