Concepts

As its name implies, Application Manager is an orchestrator for managing stateful-streaming applications in production. Where Apache Flink® allows you to submit jobs, Application Manager tracks a service across its entire lifetime. By focusing the core abstraction around applications, we allow developers to easily manage, monitor, and configure streaming jobs without worrying about the underlying infrastructure. And because an application is stateful-streaming-aware, it greatly simplifies common stream processing operations tasks such as deploying new features and changing configurations in a consistent manner.

Declarative API

Interactions with Application Manager are built around a declarative API. This allows users to provide a definition of a stateful streaming application, and Application Manager manages the complexity of coordinating all moving parts to successfully deploy the application and manage its lifecycle. Changing the definition of a streaming application will eventually lead to a change in the application, reflecting the updated definition.

Resources

Another core concept in Application Manager is a resource. A resource is an object with an identity, specification, and possibly status. Stateful streaming applications in Application Manager are managed as a Deployment resource.

A typical resource has the following structure:

  • Metadata: Auxiliary information used to identify the resource such as a system generated ID, name, labels, and optionally a modification date.
  • Spec: A specification of the resource. Everything that is used to define the resource.
  • Status: Most resources have a lifecycle that is managed by the system, which is updated by the system in the status part of the resource.

A typical workflow with Application Manager includes initially creating a resource and later modifying it by updating the spec. The system reports progress by modifying the status part of the resource.

The following is a list of the main resources that are managed by Application Manager.

Resource Description
ApiToken A resource for generating static tokens for non-interactive authentication.
ClusterRole A definition of a set of rules that represent permissions to the API and resources.
ClusterRoleBinding A role binding associates a defined role to one or more users/groups/api-tokens.
Deployment A definition of a stateful/stateless Flink application.
DeploymentTarget A resource that corresponds to a logical or physical Deployment Target.
Event An event reported by Application Manager during transitions.
Job A specific instance of a Deployment.
Namespace A resource used to partition other resources.
Role A definition of a set of rules that represent permissions to the API and resources.
RoleBinding A role binding associates a defined role to one or more users/groups/api-tokens.
Savepoint A representation of a Flink savepoint in Application Manager.
SecretValue A value containing sensitive information with restricted visibility to users.