How BYOC Works
The Ververica Cloud: Bring Your Own Cloud deployment enables you to run Apache Flink workloads in your own Kubernetes environment (such as Amazon EKS or Azure AKS), while centrally managing and monitoring those workloads through the Ververica Cloud Control Plane.
The key components of the BYOC framework are:
- Ververica Cloud Control Plane: The centralized service that handles user interactions, job orchestration, and monitoring.
- Ververica Agent: A specialized controller deployed in your Kubernetes cluster. It maintains a secure, outbound-only connection to Ververica Cloud and orchestrates Workspace Agents.
- Workspace Agent: A dedicated agent within each workspace. It manages lifecycle operations (create, update, delete) for Flink jobs and monitors their status.
- Flink Jobs: The actual stream processing or batch jobs running on Kubernetes infrastructure. These jobs access data sources, sinks, or other resources within your own cloud environment.
- Data Stores: This includes systems like Amazon S3 or Azure Blob Storage, Kafka, relational databases, or other services reachable from your Kubernetes cluster. Connectivity can be established through direct access or VPC peering.
In this architecture, you install Ververica Agent on your Kubernetes cluster to communicate with Ververica Cloud. The diagram below illustrates how the various components interact.
The diagram illustrates how the various components interact:
- The process begins by installing the Ververica Agent (Pyxis) onto your Kubernetes cluster. The agent runs alongside any workloads you plan to deploy.
- Once you install the agent, it initiates the connection to the Ververica Cloud Control Plane over HTTPS. This channel ensures that all communication between your cluster and Ververica Cloud is encrypted and bidirectional, reducing your infrastructure's exposure.
- The Ververica Cloud Control Plane sends instructions (such as creating a new workspace) to the Ververica Agent using the established secure connection. The agent then executes these instructions in your cluster.
- The Ververica Agent creates workspace agents on your behalf. The workspace agent is responsible for tasks like deploying Flink jobs and managing the underlying Kubernetes resources.
- The workspace agent also creates and manages the Flink jobs. These jobs run in your infrastructure and access data in your data stores (such as S3, Kafka, or relational databases). The jobs access the data stores directly within the same VPC or through other secure networking methods such as VPC peering and private links.
- The workspace agent reports real-time status and metrics back to the Ververica Cloud Control Plane at every step. This feedback loop provides a centralized view of your Flink applications across multiple workspaces.