Skip to main content

Private Connections

important

Private connections for BYOC workspaces are entirely customer-managed. Ververica does not provide automated support or direct control over these configurations to maintain the principles of zero trust and customer autonomy. You should consult your internal policies and AWS best practices when creating private connections in your cloud account.

Ververica Cloud offers the capability to establish private connections with AWS services. You can create such connections manually or by using the Ververica Cloud AWS Private Connection Terraform Module.

This Terraform-based approach is recommended for users who are already familiar with Terraform. The set of modules in this repository simplifies the process of creating the necessary VPC endpoint services and IAM Role in your AWS account. Once established, these enable Ververica Cloud to access AWS resources (e.g., RDS for MySQL, MSK) from Flink jobs running on Ververica Cloud.

Ververica Cloud can establish network connectivity with AWS services through two primary means:

  • Public Connection: you can grant public network access for AWS services, allowing Ververica Cloud to connect directly over the public network.
  • Private Connection: this method allows Ververica Cloud to connect with your AWS services over the intranet. This framework can be any configuration, like setting up an IAM role that facilitates communication between Ververica Cloud and AWS over the intranet.
note

The credential_process feature for private connection authentication is supported in AWS SDK for Java version 1.11.489 and later. Ververica Cloud requires Flink jobs (Data Stream JAR jobs) to use AWS SDK version 1.11.489 or later for authentication.

For more details, see the official AWS SDK for Java documentation.

Benefits of Private Connections​

Private connections offer:

  • Enhanced Security: avoid exposing services to the public internet.
  • Cost Reduction: lower network traffic costs compared to public connections.
  • Improved Performance: potentially reduced latency with internal network routes.

Connection Statuses​

Once you set up a private connection, its status can be one of the following:

  • CREATING: The creation process has started.
  • CREATED: Successfully established.
  • FAILED: Errors encountered during the creation process.
  • DELETING: The deletion process has begun.

Terminology​

  • Service Name ("service_name"): the name of the VPC endpoint service created by the user in their AWS account.
  • Service Endpoint ("service_endpoint"): represents the internal address associated with an AWS service.
  • Endpoint Group: a conceptual group of endpoints. For instance, a connection with Kafka might contain an endpoint group with entries for three Kafka brokers.

Types of Private Connections​

To implement a private connection, we provide the following two approaches:

Type-A: Binding IAM role​

  • Amazon Kinesis
  • Amazon S3
  • Apache Paimon
  • Amazon DynamoDB
  • Apache Iceberg

Type-B: Configuring VPC endpoint + (optional) Binding an IAM Role​

  • Apache Kafka
  • Apache Upsert Kafka
  • MySQL
  • MySQL CDC
  • PostgreSQL
  • PostgreSQL CDC
  • MongoDB
  • MongoDB CDC
  • Redis
  • Apache Pulsar
  • Redpanda

Limitations​

  • Disabling and re-enabling VPC Endpoints is not supported.
  • Unbinding and rebinding the IAM Role is not currently possible.
  • Editing of existing VPC Endpoint and IAM Role configurations is not supported.
On this page