Skip to main content

Create AWS Resources

Prepare Your Environment​

This guide shows you how to prepare your AWS environment for deploying the Ververica Agent on an Amazon EKS cluster. Ververica strongly recommends using the provided CloudFormation script to automate resource creation and configuration - it’s the fastest way to ensure you have the right roles, policies, and storage. However, if you need more control or want to understand each component, you can manually complete the steps outlined in this document.

You’ll learn how to:

  • Automatically set up resources with CloudFormation, including IAM roles, policies, and S3 buckets.
  • Manually create and configure your S3 bucket to store runtime artifacts and Flink checkpoints.
  • Enable Kubernetes OIDC (OpenID Connect) for IAM Roles for Service Accounts (IRSA).
  • Set up IAM roles and policies that securely manage the Ververica Agent.

Once these steps are complete, your environment will be configured for a deployment of the Ververica Agent.

Use CloudFormation script​

Ververica recommends using the provided CloudFormation script to create all required resources automatically. This approach ensures you have the right AWS Identity and Access Management (IAM) roles, policies, and S3 bucket set up with minimal manual effort.

  • Click the following link to open the AWS console with the CloudFormation template preloaded

Note: When specifying stack details, you can rename the pre-filled parameters (e.g., VVCTenantPolicy, VVCAdminRole) if needed.

Create an S3 Bucket​

  1. Create a bucket:
    Use the AWS CLI or the AWS Management Console to create an S3 bucket that stores runtime artifacts and Flink checkpoints. You can also use an existing bucket if you prefer.

    Note: S3 bucket names must be globally unique. A common practice is to include your AWS account ID in the bucket name. For example:

    aws s3 create-bucket --bucket vv-agent-bucket-1234567890 --region eu-central-1

  2. Record the bucket ARN:
    After creating the bucket, note its ARN for later use.

    image

  3. Configure CORS for the bucket:

    Configuring CORS ensures that only allowed origins can access the bucket, enforcing least privilege principles. The following configuration permits access from a Ververica website endpoint and required methods:

    1. In the AWS Management Console, open your S3 bucket.
    2. Go to Permissions.
    3. Scroll down to Cross-origin resource sharing (CORS) and click Edit.
    4. Add the following JSON and click Save changes: 
        [
            {
                "AllowedHeaders": ["*"],
                "AllowedMethods": ["GET", "POST", "PUT", "DELETE", "HEAD"],
                "AllowedOrigins": ["https://app.ververica.cloud"],
                "ExposeHeaders": ["ETag"],
                "MaxAgeSeconds": 3000
            }
        ]

Kubernetes OIDC​

The Agent uses IAM Roles for Service Accounts (IRSA) to authorize API calls to AWS. Your Amazon EKS cluster provides an OpenID Connect (OIDC) provider URL that you can use to configure pod permissions with IAM Roles.

  1. Obtain the EKS Cluster’s OIDC Provider URL:
    Run the following AWS CLI command, replacing $cluster_name with your cluster’s name:

    aws eks describe-cluster --name $cluster_name --query "cluster.identity.oidc.issuer" --output text

    You can also find it by going to your AWS Console on EKS page.

    image

  2. Configure Provider:

    1. Navigate to Identity Providers and choose Add provider.
    2. Set Provider type to OpenID Connect.
    3. For Provider URL, paste the OIDC Provider URL you obtained in the previous step.
    4. For Audience, add sts.amazonaws.com.
    5. Optionally add tags, and then click Add provider.

IAM Roles and Policies​

You need to create two IAM roles and their corresponding policies for the Ververica Agent (vv-agent):

  1. VVCTenantRole: Manages the S3 bucket.
  2. VVCAdminRole: Serves as the IRSA role for the Agent pod.
note

For zero-trust practices, restrict the policy to a specific S3 bucket (e.g., vv-agent-bucket-0123456789). If using a single agent for multiple workspaces, you may allow a wildcard pattern like vv-agent-bucket-*.

Create the Tenant Policy (VVCTenantPolicy)​

  1. In the IAM console, go to Policies and choose Create policy.
  2. On the Create policy page, switch to the JSON editor.
  3. Paste the following JSON, making sure to replace the S3 bucket ARN with your own: 
    {
       "Version": "2012-10-17",
       "Statement": [
           {
               "Effect": "Allow",
               "Action": [
                   "s3:PutObject",
                   "s3:GetObject",
                   "s3:DeleteObject"
               ],
               "Resource": "arn:aws:s3:::vv-agent-bucket-*/*"
           },
           {
               "Effect": "Allow",
               "Action": [
                   "s3:ListBucket",
                   "s3:GetBucketLocation"
               ],
               "Resource": "arn:aws:s3:::vv-agent-bucket-*"
           }
       ]
    }


  4. Click Next, review the policy, and give it the name VVCTenantPolicy.
  5. Click Create policy.

Create the Tenant Role (VVCTenantRole)​

  1. In the IAM console, go to Roles and choose Create role.
  2. On Select trusted entity, choose AWS account as the trusted entity type.
  3. Select This account, then click Next.
  4. On Add permissions, search for VVCTenantPolicy and select it.
  5. Click Next and then review the role settings.
  6. Name the role VVCTenantRole.
  7. Click Create role.

Create the Admin Policy (VVCAdminPolicy)​

Repeat the steps for creating a policy (as done for VVCTenantPolicy), but use the following JSON. Replace <REPLACE_WITH_ACCOUNT_ID> with your AWS account ID:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "sts:AssumeRole"
            ],
            "Resource": "arn:aws:iam::<REPLACE_WITH_ACCOUNT_ID>:role/VVCTenantRole"
        }
    ]
}

Name the policy VVCAdminPolicy.

Create the Admin Role (VVCAdminRole)​

  1. In the IAM console, go to Roles and choose Create role.
  2. Select Web identity as the trusted entity type.
  3. For Identity provider, select the EKS cluster OIDC provider.
  4. For Audience, select sts.amazonaws.com.
  5. Click Next.
  6. On Add permissions, search for VVCAdminPolicy and select it.
  7. Click Next, then name the role VVCAdminRole.
  8. Confirm that the trust policy and permissions are correct.
  9. Click Create role.
On this page