Building Apache Flink® Images with Support for GCS¶
Apache Flink® currently does not bundle libraries for working with GCS. There is, however, a connector from Google based on Apache Hadoop®.
In order to use the gs:// scheme for Universal Blob Storage, it is required to bundle this library along with the shaded Hadoop JAR.
This step is optional if you already have service account credentials with the Storage Admin role permissions for your designated GCS bucket.
If you do not yet have a service account, create one:
gcloud iam service-accounts create gcs-vvp-service-acc \
--description="Service account for VVP GCS"\
Assuming that you use the “uniform” bucket access control, navigate to Storage Browser,
click three dots next to your bucket -> Edit bucket permissions -> + Add member -> and paste your full service account ID
In “Select a role”, pick “Storage Admin”. Save.
Navigate to Service Accounts, click three dots next to the
gcs-vvp-service-acc service account -> Create key -> JSON -> Store as gcs-key.json.
Create a Kubernetes secret in the namespace where Ververica Platform is installed (vvp in this example):