Configuring Lake Tiering

8 min read

Tier Apache Fluss tables into Apache Iceberg: install the lake plugin, point the server at a JDBC or REST catalog, and run the tiering service.

On this page

Overview
Prerequisites
Choosing a Catalog
Installing the Lake Plugin
Configuring the Server
Verify Lake Tiering Is Configured
Running the Tiering Service
- Job Classpath
- Job Arguments
Troubleshooting
- Lake plugin directory empty after pod start
- REST catalog: Unauthorized or Failed to obtain access token
Further Reading

Applies to: fluss-bundle chart, Docker image registry.ververica.cloud/platform-images/fluss:1.0.0-vv-3

Overview

Lake tiering moves Apache Fluss™ table data into Apache Iceberg tables on object storage, where Fluss and external engines can both read it. It takes three parts: the lake plugin installed on the Fluss servers, the server's lake configuration pointing at an Iceberg catalog, and the tiering service, a long-running Flink streaming job, separate from the Fluss cluster, that reads tables from Fluss and writes them to the lake. Lake tiering does not replace remote storage: the cluster keeps writing to remote storage, and the lake warehouse is a location of its own. Fluss tiers only the tables with lake tiering enabled.

Installing the Lake Plugin and Configuring the Server prepare the Fluss servers, the coordinator and the tablet servers, to tier into your catalog, and Verify Lake Tiering Is Configured checks that they are ready. Running the Tiering Service then deploys the tiering job, which takes the same lake configuration as job arguments.

Prerequisites

A Fluss cluster deployed with the fluss-bundle chart. See Deploying Fluss on Kubernetes.
Remote storage configured on the cluster. See Configuring Remote Storage.
An Iceberg catalog service, reachable from the Fluss pods and from the tiering job, with a warehouse for the lake tables: a PostgreSQL database for a JDBC catalog, or a Lakekeeper or Polaris REST catalog. The lake warehouse can share the remote-storage bucket under another prefix, or use a bucket of its own.
Storage credentials for the warehouse location with read and write access, unless the catalog provides storage access (see Choosing a Catalog).
When the cluster has SASL enabled on its client listener, a client user for the tiering service. See Securing Fluss.
A Ververica Platform 3 workspace to run the tiering service in.

Choosing a Catalog

Catalog	Storage backends	--catalog option	Credentials Fluss holds
Catalog	Storage backends	--catalog option	Credentials Fluss holds
JDBC, backed by PostgreSQL	Amazon S3, Azure Blob Storage, S3-compatible stores	jdbc-postgres	The database password, and the warehouse's storage credentials
Lakekeeper (REST)	Amazon S3, Azure Blob Storage, S3-compatible stores	rest	A bearer token; the catalog provides storage access
Polaris (REST)	Amazon S3	rest	An OAuth2 client credential and scope, and the warehouse's storage credentials

A REST catalog is a service that holds the warehouse, including its storage location: Fluss names the warehouse, never its location. Polaris and Lakekeeper are both REST catalogs, installed with the same --catalog rest option.

Installing the Lake Plugin

Install the lake-iceberg setup on both the coordinator and the tablet servers, with the init container that installs your other plugins. Its --fs option picks the storage backend (s3 for Amazon S3 and S3-compatible stores, abs for Azure Blob Storage) and its --catalog option the catalog. The plugin installs into /opt/fluss/plugins/iceberg; mount it there in the Fluss container with the iceberg sub-path of the plugins volume.

For the tablet servers, with a REST catalog on S3:

YAML

1fluss:
2  tablet:
3    extraVolumes:
4      - name: fluss-plugins
5        emptyDir: {}
6    extraVolumeMounts:
7      # ... the mounts of your other plugins ...
8      - name: fluss-plugins
9        mountPath: /opt/fluss/plugins/iceberg
10        subPath: iceberg
11    initContainers:
12      - name: install-plugins
13        image: registry.ververica.cloud/platform-images/fluss:1.0.0-vv-3
14        command:
15          - /bin/sh
16          - -c
17          - |
18            set -e
19
20            # ... the installer calls of your other plugins ...
21
22            /opt/fluss/bin/setup/install.sh fluss lake-iceberg --fs s3 --catalog rest --force
23        volumeMounts:
24          - name: fluss-plugins
25            mountPath: /opt/fluss/plugins

Add the same mount and installer call under fluss.coordinator. For a JDBC catalog, use --catalog jdbc-postgres, which also installs the PostgreSQL JDBC driver; for Azure Blob Storage, use --fs abs. For the installer's options and a private Maven repository, see Installing Fluss Setups.

Configuring the Server

The lake configuration in this section and in Installing the Lake Plugin is for the Fluss servers only. Every catalog sets datalake.enabled and datalake.format under fluss.configurationOverrides, beside your remote-storage keys, and adds its own datalake.iceberg.* keys to the same block. With datalake.enabled set to true, datalake.format is required.

For the full set of configurable fields under fluss:, refer to the Fluss Helm chart documentation.

Storing Lake Credentials in a Secret

This manual follows the same Secret and config-provider convention as Configuring Remote Storage: each credential is a key of a Kubernetes Secret in the release namespace, referenced from values.yaml through a ${env:<NAME>} marker and a fluss.secrets.env entry. A lake credential goes in the key=value file you created the storage Secret from, keeping its remote-storage keys, or in a Secret of its own, as the Lakekeeper token does in Lakekeeper REST Catalog. The key file's lake keys, for example:

TEXT

1# ... the remote-storage keys (see Configuring Remote Storage) ...
2# JDBC catalog:
3datalake-iceberg-jdbc-password=<JDBC_PASSWORD>
4# JDBC or Polaris catalog on S3 or an S3-compatible store:
5datalake-iceberg-s3-access-key-id=<LAKE_S3_ACCESS_KEY_ID>
6datalake-iceberg-s3-secret-access-key=<LAKE_S3_SECRET_ACCESS_KEY>
7# JDBC catalog on Azure Blob Storage:
8datalake-iceberg-adls-auth-shared-key-account-key=<STORAGE_ACCOUNT_KEY>
9# Polaris catalog:
10datalake-iceberg-credential=<CLIENT_ID>:<CLIENT_SECRET>

Re-applying the Secret from the file replaces its whole data with the file's keys, so a key the file leaves out is deleted:

BASH

1kubectl create secret generic fluss-storage-credentials \
2  --from-env-file=<FILE> \
3  --namespace <NAMESPACE> \
4  --dry-run=client -o yaml | kubectl apply -f -

Each catalog's snippet below carries the fluss.secrets.env entries for its keys.

JDBC Catalog

The Fluss servers hold the database password and the warehouse's storage credentials.

On Amazon S3:

YAML

1fluss:
2  configurationOverrides:
3    # ... your remote.data.dir and storage keys (see Configuring Remote Storage) ...
4    datalake.enabled: "true"
5    datalake.format: iceberg
6    datalake.iceberg.catalog-impl: org.apache.iceberg.jdbc.JdbcCatalog
7    datalake.iceberg.uri: jdbc:postgresql://<POSTGRES_HOST>:<POSTGRES_PORT>/<DATABASE>
8    datalake.iceberg.jdbc.user: <JDBC_USER>
9    datalake.iceberg.jdbc.password: "${env:FLUSS_DATALAKE_ICEBERG_JDBC_PASSWORD}"
10    datalake.iceberg.warehouse: s3://<LAKE_BUCKET>/<WAREHOUSE_PREFIX>
11    datalake.iceberg.io-impl: org.apache.iceberg.aws.s3.S3FileIO
12    datalake.iceberg.client.region: <AWS_REGION>
13    datalake.iceberg.s3.access-key-id: "${env:FLUSS_DATALAKE_ICEBERG_S3_ACCESS_KEY_ID}"
14    datalake.iceberg.s3.secret-access-key: "${env:FLUSS_DATALAKE_ICEBERG_S3_SECRET_ACCESS_KEY}"
15  secrets:
16    env:
17      # ... the entries for remote storage ...
18      - name: FLUSS_DATALAKE_ICEBERG_JDBC_PASSWORD
19        secretName: fluss-storage-credentials
20        key: datalake-iceberg-jdbc-password
21      - name: FLUSS_DATALAKE_ICEBERG_S3_ACCESS_KEY_ID
22        secretName: fluss-storage-credentials
23        key: datalake-iceberg-s3-access-key-id
24      - name: FLUSS_DATALAKE_ICEBERG_S3_SECRET_ACCESS_KEY
25        secretName: fluss-storage-credentials
26        key: datalake-iceberg-s3-secret-access-key

On Azure Blob Storage, the warehouse is an abfss:// path and the data files go through the ADLS file IO with the storage account's shared key:

YAML

1fluss:
2  configurationOverrides:
3    # ... your remote.data.dir and storage keys (see Configuring Remote Storage) ...
4    datalake.enabled: "true"
5    datalake.format: iceberg
6    datalake.iceberg.catalog-impl: org.apache.iceberg.jdbc.JdbcCatalog
7    datalake.iceberg.uri: jdbc:postgresql://<POSTGRES_HOST>:<POSTGRES_PORT>/<DATABASE>
8    datalake.iceberg.jdbc.user: <JDBC_USER>
9    datalake.iceberg.jdbc.password: "${env:FLUSS_DATALAKE_ICEBERG_JDBC_PASSWORD}"
10    datalake.iceberg.warehouse: abfss://<LAKE_CONTAINER>@<STORAGE_ACCOUNT>.dfs.core.windows.net/<WAREHOUSE_PATH>
11    datalake.iceberg.io-impl: org.apache.iceberg.azure.adlsv2.ADLSFileIO
12    datalake.iceberg.adls.auth.shared-key.account.name: <STORAGE_ACCOUNT>
13    datalake.iceberg.adls.auth.shared-key.account.key: "${env:FLUSS_DATALAKE_ICEBERG_ADLS_AUTH_SHARED_KEY_ACCOUNT_KEY}"
14  secrets:
15    env:
16      # ... the entries for remote storage ...
17      - name: FLUSS_DATALAKE_ICEBERG_JDBC_PASSWORD
18        secretName: fluss-storage-credentials
19        key: datalake-iceberg-jdbc-password
20      - name: FLUSS_DATALAKE_ICEBERG_ADLS_AUTH_SHARED_KEY_ACCOUNT_KEY
21        secretName: fluss-storage-credentials
22        key: datalake-iceberg-adls-auth-shared-key-account-key

On an S3-compatible store, the S3 configuration adds the store's endpoint and path-style access:

YAML

1fluss:
2  configurationOverrides:
3    # ... your remote.data.dir and storage keys (see Configuring Remote Storage) ...
4    datalake.enabled: "true"
5    datalake.format: iceberg
6    datalake.iceberg.catalog-impl: org.apache.iceberg.jdbc.JdbcCatalog
7    datalake.iceberg.uri: jdbc:postgresql://<POSTGRES_HOST>:<POSTGRES_PORT>/<DATABASE>
8    datalake.iceberg.jdbc.user: <JDBC_USER>
9    datalake.iceberg.jdbc.password: "${env:FLUSS_DATALAKE_ICEBERG_JDBC_PASSWORD}"
10    datalake.iceberg.warehouse: s3://<LAKE_BUCKET>/<WAREHOUSE_PREFIX>
11    datalake.iceberg.io-impl: org.apache.iceberg.aws.s3.S3FileIO
12    datalake.iceberg.client.region: <AWS_REGION>
13    datalake.iceberg.s3.endpoint: <S3_ENDPOINT>
14    datalake.iceberg.s3.path-style-access: "true"
15    datalake.iceberg.s3.access-key-id: "${env:FLUSS_DATALAKE_ICEBERG_S3_ACCESS_KEY_ID}"
16    datalake.iceberg.s3.secret-access-key: "${env:FLUSS_DATALAKE_ICEBERG_S3_SECRET_ACCESS_KEY}"
17  secrets:
18    env:
19      # ... the entries for remote storage ...
20      - name: FLUSS_DATALAKE_ICEBERG_JDBC_PASSWORD
21        secretName: fluss-storage-credentials
22        key: datalake-iceberg-jdbc-password
23      - name: FLUSS_DATALAKE_ICEBERG_S3_ACCESS_KEY_ID
24        secretName: fluss-storage-credentials
25        key: datalake-iceberg-s3-access-key-id
26      - name: FLUSS_DATALAKE_ICEBERG_S3_SECRET_ACCESS_KEY
27        secretName: fluss-storage-credentials
28        key: datalake-iceberg-s3-secret-access-key

Warning

A JDBC catalog on an S3-compatible store might also need these environment variables on both the coordinator and the tablet servers:

YAML

1fluss:
2  tablet:
3    extraEnv:
4      - name: AWS_REQUEST_CHECKSUM_CALCULATION
5        value: when_required
6      - name: AWS_RESPONSE_CHECKSUM_VALIDATION
7        value: when_required
8  coordinator:
9    extraEnv:
10      - name: AWS_REQUEST_CHECKSUM_CALCULATION
11        value: when_required
12      - name: AWS_RESPONSE_CHECKSUM_VALIDATION
13        value: when_required

Lakekeeper REST Catalog

The Fluss servers hold a token; storage access comes through the catalog.

A Lakekeeper catalog authenticates Fluss with a bearer token issued for a principal the catalog accepts. Fluss sets no storage credentials of its own: the catalog provides storage access to the warehouse. The token is the key token of a Secret of its own, fluss-lake-client-token, in the release namespace; for an example of creating it, see A Lakekeeper Catalog in the quickstart.

On Amazon S3, and on an S3-compatible store:

YAML

1fluss:
2  configurationOverrides:
3    # ... your remote.data.dir and storage keys (see Configuring Remote Storage) ...
4    datalake.enabled: "true"
5    datalake.format: iceberg
6    datalake.iceberg.type: rest
7    datalake.iceberg.uri: http://<LAKEKEEPER_HOST>:<LAKEKEEPER_PORT>/catalog
8    datalake.iceberg.warehouse: <WAREHOUSE_NAME>
9    datalake.iceberg.token: "${env:FLUSS_DATALAKE_ICEBERG_TOKEN}"
10    datalake.iceberg.io-impl: org.apache.iceberg.aws.s3.S3FileIO
11    datalake.iceberg.client.region: <AWS_REGION>
12  secrets:
13    env:
14      # ... the entries for remote storage ...
15      - name: FLUSS_DATALAKE_ICEBERG_TOKEN
16        secretName: fluss-lake-client-token
17        key: token

On Azure Blob Storage, the data files go through the ADLS file IO, with credentials vended by the catalog:

YAML

1fluss:
2  configurationOverrides:
3    # ... your remote.data.dir and storage keys (see Configuring Remote Storage) ...
4    datalake.enabled: "true"
5    datalake.format: iceberg
6    datalake.iceberg.type: rest
7    datalake.iceberg.uri: http://<LAKEKEEPER_HOST>:<LAKEKEEPER_PORT>/catalog
8    datalake.iceberg.warehouse: <WAREHOUSE_NAME>
9    datalake.iceberg.token: "${env:FLUSS_DATALAKE_ICEBERG_TOKEN}"
10    datalake.iceberg.io-impl: org.apache.iceberg.azure.adlsv2.ADLSFileIO
11    datalake.iceberg.header.X-Iceberg-Access-Delegation: vended-credentials
12  secrets:
13    env:
14      # ... the entries for remote storage ...
15      - name: FLUSS_DATALAKE_ICEBERG_TOKEN
16        secretName: fluss-lake-client-token
17        key: token

Polaris REST Catalog

The Fluss servers hold the OAuth2 client credential and scope, and the warehouse's S3 keys.

A Polaris catalog authenticates Fluss with an OAuth2 client credential, written <CLIENT_ID>:<CLIENT_SECRET>, and a scope the catalog grants to that principal. Fluss still holds S3 credentials of its own for the data files.

On Amazon S3:

YAML

1fluss:
2  configurationOverrides:
3    # ... your remote.data.dir and storage keys (see Configuring Remote Storage) ...
4    datalake.enabled: "true"
5    datalake.format: iceberg
6    datalake.iceberg.type: rest
7    datalake.iceberg.uri: http://<POLARIS_HOST>:<POLARIS_PORT>/api/catalog
8    datalake.iceberg.warehouse: <WAREHOUSE_NAME>
9    datalake.iceberg.credential: "${env:FLUSS_DATALAKE_ICEBERG_CREDENTIAL}"
10    datalake.iceberg.scope: <OAUTH2_SCOPE>
11    datalake.iceberg.io-impl: org.apache.iceberg.aws.s3.S3FileIO
12    datalake.iceberg.client.region: <AWS_REGION>
13    datalake.iceberg.s3.access-key-id: "${env:FLUSS_DATALAKE_ICEBERG_S3_ACCESS_KEY_ID}"
14    datalake.iceberg.s3.secret-access-key: "${env:FLUSS_DATALAKE_ICEBERG_S3_SECRET_ACCESS_KEY}"
15  secrets:
16    env:
17      # ... the entries for remote storage ...
18      - name: FLUSS_DATALAKE_ICEBERG_CREDENTIAL
19        secretName: fluss-storage-credentials
20        key: datalake-iceberg-credential
21      - name: FLUSS_DATALAKE_ICEBERG_S3_ACCESS_KEY_ID
22        secretName: fluss-storage-credentials
23        key: datalake-iceberg-s3-access-key-id
24      - name: FLUSS_DATALAKE_ICEBERG_S3_SECRET_ACCESS_KEY
25        secretName: fluss-storage-credentials
26        key: datalake-iceberg-s3-secret-access-key

Verify Lake Tiering Is Configured

After applying the updated values, confirm the lake configuration reached the running server:

BASH

1kubectl exec -n <NAMESPACE> coordinator-server-0 -- \
2  cat /opt/fluss/conf/server.yaml | grep -E 'datalake\.'

The output shows the datalake.* keys you configured, with config-provider markers as written.

Confirm the lake plugin is installed:

BASH

1kubectl exec -n <NAMESPACE> coordinator-server-0 -- ls /opt/fluss/plugins/iceberg/

The directory lists the JARs of the lake-iceberg setup. Run the same check on a tablet server pod.

The server is now ready to tier, but data reaches the lake only once the tiering service runs and a table with lake tiering enabled receives writes. The next section runs the tiering service.

Running the Tiering Service

The tiering service is a long-running Flink streaming job, separate from the Fluss cluster, that reads tables from Fluss and writes them to the lake. It takes its lake configuration as job arguments: the server's datalake.* keys and values again, without datalake.enabled. For enabling tiering on a table and querying tiered tables, see the Quickstart: Fluss on Ververica Platform 3; for the credentials a read of tiered tables carries, see Lake Credentials.

Job Classpath

Install two setups' client targets into a local directory, as Installing Fluss Setups shows:

lake-tiering-service, the tiering job itself.
lake-iceberg, with the same --fs and --catalog options as the server's lake plugin.

The lake-tiering-service directory holds the job's entry JAR, fluss-flink-tiering-1.0.0-vv-3.jar. The lake-iceberg directory holds the JARs the job needs beside it: the Fluss Flink connector, the Fluss lake and filesystem plugins, Hadoop, the Iceberg JARs for that store, and, with a JDBC catalog, the PostgreSQL JDBC driver.

Warning

The S3 filesystem JAR of 1.0.0-vv-3 has a known issue on the engine version vera-4.5-jdk11-flink-1.20: reading data in remote storage fails with NoSuchMethodError: Configuration.getEnumSet. On Amazon S3 or an S3-compatible store, install the S3 filesystem from the 0.9.1-vv-1 release. Of its JARs, use only fluss-fs-s3-0.9.1-vv-1.jar, in place of the S3 filesystem JAR that lake-iceberg installs, and keep the Fluss Flink connector of 1.0.0-vv-3.

BASH

1docker run --rm -v "$PWD/<LOCAL_DIR>":/output \
2  registry.ververica.cloud/platform-images/fluss:0.9.1-vv-1 \
3  /opt/fluss/bin/setup/install.sh client fs-s3 --output-dir /output/fs-s3

Then upload these JARs, and create the JAR deployment with the entry JAR as JAR URI, the lake-iceberg JARs as Additional Dependencies, and the arguments below as Entry Point Main Arguments, with each credential a secret value referenced as ${secret_values.<name>}.

Job Arguments

Each argument is a key and the value after it. Every list starts with the connection to Fluss: the bootstrap address (see Connecting to Fluss) and, with SASL on the client listener, a client user's credentials. The lake configuration follows. The arguments are shown one key or value per line for reading; Ververica Platform 3 takes them joined on one line (see Running a JAR Deployment).

The job also reads optional lake.tiering.* arguments as its own configuration: lake.tiering.auto-expire-snapshot (default false) expires lake snapshots on each tiering commit, even for a table whose own snapshot-expiration option is off, and lake.tiering.io.tmp.dirs sets the local directories for its temporary IO files, defaulting to Flink's IO temporary directories with a fluss child directory.

For a JDBC catalog on Amazon S3, the job holds its SASL password, the database password, and the warehouse's storage credentials:

TEXT

1--fluss.bootstrap.servers
2<BOOTSTRAP_ADDRESS>
3--fluss.client.security.protocol
4SASL
5--fluss.client.security.sasl.mechanism
6PLAIN
7--fluss.client.security.sasl.username
8<SASL_USERNAME>
9--fluss.client.security.sasl.password
10${secret_values.fluss-sasl-password}
11--datalake.format
12iceberg
13--datalake.iceberg.catalog-impl
14org.apache.iceberg.jdbc.JdbcCatalog
15--datalake.iceberg.uri
16jdbc:postgresql://<POSTGRES_HOST>:<POSTGRES_PORT>/<DATABASE>
17--datalake.iceberg.jdbc.user
18<JDBC_USER>
19--datalake.iceberg.jdbc.password
20${secret_values.iceberg-jdbc-password}
21--datalake.iceberg.warehouse
22s3://<LAKE_BUCKET>/<WAREHOUSE_PREFIX>
23--datalake.iceberg.io-impl
24org.apache.iceberg.aws.s3.S3FileIO
25--datalake.iceberg.client.region
26<AWS_REGION>
27--datalake.iceberg.s3.access-key-id
28${secret_values.iceberg-s3-access-key-id}
29--datalake.iceberg.s3.secret-access-key
30${secret_values.iceberg-s3-secret-access-key}

For a JDBC catalog on Azure Blob Storage, the job holds its SASL password, the database password, and the storage account key:

TEXT

1--fluss.bootstrap.servers
2<BOOTSTRAP_ADDRESS>
3--fluss.client.security.protocol
4SASL
5--fluss.client.security.sasl.mechanism
6PLAIN
7--fluss.client.security.sasl.username
8<SASL_USERNAME>
9--fluss.client.security.sasl.password
10${secret_values.fluss-sasl-password}
11--datalake.format
12iceberg
13--datalake.iceberg.catalog-impl
14org.apache.iceberg.jdbc.JdbcCatalog
15--datalake.iceberg.uri
16jdbc:postgresql://<POSTGRES_HOST>:<POSTGRES_PORT>/<DATABASE>
17--datalake.iceberg.jdbc.user
18<JDBC_USER>
19--datalake.iceberg.jdbc.password
20${secret_values.iceberg-jdbc-password}
21--datalake.iceberg.warehouse
22abfss://<LAKE_CONTAINER>@<STORAGE_ACCOUNT>.dfs.core.windows.net/<WAREHOUSE_PATH>
23--datalake.iceberg.io-impl
24org.apache.iceberg.azure.adlsv2.ADLSFileIO
25--datalake.iceberg.adls.auth.shared-key.account.name
26<STORAGE_ACCOUNT>
27--datalake.iceberg.adls.auth.shared-key.account.key
28${secret_values.iceberg-adls-account-key}

For a JDBC catalog on an S3-compatible store, the job holds its SASL password, the database password, the warehouse's storage credentials, and the remote-storage S3 keys:

TEXT

1--fluss.bootstrap.servers
2<BOOTSTRAP_ADDRESS>
3--fluss.client.security.protocol
4SASL
5--fluss.client.security.sasl.mechanism
6PLAIN
7--fluss.client.security.sasl.username
8<SASL_USERNAME>
9--fluss.client.security.sasl.password
10${secret_values.fluss-sasl-password}
11--datalake.format
12iceberg
13--datalake.iceberg.catalog-impl
14org.apache.iceberg.jdbc.JdbcCatalog
15--datalake.iceberg.uri
16jdbc:postgresql://<POSTGRES_HOST>:<POSTGRES_PORT>/<DATABASE>
17--datalake.iceberg.jdbc.user
18<JDBC_USER>
19--datalake.iceberg.jdbc.password
20${secret_values.iceberg-jdbc-password}
21--datalake.iceberg.warehouse
22s3://<LAKE_BUCKET>/<WAREHOUSE_PREFIX>
23--datalake.iceberg.io-impl
24org.apache.iceberg.aws.s3.S3FileIO
25--datalake.iceberg.client.region
26<AWS_REGION>
27--datalake.iceberg.s3.endpoint
28<S3_ENDPOINT>
29--datalake.iceberg.s3.path-style-access
30true
31--datalake.iceberg.s3.access-key-id
32${secret_values.iceberg-s3-access-key-id}
33--datalake.iceberg.s3.secret-access-key
34${secret_values.iceberg-s3-secret-access-key}
35--fluss.client.fs.s3.endpoint
36<S3_ENDPOINT>
37--fluss.client.fs.s3.access.key
38${secret_values.fluss-s3-access-key}
39--fluss.client.fs.s3.secret.key
40${secret_values.fluss-s3-secret-key}
41--fluss.client.fs.s3.region
42<AWS_REGION>
43--fluss.client.fs.s3.path-style-access
44true

A Lakekeeper catalog on an S3-compatible store needs only the fluss.client.fs.s3.* arguments of this list, added to its Amazon S3 list below.

Warning

A JDBC catalog on an S3-compatible store might also need these JVM options on the job, the counterpart of the server's environment variables:

YAML

1env.java.opts.all: "-Daws.requestChecksumCalculation=when_required -Daws.responseChecksumValidation=when_required"

For a Lakekeeper catalog on Amazon S3, the job holds its SASL password and a token; storage access comes through the catalog:

TEXT

1--fluss.bootstrap.servers
2<BOOTSTRAP_ADDRESS>
3--fluss.client.security.protocol
4SASL
5--fluss.client.security.sasl.mechanism
6PLAIN
7--fluss.client.security.sasl.username
8<SASL_USERNAME>
9--fluss.client.security.sasl.password
10${secret_values.fluss-sasl-password}
11--datalake.format
12iceberg
13--datalake.iceberg.type
14rest
15--datalake.iceberg.uri
16http://<LAKEKEEPER_HOST>:<LAKEKEEPER_PORT>/catalog
17--datalake.iceberg.warehouse
18<WAREHOUSE_NAME>
19--datalake.iceberg.token
20${secret_values.iceberg-token}
21--datalake.iceberg.io-impl
22org.apache.iceberg.aws.s3.S3FileIO
23--datalake.iceberg.client.region
24<AWS_REGION>

For a Lakekeeper catalog on Azure Blob Storage, the job holds its SASL password and a token:

TEXT

1--fluss.bootstrap.servers
2<BOOTSTRAP_ADDRESS>
3--fluss.client.security.protocol
4SASL
5--fluss.client.security.sasl.mechanism
6PLAIN
7--fluss.client.security.sasl.username
8<SASL_USERNAME>
9--fluss.client.security.sasl.password
10${secret_values.fluss-sasl-password}
11--datalake.format
12iceberg
13--datalake.iceberg.type
14rest
15--datalake.iceberg.uri
16http://<LAKEKEEPER_HOST>:<LAKEKEEPER_PORT>/catalog
17--datalake.iceberg.warehouse
18<WAREHOUSE_NAME>
19--datalake.iceberg.token
20${secret_values.iceberg-token}
21--datalake.iceberg.io-impl
22org.apache.iceberg.azure.adlsv2.ADLSFileIO
23--datalake.iceberg.header.X-Iceberg-Access-Delegation
24vended-credentials

For a Polaris catalog on Amazon S3, the job holds its SASL password, the OAuth2 client credential and scope, and the warehouse's S3 keys:

TEXT

1--fluss.bootstrap.servers
2<BOOTSTRAP_ADDRESS>
3--fluss.client.security.protocol
4SASL
5--fluss.client.security.sasl.mechanism
6PLAIN
7--fluss.client.security.sasl.username
8<SASL_USERNAME>
9--fluss.client.security.sasl.password
10${secret_values.fluss-sasl-password}
11--datalake.format
12iceberg
13--datalake.iceberg.type
14rest
15--datalake.iceberg.uri
16http://<POLARIS_HOST>:<POLARIS_PORT>/api/catalog
17--datalake.iceberg.warehouse
18<WAREHOUSE_NAME>
19--datalake.iceberg.credential
20${secret_values.iceberg-credential}
21--datalake.iceberg.scope
22<OAUTH2_SCOPE>
23--datalake.iceberg.io-impl
24org.apache.iceberg.aws.s3.S3FileIO
25--datalake.iceberg.client.region
26<AWS_REGION>
27--datalake.iceberg.s3.access-key-id
28${secret_values.iceberg-s3-access-key-id}
29--datalake.iceberg.s3.secret-access-key
30${secret_values.iceberg-s3-secret-access-key}

Troubleshooting

Lake plugin directory empty after pod start

Inspect the init container's logs:

BASH

1kubectl logs -n <NAMESPACE> <POD_NAME> -c install-plugins

Check that the lake-iceberg installer call ran without error, and that the Fluss container mounts the plugins volume at /opt/fluss/plugins/iceberg with the iceberg sub-path, on both the coordinator and the tablet servers. For general init-container diagnostics, see Installing Fluss Setups.

REST catalog: `Unauthorized` or `Failed to obtain access token`

The catalog rejected the credential Fluss presents. Check that:

For Polaris, datalake.iceberg.credential has the form <CLIENT_ID>:<CLIENT_SECRET>, and datalake.iceberg.scope is a scope the catalog grants to that principal.
For Lakekeeper, the token is issued for a principal the catalog accepts.