Advanced Flink Job Configuration
VERA’s Enterprise Flink State Engine, powered by Gemini, provides high-performance, adaptive state management for large, stateful streaming workloads. Gemini is designed to optimize resource usage, reduce downtime, and simplify operational management. Most workloads work well with default configurations, but advanced scenarios may require parameter tuning.
Use these tables to tune the VERA engine according to your business and performance requirements.
Basic Parameters
| Parameter | Description | Data Type | Default | Notes |
|---|---|---|---|---|
table.exec.state.ttl | TTL of state data in SQL deployments | LONG | 129600000 (1.5 days) | Cannot be used with state.backend.gemini.ttl.ms |
state.backend.gemini.ttl.ms | TTL of state data in DataStream or Python deployments | LONG | (none) | Cannot be used with table.exec.state.ttl |
state.backend.gemini.savepoint.external-sort.local-storage.enabled | Store temporary savepoint data on local disk instead of DFS | BOOLEAN | false | Use true if local disk has sufficient space to speed savepoints |
Memory-Related Parameters
| Parameter | Description | Data Type | Default | Notes |
|---|---|---|---|---|
state.backend.gemini.memory.managed | Automatically allocate memory based on managed memory | BOOLEAN | true | Set false to manually configure WriteBuffer and off-heap memory |
state.backend.gemini.total.writebuffer.size | Total memory for WriteBuffer | STRING | 128 MB | Effective only if memory.managed=false |
state.backend.gemini.offheap.size | Off-heap memory size | STRING | (none) | Effective only if memory.managed=false |
Compute-Storage Parameters
| Parameter | Description | Data Type | Default | Notes |
|---|---|---|---|---|
state.backend.gemini.file.cache.type | Compute-storage separation mode | STRING | LIMITED | Use LIMITED if local disk space is constrained |
state.backend.gemini.file.cache.preserved-space | Disk space preserved for state data | STRING | 2 GB | Adjust based on TaskManager disk availability |
Key-Value Separation
| Parameter | Description | Data Type | Default | Notes |
|---|---|---|---|---|
state.backend.gemini.kv.separate.mode | Key-value separation mode | STRING | SPECIFIED_TABLE_ENABLE | Automatically adjusts based on deployment operator characteristics |
state.backend.gemini.kv.separate.value.size.threshold | Threshold for value size triggering separation | INTEGER | 200 bytes | Adaptive tuning will override if enabled |
Adaptive Parameter Tuning
| Parameter | Description | Data Type | Default | Notes |
|---|---|---|---|---|
state.backend.gemini.auto-tune.mode | Adaptive parameter tuning mode | STRING | ACTIVE | Options: DISABLED, MONITORING, ACTIVE, FORCEFUL |
state.backend.gemini.auto-tune.burst.start.x | Start time for performance-first tuning | STRING | (none) | Use yyyy-MM-dd HH:mm:ss format |
state.backend.gemini.auto-tune.burst.end.x | End time for performance-first tuning | STRING | (none) | Must be later than start.x |