Skip to main content

About the VERA Engine

At the heart of Ververica Unified Streaming Data Platform is VERA — a performance-optimized runtime built on Apache Flink® that powers enterprise-grade stream processing at scale. VERA combines the best of open-source Flink technology with Ververica’s own advanced innovations to deliver a cloud-native, resilient, and scalable platform for real-time data applications.

Designed to meet the demands of modern data workloads, VERA seamlessly blends open-core compatibility with proprietary enhancements, enabling organizations to handle large, stateful streaming applications efficiently while simplifying operations and accelerating time to value.

VERA components

VERA's Underlying Technology​

The key technology layers and advanced features that make VERA uniquely suited to power complex, mission-critical streaming use cases include:

  • Open-core technology stack that is 100% compatible with open-source Apache Flink, allowing seamless flow back and forth between Ververica Unified Streaming Data Platform and Flink.
  • Enterprise-grade technology that Ververica built to address the challenges with open-source Apache Flink® including Ververica’s Flink optimizations and advanced CDC connectors.
  • Advanced features that enable VERA’s cloud-native architecture, integration with the underlying technology stack, stream processing optimizations, and user experience improvements to meet the most demanding challenges.

VERA underlying technology

Open-Core Tech Stack​

VERA follows an open-core model, combining open-source Flink with proprietary technology and features developed by Ververica. These components complement the open-source core, ensuring VERA remains 100% compatible with Flink. VERA preserves all the benefits of Flink’s unified programming model, including low-latency processing, high throughput, and stateful stream processing. Flink applications can run on VERA without requiring modifications or compatibility issues.

Ververica-Built Tech Stack​

VERA’s enterprise-grade infrastructure is built around two key technologies: the enterprise state engine and advanced connectors.

VERA’s state management is powered internally by Ververica's proprietary, cloud-native engine called Gemini. While VERA provides the overall platform and operational experience, Gemini ensures state is managed efficiently, enabling faster job restarts, reduced downtime, and optimized resource utilization for large, stateful streaming workloads. Its key features include:

  • Compute-Storage Separation. This feature decouples state storage from local disks. It prevents failures caused by limited disk space and simplifies large deployments.
  • Adaptive Key-Value Separation. This feature automatically optimizes dual- and multi-stream JOINs. It improves throughput and resource utilization, sometimes doubling efficiency in high-demand scenarios.
  • Lightweight Savepoints. This feature accelerates the creation of checkpoints and savepoints. It provides stability even for deployments with large amounts of state data.
  • Adaptive Parameter Tuning. This feature automatically tunes internal parameters based on data access and traffic patterns. It reduces manual tuning and boosts performance.
  • Hot Job Updates. These updates allow you to modify running Flink jobs with minimal downtime. They ensure continuous stream processing without interrupting data flows.
  • Lazy State Loading. This feature loads state on demand rather than preloading it at job startup. It reduces memory usage and speeds up startup time for large, state-intensive workloads.
  • Optimized Parallelism Changes. These changes dynamically adjust the parallelism of running jobs. They reduce bottlenecks and improve throughput and resource utilization for large-scale streaming applications.

Together, these capabilities allow VERA to manage large, stateful streaming workloads efficiently while minimizing downtime and operational overhead. For more information, see the whitepaper: VERA: The Path to Cloud-Native Apache Flink.

Advanced Change Data Capture (CDC) Connectors​

VERA also provides advanced CDC connectors, optimized for enterprise use. These connectors provide scalable data ingestion and integrate seamlessly with Flink CDC, abstracting away complex tasks such as deployment, infrastructure management, fault tolerance, and scalability. This allows you to focus on streamlining workflows and simplifies the overall experience.

Advanced Features​

VERA’s advanced features enhance performance, optimize computing resource utilization, and elevate the user experience. You can:

  • Scale large stateful applications. VERA separates compute and storage resources to handle large workloads efficiently. The Enterprise Flink State Engine (powered internally by Gemini) ensures state is managed reliably at scale.
  • Enhance performance with streaming join optimization. Key-value separation improves throughput for complex streaming joins.
  • Recover faster from job failures or updates. Advanced scheduling functions speed restarts.
  • Update event processing rules on the fly. Dynamic Complex Event Processing (CEP) allows you to change event handling without stopping jobs.
  • Meet enterprise cloud-native requirements. Enhanced catalogs provide robust metadata and operational control for modern streaming workloads.
  • Unlock ML and AI capabilities. Built-in Flink ML and AI applications integrate with the open-source Apache Flink® library that provides pre-installed and pre-configured machine learning APIs.

For More Information​

If you are interested in more content on VERA, see VERA's features and benefits and check out the blog series or video series.

If you want an in-depth look at VERA, see: VERA: The Path to Cloud-Native Apache Flink.

On this page