Features and Benefits List
Ververica Unified Streaming Data Platform solves common enterprise use cases. This list is organized by the core feature sets of the platform.
- Developer Efficiency
- Operational Excellence
- Security
- Data Governance
- Multi-Tenancy
- Elasticity
- Available Everywhere
Developer Efficiency
One of the key goals of Ververica Unified Streaming Data Platform is to enhance developer productivity by offering an integrated development environment along with efficient enterprise tooling for all our customer’s development needs, maximizing performance and productivity while minimizing resource use.
| Name | Description | Benefits for Ververica Users |
|---|---|---|
| SQL IDE | Users can use the SQL Integrated Development Environment (IDE) within the product for end-to-end development tasks. The integrated SQL development features include code validation, debugging, version control, co-editing with locking/unlocking, and pre-built code templates. It supports local validation of SQL code, debugging, and collaboration. | + Lower cost and easier workflows without third-party tools. + Faster, higher-quality development with local, short-lived context checks. + Safer pipelines with visualized transformations and output views. + Simplified, secure versioning within the tenant domain. + Secure work protection with flexible sharing controls. + Faster prototyping with pre-built templates for common use cases. + Accelerated delivery through reusable solutions and libraries. |
| Rich Connector System | Users can use open-source and enterprise-grade connectors that facilitate integration with various data sources and sinks, including cloud ecosystems, databases, and messaging systems. It enables seamless integration with a diverse array of data sources and sinks, including cloud ecosystems like AWS, Azure, Google Cloud, databases, and messaging systems. | Data Engineers can use existing connectors to connect with external systems rather than investing the time to write their own implementations. Advantages include: + Pre-built connectors eliminate custom coding. + Supports diverse data sources and infrastructures. + Reduces setup time for new users. + Integrates with tools like Prometheus and Datadog. + Provides industry solutions by supporting specialized use cases, such as finance and others. + Adapts to various data types and use cases. + Ensures secure, high-performance data flows. |
| Rich Programming APIs | Users can leverage the comprehensive programing APIs for Java, Scala, Python, and SQL to facilitate relational processing and advanced analytics. It supports stream and batch processing with high-level abstractions and APIs (SQL/Table API, DataStream API, and ProcessFunction) for various use cases. | + Allows developers to use their preferred language and API. + Unified APIs for stream and batch processing support real-time analytics, machine learning, ETL, and event-driven applications. + SQL interface with familiar syntax simplifies stream processing for database professionals. + Integrated APIs eliminate the need for third-party tools, reducing CAPEX and OPEX. + High-level abstractions in Table API and SQL let developers focus on business logic rather than boilerplate code. + Supports interdisciplinary projects by enabling teams with diverse programming backgrounds to collaborate effectively. |
| Artifacts Management | Users can integrate tenant-level file storage for development artifacts within the Ververica environment. It allows users to upload, store, and access artifacts within the tenant context, enabling efficient management of deployment files. Files can be referenced in deployments and managed through a secure storage system. | + Provides file system storage near the data pipeline, reducing latency and simplifying workflows. + Eliminates the need for additional external file systems. + Keeps development artifacts within the tenant space, protected by access controls. + Enables teams to work independently by preventing shared artifact conflicts, accelerating time to market. |
| Custom Connectors | Users can register and manage custom-developed connectors within a tenant context. It supports custom connectors on top of the built-in ones, enabling users to manage proprietary or legacy connectors seamlessly. | + Enables flexible integration of data governance artifacts, allowing users to register custom connectors without waiting for built-in support. + Supports integration of legacy or proprietary connectors, maximizing the value of previous investments. |
| Custom Formats | Users can register and manage custom-developed data formats on top of built-in formats. It allows users to integrate proprietary or legacy data formats into the platform, enhancing compatibility with various data sources. | + Accelerates time to production by enabling flexible integration of data governance artifacts. Users can register custom-developed formats without waiting for built-in support. + Reduces costs and maximizes the value of prior investments by supporting integration of legacy or proprietary formats. |
| User-defined Functions | Users can perform complex data manipulation within the SQL environment with scalar, aggregate, and table user-defined functions (UDFs). They enable complex data manipulations through reusable functions within the familiar SQL environment. | + Reduces errors and boosts productivity by minimizing repetitive coding. + Allows developers to focus on strategic tasks, resulting in more robust application performance. |
| Collaborative Workspaces | Users can perform RBAC-based multi-tenant collaboration through isolated workspaces. It supports team collaboration with predefined access roles. Each workspace operates independently with dedicated resources, ensuring performance scalability and data protection. | + Provides isolated environments with dedicated computing resources, enabling teams to work independently without interference. + Dedicated resources ensure consistent performance and seamless scalability. + Predefined access roles clarify user responsibilities and reduce configuration errors. + Clear access limits minimize security risks. |
Operational Excellence
Ververica Unified Streaming Data Platform streamlines operations with features that enable continuous improvement, observability, optimization, and management of data streaming infrastructures. It allows for data-driven decisions based on enhanced efficiency, effectiveness, and agility.
| Name | Description | Benefits for Ververica Users |
|---|---|---|
| Deployment Cluster | Users can dedicate processing to specific compute clusters. Each application runs on its dedicated Flink cluster, starting and stopping with the application. | Deployment clusters ensure that critical applications have isolated resources, enhancing security and performance, perfect for high-priority tasks. |
| Session Clusters | Users can share processing across compute clusters. The pre-existing Flink cluster (session cluster) runs multiple applications, independently of job lifecycles. | Session clusters maximize resource utilization and reduce costs by allowing multiple applications to run on a single infrastructure, ideal for less critical, variable workloads. |
| Built-in Dashboards | Users leverage the visual dashboard for insights into trial info, costs, usage, and operations. It provides workspace-level insights into resource usage (storage, compute, network), costs, and health of deployments, including trial/credit information. | + Enables businesses to prevent bill shock and optimize operational expenses by comparing current hourly costs, trends, and previous billing data to make informed adjustments. + Provides a unified operations board for real-time deployment status, enabling faster issue detection and resolution. + Reduces downtime and ensures consistent time to market by identifying usage peaks and trends, allowing proactive measures to prevent system saturation and operational disruptions. |
| Application Lifecycle Management | Users manage the inventory and lifecycle of streaming applications. The lifecycle of stream processing clusters, includes starting, stopping, cloning, deleting, and stopping with savepoints or drain. It also inventories applications with multiple labels/tags. | + Exposing life cycle management through API/UI’s saves time and reduces errors, thus decreases OPEX costs. + Organizing inventory by processing clusters by teams, purpose etc. It decreases OPEX costs by decreasing time and reducing errors. |
| State Management | Users manage stream processing cluster state, backup, and recovery. It allows the creation of snapshots, resuming from specific checkpoints/savepoints (even from different deployments), and skipping non-restorable states. | + Increases resiliency of the platform by allowing a de-coupled and external store for backup and recovery and geo-redundant resiliency. + Flexible state management allows for less errors, flexibility and re-use of state for new use cases and experimentation. This increases business opportunity and revenue (new processing of same state or vice versa), quality (re-runs and checks and testing), saves OPEX (re-use), and reduces time to recovery (e.g. choosing a savepoint). |
| Built-in Monitoring (Logs, Events, Metrics) | Users can access centralized tracking of logs, events, and metrics from various applications and clusters, aggregating diagnostic data. | + Provides developers and system administrators with quick access to application performance and system health data, enabling faster issue resolution and informed decision-making. + Reduces the complexity of managing system diagnostics, allowing users to maintain application availability and performance without the need for extensive monitoring infrastructure. |
| Configuration Management | Users can access centralized management of configuration settings, including robust validation and error-checking mechanisms; defining policy configurations, and making real-time changes without requiring system restarts. | + Automates checks and validations to reduce the risk of human error in system configurations, improving reliability and stability. + Enables default configurations and policy settings, ensuring consistent standards across deployments and simplifying compliance with internal and external regulations. + Supports configuration management at both the tenant and global levels, offering control and flexibility while maintaining overall policy consistency to meet specific business needs. |
| Intelligent Diagnosis (basic) | Users can review system recommendations for performance tuning. The system detects saturation or abnormal states early and provides job runtime improvement recommendations. | Enables the customer to maintain stable SLA’s and prevents service interruptions which increases business continuity and revenue per hour. |
| Declarative Operations Management (K8s) | Users can natively manage Flink jobs with the extended Kubernetes API. | The Ververica Kubernetes Operator allows for Kubernetes-native management of Flink jobs and streamlines the CI/CD workflow by enabling: + Native Kubernetes command utilization with standard kubectl commands to manage Flink deployments and configurations, aligning with Kubernetes best practices. + Automation of CI/CD workflows by integration with Kubernetes to facilitate more robust automation pipelines, reducing reliance on REST APIs. + Synchronization to ensure that operations performed via kubectl are reflected in the Ververica Unified Streaming Data Platform Web UI to maintain a consistent user experience. + Allows users to choose their preferred interaction model. + Reduces learning curves and increases productivity. + Makes it easier to monitor, manage, and align applications with Kubernetes' state management and scaling practices. |
Security
Ververica Unified Streaming Data Platform provides mechanisms to protect sensitive, private data and ensures compliance with regulatory standards at data and user access levels.
We aim to give people access to the right data at the right time.
| Name | Description | Benefits for Ververica Users |
|---|---|---|
| Secrets Management | Users can centrally manage sensitive data like passwords, tokens, and keys to secure applications across configurations. This feature ensures that while Secret Values are referenced by name in deployment specifications, the actual values remain hidden from users. Bound to a specific Namespace, the names of Secret Values must be unique within that Namespace, reinforcing security and organization. | + Securely store sensitive information and reference it by name, minimizing exposure and adhering to best security practices. + Centralized management allows easier updates and modifications, eliminating the need to alter multiple configurations or deployment scripts. + Keeps secrets out of user interfaces and API responses, reducing the risk of accidental exposure. + Simplifies compliance with regulations protecting sensitive data, supporting audits with secure management practices. |
| Single Sign-On (SSO) | Users can access Ververica Unified Streaming Data Platform using their existing enterprise-level authentication credentials. SSO supports integration with enterprise authentication protocols (SAML, OIDC) and connects with identity providers (IdPs) for a secure, seamless login experience. | + Simplifies the user experience by reducing password fatigue and minimizing login complications. + Enhances security by leveraging existing enterprise-level authentication standards and reduces the administrative burden on IT departments by centralizing user access control. |
| Social Login | Users can access Ververica products using their existing social media accounts such as Google, Github, etc., supporting automatic account creation and synchronization. | + Simplifies the onboarding and login process, significantly reducing barriers to entry for new users. + Enhances user convenience by eliminating the need for multiple passwords and decreasing the time spent on account setup. |
| Audit Logs | System administrators can monitor 'who did what and when' by capturing events via users, REST API, and a Kubernetes Operator. All events and activities are captured and recorded, providing a detailed record of activities within the system. Administrators can view, filter, search, and export logs for security, compliance, and troubleshooting purposes. | + Audit logs record all system interactions, providing a critical tool for compliance and improving overall system security. + The system logs every action to maintain a clear record of operations, supporting transparency and accountability. + Audit logs help quickly identify actions, simplifying the troubleshooting process and system monitoring. + Detailed, exportable logs facilitate compliance with industry regulations and standards by tracking data access and changes. + System administrators can monitor audit logs to detect unauthorized access or suspicious activities within the system. + Logs help trace actions that contributed to issues or system failures, allowing for faster resolution. + Audit logs can be exported and integrated into reports to prove adherence to legal and regulatory standards. + Regular log reviews provide insights into user behavior and system usage, aiding in better system management and optimization. |
| Authorization (RBAC) | System administrators can assign specific permissions to users based on their roles. Administrators define and enforce security policies through role assignments, controlling access to resources based on user roles. | + Ensures that users access only the data and functions necessary for their roles, minimizing potential internal threats and data breaches. + Simplifies compliance with data protection regulations by systematically controlling access. |
| Authentication: Access Tokens | Users can create and manage autogenerated tokens for non-interactive API access within Ververica Unified Streaming Data Platform. | + Enables users to integrate tooling for continuous integration, delivery and deployment (CI/CD) programmable workflows. + Allows users to quickly build, test, and deploy with integrations in existing environments and CI/CD programmable workflows. + Increases quality and reduces operational (reduces OPEX) loss/debt by allowing programable CI/CD. + Increases speed and reduces errors, improving OPEX, by enabling parallel access-controlled CI/CD tracks. |
| Pluggable (TLS) Certificates | Users can generate custom SSL certificates to enable enhanced SSL support for both job and session clusters, allowing users to secure internal and external communications within Flink applications using SSL. This feature provides the flexibility to either use automatically generated SSL certificates provided by Ververica Unified Streaming Data Platform or to implement user-provided, custom certificates for a more tailored security setup. | Pluggable certificates are ideal for environments with high data security needs, such as financial services, healthcare, and public services. + Enables enterprises to deploy Flink applications securely across cloud and on-premises environments, ensuring all communications meet enterprise-grade security standards. + Ensures that all data transmitted between Flink components is encrypted, providing stronger security for sensitive information. + Meets necessary security and privacy standards, ideal for enterprises handling sensitive or regulated data. + Organizations can align security measures with internal policies or regulatory requirements by using custom certificates. + Guarantees that data flows are secure, ensuring the integrity and confidentiality of internal communications. + Reduces the need for external security tools and services, streamlining operations. |
| Private Connections | Users can connect Ververica products with cloud provider services over the intranet. It supports direct private connections to major cloud providers (AWS, Azure, GCP), ensuring secure data transfer without public internet exposure. | + Reduces exposure to public internet vulnerabilities and cyber threats by enabling secure, private connections. + Minimizes latency and network congestion, leading to more reliable and faster cloud service interactions. + Reduces expenses associated with high-volume data transfer over public networks, making it more cost-effective for users managing large datasets or requiring frequent access to cloud resources. |
Data Governance
Ververica Unified Streaming Data Platform provides a framework, or integrated business capabilities, that help our users set, enforce, and monitor policies on platform resources and data. Users can evaluate and implement diverse governance policies and monitor and enforce them across their organizations’ business systems.
| Name | Description | Benefits for Ververica Users |
|---|---|---|
| Data Lineage | Users can track their data source, processing, and origin to evaluate its value. Data lineage illustrates the relationships and outlines the flow and dependency relationships between metadata abstractions themselves (catalogs, tables, fields) and between metadata and jobs (stream, batch). | + Ensures transparency and trust in the data, by revealing the data's origin and its utilization process. + Provides visibility into the properties and relationships of table fields for better understanding and decision-making. + Helps identify the root cause of data processing errors quickly, minimizing business disruptions and labor costs. + Addresses changes or errors in data assets by identifying affected deployments, preventing costly inaccuracies. + Aids in decommissioning unused services, improving data processing quality, and lowering operational costs. |
| Built-in Catalogs | Users can create, modify, and delete catalog entries through centralized metadata management for greater data visibility, accessibility, and reliability. | + Centralizing metadata management improves data visibility and accessibility, leading to more accurate analytics and informed decision-making. + Simplifies data management processes, freeing IT staff to focus on strategic initiatives rather than routine administrative tasks. + Provides better oversight and transparency of data assets, helping organizations meet regulatory standards and reduce risks associated with data mismanagement. |
| Custom Catalogs | Users can register and manage custom-developed catalogs (proprietary or legacy) within a tenant context. It supports the integration of custom or proprietary catalogs without waiting for built-in support. Users can govern custom catalogs alongside built-in ones. | + Decreases time to production by allowing flexibility in the integration and data governance artifacts. + Saves costs and increases value of previous investments as the user can integrate legacy or special/proprietary catalogs. |
Multi-Tenancy
Ververica Unified Streaming Data Platform allows many independent applications to run in a shared environment, with the ability to isolate resources and data at the namespace and tenant level, utilizing role-based access control.
| Name | Description | Benefits for Ververica Users |
|---|---|---|
| Independent Flink Environments | Customers can manage multiple workspaces under a single business account. They are centrally managed and each workspace operates as an isolated tenant environment with independent resources designed for rapid, on-demand provisioning and de-provisioning. | + Higher density of workspaces on physical hosts reduces the cost per compute unit (CU) for customers. + Rapid, on-demand provisioning and de-provisioning of workspaces align resource usage with business needs, reducing costs during low-demand periods (e.g., weekends, off-peak hours) and scaling efficiently during peak times. + Removes the need for meticulous CAPEX planning by allowing flexible, on-demand resource adjustments based on business or operational demands, saving both time and money. + Accelerates deployment of business offerings and data pipelines by providing immediate access to ready-to-use environments. + Supports different environment types (development, testing, staging, production) under centralized control, eliminating the need for third-party platforms and reducing integration costs. |
| Secure Workspace Isolation | Workspaces with enhanced isolation at the hardware virtualization layer, ensuring separation between management APIs and other tenants. It provides hardware-level virtualization and strong security isolation with separate OS structures (I/O, memory, CPU, etc.). | + Provides predictable, stable data pipeline service SLO objectives for individual workspace (tenant). Noisy neighbor (tenant) concern is completely tackled for shared hardware resources. + Provides strong security isolation between workspaces with separate operating system structures (separate I/O path, memory structures and access, process space, cpu contexts, etc). |
Elasticity
Ververica Unified Streaming Data Platform automatically and dynamically scales managed resources up or down and in or out as needed, based on predefined SLO rules or predefined plans.
| Name | Description | Benefits for Ververica Users |
|---|---|---|
| Autopilot | Users can implement either a stable or adaptive resource management strategy. The stable strategy finds a resource plan for the entire running cycle, while the adaptive strategy adjusts resources dynamically based on real-time metrics. | + Stable Strategy eliminates trial-and-error efforts for achieving stable Service Level Objectives (SLOs) after updates, saving significant time on configuration adjustments. No tuning expertise is required, reducing knowledge-related CAPEX. This leads to quicker fulfillment of business SLAs, improving the likelihood of meeting SLA contracts and boosting revenue. + Adaptive Strategy balances OPEX savings with more predictable SLAs. Enables users to offer stricter SLAs, such as those with latency requirements, allowing them to capture a broader market, generate more revenue, and optimize resource usage. |
| Planned Resources Management | Users can create and execute resource management plans for fixed periods based on frequency, duration, and start time, helping balance resource allocation for different workloads. | + Enables planned balancing of shared infrastructure resources across different pipelines and workloads, optimizing resource utilization and reducing costs (CAPEX and OPEX). + Allows resource allocation flexibility, for instance, during nightly build runs, testing deployments can be scheduled with reduced capacity to allocate more resources to the build, ensuring efficient resource use. + Helps plan for predictable traffic spikes and burst events, such as high CPU parallelism requirements on New Year's Eve, or scheduled maintenance and backups, ensuring business continuity. + Allows scaling down resources during low utilization periods, such as off-hours or weekends, to save on operational costs. + Balances resources across different time zones or business priorities, adjusting resource configurations based on peak or off-peak periods. |
Available Everywhere
Ververica Unified Streaming Data Platform provides options for the service model and deployment model based on the customer’s requirements.
| Name | Description | Benefits for Ververica Users |
|---|---|---|
| Multiple Deployment Options | Users select their preferred deployment option and level of required management. See deployment options for more information. | + Ververica Platform: Self-Managed offers the highest control, security, and flexibility. It allows reuse of CAPEX investment while keeping the value of both CAPEX and OPEX investments. It saves costs of invested infrastructure and devops personnel. + Ververica Cloud: Managed Service offers serverless, fully managed service. It is our fully managed (full stack), serverless deployment option. It provides quicker time to value. Start building business use cases, without knowledge of underlying stack (Flink, CaaS, IaaS etc.) costs, operations, investments. Significant CAPEX savings. + Ververica Cloud: Bring Your Own Cloud combines partial management by Ververica with customer control of IaaS. It keeps the value of previous CAPEX investments, but lowers future OPEX investments, saving cost by allowing co-location / workload proximity, and avoiding redundant IaaS costs (network, compute etc.) It increases security by allowing the customer to keep data sovereignty compliance. |
| Multi-cloud Support | Users can choose between major cloud providers (Amazon, Microsoft, Google), enabling cross-cloud data integration and real-time processing. Ververica supports deployment on different cloud platforms, offering compatibility and cross-cloud data integration, ensuring unified management across providers. | + Multi-Cloud flexibility avoids vendor lock-in, allowing businesses to choose or switch cloud providers based on pricing, performance, geographic location, or specific service offerings. + Enables businesses to leverage the best offerings from each cloud provider, leading to significant cost reductions and optimized performance. + Distributes assets across multiple clouds, mitigating risks associated with disruptions from any single cloud provider and improving continuity planning. |