Ververica celebrates as Apache Paimon becomes top-level project

Ververica, the comprehensive streaming data platform, is celebrating the news that Apache Paimon has graduated to a top-level project. This update integrates a lake format and a Log-structured merge-tree (LMS) structure, enabling the construction of a real-time lakehouse architecture. The architecture encompasses both streaming and batch operations while embedding real-time streaming updates into the lake.

This graduation celebrates the strength of the Paimon community and the project's maturity, testifying to its evolution. The Paimon community achieved this landmark not only by adhering to the Apache community guidelines but also by releasing major feature-rich updates and fostering an inclusive community that welcomes new contributors.

Prior to graduation, the community rolled out key features, which included substantial real-time updates and robust support for handling appended data processing. For instance, they added primary key table support for large-scale updates that set new performance standards. The platform allows users to update records as they wish through Merge Engine, coupled with Changelog-producer management support, thereby ensuring correct and complete change log updates. Additional highlights were the appending of table (no-primary-key) for large-scale batch and streaming processing capabilities and data compaction with z-order sorting to optimise layouts and queries.

Reflecting on Apache Paimon's accomplishment, Vladimir Jandreski, Chief Product Officer at Ververica, said, "Having Paimon as a cornerstone of the streaming lakehouse (Streamhouse) support in Ververica Cloud represents more than a technical achievement. It is a progression towards a future where stream processing on a data lake is accessible, efficient and necessary for businesses worldwide. We are thrilled to persist in contributing towards, and supporting Apache Paimon's journey, delivering leading-edge solutions to our customers."

Platforms like Ververica Cloud and Apache Paimon empower businesses to focus on crafting valuable data products without the burden of intricate tech stack maintenance in their streaming lakehouse projects. The platform provides enhanced flexibility, enabling firms to tailor their data products within Ververica Cloud while rapidly adjusting to changing data needs. Additionally, businesses can streamline operations thanks to Ververica's expertise in stream processing, which ensures efficient and scalable operations. Last but not least is the vibrant community and professional support that guarantees success.

Ververica remains committed to pushing the boundaries of real-time data processing technologies, with the knowledge that the most outstanding innovations stem from collaborative, community-driven efforts. As Apache Paimon takes its next step as a top-level project, Ververica is eager to support its growth and integration, confirming that Ververica Cloud continues to be the prime platform for harnessing the power of real-time data.