Apache Hudi Table Architecture
Apache Hudi Table Architecture Apache Hudi (Hadoop Upserts Deletes and Incrementals) is a data management framework that provides support for incremental data processing and efficient upserts (updates and inserts) and deletes on large datasets stored in data lakes. Hudi’s architecture is designed to provide near real-time data ingestion and supports data versioning and rollback capabilities. Here’s a detailed look at the key components of Hudi’s table architecture: Hudi Table Types Hudi supports two primary table types: Copy-on-Write (CoW) : In this type, data is written to new files during updates, and old files are replaced. This is suitable for read-heavy workloads. Merge-on-Read (MoR) : Here, updates are written to delta files (log files), which are later compacted with base files. This type is suitable for write-heavy workloads where compaction can be deferred. Key Components of Hudi Tables 1. Commit Timeline Timeline Management : Hudi maintains a timeline of commits, savepo...