OneTable

Seamlessly interoperate between Apache Hudi, Delta Lake, and Apache Iceberg.

Omni-Directional Interoperability

What is OneTable?

  • Onetable provides omni-directional interop between lakehouse table formats
  • OneTable is NOT a new or separate format, OneTable provides abstractions and tools for the translation of lakehouse table format metadata
Choose your source format
Choose your destination format(s)
Onetable will translate the Metadata layers

What is OneTable?

  • OneTable is an omni-directional interop of lakehouse table formats
  • OneTable is NOT a new or separate format, OneTable provides abstractions and tools for the translation of lakehouse table format metadata
Choose your source format
Choose your destination format(s)
Onetable will translate the Metadata layers

Why build OneTable?

  • Choosing a table format is a costly evaluation
  • Each project has rich features that may fit different use-cases
  • Some vendors use a table format as a point of lock-in
  • Your data should be UNIVERSAL

Let's build together

OneTable is a community driven open source project. Come join us on Github and find easy ways to contribute.
Try it on GitHub

FAQs

Got questions? We've got answers.

How does it work?

keyboard_arrow_down

OneTable reads the existing metadata of your table and writes out metadata for one or more other table formats by leveraging the existing APIs provided by each table format project. The metadata will be persisted under a directory in the base path of your table (_delta_log for Delta, metadata for Iceberg, and .hoodie for Hudi). This allows your existing data to be read as though it was written using Delta, Hudi, or Iceberg. For example, a Spark reader can use spark.read.format(“delta | hudi | iceberg>”).load(“path/to/data”). 

How is OneTable different from Delta Lake Uniform?

keyboard_arrow_down

OneTable provides abstraction interfaces that allow omni-directional interoperability across Delta, Hudi, Iceberg, and any other future lakehouse table formats such as Apache Paimon. OneTable is a standalone github project that provides a neutral space for all the lakehouse table formats to constructively collaborate together.

Delta Lake Uniform is a one-directional conversion from Delta Lake to Apache Hudi or Apache Iceberg. Uniform is also governed inside the Delta Lake repo.

When should I consider OneTable?

keyboard_arrow_down

OneTable can be used to easily switch between any of the table formats or even benefit from more than one simultaneously. Some organizations use OneTable today because they have a diverse ecosystem of tools with polarized vendor support of table formats. Some users want lightning fast ingestion or indexing from Hudi and photon query accelerations of Delta Lake inside of Databricks. Some users want managed table services from Hudi, but also want write operations from Trino to Iceberg. Regardless of which combination of formats you need, OneTable ensures you can benefit from all 3 projects.

Does OneTable work in every cloud?

keyboard_arrow_down

Yes, anywhere that Delta, Iceberg, or Hudi work, OneTable works.

What are the current limitations?

keyboard_arrow_down

1. Hudi and Iceberg MoR tables not supported
2. Delta Delete Vectors are not supported
3. Synchronized transaction timestamps

With OneTable you pick one primary format and one or more secondary formats. The write operations with the primary format work as normal. OneTable than translates the metadata from the primary format to the secondaries. When committing the metadata of the secondary formats, the timestamp of the commit will not be the exact same timestamp as shown in the primary.

How can I contribute?

keyboard_arrow_down

Come check out the project on Github and add a little star. There are some low hanging fruit features, bugs, and documentation that can be added. Reach out directly to any of the contributors on Github to ask for help.

How can I learn more?

keyboard_arrow_down

Follow OneTable community channels on Linkedin and Twitter. Become a watcher on Github or reachout directly to any of the Github contributors to learn more.

What are the future plans for open source governance?

keyboard_arrow_down

Current contributors across Onehouse, Microsoft, Google, and others are planning to incubate OneTable into the Apache Software Foundation. Stay tuned for more updates.