Ozone - Evolution of HDFS scalability to trillions of file system objects

Dinesh Chitlangia from Cloudera

May 13th, 2019, 2:10 PM (duration: 40 minute)

Location: Revolution Brewing- Brewpub

Apache Hadoop Ozone is a robust, distributed key-value object store for Hadoop with layered architecture and strong consistency. It separates the namespace management from block and node management layer, which allows users to independently scale on both axes. Ozone is interoperable with Hadoop ecosystem as it provides OzoneFS (Hadoop compatible file system API), data locality and plug-n-play deployment with HDFS as it can be installed in an existing Hadoop cluster and can share storage disks with HDFS. Ozone solves the scalability challenges with HDFS by being size agnostic. Consequently, it allows users to store trillions of files in Ozone and access them as if they are on HDFS. Ozone plugs into existing Hadoop deployments seamlessly, and programs like MapReduce, Spark, Hive, Hive Interactive and work without any modifications. This presentation gives us an overview of Ozone and talks about the design, architecture, and future roadmap.