This topic mainly covers the architecture, implementation principles and application scenarios of NetEase’s open source big data component Kyuubi project, and demonstrates Kyuubi’s ability to help businesses implement Serverless Spark within NetEase and the corresponding process and thinking through practical cases. Cases.

1. Implementing seamless migration from Hive QL to Spark SQL, achieving an overall 6~8x performance improvement
2. Mixed deployment of YARN and K8S clusters to achieve smooth business migration, elasticity to the cloud and converged scheduling requirements

At the same time, we will introduce how we are directly involved in the Spark open source community and complete the corresponding problem solving and feature optimization in the process."

Speakers:

Kent Yao: Primary Author of Kyuubi Project / Apache Spark Committer / Apache Submarine Committer. @NetEase Bigdata Team