Tales at Scale: Analytics at 1000 QPS and Beyond

Gian Merlino

English Session 2022-07-31 16:50 GMT+8  (ROOM : A) #bigdata

How do you build and operate systems that can ingest millions of events per second, store petabytes of historical data, and run thousands of queries per second, all at subsecond response times? It’s not easy, but it has been accomplished using the right mix of compute-storage design, scatter/gather query engines, and cluster management.

Gian Merlino, Apache Druid® committer and co-founder of Imply will share tales of scale, showing how high-performance systems for interactive data conversations with high concurrency and low latency combining stream and batch data are built and used today.

  • How to build and operate systems that can ingest millions of events, store petabytes of historical data, and run thousands of queries
  • What’s the right mix of compute-storage design, scatter/gather query engines, and cluster management
  • How Apache Druid delivers high-performance for interactive data conversations with high concurrency and low latency, for both streaming and batch data

Speakers:


Gian Merlino: Imply, Co-Founder and Chief Technology Officer, Gian is a co-founder and CTO of Imply. Gian is also one of the main committers of Apache Druid. Previously, Gian led the data ingestion team at Metamarkets and held senior engineering positions at Yahoo. He holds a B.S. in Computer Science from Caltech.