Apache Druid real-time ingestion challenges and best practices

Tijo Thomas

English Session 2021-08-08 16:50 GMT+8  #streaming

Modern businesses make real-time data-driven decisions with Apache Druid. One of the key challenges is to design a reliable ingestion pipeline. Some of these challenges are primarily because the nature of the queries varies from customer to customer and use case to use case.

In this talk, we will cover some of the best practices in setting up real-time ingestion Apache Druid with Apache Kafka. We will also discuss some of the advanced tips on optimizing real-time ingestion, query performance and reliability.


Tijo Thomas: Tijo Thomas is a Sr. Solutions Architect at Imply and an experienced Data Engineer. He has over 18 years of experience in software development, mostly in big data and streaming technologies. He has been helping customers in setting up their stream processing infrastructures using Apache Druid over the last couple of years. During this time he has collected best practices, patterns and anti-patterns applied in production environments.