The Streaming computing engine for Tencent’s advertising business is gradually switching from Spark Streaming to Apache Flink. In this talk, we will cover our business scenarios in feature production, training samples, strategy calculation, and some of the challenges, experiences and lessons learned in Spark Streaming switching to Apache Flink. It includes some upgrade and optimization of Flink kernel under the scenario of massive data (40TB), so that it can meet the characteristics of abnormal fast recovery in the large state of 40TB, and stream as batch (not stream as one batch).

Speakers:

Liwei Lin: Tencent Technology (Beijing) Co. LTD, Tencent advertising feature production, sample data, strategic framework technology leader, Lin Liwei, head of Tencent advertising feature production, sample data, strategic framework technology, head of Tencent Advertising Spark, Flink; 10 years of big data experience, active contributor to Apache Spark/Apache Flink open Source community, author of Spark Streaming Source Code Analysis Series (Github Star 3K +).