In this talk, we share some of the most exciting achievements of Flink at Alibaba in recent years, including two main topics: one is the architecture evolution of stream-batch unification; the other is the recent efforts to improve high availability for streaming processing.

Flink has been greatly improved since its first appearance at Alibaba in 2016 and has become the de facto real-time computing standard. Last year for the first time, Flink’s unified stream-batch processing was officially used in Tmall’s big-screen marketing analysis during Double 11 — one of the Double 11’s core scenarios. We will share the architecture evolution beneath to help us achieve this. In addition, Flink never stops expanding its application spectrum to extreme real-time processing. We will discuss some of the efforts in this area as well.

Speakers:

Yuan Mei: Yuan Mei is currently the architect of Flink Engine at Alibaba, and she was a research scientist at Facebook before joining Alibaba. She was one of the main contributors of Turbine: Facebook’s Service Management Platform for Stream Processing (ICDE2020). She has various experiences building Stream Processing Systems (Puma & Stylus, VLDB2018) and many other data systems at Facebook (Presto). She holds a Ph.D. from MIT CSAIL, under the supervision of Prof. Samuel Madden & Prof. Michael Stonebraker.