In this presentation, Dr. Yun Gao and Dr. Zhipeng Zhang will introduce the recent development of Apache Flink Machine learning library (Flink ML), as well as the development vision of Flink ML.

We have designed an algorithm interface that natively supports real-time machine learning, making it easier for algorithm users to configure, combine, and deploy online prediction and learning algorithms. The algorithm interface can support multiple input and multiple output, and the algorithm modules can be combined in the way of directed graph. We designed and implemented an iteration engine based on DataStream to replace the iteration engine based on DataSet. To meet the needs of various algorithms, we design iterative engine interfaces that are easier to use and provide algorithm developers with richer interface choices to optimize algorithm performance. Based on this, we implemented 15+ efficient and easy-to-use offline/online machine learning algorithms.

According to the newly designed algorithm interface and iteration engine, we plan to transform the Alink algorithm library developed by Ali Cloud for many years and contribute to Flink ML. By combining The strong community ecosystem of Apache Flink, the technologically leading Alink library, and the newly designed algorithmic interface, we hope to complement the strengths of Flink ML to help make it the easiest to use, most algorithmic, and most widely used stream batch integrated machine learning library.

Speakers:

Yun Gao: alibaba, Technical experts, Alibaba Technical Specialist, Apache Flink PMC/ Committer

Dr. Yun Gao graduated from University of Chinese Academy of Sciences and joined Alibaba real-time Computing team, mainly engaged in the development and improvement of Flink Runtime/DataStream.

Zhipeng Zhang: alibaba, Senior Algorithm Engineer, Apache Flink committer.

Dr. Zhipeng Zhang graduated from Peking University. His research interests are distributed machine learning systems/algorithms. After graduation, I joined alibaba machine Learning team, mainly engaged in flink-related machine learning development and improvement.