China Unicom's large-scale real-time computing production practice based on Apache StreamPark

穆纯进

Chinese Session 2023-08-18 14:30 GMT+8  #streaming
  1. The big data real-time computing platform supports event-based low-latency processing and stream-and-batch data processing, supports the real-time business of 30+ internal and external organizations and 10,000 + data service subscriptions, processes 2.3 trillion pieces of data and 600TB+ data volume every day, has 480+ servers in the cluster, and serves more than a dozen business production lines
  2. A one-stop management platform for real-time computing jobs based ON Apache StreamPark, which supports the production environment 500+Flink ON YARN real-time computing job management. Through the visual and concise operation process, the management functions such as project management, job management, team management, permission management, alarm management, log management, version management, cluster management, resource configuration, Flink JAR, Flink SQL, and monitoring large screen are completed, and the real-time job life cycle management is realized. Help the team to solve the operation and maintenance mire, improve management efficiency, reduce the failure rate, improve the quality of business support, and fully realize the integration of real-time computing and platform management

Speakers:


Mu Chunjin: Unicom Digital Technology Co. LTD, Head of big data real-time computing platform research and development, Head of Apache StreamPark PMC, Big Data real-time computing platform R&D, responsible for trillion-level Flink real-time computing development, operation and maintenance and platform construction