Secondary development and practice of Apache DolphinScheduler scheduling in Unicom big data

刘武

Chinese Session 2022-07-29 14:50 GMT+8  #workflowdatagovernance

Based on the secondary development of version 1.3.2, 80+ servers have been deployed in the company for operation and maintenance, and the functions of integration and interaction with other components have been expanded, such as background multi-machine, version upgrade, online batch management, etc

China Unicom’s big data is developed on the basis of Apache Dolphinscheduler 1.3.2, and more than 80 servers are deployed in the company for operation and maintenance

  1. Contributed many functions to the community, including the extension of global variables, the definition of input and output parameters of workflow components, the definition of workgroups and other functions are reflected in the 2.0.x version 2, because the deployment of more servers to expand the background multi-machine, master and worker version upgrade, online and shutdown services, batch management and other functions
  2. Added the execution of retained tasks on deployed servers
  3. Optimize the overall execution performance, etc

Speakers:


Wu Liu: Unicom Digital Technology Co. LTD, Data Development Engineer, Unicom Digital Technology Co., LTD. Participated in the integration and secondary development of DolphinScheduler scheduling in the company