1. The backgrand (background) With the development and deepening of business, more and more data users, from the original senior management, analysts to front-line business personnel, how to quickly obtain data analysis results is more and more important for business development. Simplified data processing/data analysis is critical for front-line business analysts. Optimized data processing tools/processes help non-technical users capture and automate the embedding of business analysis data into the data processing process, and simplify the cost of using data through configuration.
2. The purpose (goal) Let business users almost no code to achieve part of the data processing logic.
3. The method (method) The existing tools and tool combinations should be properly encapsulated, only the logic processing related to the core process should be reserved, and the processing process related to other technologies should be preset by the system to simplify the user operation process. Encapsulation is based on the existing scheduling system, and the environment, data source management and task scheduling functions provided by the system scheduling system are reasonably integrated.
4. Design and implementation In terms of data processing, among the three steps of data collection, processing and distribution, processing is the most complex, but the collection and distribution can be simplified and optimized by configuration processing. Reasonable data model design, processing the data model to a reasonable data granularity, and then processing the follow-up content of dimension modeling through automatic coding tools, so as to simplify the data processing process, simplify the data model to report forms, visual analysis of the data gap. In this process, especially from model to data application layer, users are involved, so workflow system should reasonably support the production of configuration jobs, and automatically embed into the existing workflow for daily scheduling execution.
5. Benfit and Impaction Data processing personnel pay more attention to the core processes and models of data processing. After completion, they interact model data with users with the assistance of modeling tools. Users can realize self-service analysis and data distribution on the basis of model data.
6. Question (answer)

Speakers:

Wei Cheng: Sichuan Xinwang Bank, Big data architect, 11 years of work experience, engaged in the construction of data warehouse in the early stage, then shifted to the construction of big data basic platform and scheduling system, etc., with experience in the construction of traditional financial industry, Internet data warehouse and data mart, years of scheduling system construction experience, migu Culture Analysis cloud scheduling system design, report platform design, etc. Currently, I am mainly responsible for the system construction of DataOps system of Xinwang Bank (offline development, indicator system, label system).