Maturity Assessment Model for Industrial Data Pipelines
Paper i proceeding, 2023

Data pipelines can be defined as a complex chain of interconnected activities that starts with a data source and ends in a data sink. They can process data in multiple formats from various data sources with minimal human intervention, speed up data life cycle operations, and enhance productivity in data-driven organizations. As a result, companies place a high value on strengthening the maturity of their data pipelines. The available literature, on the other hand, is significantly insufficient in terms of providing a comprehensive roadmap to guide companies in assessing the maturity of their data pipelines. Therefore, this case study focuses on developing a data pipeline maturity assessment model that can evaluate the maturity of data pipelines in a staged manner from maturity level 1 to maturity level 5. We conducted empirical research in order to develop the maturity assessment model on the basis of five different determinants to address the specific needs of each data pipeline maturity level. Accordingly, it aims to support organizations in assessing their current data pipeline maturity, determining challenges at each stage, and preparing an extensive roadmap and suggestions for data pipeline maturity improvement. In future work, we plan to employ the maturity model in different companies as a case study to evaluate its applicability and usefulness.

roadblocks

challenges

data pipeline maturity improvement

benefits

recommendations

factors

roadmap

Maturity assessment

Data Pipelines

determinants

Författare

Aiswarya Raj Munappy

Software Engineering 1

Jan Bosch

Software Engineering 1

Helena Holmström Olsson

Malmö universitet

Proceedings - Asia-Pacific Software Engineering Conference, APSEC

15301362 (ISSN)

Vol. 30th Asia-Pacific Software Engineering Conference, APSEC 2023 503-513
9798350344172 (ISBN)

30th Asia-Pacific Software Engineering Conference, APSEC 2023
Seoul, South Korea,

Ämneskategorier

Programvaruteknik

DOI

10.1109/APSEC60848.2023.00062

Mer information

Senast uppdaterat

2024-04-23