Empirical evaluation of an architectural technical debt index in the context of the Apache and ONAP ecosystems
Journal article, 2022

Background. Architectural Technical Debt (ATD) in a software-intensive system denotes architectural design choices which, while being suitable or even optimal when adopted, lower the maintainability and evolvability of the system in the long term, hindering future development activities. Despite the growing research interest in ATD, how to gain an informative and encompassing viewpoint of the ATD present in a software-intensive system is still an open problem. Objective. In this study, we evaluate ATDx, a data-driven approach providing an overview of the ATD present in a software-intensive system. The approach, based on the analysis of a software portfolio, calculates severity levels of architectural rule violations via a clustering algorithm, and aggregates results into different ATD dimensions. Method. To evaluate ATDx, we implement an instance of the approach based on SonarQube, and run the analysis on the Apache and ONAP ecosystems. The analysis results are then shared with the portfolio contributors, who are invited to participate in an online survey designed to evaluate the representativeness and actionability of the approach. Results. The survey results confirm the representativeness of the ATDx, in terms of both the ATDx analysis results and the used architectural technical debt dimensions. Results also showed the actionability of the approach, although to a lower extent when compared to the ATDx representativeness, with usage scenarios including refactoring, code review, communication, and ATD evolution analysis. Conclusions. With ATDx, we strive for the establishment of a sound, comprehensive, and intuitive architectural view of the ATD identifiable via source code analysis. The collected results are promising, and display both the representativeness and actionability of the approach. As future work, we plan to consolidate the approach via further empirical experimentation, by considering other development contexts (e.g., proprietary portfolios and other source code analysis tools), and enhancing the ATDx report capabilities.

Software architecture


Software metrics

Empirical evaluation

Software portfolio analysis

Technical debt


Roberto Verdecchia

Vrije Universiteit Amsterdam

Ivano Malavolta

Vrije Universiteit Amsterdam

Patricia Lago

Vrije Universiteit Amsterdam

Chalmers, Computer Science and Engineering (Chalmers), Interaction Design and Software Engineering

Ipek Ozkaya

Carnegie Mellon University (CMU)

PeerJ Computer Science

23765992 (eISSN)

Vol. 8 e833

Subject Categories

Software Engineering

Information Science

Information Systemes, Social aspects



More information

Latest update

3/8/2022 2