Deep learning model for end-to-end approximation of COSMIC functional size based on use-case names

Journal article, 2020

Context: COSMIC is a widely used functional size measurement (FSM) method that supports software development effort estimation. The FSM methods measure functional product size based on functional requirements. Unfortunately, when the description of the product's functionality is often abstract or incomplete, the size of the product can only be approximated since the object to be measured is not yet fully described. Also, the measurement performed by human-experts can be time-consuming, therefore, it is worth considering automating it. Objective: Our objective is to design a new prediction model capable of approximating COSMIC-size of use cases based only on their names that is easier to train and more accurate than existing techniques. Method: Several neural-network architectures are investigated to build a COSMIC size approximation model. The accuracy of models is evaluated in a simulation study on the dataset of 437 use cases from 27 software development projects in the Management Information Systems (MIS) domain. The accuracy of the models is compared with the Average Use-Case approximation (AUC), and two recently proposed two-step models—Average Use-Case Goal-aware Approximation (AUCG) and Bayesian Network Use-Case Goal AproxImatioN (BN-UCGAIN). Results: The best prediction accuracy was obtained for a convolutional neural network using a word-embedding model trained on Wikipedia+Gigaworld. The accuracy of the model outperformed the baseline AUC model by ca. 20%, and the two-step models by ca. 5–7%. In the worst case, the improvement in the prediction accuracy is visible after estimating 10 use cases. Conclusions: The proposed deep learning model can be used to automatically approximate COSMIC size of software applications for which the requirements are documented in the form of use cases (or at least in the form of use-case names). The advantage of the model is that it does not require collecting historical data other than COSMIC size and names of use cases.