Machine Learning Algorithms for Labeling: Where and How They are Used?

Paper i proceeding, 2022

With the increased availability of new and better computer processing units (CPUs) as well as graphical processing units (GPUs), the interest in statistical learning and deep learning algorithms for classification tasks has grown exponentially. These classification algorithms often require the presence of fully labeled instances during the training period for maximum classification accuracy. However, in industrial applications, data is commonly not fully labeled, which both reduces the prediction accuracy of the learning algorithms as well as increases the project cost to label the missing instances. The purpose of this paper is to survey the current state-of-the-art literature on machine learning algorithms that are used for assisted or automatic labeling and to understand where these are used. We performed a systematic mapping study and identified 52 primary studies relevant to our research. This paper provides three main contributions. First, we identify the existing machine learning algorithms for labeling and we present a taxonomy of these algorithms. Second, we identify the datasets that are used to evaluate the algorithms and we provide a mapping of the datasets based on the type of data and the application area. Third, we provide a process to support people in industry to optimally label their dataset. The results presented in this paper can be used by both researchers and practitioners aiming to improve the missing labels with the aid of machine algorithms or to select appropriate datasets to compare new state-of-the art algorithms in their respective application area.