Offline Accuracy: A Potentially Misleading Metric in Myoelectric Pattern Recognition for Prosthetic Control

Paper in proceeding, 2015

Offline accuracy has been the preferred performance measure in myoelectric pattern recognition (MPR) for the prediction of motion volition. In this study, different metrics relating the fundamental binary prediction outcomes were analyzed. Our results indicate that global accuracy is biased by 1) the unbalanced number of possible true positive and negative outcomes, and 2) the almost perfect specificity and negative predicted value, which were consistently found across algorithms, topologies, and movements (individual and simultaneous). Therefore, class-specific accuracy is advisable instead. Additionally, we propose the use of precision (positive predictive value) and sensitivity (recall) as a complement to accuracy to better describe the discrimination capabilities of MPR algorithms, as these consider the effect of false predictions. However, all the studied offline metrics failed to predict real-time decoding, and therefore real-time testing continue to be necessary to truly evaluate the clinical usability of MPR.