Non-Functional Requirements for Machine Learning: An Exploration of System Scope and Interest

Paper in proceeding, 2022

Systems that rely on Machine Learning (ML systems) have differing demands on quality—non-functional requirements (NFRs)— compared to traditional systems. NFRs for ML systems may differ in their definition, scope, and importance. Despite the importance of NFRs for ML systems, our understanding of their definitions and scope—and of the extent of existing research—is lacking compared to our understanding in traditional domains.Building on an investigation into importance and treatment of ML system NFRs in industry, we make three contributions towards narrowing this gap: (1) we present clusters of ML system NFRs based on shared characteristics, (2) we use Scopus search results— as well as inter-coder reliability on a sample of NFRs—to estimate the number of relevant studies on a subset of the NFRs, and (3), we use our initial reading of titles and abstracts in each sample to define the scope of NFRs over parts of the system (e.g., training data, ML model). These initial findings form the groundwork for future research in this emerging domain.