REST-at: An LLM-Based Tool for Automating Traceability between Requirements and Test Cases

Paper i proceeding, 2026

Large Language Models (LLMs) are increasingly used across Software Engineering tasks, and their ability to process natural language makes them particularly well suited for aligning requirements and testing artifacts. In this study, we investigate AI-supported traceability in software development through the creation and evaluation of REST-at, a tool that facilitates alignment between requirements engineering and software testing (REST) by automatically generating trace links between requirements and manual system-level tests. To evaluate its effectiveness, we compare five LLMs, including proprietary models such as GPT-4 and open-weight alternatives like Mixtral 8x7B and LLAMA 3 70B. The results indicate that open-weight models can achieve comparable or even superior levels of precision and recall relative to larger proprietary models. Among them, LLAMA 3 70B consistently provided robust and reliable link generation between requirements and tests. We also observe a high sensitivity to prompt design, with model performance varying according to prompt structure. Based on these insights, we provide practical recommendations for researchers and practitioners concerning hardware demands, governance and transparency, and the importance of standardized documentation when applying open-weight models to software engineering artifacts.