Dual-layer prompt ensembles: Leveraging system- and user-level instructions for robust LLM-based query expansion and rank fusion

Artikel i vetenskaplig tidskrift, 2026

Large Language Models (LLMs) show strong potential for query expansion (QE), but their effectiveness is highly sensitive to prompt design. This paper investigates whether exploiting the system-user prompt distinction in chat-based LLMs improves QE, and how multiple expansions should be combined. We propose Dual-Layer Prompt Ensembles, which pair a behavioural system prompt with varied user prompts to generate diverse expansions, and aggregate their BM25-ranked lists using lightweight SU-RankFusion schemes. Experiments on six heterogeneous datasets show that dual-layer prompting consistently outperforms strong single-prompt baselines. For example, on Touche-2020 a dual-layer configuration improves nDCG@10 from 0.4177 (QE-CoT) to 0.4696, and SU-RankFusion further raises it to 0.4797. On Robust04 and DBPedia, SU-RankFusion improves nDCG@10 over BM25 by 24.7% and 25.5%, respectively, with similar gains on NFCorpus, FiQA, and TREC-COVID. These results demonstrate that system-user prompt ensembles are effective for QE, and that simple fusion transforms prompt-level diversity into stable retrieval improvements.