Statistical Models for the Analysis of Optimization Algorithms with Benchmark Functions
Artikel i vetenskaplig tidskrift, 2021

Frequentist statistical methods, such as hypothesis testing, are standard practices in studies that provide benchmark comparisons. Unfortunately, these methods have often been misused, e.g., without testing for their statistical test assumptions or without controlling for family-wise errors in multiple group comparisons, among several other problems. Bayesian Data Analysis (BDA) addresses many of the previously mentioned shortcomings but its use is not widely spread in the analysis of empirical data in the evolutionary computing community. This paper provides three main contributions. First, we motivate the need for utilizing Bayesian data analysis and provide an overview of this topic. Second, we discuss the practical aspects of BDA to ensure that our models are valid and the results are transparent. Finally, we provide five statistical models that can be used to answer multiple research questions. The online appendix provides a step-by-step guide on how to perform the analysis of the models discussed in this paper, including the code for the statistical models, the data transformations, and the discussed tables and figures.

Benchmark Comparison

Black-Box Optimization

Bayesian Data Analysis

Statistical models


David Issa Mattos

Chalmers, Data- och informationsteknik, Software Engineering

Jan Bosch

Chalmers, Data- och informationsteknik, Software Engineering

Helena Holmström Olsson

Malmö universitet

IEEE Transactions on Evolutionary Computation

1089-778X (ISSN) 19410026 (eISSN)

Vol. 25 6 1163-1177


Annan data- och informationsvetenskap

Bioinformatik (beräkningsbiologi)

Sannolikhetsteori och statistik



Mer information

Senast uppdaterat