The large sample size fallacy
Journal article, 2013
Background: Significance in the statistical sense has little
to do with significance in the common practical sense.
Statistical significance is a necessary but not a sufficient
condition for practical significance. Hence, results that are extremely statistically significant may be highly nonsignificant in practice. The degree of practical significance is generally determined by the size of the observed effect, not the p-value. The results of studies based on large samples are often characterized by extreme statistical significance despite small or even trivial effect sizes. Interpreting such results as significant in practice without further analysis is referred to as the large sample size fallacy in this article.
Aim: The aim of this article is to explore the relevance of the large sample size fallacy in contemporary nursing research.
Results: Relatively few nursing articles display explicit
measures of observed effect sizes or include a qualitative
discussion of observed effect sizes. Statistical significance is often treated as an end in itself.
Conclusion: Effect sizes should generally be calculated and
presented along with p-values for statistically significant
results, and observed effect sizes should be discussed
qualitatively through direct and explicit comparisons with
the effects in related literature.