Bonferroni Correction: A Practical Guide

When a researcher conducts several statistical tests on the same data set, the chance of obtaining at least one significant result purely by chance increases. The Bonferroni correction is one of the simplest and most widely used methods for controlling this inflation of falsepositive findings, known as the familywise error rate (FWER).

Why Adjust for Multiple Comparisons?

Assume each test is performed at a conventional significance level of = 0.05. If ten independent null hypotheses are tested, the probability that at least one test produces a falsepositive result is

That 40% chance is far larger than the nominal 5% error rate. Adjusting the threshold for each test keeps the overall risk of any false positive at the desired level.

How the Bonferroni Method Works

Suppose a set of m hypotheses is examined. The classic Bonferroni rule replaces the original significance level with a stricter pertest level:

Each individual pvalue is compared with / m. Equivalently, you may multiply each pvalue by m and compare the result with the original . Both approaches give the same decision.

StepbyStep Example

1. Choose an overall error rate, e.g., = 0.05.
2. Count the number of tests. In this example m = 4.
3. Compute the Bonferroni threshold: 0.05 / 4 = 0.0125.
4. Compare each pvalue with 0.0125.

Only tests 1 and 3 survive the correction.

Assumptions and Limitations

• Independence (or weak dependence) The bound is exact when tests are independent. With strong positive correlation the correction can be overly conservative.
• Uniformity under the null The pvalues must be valid (uniformly distributed) when the null hypothesis holds.
• Conservativeness Because the adjustment ignores any correlation structure, power can be substantially reduced, especially when m is large.

When to Use Bonferroni

The method shines in situations with a modest number of tests and where controlling any false positive is critical, such as:

• Clinical trials with a few prespecified outcomes.
• Genetic association studies focusing on a small set of candidate variants.
• Psychological experiments testing multiple related hypotheses.

For highdimensional data (e.g., genomewide scans, brain imaging voxel analyses) researchers often turn to less stringent procedures like the false discovery rate (FDR) control.

Bonferroni vs. Other Adjustments

Method	Control	Typical Power	When Preferred
Bonferroni	FWER	Low (conservative)	Few tests, strict error control
HolmBonferroni	FWER (stepdown)	Slightly higher	Any number of tests, still FWER
idk	FWER (assuming independence)	Similar to Bonferroni	Independent tests, exactness desired
BenjaminiHochberg	FDR	Higher	Largescale testing, tolerate some false positives

Implementation in Popular Software

R

p.adjust(pvalues, method = "bonferroni")

Python (statsmodels)

from statsmodels.stats.multitest import multipletestsreject, pvals_corrected, _, _ = multipletests(pvals, alpha=0.05, method='bonferroni')

SPSS Use the Bonferroni option under Multiple Comparisons.

Practical Tips

• Plan the correction before looking at the data to avoid phacking.
• Report both raw and adjusted pvalues so readers can assess the impact.
• If the number of tests is very large, consider a hierarchical approach: prefilter hypotheses, then apply Bonferroni on the reduced set.
• Document the rationale for the chosen correction method in the methods section.

Summary

The Bonferroni correction offers a transparent, easytoapply way to protect against inflated type I error when several hypotheses are tested simultaneously. By dividing the desired overall by the number of comparisons, it guarantees that the probability of any false positive does not exceed the prespecified level. While its simplicity is appealing, the method can be overly conservative, especially with many or dependent tests. Researchers should weigh the tradeoff between strict error control and statistical power, and consider alternative procedures when appropriate.

References: Bonferroni, C. (1936). Teoria statistica delle classi e delle prove. Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics. Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate. Journal of the Royal Statistical Society.