What Is the Bonferroni Test?

A Bonferroni test is a type of multiple comparison test used in statistical analysis. When performing a number of hypothesis tests with multiple comparisons, eventually, a result could occur that shows the statistical significance of the dependent variable, even if there is none.

If a particular test yields correct results 99% of the time, running 100 tests could lead to a false result somewhere in the mix. The Bonferroni test attempts to prevent data from incorrectly appearing to be statistically significant by making an adjustment during comparison testing.

The Bonferroni test, also known as the "Bonferroni correction" or "Bonferroni adjustment" suggests that the "p" value for each test must be equal to alpha divided by the number of tests.

Key Takeaways

  • A Bonferroni test is a type of multiple comparison test used in statistical analysis.
  • During hypothesis testing with multiple comparisons, errors or false positives can occur.
  • Bonferroni designed a test or an adjustment to prevent data from incorrectly appearing to be statistically significant.

Understanding the Bonferroni Test

The Bonferroni test is named for the Italian mathematician who developed it, Carlo Emilio Bonferroni (1892–1960). Other types of multiple comparison tests include Scheffe's test and the Tukey-Kramer method test. A criticism of the Bonferroni test is that it is too conservative and may fail to catch some significant findings.

In statistics, a null hypothesis is essentially the belief that there's no statistical difference between two data sets being compared. Hypothesis testing involves testing a statistical sample to confirm or reject a null hypothesis. The test is performed by taking a random sample of a population or group. While the null hypothesis is tested, the alternative hypothesis is also tested, whereby the two results mutually exclusive. 

However, with any testing of a null hypothesis, there's the expectation that a false-positive result could occur. This error is called a Type-1 error, and as a result, an error rate is assigned to the test. In other words, a certain percentage of the results will likely yield an error.

For example, an error rate of 5% might typically be assigned to a test, meaning that 5% of the time, there'll be a false positive. The 5% error rate is called the alpha level. However, when many comparisons are being made in a test, the error rate for each comparison can impact the results, creating multiple false positives.

Bonferroni designed a method of correcting for the increased error rates in hypothesis testing that had multiple comparisons. Bonferroni's adjustment is calculated by taking the number of tests and dividing it into the alpha value. Using the 5% error rate from our example, two tests would yield an error rate of 0.025 or (.05/2) while four tests would have an error rate of .0125 or (.05/4).