Tukey's test

Tukey's test, named after John Tukey, is a statistical test generally used in conjunction with an ANOVA to find which means are significantly different from one another. It compares all possible pairs of means, and is based on a "studentized range" distribution "q" (this distribution is similar to the distribution of "t" from the t-test).Linton, L.R., Harder, L.D. (2007) Biology 315 - Quantitative Biology Lecture Notes. University of Calgary, Calgary, AB]

The test compares the means of every treatment to the means of every other treatment, and identifies where the difference between two means is greater than the standard error would be expected to allow.

Assumptions of Tukey's test

#The observations being tested are independent
#The means are from normally distributed populations

The test statistic

Tukey's test is based on a formula very similar to that of the t-test. In fact, Tukey's test is essentially a t-test, except that it corrects for experiment-wise error rate (when there are multiple comparisons being made, the probability of making a type I error increases - Tukey's test corrects for that, and is thus more suitable for multiple comparisons than doing a number of t-tests would be).

The formula for Tukey's test is:

:$q\_s\; =\; frac\{Y\_A\; -\; Y\_B\}\{SE\},$

where "Y"_A is the larger of the two means being compared, "Y"_B is the smaller of the two means being compared, and SE is the standard error of the data in question.

This q_s value can then be compared to a q value from the "studentized range" distribution. If the q_s value is "larger" than the q_critical value obtained from the distribution, the two means are said to be significantly different.

Since the null hypothesis for Tukey's test states that all means being compared are from the same population (ie. μ₁ = μ₂ = μ₃ = ... = μ_"n"), the means should be normally distributed (according to the central limit theorem). This gives rise to the normality assumption of Tukey's test.

The "q"-distribution

Tukey's test is based on the comparison of two samples from the same population. From the first sample, the range (calculated by subtracting the smallest observation from the largest, or $scriptstyle\; ext\{range\},\; =,\; max\_i(Y\_i),\; -,\; min\_i(Y\_i)$, where "Y"_"i" represents all of the observations) is calculated, and from the second sample, the standard deviation is calculated. The "studentized range" ratio is then calculated:

: $q\; =\; frac\{\; ext\{range\{s\},$

where "q" = "studentized range", and "s" = standard deviation of the second sample.

This value of "q" is the basis of the critical value of "q", based on three factors:
#α (the Type I error rate, or the probability of rejecting a true null hypothesis)
#"n" (the number of degrees of freedom in the first sample (the one from which range was calculated))
#"v" (the number of degrees of freedom in the second sample (the one from which "s" was calculated))

Order of comparisons

If there are a set of means ("A", "B", "C", "D"), which can be ranked in the order "A" > "B" > "C" > "D", not all possible comparisons need be tested using Tukey's test. To avoid redundancy, one starts by comparing the largest mean ("A") with the smallest mean ("D"). If the "q"_"s" value for the comparison of means "A" and "D" is less than the "q" value from the distribution, the null hypothesis is accepted, and the means are said have no statistically significant difference between them. Since there is no difference between the two means that have the largest difference, comparing any two means that have a smaller difference is futile. As a result, no other comparisons need to be made.

Overall, it is important when doing a Tukey's test to always start by comparing the largest mean to the smallest mean, and then the largest mean with the next smallest, etc., until the largest mean has been compared to all other means (or until no difference is found). After this, compare the second largest mean with the smallest mean, and then the next smallest, and so on. Once again, if two means are found to have no statistically significant difference, do "not" compare any of the means between them.

References