# Familywise error rate

Familywise error rate

In statistics, familywise error rate (FWER) is the probability of making one or more false discoveries, or type I errors among all the hypotheses when performing multiple pairwise tests [Shaffer J. P. Multiple Hypothesis Testing, Annual Review of Psychology, January 1995, Vol. 46, Pages 561-584 http://dx.doi.org/10.1146/annurev.ps.46.020195.003021] [Benjamini, Y., and Hochberg Y. (1995). "Controlling the false discovery rate: a practical and powerful approach to multiple testing". "Journal of the Royal Statistical Society. Series B (Methodological)" 57 (1), 289&ndash;300. [http://www.math.tau.ac.il/~ybenja/MyPapers/benjamini_hochberg1995.pdf] ] .

Classification of "m" hypothesis tests

The following table defines some random variables related to the m hypothesis tests.

The "m" specific hypotheses of interest are assumed to be known, but the number of true null hypotheses "m"0 and of alternative hypotheses "m"1, are unknown. "V" is the number of Type I errors (hypotheses declared significant when they are actually from the null distribution). "T" is the number of Type II errors (hypotheses declared not significant when they are actually from the alternative distribution). "R" is an observable random variable, while "S", "T" , "U", and "V" are unobservable random variables.

In terms of random variables,

:$mathrm\left\{FWER\right\} = Pr\left(V ge 1\right), ,$

or equivalently,

:$mathrm\left\{FWER\right\} = 1 -Pr\left(V = 0\right).$

What constitutes a family?

In confirmatory studies (i.e., where one specifies a finite number of a priori inferences), families of hypotheses are defined by which conclusions need to be jointly accurate or by which hypotheses are similar in content/purpose. As noted by Hochberg and Tamhane (1987), "If these inferences are unrelated in terms of their content or intended use (although they may be statistically dependent), then they should be treated separately and not jointly" (p. 6).

For example, one might conduct a randomized clinical trial for a new antidepressant drug using three groups: existing drug, new drug, and placebo. In such a design, one might be interested in whether depressive symptoms (measured, for example, by a Beck Depression Inventory score) decreased to a greater extent for those using the new drug compared to the old drug. Further, one might be interested in whether any side effects (e.g., hypersomnia, decreased sex drive, and dry mouth) were observed. In such a case, two families would likely be identified: 1) effect of drug on depressive symptoms, 2) occurrence of any side effects.

Thus, one would assign an acceptable Type I error rate, alpha, (usually .05) to each family and control for family-wise error using appropriate multiple comparison procedures. In the case of the first family, effect of antidepressant on depressive symptoms, pairwise comparisons among groups (here, there would be three possible comparisons) would be jointly controlled using techniques such as Tukey's Honestly Significant Difference (HSD) comparison procedure or a Bonferroni correction. In terms of the side effect profile, one would likely be interested in controlling for Type I error in terms of all side effects considered jointly so that decisions about the side effect profile would not be erroneously inflated by allowing each side effect and each pairwise comparison among groups to receive its own uncorrected alpha. By the Bonferroni inequality, allowing each side effect and comparison its own alpha would result in a Type I error of .05 * 3 side effects * 3 pairwise comparisons per side effect = 0.45 (i.e., 45% chance of making a Type I error). Thus, a more appropriate control for side effect family-wise error might divide alpha by three (.05/3 = .0167) and allocate .0167 to each side effect multiple comparison procedure. In the case of Tukey's HSD (a strong control multiple comparison procedure), one would determine the critical value of Q, the studentized range statistic, based on the alpha of .0167.

ee also

*Experimentwise error rate
*False discovery rate
*Holm-Bonferroni method
*Closed testing procedure

References

*Hochberg, Y., & Tamhane, A. C. (1987). "Multiple comparison procedures". New York: Wiley.

Wikimedia Foundation. 2010.

### Look at other dictionaries:

• Experimentwise error rate — In statistics, during multiple comparisons testing, experimentwise error rate is the probability of at least one false rejection of the null hypothesis over an entire experiment. The α (alpha) that is assigned applies to all of the hypothesis… …   Wikipedia

• False positive rate — When performing multiple comparisons in a statistical analysis, the false positive rate is the probability of falsely rejecting the null hypothesis for a particular test among all the tests performed. If the false positive rate is a constant α… …   Wikipedia

• False discovery rate — (FDR) control is a statistical method used in multiple hypothesis testing to correct for multiple comparisons. In a list of rejected hypotheses, FDR controls the expected proportion of incorrectly rejected null hypotheses (type I errors).… …   Wikipedia

• Multiple comparisons — In statistics, the multiple comparisons or multiple testing problem occurs when one considers a set of statistical inferences simultaneously.[1] Errors in inference, including confidence intervals that fail to include their corresponding… …   Wikipedia

• Closed testing procedure — In statistics, the closed testing procedure[1] is a general method for performing more than one hypothesis test simultaneously. Contents 1 The closed testing principle 2 Example 3 Special cases …   Wikipedia

• List of statistics topics — Please add any Wikipedia articles related to statistics that are not already on this list.The Related changes link in the margin of this page (below search) leads to a list of the most recent changes to the articles listed below. To see the most… …   Wikipedia

• Statistical significance — In statistics, a result is called statistically significant if it is unlikely to have occurred by chance. A statistically significant difference simply means there is statistical evidence that there is a difference; it does not mean the… …   Wikipedia

• List of mathematics articles (F) — NOTOC F F₄ F algebra F coalgebra F distribution F divergence Fσ set F space F test F theory F. and M. Riesz theorem F1 Score Faà di Bruno s formula Face (geometry) Face configuration Face diagonal Facet (mathematics) Facetting… …   Wikipedia

• Holm-Bonferroni method — In statistics, the Holm Bonferroni method [Holm, S (1979): A simple sequentially rejective multiple test procedure , Scandinavian Journal of Statistics , 6:65 70] performs more than one hypothesis test simultaneously. It is named after Sture Holm …   Wikipedia

• Bonferroni correction — In statistics, the Bonferroni correction states that if an experimenter is testing n dependent or independent hypotheses on a set of data, then the statistical significance level that should be used for each hypothesis separately is 1/ n times… …   Wikipedia