6.3 The Behrens-Fisher problem (BFP)

Overview

The Behrens-Fisher problem (BFP) is one of the oldest puzzles in statistics ( Behrens (1929) ; Fisher (1935) ; Welch (1938) ). The essence of this problem is how validly to compare the means of two or more populations (groups) when their variances differ. It is clear how the assumption of common variance is built right in to the ANOVA $F$ statistic itself. For example, consider the one-way case, where the $F$ ratio is built using a single common estimate of the error variance (i.e., the residual mean square) as its denominator.

There are quite a few solutions to the Behrens-Fisher problem for univariate data (e.g., see Wang (1971) , Brown & Forsythe (1974) , Clinch & Keselman (1982) , Weerhandi (1993) and Ghosh & Kim (2001) ), yet all generally assume normality of errors.

Below we shall outline a solution to the univariate BFP proposed by Brown & Forsythe (1974) , as it points the way towards a more generalised solution to the BFP for multivariate data in dissimilarity-based analyses, using PERMANOVA.

The Brown & Forsythe (1974) solution to the BFP

Brown & Forsythe (1974) proposed a modification of the classical univariate $F$ ratio such that the means are weighted by $n_i / s_i^2$ (rather than being weighted only by $n_i$) and the denominator is chosen in order to ensure that numerator and denominator have the same expectation under a true null hypothesis, after this adjustment in the weights.

The resulting modified test-statistic is given by them as: $$ F_{\tiny{BF}} = \frac{ \sum_{i=1}^a n_i (\bar{y}_ {i \cdot} - \bar{y}_ {\cdot\cdot})^2 } { \sum_{i=1}^a (1 - n_i / N) s_i^2 } $$ Under the usual classical ANOVA assumptions, a p value can be obtained by comparing this modified test-statistic to an $F_0$ distribution having $(a-1)$ and $f$ degrees of freedom (defined implicitly by the Satterthwaite (1941) approximation), where: $$ f = \frac{1} { \sum_{i=1}^a c_i^2 / (n_i - 1)} $$ and $$ c_i = \frac { (1-n_i/N)s_i^2 } { \sum_{i = 1}^a (1 - n_i/N)s_i^2 } $$

Next, we shall see how a similar modification to the PERMANOVA pseudo F statistic can be constructed to allow heterogeneous dispersions in dissimilarity-based settings as well.