2.5 $P$-values by permutation

The other hurdle that must be cleared is to recognise that, in line with the philosophy of all of the routines in the PERMANOVA+ add-on, we have no particular reason to assume that the distribution of the z’s will be necessarily normal. Yet, to use tabled P-values for Fisher’s F distribution requires this assumption to be made. Of course, we wish to cater for any situation where the z’s may be non-normal. This can be achieved by using a permutation procedure.

Note that, unlike the PERMANOVA one-way analysis, it does not make sense to simply permute the labels randomly among the groups to test the null hypothesis of homogeneity. This is because such permutations will cause potential differences in location among the groups to suddenly be included as part of the “within-group” measures. However, differences in location do not form part of the null hypothesis and so it is not logical to be mixing samples together from groups having different locations as possible alternative outcomes if the null hypothesis were true. Before proceeding, the observations from different groups must therefore be centered onto a common group centroid. In other words, it is the residuals obtained after removing any location differences that are actually exchangeable under the null hypothesis of homogeneity of dispersions. Permutation of residuals ( Freedman & Lane (1983) ) is an approach that has been demonstrated to have good asymptotic and empirical properties ( Anderson & Legendre (1999) , Anderson & Robinson (2001) ). PERMDISP uses permutation of residuals (i.e., permutation of samples among groups after centering all groups onto a common location) in order to generate P-values for the test.