Biology, images, analysis, design...
Use/Abuse Principles How To Related
"It has long been an axiom of mine that the little things are infinitely the most important" (Sherlock Holmes)

Search this site



(mixed and random effects, distribution of errors, confusion with one way ANOVA, erroneous pooling of lower error terms)

Statistics courses, especially for biologists, assume formulae = understanding and teach how to do  statistics, but largely ignore what those procedures assume,  and how their results mislead when those assumptions are unreasonable. The resulting misuse is, shall we say, predictable...

Use and Misuse

A common extension of one way ANOVA is to have additional nominal variables (factors) nested within the main factor of interest. By nested we mean that each level of the 'lower' nominal variable occurs in only one level of the 'higher' nominal variable. If the top level nominal variable (in this case treatment) is a fixed factor (for example treatment), and the lower level nominal variable is a random variable, then we are dealing with a mixed effects nested ANOVA. If the top level nominal variable is a random factor, and the lower level nominal variable is a random variable, then we have a random effects nested ANOVA. In nested ANOVA all lower level nominal variables are usually random factors. In a nested ANOVA you have several different error terms reflecting each level of the hierarchy. Parametric nested ANOVA assumes that the distribution of errors in each of the subgroups groups is normal and that effects are additive.

Nested ANOVA is widely used in many types of life sciences research especially in the fields of psychology, behaviour, genetics and ecology. It is less used in medical research and indeed most texts on medical statistics scarcely mention the topic. This is partly because ANOVA itself is less predominant (less use of continuous measurement variables) and partly because nested observations are wrongly analyzed as independent observations. We do give  a few medical examples but in two of these the factors were actually crossed and not nested - the authors considered the fixed factors as (random) nested factors to allow estimation of variance components.

One important misuse in ANOVA is to use ordinary one-way ANOVA when one should be using nested ANOVA. Good indicators of such misuse are very small P-values and/or very small error bars. In one veterinary example multiple faecal samples from individual donkeys were wrongly taken as independent replicates of management systems. In fact samples were nested within donkeys, donkeys were nested within owners, and owners were nested within management type. Even when nested ANOVA is used, researchers sometimes use the wrong error term to test the treatment effect. In one aquaculture experiment the 'within ponds' mean square was used rather than the (significant) 'ponds within treatment' mean square. Use of the correct denominator would have rendered all treatment effects non-significant. There is also the (erroneous) practice of pooling non-significant lower terms (advocated in several major texts) for which we have given several veterinary and wildlife examples.

It is rare indeed to find any evidence that the assumptions of nested ANOVA have been checked. This is true even when the variables under consideration are very unlikely to be normally distributed (for example time to pupation and counts of insects). In some examples quoted standard errors certainly suggested heterogeneity of variances. In one case the 'significant' differences  in means were probably due in part to differences in spread. Most studies wisely used equal numbers of replicates, but we give one example where a very unbalanced design was used with no acknowledgement of the likely inaccuracy of the quasi-F-ratios. The ever present issues of convenience  sampling and non-independence of sampling units recurs in several of the ecology and wildlife examples. The assumption in nested ANOVA is that (at least) at the uppermost level the sampling units are independent; similarly in experiments one has to assume random allocation of treatment to experimental units.


What the statisticians say

For the medical sciences Andersen (1990) covers the misuse of one-way ANOVA in situations where nested ANOVA would be more appropriate in his classical compilation of methodological errors in medical research. Extensive treatment of nested ANOVA using R is provided by Logan (2010) and Crawley (2007), (2005). Bailey (2008) consider nested ANOVA in Chapter 8 entitled 'small units inside large units'. Doncaster & Davey (2007) cover nested ANOVA in Chapter 2 whilst Quinn & Keough (2002) deal with the topic in Chapter 9. Older texts include Underwood (1997) in Chapter 9 and Sokal & Rohlf (1995) in Chapter 10.

In medical research German et al. (2008) stress the need for a heirarchical approach to analysis of electromyographic signals to avoid pseudoreplication. The same point is made by Palmer & Airey (2003) in relation to a genetics study on trait loci in mice. Patterson & Lello - note that nested ANOVA ok for orthogonal designs. Riley & Edwards (2008) stress the need to use the right unit of analysis for pond experiments in aquaculture research. Bliese & Hanges (2004) highlight the perils of treating grouped data as though they were independent in the field of organizational research. Kroodsma et al. (2001) advocate the use of nested ANOVA to avoid the problems of pseudoreplication in analysing the results of playback experiments. Ruohonen (1998) notes that in fish experiments nested models which take account of individual measurements perform better than those which use tank means.

Several authors have considered the issue of pooling in nested ANOVA. Hurlbert (2009) refers to the practice as 'the ancient black art of pseudoreplication'. Jenkins (2002) refutes the claim by Leger & Didrichsons (1998) that pooling does not increase the probability of type I error. Janky (2000) shows that pooling generally inflates Type I error and offers at best an insubstantial gain in power relative to the nominal test. Kromrey & Dickinson (1996) all stress the inadvisability of pooling in nested designs. Schwarz (1993) highlights the problems of the mixed model ANOVA especially for unbalanced designs.

NIST/SEMATECH e-Handbook of Statistics and the Handbook of biological statistics have sections on nested ANOVA. Alexander Kerr provides an excellent lecture on nested ANOVA in the marine biology context, albeit he (unjustifiably) pools mean squares. Other lectures on nested ANOVA are provided by Karl Broman, and the University of Southern Maine.