InfluentialPoints.com
Biology, images, analysis, design...
Use/Abuse Principles How To Related
"It has long been an axiom of mine that the little things are infinitely the most important" (Sherlock Holmes)

 

 

Covariance analysis ANCOVA: Use & misuse

(analysis of variance, regression, precision, homogeneity of slopes, linearity of responses)

Statistics courses, especially for biologists, assume formulae = understanding and teach how to do  statistics, but largely ignore what those procedures assume,  and how their results mislead when those assumptions are unreasonable. The resulting misuse is, shall we say, predictable...

Use and Misuse

Analysis of covariance (ANCOVA) combines the techniques of analysis of variance and regression by incorporating both nominal variables (factors) and continuous measurement variables (covariates) into a single model.

The primary use of covariance analysis is to increase precision in randomized experiments. A covariate X is measured on each experimental unit before treatment is applied. The eventual treatment means are then adjusted to remove the initial differences, thus reducing the experimental error and permitting a more precise comparison among treatments. Such adjustment is dependent on the slope of the relationship between the covariate and the response variable being the same for each treatment level. The other main use of ANCOVA is to model relationships especially where one wants to compare regression relationships at different levels of some treatment variable. In this use the aim is to assess which model is most appropriate to describe the data - whether separate slopes for each treatment level, a common slope but different intercepts or a common intercept but different slopes. This use clearly does not depend on the parallel slopes assumption - that is simply one of the model simplifications that can be made if justified - and because of this some authorities do not include such an analysis under the name 'analysis of covariance'. A third somewhat controversial use of covariance analysis is to adjust for sources of bias in observational studies.

Analysis of covariance is widely used (and misused) across many disciplines. Medical and veterinary researchers use it both to increase precision in randomized trials and to (attempt to) adjust for sources of bias in observational studies. Ecological and wildlife researchers also use covariance analysis for these purposes as well as to model relationships. We have attempted to cover all the main uses of analysis of covariance in the various examples.  We give two medical examples of randomized trials where precision of the comparison was increased by incorporating key covariates. Similarly ecological researchers used covariance analysis to increase precision of comparison of the effect of different levels of parasitism on consumptions of alkaloids by caterpillars. We also give a number of examples where covariance analysis was used to adjust for bias in observational studies - a more controversial use of the technique. There is no problem if distributions of the confounder can be assumed to be the same between treatments - an important assumption specific to analysis of covariance. But using covariance analysis to correct for bias when covariate distributions clearly do differ (for example previous milk production in a comparison of lame versus non-lame cows, or level of pruning in trees in different climatic zones) is fraught with dangers. The last use of covariance analysis - to model relationships - is uncontroversial and we give several ecological examples of this.

When it comes to the other assumptions of ANCOVA, homogeneity of slopes and linearity of responses are both extremely important if you are estimating adjusted means. Yet these assumptions appear to be seldom tested. All too often one reads that some variable is adjusted by covariance analysis (often for age), yet no checks have been made that there is any relationship between the response variable and the covariate, let alone a linear one. It does not matter if one 'knows' that in a large sample there will be a relationship - the adjustment will be done using the data at hand and if there happens to be no linear relationship the 'adjustment' may be grossly misleading. We also note the importance of linearity when ANCOVA is being used to compare regression slopes - apparent heterogeneity in slopes may result from non-linear relationships rather than true heterogeneity. One of the issues we noted for the various ANOVA models also recurs with covariance analysis - namely the reporting of interactions. In general reporting a significant main effect is not meaningful if there is a significant covariate factor interaction. If one wants to compare mean values when there is a significant interaction, then appropriate techniques must be used.

Two of the examples (where time was the response variable ) would probably have better analyzed using some form of survival analysis. This would deal both with the non-normal distribution of times and with the presence of censored observations. Other issues are ones that commonly rear their head(s) in many types of analysis. We give two examples (curlew numbers and reproductive capacity of African mahogany trees) where the sampling procedures would not have produced independent observations - yet analyses are made on that basis.

What the statisticians say

Huitema (1980) is the standard text on analysis of covariance. Other texts include Wildt & Ahtola (1978) Doncaster & Davey (2007) considers covariance analysis models for a variety of designs. Logan (2010) and Crawley (2007), (2005) covers analysis of covariance using R. Scheiner & Gurevitch (2001) cover analysis of covariance in Chapter 7. Maxwell et al. (1993) gives a widely quoted account of ANCOVA. Extensive treatment of covariance analysis is given by Underwood (1997) in Chapter 13, Sokal & Rohlf (1995) in Chapter 14. Winer et al (1991) and Neter et al (1985) provide a more mathematical treatment.

Frison & Pocock (2007) recommend using analysis of covariance of pre and post-treatment means, rather than analysis of post-treatment means or analysis of mean change. Van Breukelen (2006) argues that analysis of covariance versus change from baseline has more power in randomized studies but more bias in nonrandomized studies. Wright (2006) examines why the paired t test and covariance analysis can produce different results when comparing groups in a before-after design, the so-called Lord's paradox. Vickers (2005) argues in favour of covariance analysis rather than non-parametric tests for analyzing randomized trials with baseline and post-treatment measures. Pocock (2002) stresses that adjustment is essential if a covariate is strongly related to outcome. Miller & Chapman (2001) explain why for nonrandomized studies it is often not possible to correct for real group differences on a potential covariate. Senn (2001), Vickers & Altman (2001) and Vickers (2001) all emphasize that controlled trials with baseline and follow up measurements should be analyzed with analysis of covariance.

Ceyhan & Goad (2009) , Serrano et al. (2008), Garcia-Berthou (2004), and Darlington & Smulders (2001) and Freckleton (2002) all explain why analysis of covariance (or the equivalent general linear model or multiple regression) is a better approach for ecologists and animal behaviourists than using residuals from a regression or ratios. Liermann et al. (2004) put the case in favour of analyzing ratios whilst Beaupre & Dunham (1995), Horton & Redak (1993), and Raubenheimer & Simpson (1992) ) all argue that analysis of covariance is superior to ratio-based indices for nutritional data.

McCoy et al. (2006) notes that when the covariate is measured with error, which is presumably true in most studies of size correction, ANCOVA may produce biased estimates of effect sizes. However Chan (2004) argues that adjustment for baseline measurement error in randomized controlled trials induces bias whilst Jamieson (2005) notes that covariance analysis is biased for for comparing non-randomized groups using either the posttest score or the posttest minus pretest difference score because of measurement error. Federer & Meredith (1992) provide appropriate covariance analysis models for data from split- plot designs.

Hayes & Matthes (2009) and Bauer & Curran (2006) are the two most useful references for analysis of covariance for nonparallel slopes including the 'pick-a-point' approach and the Johnson-Neyman technique. See also Preacher et al. (2006), Fraas & Newman (2005), Kowalski et al. (1994) and Wilcox (1987). Vanderburgh et al. (1998) point out that nonparallel slopes may result from not using an appropriate transformation. Olejnik & Algina (1985) review nonparametric alternatives to analysis of covariance.

Wikipedia provides a rather brief section on analysis of covariance. The Handbook of biological statistics also gives an introduction to covariance analysis. Newcastle University provides an introduction to analysis of covariance for medical researchers using Minitab. Julian Faraway provides a useful guide to ANCOVA using R in Chapter 15. Andy Field (2008) provides his usual lively treatment of the topic for pschologists mainly using SPSS.