Biology, images, analysis, design...
Use/Abuse Principles How To Related
"It has long been an axiom of mine that the little things are infinitely the most important" (Sherlock Holmes)




Efficiency of Tests

You can compare the power of different statistical tests in terms of the relative number of observations each needs to obtain a given power for a given set of data. This comparison assumes the test is applicable for that type of data. For instance, you cannot compare parametric and non-parametric tests for analyzing categorical data.

Efficiency is of importance when there are more than one way of measuring a treatment effect. For instance, you could use either the difference between means or the difference between medians to estimate a treatment effect. The difference between medians has the advantage of being less affected by non-normal data, but has less power. The reason for this is simply because a median is the location of the mean ranked value - and transforming continuous measurements to their ranks loses information. As a result the median is a more variable (less precise) estimator of its population median.

A simple way to compare precision is to set up a population of observations, from which repeated samples are taken, and the alternate statistics calculated - from which the standard deviation of each statistic can then be compared. To compare power, a treatment effect may have to be specified, for which the power of each test can be found from its estimated distribution - and the least powerful test is expressed as a percentage of the more powerful one.

To obtain a less relative comparison, the power of each test is compared against that of the least variable statistic - and expressed as a percentage. To make this more generalised, and to take advantage of central limit effects, this comparison is commonly done for samples whose size approaches infinity - and is known as the asymptotic relative efficiency (ARE).

The efficiency of a test is estimated by varying the number of observations to obtain an ARE of 100%, then expressing the efficiency as the ratio of these two sample sizes.