InfluentialPoints.com
Biology, images, analysis, design...
 Use/Abuse Principles How To Related
"It has long been an axiom of mine that the little things are infinitely the most important" (Sherlock Holmes)

# Efficiency of Tests

You can compare the power of different statistical tests in terms of the relative number of observations each needs to obtain a given power for a given set of data. This comparison assumes the test is applicable for that type of data. For instance, you cannot compare parametric and non-parametric tests for analyzing categorical data.

Efficiency is of importance when there are more than one way of measuring a treatment effect. For instance, you could use either the difference between means or the difference between medians to estimate a treatment effect. The difference between medians has the advantage of being less affected by non-normal data, but has less power. The reason for this is simply because a median is the location of the mean ranked value - and transforming continuous measurements to their ranks loses information. As a result the median is a more variable (less precise) estimator of its population median.

A simple way to compare precision is to set up a population of observations, from which repeated samples are taken, and the alternate statistics calculated - from which the standard deviation of each statistic can then be compared. To compare power, a treatment effect may have to be specified, for which the power of each test can be found from its estimated distribution - and the least powerful test is expressed as a percentage of the more powerful one.

To obtain a less relative comparison, the power of each test is compared against that of the least variable statistic - and expressed as a percentage. To make this more generalised, and to take advantage of central limit effects, this comparison is commonly done for samples whose size approaches infinity - and is known as the asymptotic relative efficiency (ARE).

The efficiency of a test is estimated by varying the number of observations to obtain an ARE of 100%, then expressing the efficiency as the ratio of these two sample sizes.