Biology, images, analysis, design...
|"It has long been an axiom of mine that the little things are infinitely the most important" |
Binomial and related testsOn this page: Binomial test Sign test McNemar's test, for significance of change, for matched pairs Cox & Stuart test for trend
This More Information page describes several closely-related tests, namely the binomial test, the sign test, McNemar's test and the Cox & Stuart test for trend. All these tests are essentially variants of the binomial test, although they are used for different types of data.
The binomial test is a one-sample test used to assess whether an observed proportion derived from a single random sample differs from an expected parametric proportion. It can be used to assess outcomes of encounters in behavioural studies. For example to assess whether the probability that a male already resident in a territory wins a fight with a non-resident male can be accepted as 0.5 or not. Another example comes from choice experiments. Odour may be presented in one arm of an olfactometer and not in the other. The probability of an insect choosing one arm or the other can be tested against a null hypothesis of P = 0.5 to establish whether the odour is attractant, repellent or has no
To carry out the large sample version of the test (n > 25), we first calculate the difference between our sample proportion and the parametric value. We then convert this difference to a standard normal deviate by dividing it by the estimated standard error of the proportion under your null hypothesis. The standard error of a proportion is given from the binomial distribution as √PQ/n, where P is the true proportion, Q is 1−P and n is the number of observations. Thus
The z-test assumes that the distribution of the estimate is continuous. However the estimate of a proportion can only take a finite number of values depending on the sample size, and so the distribution is discrete. Some statisticians argue that you should therefore use a continuity
For small samples you need to work out probabilities directly using the mass probability function for the binomial
The assumptions of the binomial test are:
The Sign test
The sign test is used for paired data where quantitative measurements are not possible, but where it is possible to rank each member of a pair (or the same individual before and after a treatment) for some characteristic with respect to each other. In other words measurement is only possible on the ordinal scale. It also has an important use for testing paired data where quantitative measurements are possible, but where the distribution of differences is neither normal (where you could use the paired t-test) nor symmetrical (where you could use Wilcoxon's matched pairs signed ranks test).
We might for example compare preliminary results on species richness between two types of habitat. We do not feel we have sufficient data to precisely quantify species richness in each habitat type - but we do have sufficient data to say which habitat has the greater species richness. Those cases where habitat type 1 has a greater species richness than habitat type 2 are classified as +, those where the converse is true as −, and those where there is no difference are dropped from the analysis.
The sign test tests the null hypothesis (H0) that the number of pairs where habitat type 1 has a greater species richness than habitat type 2 is the same as the number of pairs where the converse is true. In other words the null hypothesis is that the probability of each is equal to 0.5. The sign test is thus equivalent to the binomial test where the parametric proportion is equal to 0.5.
Again for small samples you need to work out probabilities directly using the mass probability function for the binomial distribution. The probabilities of the observed result and of results more extreme are summed to give the required probability. The large sample test is given below:
Again you may wish to apply a continuity correction to obtain conventional P-values:
The assumptions of the sign test are:
McNemar's test (for significance of change or for matched pairs)
McNemar's test is essentially the sign test under another name but applied when the response variable is binary. It is therefore used to compare paired proportions. This arises in several types of study designs such as 'before-and-after' studies and matched cohort or case-control designs. Say we are carrying out a 'before-and-after' study on 10 individuals which gives the following result:
Such data are often wrongly presented in a 2 by 2 contingency table thus:
Such a presentation looses the information on the individual pairs. Instead data from matched pairs studies should be presented so you can see the number of concordant pairs (each with the same characteristics) and the number of discordant pairs (each with different characteristics). To distinguish these from unpaired data we use e,f,g,h rather than a,b,c,d
McNemar's test is also used for observational studies where each case is matched to one control. Here a 2 × 2 table is used to display whether each member of each pair is exposed or not to the risk factor.
The same simplified version of the binomial test formula for P=0.5 is used as in the sign test:
Again we can apply a continuity correction if desired, namely:
We have used the standardised normal deviate as the test statistic for McNemar's test to demonstrate that it is derived from the sign test. However, you will also come across the test where X2 is used as the test statistic and tested against χ2 with one degree of freedom. For this we simply square the expression used to obtain 'z'.
Again for small samples you need to work out probabilities directly using the mass probability function for the binomial distribution. The probabilities of the observed result and of results more extreme are summed to give the required probability.
The significance test is based entirely on the number of changes and does not take account of total sample size (e+f+g+h). However, to attach a confidence interval to the magnitude of the change, we must estimate the standard error to the difference between the proportions - and this does take account of total sample size:
To apply a continuity correction add 1/n to that standard error.
This method is adequate for large samples, but otherwise performs poorly. There is a conventional conditional exact conditional interval, but it yields an interval which is prone to artefacts and has poor coverage. Exact unconditional intervals, profile likelihood intervals, and score intervals have better properties but are seldom used. See Newcombe
The assumptions of the McNemar test are:
Cox and Stuart test for trend
An interesting, if little used, application of the sign test is to test for trend. Under certain circumstances it can be used to determine if there is a trend in observations on a sequence of random variables.
One possible application would be to test the trend in mean maximum temperature over a period of say 100 years. The data are divided up into pairs (Y1, Y1 + c), (Y2, Y2 + c) etc where c = n/2 if n is even, and c = (n+1)/2 if n is odd. For example, for c=100/2=50 years, the year 1 observation would be paired with the year 51 observation, year 2 observation would be paired with the year 52 observation and so on. The middle observation is eliminated if there is an odd number of data points. If the reading is greater for the second observation in the pair, it is denoted as a '+'; if less it is a '−'. Ties are eliminated from the data.
The formula used is the same as for the sign test. The upper-tailed test is used to detect an upwards trend; the lower-tailed test to detect a downwards trend. A two-tailed test if the alternative hypothesis is that there is any type of trend. Assumptions are as follows: