Models and study designs
As with Pearson's chi square test, the purpose of Fisher's exact test is to determine if there is a significant difference between two proportions or to test association between two characteristics. However, Fisher's exact test assumes a quite different model. As before the frequencies in each category are arranged in a 2x2 contingency table. But in this case both row and column totals are assumed to be fixed - not random.
We will take an example to illustrate the model:
|Captive-reared|| a|| b||a+b
|Totals||a+c||b + d||N = a+b+c+d
A fixed number of captive-reared and wild-caught individuals of a bird are given access to two types of nest sites (high and low). Each bird can only choose one nest site. Individuals are classified according to whether they are captive-reared or wild-caught and whether they nest high or low. We want to compare the proportion of captive-reared choosing the high sites with the proportion of wild-caught choosing the high sites - or equivalently test for association between origin and nest site.
This model is much rarer than the two models we discussed in the More Information page on Pearson's chi square test, but it is applicable to choice experiments.
Assuming the marginal totals are fixed greatly simplifies the mathematics and means that probabilities can be estimated using the hypergeometric distribution with four classes.
To apply Fisher's test to a sample:
Work out all the 2 × 2 tables that deviate from the expected values more than your sample.
Use Fisher's formula to estimate the probability of each of these 2 × 2 tables.
The distribution of cell frequencies when the null hypothesis is true (that is no association between categories) is given by the hypergeometric distribution. The formula for calculating the probability of a given cell combination (a,b,c,d) arising is:
Algebraically speaking -
||(a+b)! (c+d)! (a+c)! (b+d)!
|N! a! b! c! d!
P is the probability that a particular cell combination will arise
The cell frequencies (and their margin totals) are -
|a||b||a + b
|c||d||c + d
|a + c||b + d||N
And ! indicates a factorial
Add up the probabilities of the tables as extreme or more extreme than that observed, and (for a 2-tailed test) double that summed probability. Some authorities prefer to additionally estimate probabilities in the other tail of the distribution and sum them all to get the two-tailed value.
The row and column totals are fixed, not random.
Sampling or allocation are random and observations are mutually independent within the constraints of fixed marginal totals.
Each observation is mutually exclusive - in other words each observation can only be classified in one cell.
Despite these assumptions, it is widely recommended that Fisher's exact test should be used in place of Pearson's chi square test when rows and/or columns are random if frequencies are very small. Such usage is, however, controversial. Conover (1999) states that it is valid (at least as regards controlling the Type I error rate at less than 5%), although its power is less than that of the more appropriate Pearson's chi square test when row totals, column totals or both are not fixed. However, our own simulations suggest that the Type I error rate may exceed 5% if one or more expected frequencies is very small. Whatever its (statistical) validity, the test should certainly not be described as 'exact' when it is applied to data from other than a Model III design.