Biology, images, analysis, design...
Use/Abuse Principles How To Related
"It has long been an axiom of mine that the little things are infinitely the most important" (Sherlock Holmes)

Search this site




On this page: Observational studies  Experimental studies 

Observational studies

    Pseudoreplication is especially a risk when using multiple group/period studies so we will concentrate on this type of design.
  1. Define the limits of the statistical population you are considering in all dimensions. For example if you are comparing quality of service between public and private health providers, or bird species richness on organic and conventional farms, define the geographical area and time period within which you will operate.
  2. Draw up a sampling frame for your primary sampling unit - for example all public and private health providers, or all organic and conventional farms within the area. If you cannot produce such a list, you will have to take a haphazard sample - but this may be biased and will diminish (or eliminate) the external validity of the study. Ensure units are spatially and temporally independent.
  3. Select a sample of each type using random or stratified random sampling. These will be your replicates for examining the 'treatment' effect, so ensure your sample size is adequate. If you anticipate 'non-response' from some units (for example uncooperative farmers) select a larger sample to allow for this.
  4. Measure the response variable with sufficient precision within each primary unit. This may require sampling multiple secondary units, whether patients (for health providers) or transects (for bird species richness).
  5. Analyse the data using the average value for each primary unit to evaluate the 'treatment' effect.


Experimental studies

  1. Select your experimental units. Ideally these would be a random selection from a known statistical population but in practice this is rarely possible. Most experiments lack external validity, but have high internal validity because of random allocation.
  2. Allocate units to treatments randomly using an appropriate experimental design to ensure adequate interspersion of treatments amongst units. If you have a large number of experimental units a completely random allocation of units to treatments may be adequate. But for a small number of units, pairing or stratification may be necessary to ensure balance of known confounding factors. If using a Latin square design with repeated measures over time ensure that treatments are independent - do not use exactly the same treatment applied to different units (as in the trap example above).
  3. After treatment allocation, ensure that each experimental unit is kept independent from all other units. For example: if an individual fish is the experimental unit, do not put all fish allocated to one treatment in one tank, and all fish allocated to the other treatment in another tank. Similarly, if the treatment is temperature, you cannot just use one constant temperature room or cabinet for each temperature. If all units cannot practically be kept separate (temperature controlled cabinets or rooms are often a limited resource!) or if the natural situation is that units interact, then make a group of individuals your experimental unit, and allocate treatments to groups (cluster randomization). Your sample size will then be the number of groups, but at least your estimate for each group will be relatively precise.
  4. Analyse the data using the average value for each experimental unit to evaluate the 'treatment' effect.