InfluentialPoints.com
Biology, images, analysis, design...
Use/Abuse Principles How To Related
"It has long been an axiom of mine that the little things are infinitely the most important" (Sherlock Holmes)

 

 

Multiple group / period designs

Characteristics  Pros & cons 

Characteristics

Medical epidemiologists lump together multiple group & multiple time period designs as 'ecologic studies'. These two comparative group designs do have one important feature in common - namely that the main explanatory variables (and sometimes the response variable) are measured at the group rather than individual level. The explanatory variables may be aggregate measures, for example the average income of people in an area. Or they may be environmental measures such as the level of pollution. All individuals in a given group are then assumed (often quite wrongly!) to experience the same level of that explanatory variable.

In other respects the two designs differ quite considerably:

  • In a multiple group design, data are usually collected at one point in time from several groups or 'populations'. The groups are usually defined by the area in which they live and are selected to represent different levels of the explanatory variable - for example diet or pollution in medical studies, or intensity of control or presence of an invasive species in ecological studies. Ecologists sometimes use the term mensurative experiment for an observational multiple group design. The response variable is often measured using some form of probability sampling within each group. In medical studies records may exist of the total number of cases occurring over a period of time.

  • In a multiple time period design or before-and-after design (or time-trend design), data are collected from just one group or population. The different time periods are selected to represent different levels of the explanatory variable, either before and after some natural event (say an El Nino event) or before and after a human intervention (such as introduction of a vaccination programme). Ecologists sometimes use the term quasi-experiment for an observational multiple time period study where there is manipulation but there is no random allocation or replication. Probability sampling can be carried out in each period, or alternatively the same cohort of sampling units (whether individuals or quadrats) can be monitored throughout the period.

 

As with analytical surveys at the group level, it is important to realise that it may not be valid to extrapolate a relationship between variables at the group level to the individual level. The problem is that you only know that the average level of an explanatory variable differs between two areas or two time periods - you do not know exactly what each individual has experienced. This gives rise to the ecologic fallacy . This is much less of a problem for environmental measures - where all individuals may indeed experience the same level - than for aggregate measures - which may be quite useless for describing what individuals experience.

Inference from this type of design is generally considered very weak, but need not necessarily be so.

  • For multiple group studies, the important thing to remember is that the group is the sampling unit - even if you are gathering data on individuals by subsampling. One cannot use measurements on individuals as replicates if all of individuals for one level of the explanatory variable live in one area and all of the other level live in another area. A multiple group design is only really worthwhile if one (a) takes samples from replicated groups for each level of the explanatory variable and/or (b) uses multiple levels of the explanatory variable (say areas selected to represent different levels of alcohol abuse or farming intensification).

  • Multiple time period studies tend to suffer even more severely from lack of replication and/or pseudoreplication because it may be impossible to replicate large scale interventions. They can, however, be improved by adding one or more control groups where there is no intervention. Such control groups can be randomly selected from available groups or matched to the intervention group(s). Where a control or reference group is included the design has been termed a before-after control-impact (BACI) study. Further improvements are to take multiple samples over time both before and after the intervention, or to use multiple interventions over time - applying the intervention for a period, stopping, and then reapplying it.

 

 

Pros and cons of multiple group/period designs

Multiple group designs
    Advantages
    • If routine records are available for the response variable (for example area wide mortality rates), they are relatively simple and inexpensive to implement. Also being cross-sectional, no follow up is required.
    • You can estimate your required sample size in advance - but remember that the unit of study is the group so the sample size is the number of groups required.

    Disadvantages

    • For ecological studies the cost of studying multiple groups is often prohibitive - leading to numerous one-replicate studies.
    • The ecologic fallacy may be a serious drawback to using the design if we are investigating whether there is a causal relationship between (say) pollution and individual disease.
    • Since only specific groups are compared, there is a risk of only comparing extremes. For example one might compare heart disease rates in 10 countries where they eat a lot of vegetables with rates in 10 countries where they eat very few - ignoring the countries where a moderate amount is eaten may lead to bias. An analytical survey (where a random sample of groups is considered) is less biased in this respect.
    • All the constraints of cross-sectional studies listed under analytical surveys also apply for multiple group studies. They cannot be used for transient risk factors, and there is a high risk of both incidence-prevalence bias and recall bias.
    • Confounding factors will not be equally distributed amongst groups. Whilst known confounding factors can be corrected for, unknown ones obviously cannot.

  Multiple time period designs

    Advantages
    • Again they are relatively simple to implement because routine records are often available. However, follow up is required to evaluate any long term changes as a result of an intervention.
    • Unlike the multiple group design, the temporal sequence is clear and unambiguous with the intervention coming before the assessment of outcome.
    • Providing there is a random selection of (multiple) control sites and multiple sampling over time before and after the intervention, the strength of inference is higher than for most other observational designs.

    Disadvantages

    • The ecological fallacy applies as much for multiple time period studies as it does for multiple group studies. Again this is not a problem if the outcome is a group characteristic (for example density of an insect pest), but is a constraint in the medical context.
    • Confounding factors will not be equally distributed amongst groups. Whilst known confounding factors can be corrected for, unknown ones obviously cannot.