Does organic farming benefit biodiversity

What does evidence-based mean? The concept of an evidence-based approach probably originated in medicine in early 19th century Paris, but it was not formalised unitil the 1990s. Sackett et al. (1996) defined evidence-based medicine as integrating individual clinical expertise with the best available external clinical evidence from systematic research.

"The man of science has learned to believe in justification, not by faith, but by verification." Thomas H. Huxley (1825-95) English biologist.

At its core is the belief that there should be good scientific evidence that the treatment (say a drug) caused the observed outcome (say cure rate). The 'strong inference' that treatment explains response can only obtained by carrying out a number of properly conducted randomized trials - in other words treatments are randomly assigned to a certain number of independent study units (in this case patients).

Such evidence may be supplemented (or sometimes replaced) with evidence from well-designed observational studies where exposure to the explanatory variable of interest (the 'treatment') is not randomly assigned, and indeed is often self-assigned. Observational studies are commonly used by epidemiologists, where it is not ethical to randomly assign treatments known to be harmful (such as smoking). The researcher selects contrasting groups to study, usually (but not always) by the level of a selected explanatory variable. Often one of these groups will be a control.

Control, or untreated, subjects are vital in both randomized trials and observational studies, because without them it can be impossible to argue any effect was due to the treatment applied. For instance any clinician knows that some subjects recover spontaneously without treatment, others do so because they merely think they have received an effective treatment, and not everyone responds to a known effective treatment.

Unfortunately, without random allocation, there is a much greater potential for bias in the outcome. Some forms of bias can be eliminated by using (genuinely) random sampling, but even so the individuals in your study groups are likely to differ in all sorts of characteristics other than the key explanatory variable under investigation. Those characteristics may also be causally related to your response variable. If they are not distributed similarly between your groups (as they would be, on average, given random allocation), they may obscure or confound the effects of your key explanatory variable.

Hence although observational studies can be used to contribute to the body of evidence, they need to be carefully screened to ensure that only well designed studies are included and the results should be given considerably less 'weight' than randomized trials. In the evidence-based approach, such weighting may either be done informally through a systematic review of work done, or formally with a meta-analysis where one attempts to quantify the 'treatment' effect together with a measure of uncertainty.

 

The spread of randomized trials

The desire (or claim) for an evidence-based approach has now spread to many other disciplines such as education (Thomas & Pring, 2004), vector control (Croft et al., 2001), conservation management (Stewart et al., 2005) and forestry (Petrokofsky, 2008). In recent years a start has been made to apply the 'evidence-based' approach to livestock disease control. Thus Donnelly et al. (2007) reported on the first randomized trial (known as the Krebs' trial) to assess the effectiveness of badger culling to reduce the incidence of bovine tuberculosis in English cattle.

The Krebs' trial certainly provided the first 'high quality' evidence about the effects of culling, but it also highlighted some of the implementation difficulties of large-scale randomized trials (for more details see InfluentialPoints). These included (1) non-random allocation of treatment in one set of replicates because of 'security concerns', (2) failure to meet treatment and monitoring schedules because of activist protests and an outbreak of foot and mouth disease, and (3), probably most importantly, the very high cost of such trials. A freedom of information request by InfluentialPoints to the UK Department for Environment, Food and Rural Affairs (DEFRA) revealed the total cost of the randomized badger trial as 49,030,000 UK pounds (DEFRA, 2011).

In drug trials that high cost is met from the profits of the drug companies. But funds are not so readily available for trials to test new surgical procedures, and it can be very difficult to get funds in other disciplines. Costs therefore have to be met from public funds. Given which, we should not expect any more randomized trials from the UK government on the issue of badger culling - or perhaps on any other environmental issue...

 

Why we still need observational studies

Not surprisingly the results of the Krebs badger culling trial have now been pounced upon by both pro- and anti-cull groups with messianic zeal. But unfortunately a single randomized trial can be just as misleading as an observational study, especially if the number of replicates is small. This is partly because of simple random error - random allocation can only be guaranteed to provide similar distributions of confounding factors in each treatment group on average, and only approaches this happy situation when there are hundreds of experimental units in each treatment group.

Hence nearly all medical trials now use hundreds, or thousands, of patients. This degree of replication is much more difficult to achieve when the study unit is an area of land. The badger culling trial referred to above had just ten replicates of each treatment! Further problems arise when experimental units are unrepresentative of the population at-large. This is always a potential problem, since experimental units (whether patients or land areas) are never chosen at random, and is why many drugs cannot be given to pregnant women (they are usually excluded from randomized trials on safety grounds).

One is forced to conclude that, despite the benefits of properly conducted randomized trials, many government policies will be based on (at best) one or two such trials, together with the results of a much larger number of observational studies. But they do need to be well designed observational studies! Unfortunately, in the clamour for randomized trials, design issues for observational studies have been largely ignored. Yet in many important areas no randomized trials have been carried out, and decisions are instead based on many badly designed observational studies. We suggest below that one such area is the effects on biodiversity of organic farming versus conventional farming...

 

Conventional versus organic farming

In 'developed' economies conventional farming has long used a high level of synthetic inputs - including manufactured fertilizers, synthetic pesticides and fungicides. Such methods have been adopted worldwide for the commercial production of most crops and livestock.

The image below shows the worst side of conventional farming - a local farmer in Kenya mixing up synthetic insecticides (profenofos, cypermethrin and carbosulfan) and fertilizers for application to a melon crop. The mixture is then sprayed on the crop using a standard backpack sprayer - in this case no protective clothing was worn at either mixing or application. Despite the risks to the farmers and to local water supplies, farmers find the much higher yields worth the costs.

Mixing insecticides and fertilizer with no protective clothing
(Photo: InfluentialPoints)

In reaction to the abuse of chemical pest control, some conventional farming is now done used integrated pest control where attempts are made to minimize pesticide use and to use more selective pesticides. Also over recent years there has been a marked increase in the amount of land farmed 'organically' rather than conventionally. Organic farming uses techniques such as crop rotation and compost rather than manufactured fertilizers to maintain soil productivity. It also benefits biological control by using mechanical weeding and intercropping to encourage the presence of predators and parasitoids of pests . If insecticides are required, only 'natural' rather than synthetic insecticides are used.

The image below shows a coccinellid larva (below, center) eating a cherry aphid (left), a serious pest of cherry trees.

Coccinellid larva eating a cherry aphid
(Photo: InfluentialPoints)

Crop yields tend to be markedly lower in organic than conventional farming, but its advocates claim this disadvantage is more than offset by the greater sustainability of the organic farming system. Although initially the preserve of the wealthy elite in Western countries, organic food stores can now be found in many parts of the world.

Aside from benefiting biological control, organic farming is supposed to benefit biodiversity. This is most commonly and simply defined as species richness, which is the number of species (whether of plants or birds or butterflies) present in an area. The use of species richness as an index of biodiversity can create problems from a conservation point of view because there is no advantage to replacing a small number of (regionally) rare species with a large number of (nationally) common species. But let us accept this simple definition for the time being, and assess what the level of evidence is for the assertion that organic farming benefits biodiversity.

 

What is the evidence that organic farming benefits biodiversity?

Perhaps the first point to make is that, as far as we know, no randomized trials have been carried out to test the hypothesis that converting an area from conventional to organic farming increases species richness or any other measure of biodiversity. A large number of studies have been carried out, but they nearly all use what is best described as a comparative group design: Species richness in one or more taxonomic groups is compared between convenience-sampled organic farms and a similar number of conventional farms.

A convenience sample is one where the sampling units are selected from those most conveniently-available, rather than randomly from the entire population of interest. Aside from invalidating most commonly-used statistical analyses, convenience-sampled data tends to produce seriously biased conclusions.

Usually (but not always) a paired design is used, where each organic farm is matched with a conventional farm on the basis of proximity and/or size, land use in surrounding areas, and farming practices. Other confounding factors (for example time-elapsed since organic-farm conversion) are usually not considered. Type-of-farming is taken as the explanatory variable, and species richness of a specified taxonomic group (or groups) as the response variable.

Whether this should really be given the status of the word "design" is debatable, since most such studies are akin to a social researcher convenience-sampling (say) 20 white people and 20 black people, possibly matched on where they live, and comparing intelligence between the two groups. Some may find this analogy offensive, but this really is the design and level of replication most commonly used for comparing species richness on organic and conventional farms! This type of study has been done so frequently that there have been several reviews of results.

1. Hole et al. (2005) reviewed 76 studies that compared single or multiple taxonomic groups on organic and conventional farms. Summing comparisons for all taxa, they showed a positive effect of organic agriculture on species abundance and/or richness in 66 cases; 25 had neutral or mixed outcomes, and only eight showed a negative effect. No methodological quality measures were used for inclusion / exclusion of a particular study, but some problems in comparing studies were reviewed. These were:
   • The definition of organic farming standards varies between and within countries.
   • Differences may be caused by landscape characteristics because of lack of adequate pairing, rather than farming regime.
   • Some studies were only carried out over a single season or year.
   • Considerable variation existed in the spatial scale at which comparisons were made.
   • Different studies used different measures of biodiversity.

   Hole et al. also noted several aspects which may bias results in favour of conventional farms namely (1) avoiding the largest, most intensively managed conventional units, because of the lack of similarly sized organic units for comparison and (2) by pairing or statistically controlling for the majority of inter-regime variability in management practice, some studies may have excluded the very differences that were responsible for creating the observed varition in biodiversity in the first place - an effect known as over-matching.

   On the other hand farmers who choose to convert to organic farming may be pre-disposed to environmentally friendly farming practices, or may farm land that has previously been managed less intensively and is therefore easier to convert successfully to organic farming - both of which would introduce a bias in favour of the organic system.

2. Bengtsson et al. (2005) carried out a meta analysis of 66 published studies and found that species richness was, on average, 30% higher on organic farms, with stronger effects likely in intensively managed landscapes. Bengtsson et al. also used no methodological quality measures for inclusion / exclusion of a particular study, but noted that many studies comparing organic and conventional farming were poorly designed.

   Large numbers of studies had low numbers of replicates or failed to include factors other than farming system in the design. They questioned whether studies performed using 'plot' as the sampling unit are relevant at all to the wider question of whether organic farming enhances biodiversity, and recommended studies should be carried out at the farm scale or larger. They also noted that criteria for the selection or farms or fields to be studied were important - but then focused only on the problems of over-matching.

3. There have also been some less 'rigorous' reviews, namely by Smith et al. (2011) which concluded there is 'overwhelming evidence that organic farming provides more biodiversity than conventional farming' and Mondelaers et al. (2009) who came to same conclusion.

 

What the reviews don't mention

Surprisingly none of these reviewers have commented on two obvious design problems common to nearly all these studies:

1. Cross-sectional designs provide very weak evidence for causality

   In terms of demonstrating causality between organic farm management and biodiversity, all these studies had a fundamental design weakness - they are all cross-sectional. In other words sampling is done at one point in time. A longitudinal study could compare diversity on one (or more) farms before and after conversion to organic. If farms already with a high biodiversity are more likely to convert to organic farming than farms with low biodiversity, a cross-sectional study would suggest a difference whilst a longitudinal study may not.

   There are many reasons why farms which have already converted to organic may always have been more diverse - for instance because the farmer was 'environment-friendly' even when using conventional agriculture, because the farm was more structurally diverse, or because of their surrounding areas. Some of these confounding factors can be allowed for in the design (by matching), or statistically (providing the factor is measured) - but you cannot allow or adjust for unknown confounding factors. That is why a randomized trial - a longitudinal design with random treatment assignment - is always the best approach. However, there are obvious practical difficulties. Farmers would not agree to a random assignment of their farming management system, and few governments would give such an experiment enough priority to implement it at a sufficient scale on 'state-owned' farms.

   The 'next-best' option is to combine a longitudinal with a cross-sectional design to give a 'Before-After-Control-Impact' (BACI) design. If we monitor diversity of one or more animal or plant taxa on several farms that converted to organic farming, and find an increase in diversity, we then know that the putative causative factor preceded the observed effect. This is the crucial element in a medical 'cohort' study, where a group of individuals are followed over a period of time. If we also monitor some farms that do not convert to organic farming over the same time period, then those farms can serve as controls.

   The converting farms should be a random sample of farms converting in the study area (or all such farms) and the conventional farms should be matched to the converting farms using strict matching criteria. One then compares the degree of change in diversity over the time period for the two types of farm. A BACI design was proposed for comparing organic and conventional farming systems ten years ago (Fairweather & Campbell, 2001). At least one such study has been started (Mondot, 2007), but results after completion of conversion have yet to be published.

    

2. Avoiding bias in selection of sample units (farms)

   One may decide against a BACI design on the basis that it will take a long time period for diversity to increase. If this is the case, then a comparative group design may be the only approach. However, it is essential to avoid bias in the selection of farms for comparison. Yet as far as we can tell none of the studies reviewed by Hole et al. (2005) or Bengtsson et al. (2005) selected a random sample of organic farms.

   Does this matter?

   Well yes, on several levels:
   • If you wish to make any statistical (rather than hunch-based) inferences about the organic / conventional comparison in your study area, you must make a random selection of the organic farms. Convenience sampling can perhaps be regarded as a form of pseudoreplication (although Hurlbert (1984) did not use the term in this way) since the replicates are clearly not independent. But probably the main problem with convenience sampling is that it wide open to bias. It has already become a paradigm that organic farms are more diverse than conventional farms - in other words we all have to believe it whether it is true or not. Hence there is a clear danger that researchers will (consciously or subconsciously) tend to select very 'diverse' organic farms - a phenomenon known as confirmation bias.

   • Moreover many researchers gave only vague criteria of how the (matched) conventional farm was selected. Strict criteria are essential so we know which (possibly confounding) factors are being allowed for by pairing, and which still need to be corrected for statistically. If strict criteria are not specified, then the pair of farms can no longer be regarded as a random sample of pairs under those conditions, based on the original random selection of the organic farm. Again there is a risk of confirmation bias - if there is an option to select one of say three conventional farms, the researcher may choose the one that looks the least diverse. The risk of bias is probably even greater if the owners of each organic farm are asked themselves to suggest the nearest appropriate conventional farm for comparison (see for example Wickramasinghe et al., 2003).

Be very clear - we are not arguing that that organic farms do not benefit biodiversity - merely that most research so far done has ignored all the principles of study design in terms of obtaining evidence for causality and avoiding bias. Such bias may well lead to the effect size being overestimated.

 

Can the comparative group design be improved?

The comparative group design could be greatly improved if all or a random sample of organic farms in an area (obtained from a listing of organic farms) were compared with either a random sample of conventional farms, or with a matched sample obtained using rigorous criteria (say farms randomly selected from the five nearest conventional farms within specified size limits). But the use of convenience sampling or vague matching criteria is simply not acceptable in scientific study. These problems should come as no surprise - Boyce (2002) pointed out that sampling and study design problems plague biological research programmes both at the student and faculty level, and felt that this continues to be one of the greatest needs for statistical training.

Of late we are starting to see improved (less biased) methods of site selection.

• Chamberlain et al. (2010) compared winter bird species richness between organic and conventional farms for a group of Farmland Bird Indicator species. They attempted to include all organic farms within their specified inclusion criteria - albeit they did not specify what percentage was actually included. Conventional farms were matched to organic farms by location, and presumably by growing cereals since each comparison was centred on cereal plots. None of the matched farms was adjacent to the organic farm and none was included on the basis of recommendation by the organic farmer.

  Chamberlain was also one of the few authors to give the participation rate of farmers, at least of the conventional farmers (only 25% of those selected agreed to take part). However they did not investigate this for 'non-participation bias'. Note, most of these methodological details were in 'supplementary material', presumably on the basis that most readers are not interested in such (essential) details.

  A yellowhammer, one of the species covered in Chamberlain's study
  (Photo: courtesy of Wikipedia/Eigenes Werk under a Creative Commons Attribution 3.0 Licence)

  

  The study did not demonstrate a strong effect of farm type on species richness, although there was a greater total relative density of birds on organic farms. Variation in structural habitat was more important for species richness than whether farms were organic or conventional.

   

• Another study that stands out in recent years in terms of site selection, albeit with a somewhat different aim, is that by Reid et al. (2007). They compared (relative) population sizes of the three farmland mammal species in areas subject to Environmentally Sensitive Area (ESA) prescriptions with those in matched non-ESAs in the wider countryside. The species were the Irish hare - a priority species for conservation action - and two 'pest' species, the European rabbit and the red fox. A sample of 150 1-km² ESA survey squares was randomly selected in proportion to the area of each ESA (known as probability proportional to size). A sample of 50 non-ESA squares was frequency matched for land class, altitude, category of bisecting road and distance from ESA boundary, as represented in the ESA sample.
  Frequency matching is where controls are selected so that the overall make up of the control group is similar to that of the ESA group. This is in contrast to individual matching where each organic farm would be matched to a conventional farm. Frequency matching is nonetheless perfectly adequate and often more practicable.

  The ESA scheme had no demonstrable effect on the abundance of the conservation priority species (Irish hare), but the abundance of pest species (considered as rabbits and foxes) was significantly greater within ESAs than the wider countryside. The authors noted that targeted and evidence-based agri-environment prescriptions are clearly required in order to ensure the realization of species-specific conservation targets - which is perhaps a rather academic way of saying we need randomized trials of the ESA prescription using hare density as the response variable!

• Lastly we consider the work of Gabriel et al. (2010). They used a novel multiscale hierarchical sampling design where stratified random sampling was used to obtain contrasting landscapes and contrasting farm types within landscapes. One of the authors (Benton, pers. comm., 2011) noted that the whole point of the design was to avoid any bias in the selection of farms. On average they found that the positive effects of organic farming on biodiversity were not as strong as were previously thought. They suggested this was because the effect of landscape had not been properly taken into account. We only note that when a real effort is made to eliminate the risk of bias in site selection, one would also expect the positive effect size of organic farming on biodiversity to decrease.

In summary the observational comparative design can be improved providing the basic rules of sampling are followed, although strength of inference will always be less than that of a well-designed BACI study.

 

What about using other 'cross-sectional' designs

We next consider whether alternative 'cross-sectional' designs could be used.

• The obvious design to try is a simple analytical survey. One takes a completely or stratified random sample of all arable farms in the study area. For this one needs a listing of all farms in the study area. This unfortunately may not be available in all countries, but could presumably be collated for a particular region. Species richness (the response variable) is then assessed on each sampled farm together with a range of appropriate explanatory variables. In this design one would have less power for comparing organic versus conventional, because organic farms may only comprise a small percentage of the total. But it would anyway be better to focus on the different components of farm management such as usage of fertilizers, composting, crop rotation, synthetic insecticides, 'natural' insecticides, and so forth.

  One advantage of the analytic survey approach is that enables one to assess the effect of the whole range of farm management types on species richness. The comparative group design instead forces one to make a rather arbitrary distinction between certified organic farms and conventional farms. This invariably introduces a degree of bias since one has to exclude certain farms that do not fit neatly in one or other category. In addition evidence of causality is stronger if there is a dose response relationship - in other words if species richness increases as insecticide usage decreases (this is impossible for a simple binary, organic / conventional classification). The approach would also enable one to compare the effect of 'natural' versus synthetic insecticides, whilst controlling statistically for other factors. Some work has been done on this, for example Bahlai et al. (2010) showed that, on soybeans, certified organic insecticides had a similar or greater impact on natural enemy species than several new synthetic insecticides which have been developed specifically to minimise predator mortality.

• The case-control design is a very different type of design commonly used by epidemiologists. In this design comparison groups are defined not by the explanatory variable but by the response variable. So instead of comparing species richness in a group of organic farms with species richness in a group of conventional farms, one compares farm management in a group of high diversity farms with farm management in a group of low diversity farms.

  Ecologists already use this design when, for example, they want to find out the factors responsible for site selection by animals - either nesting site, resting site, or sometimes prey kill site. In that situation the response variable is binary (the site is used or not) and the explanatory variables are a group of environmental variables (vegetation in site, topography, and suchlike). Note one has to assume that the explanatory factor is constant over the time it takes the disease (or difference in biodiversity) to develop.

  The advantage of this approach is that one has to focus much more directly on all the factors affecting species richness. The disadvantage of course is that one has to know in advance which are the 'high diversity' and 'low diversity' farms - which is not very practicable, other than on farms which have been examined in other studies. Nevertheless if one wants to focus on presence / absence of certain key indicator species this may be a more productive way to go.

 

Is species richness the best response variable?

What do we want from our agricultural system(s)? Most would agree on a sustainable system that gives high yields at low cost; many would also want systems that use biological rather than pesticidal / herbicidal control of pests, and that make a significant contribution to wildlife conservation. Species richness is a relatively easy response variable to measure (albeit it is time-consuming). But is species richness a good proxy variable, either for effective biological pest control, or for conservation value?

A proxy variable is something which is measured instead of what you are you are actually interested in measuring - usually because the proxy variable is easier, cheaper, or less controversial, than the variable of interest.

Despite the frequency of claims in the literature that biodiversity enhances biological control, rather few studies have measured biodiversity effects upon pest control and yield on organic farms compared to conventional farms. Letourneau & Bothwell (2008) reviewed relevant studies and concluded that increase in the diversity of insect predators and parasitoids can have a positive or negative effect upon prey consumption rates. They therefore called for a stronger scientific basis for evaluating pest suppression effects due to enhanced natural enemy diversity - a call we would strongly support.

 

So is species richness a good proxy variable for conservation value? Well not necessarily! It is often found that low-intensity use agricultural land has a higher species richness than (for example) mature native woodland. This is not at all surprising since disturbed areas often have a high level of structural heterogeneity that is utilized by both remnant populations of k-strategists (a rather dated but sometimes appropriate terminology) and invading populations of r-strategists. Unfortunately the latter species tend to be very widespread (many are pests), and hence tend to have low conservation value. If one is serious about using species richness as a proxy variable for conservation value, then one needs to weight any measure of species richness with area-wide rarity (for more on this see Fattorini et al., 2006 and Basset et al., 2008).

We also need to consider the issues raised by Green et al. (2005), Gabriel et al. (2009) and Hodgson et al. (2010). Land-sparing describes an approach whereby agricultural land is farmed as intensively as possible, with the conservation of wildlife within separate nature reserves. In contrast, land-sharing describes wildlife-friendly farming such as organic systems, where biodiversity is managed. Given the 'land sparing vs. land sharing' debate, it has been suggested organic farming should be concentrated in areas where there is already a concentration of organic farms, whilst conventional farming should be intensified in other areas:

"the most efficient way to conserve farmland biodiversity under food production constraints will be to further extensify those landscapes that are naturally rich in organic farming. Consequently, areas supporting few or no organic farms can be encouraged to further intensify, even at the cost of biodiversity in these areas." Gabriel et al. (2009)

This approach was dismissed by the Organic Research Centre thus:

"Ecological thinking should not be limited to the biological world. It has a wider relevance - certainly to the whole food system and relations within it. The concept of "land sharing or land sparing" is not ecological; it is mechanistic, inappropriate and out of time." Organic Research Centre (2010)

Describing these approaches as 'inappropriate and out of time' seems a rather weak critique of what is an innovative but potentially dangerous approach. The land sharing or land sparing approach seems to fall into the trap of thinking that organic farms can serve all conservation needs - which they clearly cannot! The essence of agriculture is that is a man-managed environment. Whilst some species are well suited to exploit such disturbed environments, many more are not. If you consider British butterflies, only about half the species may occur in any numbers on the most environment-friendly farm. The other half tend to be very specialized in their habitat requirements, often with much higher conservation value, and will only survive in nature reserves. An example of this group is the Heath Fritillary shown below.

Heath Fritillary (Melitaea athalia) in Blean Woods, Kent
(Photo: InfluentialPoints)

Hence there is a need for more nature reserves (not over-run by dog-walkers, cyclists or quad bikers) both in organic and conventional farming areas - the advantage of organic farming areas is that they will have a greater capacity to support metapopulations of rare species as individuals will be more readily able to move between suitable habitat patches.

 

A postscript

The worrying aspect of what we have described above is the scientific community appears to have accepted the organic paradigm so unquestioningly. At any rate very little attention has been paid to the study design issues described above. Perhaps the central (albeit embarrassing) question we have to ask the reader is whether you think the arguments for organic farming are philosophical (natural vs scientific) or empirical (in other words does this method do what it claims to do?).

If your position is the former, you may find our arguments about quality of scientific evidence (at very best) irrelevant, and approve of organic farming simply because it makes you feel good. Let's face it, many people operate on that basis, as the cosmetics industry (for example) is highly aware. If on the other hand, you believe like us that government policy should be evidence-based, you might ask why the scientific community has produced so few high quality studies on the topic in recent years. Perhaps scientists themselves are not immune to the faith bug...

"Faith is the great cop-out, the great excuse to evade the need to think and evaluate evidence. Faith is belief in spite of, even perhaps because of, the lack of evidence." Richard Dawkins. Untitled Lecture, Edinburgh Science Festival 1992.

If you are interested in an evidence-based approach to conservation issues see our hyperbook: Avoiding and Detecting Statistical Malpractice.

Robert D. Dransfield (Senior Partner, InfluentialPoints.LLP)  

Feedback & comments We would be delighted to receive feedback and comments at info@InfluentialPoints.com Comments from authors quoted on this page (whether positive or negative) are displayed here, in the order received. Prof Tim Benton "Champion" for Global Food Security (from Nov 1), Institute of Integrative and Comparative Biology, University of Leeds, UK 10 Oct 2011 Interesting piece... I hosted a workshop in Warsaw last week where we discussed some of these issues. I think even the dyed-in-the-wool academics (cf me, who they see as having gone over to the dark side to critique this area) appreciate the methodological issues (perhaps more than comes across in some of the papers). However, longitudinal studies, whilst desirable, are too expensive to fund de novo (though I know Jan Bengtsson is following some farms thru time). The Gabriel et al study cost 1 million pounds to do the 32 farm design over 3 years. Doing a reasonable experimental study of converting over the 20-50 years is both expesnive and time consuming... The Warsaw workshop was about finding solutions NOW for the impending food security crisis of doubling world food supply, in 30-40 years, in a sustainable way... John Fairweather (PhD) Research Professor of Rural Sociology, Lincoln University, NZ 11 Oct 2011 Thank you for sending this link to your discussion of research designs best suited to causal inference about agricultural management systems. I think you have canvassed the important issues very thoroughly and provide good leads to the better designs. Members of the the team of researchers in ARGOS have often discussed these issues, and we are well aware of the shortfalls in our application of BACI. We have been overly attentive to publishing substantive results and given less attention to publishing on our research design. Our work in 2001 did lead to a major study which still continues (see http://www.argos.org.nz/). Mondot (2997) is one publication from this research. Dr G.B. Stewart University of York, UK 11 Oct 2011 Your reference to my work is rather peripheral but I thought I would comment more generally. I like the points you are making but wonder if you are focusing on the most important biases? My major concerns addressing these types of question are about the potential for bias associated with the definition of biodiversity. Clearly, biodiversity can be defined and measured in lots of ways, thus it can be manipulated and reported in lots of ways creating substantial risk with respect to multiple testing and selective reporting. How many of the syntheses you discuss had protocols which reported how these issues would be handled prior to analysis? The variation in definitions and both temporal and spatial scales over which change (or difference) is measured can also pose serious analytical problems in meta-analysis which are typically ignored or handled with sensitivity analyses although more advanced hierarchical modelling methods could allow the full complexity to be realistically modelled particularly if raw data were available from all or some of the studies. Given the problems of randomisation and replication, separation of within and between unit covariate interactions could also be important if the objective is to assess the generality of the relationships between biodiversity and farming type across multiple sites. A final point is that ideally, the synthesis should be embedded in a decison framework in order to make it informative. Quantifying the "average magnitude" of the difference between organic and inorganic agriculture or understanding how the relationship varies isn't particularly interesting from a theoretical perspective but neither is it likely to be useful to a decision-maker unless the influence of these relationships on the decision is explicit.

 

References  Bahlai, C.A. et al. (2010). Choosing organic pesticides over synthetic pesticides may not effectively mitigate environmental risk in soybeans. PLoS ONE 5 (6), e11250. Abstract  Full text  Basset, Y et al. (2008). Choice of metrics for studying arthropod responses to habitat disturbance: one example from Gabon. Insect Conservation and Diversity 1, 55-66.  Abstract  Full text  Bengtsson, J. et al. (2005). The effects of organic agriculture on biodiversity and abundance: a meta-analysis. Journal of Applied Ecology 42, 261-269.  Abstract Full text  Boyce, J. (2002). Statistics as viewed by biologists. Journal of Agricultural, Biological and Environmental Statistics 7 (3), 306-312.  Abstract  Full text  Chamberlain, D.E. et al. (2010). Does organic farming benefit farmland birds in winter? Biological Letters 6, 82-84. Abstract   Full text   Supplement  Croft, A.M. et al. (2001). An evidence-based vector control strategy for military deployments: the British Army experience. Medecine Tropicale 61 (1), 91-98.   Full text  Donnelly, C.A. et al. (2007). Impacts of widespread badger culling on cattle tuberculosis: concluding analyses from a large-scale field trial. International Journal of Infectious Diseases 11, 300-308.  Abstract  Full text  Fairweather, J.R. & Campbell, H.R. (2001). Research on the consequences of converting to organic production: A review of international literature and outline of a research design for New Zealand. Studies in Land Use Change and Socio-Economic Consequences. Lincoln University, Canterbury, New Zealand . Full text  Fattorini, S. et al. (2006). A new method to identify important conservation areas applied to the butterflies of the Aegean Islands (Greece). Animal Conservation 9 (1), 75-83.  Abstract  Gabriel, D. et al. (2009). The spatial aggregation of organic farming in England and its underlying environmental correlates. Journal of Applied Ecology 46, 323-333. Abstract Full text  Gabriel, D. et al. (2010). Scale matters: the impact of organic farming on biodiversity at different spatial scales. Ecology Letters 13 (7), 858-86.  Abstract  Green, R.E. et al. (2005). Farming and the fate of wild nature. Science 307, 550-555.  Abstract  Hodgson, J.A. et al. (2010). Comparing organic farming and land sparing: optimizing yield and butterfly populations at a landscape scale. Ecology Letters 13 (11), 1358-1367. Abstract  Hole, D.G. (2005). Does organic farming benefit biodiversity? Biological Conservation 122, 113-130.  Abstract  Full text  Hurlbert, S.H. (1984). Pseudoreplication and the design of ecological field experiments. Ecological Monographs 54 (2), 187-211.  Abstract  Full text  Letourneau, D.K. & Bothwell, S.G. (2008). Comparison of organic and conventional farms: challenging ecologists to make biodiversity functional. Frontiers in Ecology and the Environment 6 (8), 430-438.  Abstract  Full text  Mondelaers, K. et al. (2009). A meta-analysis of the differences in environmental impacts between organic and conventional farming.. British Food Journal 111 (10), 1098-1119.   Abstract   Full text  Mondot, M. et al. (2007). Baseline data on the diversity and abundance of selected bird species on conventional and converting organic dairy farms. Proceedings of the New Zealand Grassland Association 69, 245-251.  Full text  Organic Research Centre (2010). The biodiversity benefits of organic farming  Full text  Petrokofsky, G. (2008). T10Q In the context of evidence-based forestry - A brief discussion note.    Full text  Reid, N. et al. (2007). Mammals and agri-environment schemes: hare haven or pest paradise? Journal of Applied Ecology 44, 1200-1208. Abstract  Full text  Sackett D.L. et al. (1996). Evidence based medicine: what it is and what it isn't. BMJ 312 (7023), 71-72.  Full text  Smith, J. et al. (2007). Organic farming and biodiversity: A review of the literature. Organic Centre Wales, Aberystwyth.  Full text  Stewart, G.B. et al. (2005). Applying evidence-based practice in conservation management: Lessons from the first systematic review and dissemination projects. Biological Conservation 126 (2), 270-278.  Abstract  Full Text  Thomas, G & Pring, R. (eds) (2004). Evidence-based practice in education. Open University Press, Buckingham,    Wickramasinghe, L.P. (2003). Bat activity and species richness on organic and conventional farms: impact of agricultural intensification. Journal of Applied Ecology 40 , 984-993.  Abstract  Full text