Questionnaires

Questionnaires have long been used by medical epidemiologists as a means of collecting data from individuals on exposure to risk factors. They are now being increasingly used by veterinary epidemiologists and researchers in other disciplines such as forestry and wildlife biology. In some areas the increasing use reflects greater interest in obtaining the views of the public, for example perceptions or attitudes towards conservation strategies. In other areas it reflects the rising cost of field research and the (often mistaken) belief that one can get reliable information just by sending round a questionnaire rather than going to the field oneself.

Sample selection for a questionnaire is covered elsewhere and will not be considered further here - except to emphasize the need for either a complete census or (more usually) probability sampling. Here we consider questionnaire construction, methods of implementation, and sources of bias.

1. Questionnaire construction

   There are three basic type of questions that can be asked.
   (a) The first type is the closed question where the answer is either binary, multiple choice, or requires some sort of rating on a scale. If multiple choice categories are given they should be mutually exclusive (categories should not overlap) and exhaustive (covering all possible answers). The latter often requires a dumping category such as 'others', or 'don't know'. Avoid an excessively large number of categories (more than ten) and try to minimize the number of responses in the dumping category.
   (b) The second type is the open question where the respondent can answer in his or her own words. Closed questions are both quicker and easier to answer and to analyse, but they can unnecessarily constrain responses on some issues.
   (c) A third approach is to set a discussion topic which is then put to a focus group (see below).

   Questions should be clear and concise and each should focus on one issue. Vague adjectives such as 'many' and 'occasionally' should be avoided - use specific terms instead such as 'more than 20' or 'less than once a week'. Related questions should be grouped together so that it is easier for the respondent to follow. Keep the questionnaire as short as possible to adequately cover the topic - however large the topic it is very unwise to exceed 30 minutes per respondent as the response rate will fall and measurement error will increase.

   If possible the questionnaire should be developed in the language of the respondent group. If translation is necessary, then the translated version should be always back-translated by someone blinded to the original questionnaire, and the result compared with the original.

   Any questionnaire should be piloted with a small random sample of the population to test how well it performs in terms of comprehension and time for completion. A questionnaire should only be implemented after it has gone through this test.

2. Methods of implementation

   There are several methods of implementing a questionnaire:
   (a) A postal survey self-completed by the respondent is still the most common method used for large scale surveys. It has the advantages of low cost, and elimination of interviewer bias (see below). But it tends to produce a lower response rate (sometimes much lower), and you cannot be certain that views are not influenced by others receiving the questionnaire (in small communities you may well end up with everyone having the same views as some influential leader).
   (b) Telephone interviews have been increasingly used in recent years. They retain most of the advantages of in-person interviews and are much cheaper to implement - but carry with them a high risk of selection bias if not everyone is on the phone, or equally willing to be interviewed that way (see below).
   (c) In-person interviews are the method of choice where it is felt that the presence of an interviewer will make the questionnaire more comprehensible or increase the response rate. They are sometimes the only method that can be used in areas where telecommunications services are not well developed.
   (d) In all the above methods the sampling unit is the individual. If one wishes to investigate motivations or perceptions it may be more appropriate to use the focus group approach. Here a topic / questionnaire is discussed by a small groups of respondents. Ideally the individuals in the groups would be randomly selected, but commonly they are purposively selected either to be 'representative' of the community, or to specifically include key opinion leaders.
   

3. Sources of error and bias

   High levels of error and bias are commonly associated with information obtained by questionnaire:
   

   1. Non-response bias is a form of selection bias which we considered in Unit 2. It is reduced by keeping the questionnaire short, by having a respected sponsor connected with study, by the subject being of importance to the respondent, and by respecting confidentiality. For in-person interviews it can be reduced by checking through responses immediately; for postal questionnaires, reminders should be sent to non-respondents, or they should be contacted by telephone. An attempt can be made to quantify the impact of non-response by surveying non-respondents for comparison with respondents, but usually information can only be obtained on a small subset of the questions - and just because there was no difference on those does not mean there is no difference concerning the other topics.
      

   2. Measurement bias applies to questionnaires no less than any other data. Interviewer bias (a form of observer bias) occurs when the opinions of the respondent are affected by the interviewer. We have found this a serious problem when the survey is already well underway, when interviewers may just fill in some responses for the respondent if the majority of people are answering a certain way. Recall bias is particularly common if you ask people about events that occurred a long time ago. Social desirability bias can occur especially if people view the interviewer as holding certain moral positions.

      The only way to deal with most types of measurement bias is through some form of data validation/verification. The consistency of responses can be checked by asking the same question in a quite different way in a later part of the questionnaire, or by repeating the questionnaire at a later time. However, for questions involving factual matters this is always second best to ground truthing. A random subsample of interviewees should be visited and the truth, or otherwise, of the responses confirmed on the ground.