How will the data be collected?
How the information will be collected is important because the method impacts the response rate as well as the quality of the responses.
The most common collection methods include the following:
- Interviewers ask the survey questions in a telephone interview or in a face-to-face (personal) interview.
- Respondents self-complete the questionnaire without the assistance of an interviewer via traditional mail, email or on-line.
A survey may use one or more of these approaches. For example, a paper questionnaire sent via traditional mail may use a telephone follow-up if responses are not received within a certain timeframe.
Each of these methods has various advantages and disadvantages. Inappropriate use can introduce unintended errors, which can make survey results less than reliable, if steps are not taken to reduce such errors.
Will interviewers be fully trained for this particular survey?
A survey provider's interviewing staff is the backbone of its data collection effort. The interaction between interviewer and respondent is a crucial element in the success of your survey.
If your questions are unclear to interviewers, then they will likely be unclear to respondents, and interviewers will struggle to help respondents understand what they are being asked. Make sure everyone understands what the questions mean.
Verify that the organization employs experienced, well-trained interviewing staff.
- Check that training manuals are provided to the interviewers and cover all field procedures.
- Ask about the amount of time the interviewers have been working for the organization conducting surveys.
- Ask about the types of surveys that the interviewers are experienced in collecting.
- Ensure that interviewers are provided with a good introduction to the survey for their initial approach.
Will the survey provider apply a range of best practices to ensure the highest possible response rate?
Key points to focus on include the following:
- A well designed and planned survey should incorporate procedures for following up with the people who have not responded on the first attempt.
- If interviewers collect the survey information, they should make more than one attempt to contact respondents who are not available on the first try.
- Call-backs should be made on different days of the week and at different times of the day.
- The collection period should be long enough to ensure maximum response rates.
- The survey provider should use industry-standard methods to calculate response rates.
Will the information be kept confidential?
Determine what steps the survey provider will take to respect respondent confidentiality. These steps may take the form of an interviewer oath of secrecy, documentation of how personal information will be used, infrastructure that guards against unintended information uses or sharing, or a combination of these and other safeguards.
What to look for in your questionnaire?
Always ask for a copy of the survey questionnaire and take the time to review the exact wording of all the questions that will be asked.
The wording should be fair and unbiased. Look for any evidence of leading or loaded questions, and verify that the questionnaire presents a balanced set of response choices. The readability level is also important. Most people should be able to understand the questionnaire wording easily.
Pay attention to the order of the questions to make sure the sequence doesn't inadvertently bias the results. Seemingly minor points like this can seriously undermine the quality of your survey results.
Use short words and simple, direct sentences so the questions will be understood uniformly by most people. This will also ensure more accurate translation into official and, if applicable, minority languages.
Make the questionnaire as short as possible to meet your information needs. Keep the "need to know" questions. Remove extraneous questions that may distract from your survey's focus.
Thoroughly test the questionnaire in all language versions. Be prepared to edit the questionnaire for issues uncovered during testing.