From what population (area or group) will the sample be selected?
The population of interest for the survey, or target population, must be carefully identified. The information, called the "sampling frame" by statistical agencies and the "call-list" by public opinion research organizations, used to identify members of the target population should be up-to-date and well documented. If the sampling frame does not cover the desired target population accurately, the survey results may be severely biased. If the survey targets a specific group of the population or a specific geographical area, the results should not be interpreted as representing people outside of that group or area. A specific group might be men, women, Aboriginals, teachers, political party supporters and so on. A specific area might be a province, a region, a city, and so on.
How will people be selected for interviewing?
To avoid sample bias, some important questions must be asked about how people will be selected to participate in the survey. The survey documentation should indicate whether the sample will be chosen using a probability or non-probability sampling method.
If a probability sampling method is used, you should verify the following:
- Respondents will be selected objectively, that is, randomly
- All members of the target population will have a known chance to be selected in the survey
You should also enquire about the general structure of the sampling design, such as stratification, clustering, multi-stage or multi-phase design, as applicable.
If a non-probability approach is used, the way respondents are selected should also be explained.
- Will the selection of people to be interviewed be left up to an interviewer, such as in quota sampling?
- Will respondents select themselves in some way such as by participating in a phone-in poll, responding to a questionnaire in a book or magazine, or by joining an on-going panel?
Note that some surveys use a combination of probability and non-probability sampling. An example of this might be overlaying a quota sampling constraint onto an initially probability-based design.
Will the sample selected from a population be representative of that population?
To ensure the sample selected for your survey represents the population, you should ensure that key characteristics within the selected sample are similar to those characteristics in the population. It is also important to verify that the characteristics among the actual survey respondents are similar to the characteristics in the selected sample. Key characteristics within a population might include age, sex, education, marital status, or any other available profiling information to help answer questions important to the survey subject.