When undertaking any sample survey, it will be subject to what is known in statistics as sampling error.
Sampling error arises from estimating a population characteristic by looking at only one portion of the population rather than the entire population. It refers to the difference between the estimate derived from a sample survey and the 'true' value that would result if a census of the whole population were taken under the same conditions. There is no sampling error in a census because the calculations are based on the entire population.
As a general rule, the more people being surveyed (sample size), the smaller the sampling error will be. Many people are surprised by the small size of well-known surveys. For example, polls that try to predict voting patterns are taken from sample sizes ranging from 1,000 to 2,000 people, with samples of about 1,000 people being the most common. Ratings for television programs are estimated from approximately 2,000 viewers. This small sample represents the television preferences of a total population of 12 million Canadian households! Despite a widely-held perception that such polls are reliable, some statisticians question their accuracy because of the small sample size.
If one of the survey objectives is to look at sub-populations or measure rare events, then a larger sample will be needed. However, it is important to note that increasing the sample size also means increasing costs.
Except for very small populations where the relationship is more direct, the size of a sample does not increase in proportion to the size of the population. In fact, the population size plays an almost non-existent role as far as large populations are concerned.
In general, the greater the difference between the population units, the larger the sample size required to achieve a specific level of reliability. For example, if you were to conduct a survey on work environments for a population where the income varies from $30,000 to $50,000, you would use a smaller sample size to achieve the same level of reliability than you would use for a population of equal size for which income varies from $5,000 to $1,000,000.
It is important to develop an efficient sampling plan, which includes a sample design and an estimation procedure. The method of sampling, called "sample design", can greatly affect the size of the sampling error. Many surveys involve a complex sample design that often leads to more sampling error than a simple random sample design. The estimation procedure also has a major impact on the sampling error. (These concepts are examined in greater detail in the chapter entitled Sampling methods.)
There are methods that estimate sampling error for probability sample surveys. The sampling variance is the most commonly used measure to quantify sampling error, and like the other methods, it is derived directly from the sampling and estimation methods used in the survey. (Sampling variance is examined in more detail in the chapter entitled Sampling methods.)