Data quality, concepts and methodology: Response and non-response
Archived Content
Information identified as archived is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please "contact us" to request a format other than those available.
Response and non-response
Despite the best efforts of survey managers and operations staff to maximize response in the MRTS, some non-response will occur. For statistical establishments to be classified as responding, the degree of partial response (where an accurate response is obtained for only some of the questions asked a respondent) must meet a minimum threshold level below which the response would be rejected and considered a unit non-response. In such an instance, the business is classified as not having responded at all.
Non-response has two effects on data: first it introduces bias in estimates when non-respondents differ from respondents in the characteristics measured; and second, it contributes to an increase in the sampling variance of estimates because the effective sample size is reduced from that originally sought.
The degree to which efforts are made to get a response from a non-respondent is based on budget and time constraints, its impact on the overall quality and the risk of non-response bias.
The main method to reduce the impact of non-response at sampling is to inflate the sample size through the use of over-sampling rates that have been determined from similar surveys.
Besides the methods to reduce the impact of non-response at sampling and collection, the non-responses to the survey that do occur are treated through imputation. In order to measure the amount of non-response that occurs each month, various response rates are calculated. For a given reference month, the estimation process is run at least twice (a preliminary and a revised run). Between each run, respondent data can be identified as unusable and imputed values can be corrected through respondent data. As a consequence, response rates are computed following each run of the estimation process.
For the MRTS, two types of rates are calculated (un-weighted and weighted). In order to assess the efficiency of the collection process, un-weighted response rates are calculated. Weighted rates, using the estimation weight and the value for the variable of interest, assess the quality of estimation. Within each of these types of rates, there are distinct rates for units that are surveyed and for units that are only modeled from administrative data that has been extracted from GST files.
To get a better picture of the success of the collection process, two un-weighted rates called the ‘collection results rate’ and the ‘extraction results rate’ are computed. They are computed by dividing the number of respondents by the number of units that we tried to contact or tried to receive extracted data for them. Non-monthly reporters (respondents with special reporting arrangements where they do not report every month but for whom actual data is available in subsequent revisions) are excluded from both the numerator and denominator for the months where no contact is performed.
In summary, the various response rates are calculated as follows:
where i = units that have either reported data that will be used in estimation or are converted refusals, or have reported data that has not yet been resolved for estimation.
where ii = units that have data that was extracted from administrative files and are usable for estimation.
where iii = units that have either reported data (unresolved, used or not used for estimation) or are converted refusals.
where iv = all of the above plus units that have refused to respond, units that were not contacted and other types of non-respondent units.
where vi = in-scope units that have data (either usable or non-usable) that was extracted from administrative files
where vii = all of the above plus units that have refused to report to the administrative data source, units that were not contacted and other types of non-respondent units.
(% of questionnaire collected over all in-scope questionnaires)
where iii = same as iii defined above
where viii = same as iv except for the exclusion of units that were contacted because their response is unavailable for a particular month since they are non-monthly reporters.
where ix= same as vi with the addition of extracted units that have been imputed or were out of scope
where vii = same as vii defined above
(% of questionnaires collected over all questionnaire in-scope we tried to collect)
All the above weighted and un-weighted rates are provided at the trade group, geography and size group level or for any combination of these levels.
Use of Administrative Data
Managing response burden is an ongoing challenge for Statistics Canada. In an attempt to alleviate response burden and survey costs, especially for smaller businesses, the MRTS has reduced the number of simple establishments in the sample that are surveyed directly and instead derives sales data for these establishments from Goods and Service Tax (GST) files using a statistical model. The model accounts for differences between sales and revenue (reported for GST purposes) as well as for the time lag between the survey reference period and the reference period of the GST file.
For more information on the methodology used for modeling sales from administrative data sources, refer to ‘Monthly Retail Survey: Use of Administrative Data’ under ‘Documentation’ of the IMDB.
Table 3 contains the weighted response rates for all trade groups as well as for total retail trade for each province and territory. For more detailed weighted response rates, please contact the Marketing and Dissemination Section at (613) 951-3549, toll free: 1-877-421-3067 or by e-mail at statcan.retailinfo-infodetaillants.statcan@canada.ca.
Methods used to reduce non-response at collection
Significant effort is spent trying to minimize non-response during collection. Methods used, among others, are interviewer techniques such as probing and persuasion, repeated re-scheduling and call-backs to obtain the information, and procedures dealing with how to handle non-compliant (refusal) respondents.
If data are unavailable at the time of collection, a respondent's best estimates are also accepted, and are subsequently revised once the actual data become available. To minimize total non-response for all variables, partial responses are accepted. In addition, questionnaires are customized for the collection of certain variables, such as inventory, so that collection is timed for those months when the data are available.
Finally, to build trust and rapport between the interviewers and respondents, cases are generally assigned to the same interviewer each month. This action establishes a personal relationship between interviewer and respondent, and builds respondent trust.
- Date modified: