Documentation

Scope and purpose

Documentation constitutes a record of the statistical activity, including the concepts, definitions and methods used to produce the data. It also includes descriptions of factors affecting data comparability and quality.

Documentation may serve as a record for users of what was done in order to provide a context for effective and informed use of the data. During implementation, documentation is a means of communication to assure effective development of the statistical activity. In addition, documentation includes not only what decisions were made, but also why they were made, and provides information which will be useful for future development and implementation of the same, a similar or a redesigned statistical activity.

The goal of documentation is to provide a complete, unambiguous and multi-purpose record of the survey, including the data produced from the survey. Documentation of any step of the survey may be aimed at any of several different groups, such as management, technical staff, planners of other surveys, and users. Documentation should be up-to-date, well organized, easily retrievable, concise and precise. Effective presentation of results is an important part of documentation. Documentation may be multi-media (e.g., paper, electronic, visual), and different documents may be prepared for different audiences and purposes.

Guidelines

• The level of detail provided in the documentation will depend on its intended audience, the type of data collection, the data sources, the analysis, the method of dissemination, the range of uses of the data and their impact, and the total budget of the statistical program. Ensure that all statistical products meet the requirements of the Policy on Informing Users of Data Quality and Methodology (Statistics Canada, 2000d).
  
  
• Depending on the audience and purpose, documentation may include the following:
  
  
  • Objectives: Include information on the objectives and uses of the data, timeliness, frequency, and data quality targets; these may have changed as work proceeded on the survey (for example: owing to budgetary constraints, perceived feasibility, results of new pilot studies, or new technology), and these changes need to be documented because they have an impact on the design of the questionnaire and the analysis of test results.
    
  • Content: Include the questionnaire used and concepts and definitions. In the case of computer assisted interviews, provide the development specifications for the software application. To facilitate integration with other sources, use standardized concepts, questions, processes and classifications. Highlight differences, if warranted. Mention the role of advisory committees and users.
    
  • Tests: Describe cognitive tests, field tests or pilot surveys, and report on results as to how specifications were met, complete with recommendations.
    
  • Methodology: Set out design alternatives. Deal with issues such as target population, frame, coverage, reference period, sample design, sample size and selection, collection method and follow-up procedures for nonresponse, estimation, editing and imputation, benchmarking and revision, seasonal adjustment, confidentiality and evaluation. Emphasize different aspects for different readers. Provide a consolidated document on technical issues for professionals. Provide a methodological overview.
    
  • Systems: Include information on data files (capture method, layouts, explanation of codes, basic frequencies, edit procedures), systems documentation (construction, algorithms, use, storage and retrieval), and monitoring reports (time spent and where, trouble areas, scheduling of runs to determine whether processing is on time).
    
  • Operations: Include or cite references for training manuals, operator and interviewer manuals, feedback and debriefing reports.
    
  • Implementation: Document operations, with inputs and outputs clearly specified. Attach schedules for each implementation step.
    
  • Quality control: Include the instructions and/or a manual for supervisors and editing staff.
    
  • Data quality: For general use, include coverage, sampling error, nonsampling error, response rates, the rates and effects of edit and imputation, comparability over time and with other data, validation studies and any other relevant measures specific to the particular statistical activity. Describe any unexpected events affecting data quality (e.g., floods, high nonresponse). For technical users, include total variance or its components by source, nonresponse and response biases, and the impact and interpretation of seasonal adjustment.
    
  • Resources: List the actual resources consumed, as a function of time. Account for all expenditures in terms of money and time. Comment on expenditures vs. budgets.
    
  • References: Organize and document references (theoretical and general papers and documents relevant to, but not produced as part of, the project).
    
    
• Provide documentation elements required for the Integrated Metadatabase (Statistics Canada, 2000c). As the archive for information on Statistics Canada surveys and programs, the IMDB contains most of the information pertaining to methodology and data accuracy. Electronic products contain a link to the IMDB, which is used to access documentation relevant to the product. As for hard copy products, the IMDB ensures adequate documentation, in compliance with the Policy on Informing Users of Data Quality and Methodology (Statistics Canada, 2000d).
  
  
• Consider the readership of each document. Subject the document to extensive review by managers, representatives of the intended readers and peers to ensure quality and readability (Statistics Canada, 1995). Edit documents meticulously. Date each version of the document. Check references.
  

Top of Page

Statistics Canada (1995). Policy on the Review of Information Products. Policy Manual, 2.5.

Statistics Canada (1998b). Policy on Standards. Policy Manual, 2.10.

Statistics Canada (2000c). Integrated Metadatabase: http://stdsweb/standards/imdb/imdb-menu.htm (STC intranet site).

Statistics Canada (2000d). Policy on Informing Users of Data Quality and Methodology. Policy Manual, 2.3.

Statistics Canada (2001d). Standards and Guidelines for Reporting of Nonresponse Rates. Statistics Canada technical report.

Statistics Canada (2002c). Statistics Canada’s Quality Assurance Framework – 2002. Statistics Canada Catalogue no. 12-586-XIE.

Statistics Canada (2003e). The Official Style Guide. Editorial Services, Communications Division. See http://icn-rci.statcan.ca/10/10d/10d_000_e.htm (STC intranet site). Publication updated regularly.

United Nations (1964). Recommendations for the preparation of sample survey reports (provisional edition). Statistical Series C, 1, New York.

United Nations, Conference of European Statisticians (1983). Draft Guidelines for the Preparation of Presentations of the Scope and Quality of Statistics for Users. Geneva, Switzerland.