Data quality

2011 Census of Agriculture — concepts, methodology and data quality

Using the following information will ensure a clear understanding of the basic concepts that define the data provided in this product, and of the underlying census methodology and key aspects of the data quality. It will give you a better understanding of how the data can be effectively used and analysed according to their strengths and limitations. The information may be particularly important when making comparisons with data from other surveys or sources of information, and in drawing conclusions regarding change over time.

Data sources and methodology

The Census of Agriculture collects and disseminates a wide range of data on the agriculture industry such as number and type of farms, farm operator characteristics, business operating arrangements, land management practices, crop areas, numbers of livestock and poultry, farm capital, operating expenses and receipts, and farm machinery and equipment. These data provide a comprehensive picture of the agriculture industry across Canada every five years at the national, provincial and territorial levels as well as at lower levels of geography.

General methodology

Target population

The target population is all census farms in Canada. In 2011, a census farm was defined as an agricultural operation that produces at least one of the following products intended for sale: crops (hay, field crops, tree fruits or nuts, berries or grapes, vegetables, seed); livestock (cattle, pigs, sheep, horses, game animals, other livestock); poultry (hens, chickens, turkeys, chicks, game birds, other poultry); animal products (milk or cream, eggs, wool, furs, meat); or other agricultural products (Christmas trees, greenhouse or nursery products, mushrooms, sod, honey, maple syrup products). However, the definition of a census farm has changed over time; for a summary of these changes since 1921, please refer to Census farm.

The Census of Agriculture also collects and disseminates data pertaining to a related sub-population — farm operators. In 2011, "farm operators" were defined as those persons responsible for the day-to-day management decisions made in the operation of a census farm or agricultural operation. Up to three farm operators could be reported per farm. Prior to the 1991 Census of Agriculture, the farm operator referred to only one person responsible for the day-to-day decisions made in running an agricultural operation.

Collection

In 2011, Census of Agriculture forms were delivered to farm operations by Canada Post. Once completed, the questionnaire was mailed back to the Data Operations Centre for processing. Respondents also had the option to complete and submit a questionnaire via the Internet. If it was determined that a questionnaire had not been received, or if data were missing, a follow-up was conducted by telephone. For a more detailed description of the collection process, please refer to Data collection.

Data processing

Once the questionnaires were received at the Data Operations Centre they were electronically scanned and had their data automatically captured from the image using intelligent character recognition (ICR) technology. The captured data were then subjected to many rigorous quality control and processing edits to identify and resolve problems related to inaccurate, missing or inconsistent data. Subject-matter analysts also reviewed the aggregated data and individual values so that any remaining errors due to coverage, misreporting, data capture or other reasons were identified and corrected. For a more thorough explanation, please refer to Data processing.

Reference period

The Census of Agriculture has been conducted concurrently with the Census of Population every five years since 1951. The 2011 Census of Agriculture was conducted as of May 10, 2011.

Revisions

Data from the Census of Agriculture are not subject to revision.

Adjustments

Data from the Census of Agriculture are not subject to seasonal adjustments or benchmarking to other data sources.

Concepts and variables measured

For a full description of census concepts, derived variables and geographic levels, please refer to Census terms and Geographic definitions.

Data accuracy

An integral part of each Census of Agriculture is the implementation of new or enhanced methods, procedures and technologies that improve not only the collection, but also the processing, validation and dissemination of the data. New methods, procedures and technologies adopted for the 2011 Census of Agriculture included significant updates to the Statistics Canada Farm Register in preparation of the Census, mailing questionnaires to the entire farm population with a recognized mailing address and an enhanced centralized telephone follow-up operation to resolve non-response as well as inconsistencies within questionnaires returned by respondents. In addition, to help ensure that data from the 2011 Census of Agriculture would be of consistently high quality, improved quality assurance and control procedures were incorporated into each of the collection and data processing stages.

Primarily as a result of adopting these methods, procedures and technologies, the 2011 Census of Agriculture data are of very good quality, with data for the major commodities being of the highest quality. A response rate of 95.9% and an estimated 1.8% undercoverage rate of farms indicate the overall success of the 2011 Census of Agriculture. Note that close to half of the estimated undercoverage was of farms with sales below $10,000 in 2010. As a result, the undercoverage rate for major commodities is below 1%.

With projects as large and complex as the Censuses of Agriculture and Population, the estimates produced from them are inevitably subject to a certain degree of error. Knowing the types of errors that can occur and how they affect specific variables can help users assess the data's usefulness for their particular applications as well as assess the risks involved in basing conclusions or decisions on them.

Errors can arise at virtually every stage of the census process, from preparing materials, through collecting data, to processing. Moreover, errors may be more predominant in certain areas of the country or vary according to the characteristic being measured. Some errors occur at random, and when individual responses are aggregated for a sufficiently large group they tend to cancel each other out. For errors of this nature, the larger the group, the more accurate the corresponding estimate. For this reason, data users are advised to be cautious when using estimates based on a small number of responses. Some errors, however, might occur more systematically and result in "biased" estimates. Because the bias from such errors is persistent no matter how large the group for which responses are aggregated, and because bias is particularly difficult to measure, systematic errors are a more serious problem for most data users than random errors.

The most common types of errors are described below.

Coverage errors

In spite of efforts to enumerate all farm operations in Canada, each Census of Agriculture misses some farms. To reduce undercoverage, an agriculture operator screening question is included on the Census of Population questionnaire to help identify farm operators not on Statistics Canada's Farm Register (FR). If a Census of Population questionnaire was returned with this question marked "yes" and no household members were found on the FR, those households were followed up by phone to complete a Census of Agriculture questionnaire. In addition, the operators of farms on the FR that did not respond to the mail-out or that did not receive a questionnaire due to an incorrect mailing address were contacted by phone to complete a questionnaire. Finally, the Coverage Evaluation Study showed an estimated 1.8% undercoverage rate for the 2011 Census of Agriculture.

Non-response errors

Some Census of Agriculture and Census of Population questionnaires are only partially completed or not completed at all, usually because of the respondent's absence during the census period or unwillingness to complete the questionnaires. In either case, if the follow-up attempt to obtain the appropriate information is unsuccessful, missing responses are approximated using an automated imputation procedure during data processing. This procedure replaces a missing or inconsistent response, either with a value that is consistent with the other data provided on the questionnaire or with a response obtained from a similar agricultural operation. Data resulting from this procedure generally have little impact on the final figures released.

Response errors

Respondents sometimes provide inaccurate responses on the questionnaire, perhaps as a result of misinterpretation of a question, incorrect placement of a response or approximation of a response. In the Census of Agriculture, implausible or inconsistent responses are confirmed or corrected by contacting the respondents, since they could have a significant impact on totals at either the provincial or the sub-provincial level.

Processing errors

Errors can arise at any stage of data processing, including scanning or character recognition errors during data capture, coding and classification errors, and errors due to limitations in the imputation procedure (to correct missing or inconsistent responses, as described in "Non-response errors"). A detailed set of computerized checks at each stage of processing identifies such errors for corrective action. In addition, quality assurance procedures were developed for all processing steps.

Comparability of data and related sources

The data validation process identified some instances in which data either were not directly comparable to those from previous censuses or were of reduced quality, primarily because of coverage or response errors. After thoroughly investigating each case, notes were developed to identify the variables affected and explain the situation associated with each.

Following each Census of Agriculture, other agricultural surveys use Census of Agriculture data as a basis, or benchmark, for the production of regularly published estimates of the agriculture industry.

Other quality indicators and assessments

Coverage Evaluation Study

The purpose of the Coverage Evaluation Study (CES) is to estimate the coverage of the 2011 Census of Agriculture that was conducted as of May 10, 2011.

Coverage is a problem that affects the quality of estimates of all censuses. For the Census of Agriculture, coverage errors occur when farms are missed, incorrectly included or double counted. The CES measures the level of coverage and is one way to assess the quality of the Census of Agriculture estimates.

The collection method provides farm operators the opportunity to mail back their completed questionnaire or to complete and submit a questionnaire via the internet. Those that do not are eligible to be followed-up.  Not all non-responding farms were followed up, nor were all household responding yes to the agriculture operator screening question on the Census of Population. The CES selects a random sample of smaller farm operations from Statistic Canada's Farm Register for which no Census of Agriculture questionnaire was received, as well as a random sample of households responding yes to the agriculture operator screening question. The farms and households selected for the CES are followed up by telephone in order to complete a Census of Agriculture questionnaire, and their data is used to estimate the undercoverage resulting from not following up all farms and households. In addition, an estimate of the undercoverage resulting from larger farm operations that could not be contacted during non-response follow up is calculated. The final net undercoverage estimates combine the estimates of these different sources.

Please note that there are no estimates of undercoverage for Yukon Territory, the Northwest Territories and Nunavut.

Table 1 Farm undercoverage: breakdown by province
Province Enumerated farms Non-enumerated farms (estimated) Undercoverage Standard Error
number of farms %
Newfoundland and Labrador 510 20 3.9 0.4
Prince Edward Island 1,495 33 2.2 0.1
Nova Scotia 3,905 86 2.2 0.1
New Brunswick 2,611 61 2.3 0.2
Quebec 29,437 407 1.4 0.1
Ontario 51,950 836 1.6 0.1
Manitoba 15,877 278 1.8 0.1
Saskatchewan 36,952 287 0.8 0.1
Alberta 43,234 831 1.9 0.1
British Columbia 19,759 779 3.9 0.2
Canada 205,730 3,618 1.8 0.1
Table 2 Total farm area undercoverage: breakdown by province/region
Province/region Enumerated farms Non-enumerated farms (estimated) Undercoverage Standard Error
acres %
Atlantic 2,627,577 38,148 1.5 0.2
Quebec 8,256,614 53,610 0.6 0.1
Ontario 12,668,236 93,714 0.7 0.1
Manitoba 18,023,472 149,501 0.8 0.1
Saskatchewan 61,628,148 238,652 0.4 0.1
Alberta 50,498,834 481,379 1.0 0.1
British Columbia 6,452,867 102,409 1.6 0.2
Canada 160,155,748 1,157,413 0.7 0.1
Table 3 Total gross farm receipts undercoverage: breakdown by province/region
Province/region Enumerated farms Non-enumerated farms (estimated) Undercoverage Standard Error
$ %
Atlantic 1,740,326,096 12,481,311 0.7 0.1
Quebec 8,425,943,743 33,827,804 0.4 0.1
Ontario 11,902,171,357 77,715,355 0.7 0.2
Manitoba 5,290,437,772 35,855,054 0.7 0.1
Saskatchewan 9,376,064,350 32,796,087 0.3 0.1
Alberta 11,438,786,537 90,525,031 0.8 0.1
British Columbia 2,940,420,649 46,055,545 1.6 0.3
Canada 51,114,150,504 329,256,187 0.6 0.1
Table 4 Farm undercoverage: breakdown by total gross farm receipts
Total gross farm receipts Enumerated farms Non-enumerated farms (estimated) Undercoverage Standard Error
number of farms %
0 to $9,999 43,364 1,644 3.8 0.1
$10,000 to $24,999 33,049 691 2.1 0.1
$25,000 to $99,999 51,506 759 1.5 0.1
$100,000 to $249,999 31,713 255 0.8 0.1
Greater than or equal to $250,000 46,098 269 0.6 0.1
Date modified: