Quality assurance
Filter results by
Search HelpKeyword(s)
Type
Results
All (18)
All (18) (0 to 10 of 18 results)
- 1. Challenges and results in using Audit trail data to monitor Labour Force Survey data quality ArchivedSurveys and statistical programs – Documentation: 11-522-X201700014707Description:
The Labour Force Survey (LFS) is a monthly household survey of about 56,000 households that provides information on the Canadian labour market. Audit Trail is a Blaise programming option, for surveys like LFS with Computer Assisted Interviewing (CAI), which creates files containing every keystroke and edit and timestamp of every data collection attempt on all households. Combining such a large survey with such a complete source of paradata opens the door to in-depth data quality analysis but also quickly leads to Big Data challenges. How can meaningful information be extracted from this large set of keystrokes and timestamps? How can it help assess the quality of LFS data collection? The presentation will describe some of the challenges that were encountered, solutions that were used to address them, and results of the analysis on data quality.
Release date: 2016-03-24 - Surveys and statistical programs – Documentation: 11-522-X201700014716Description:
Administrative data, depending on its source and original purpose, can be considered a more reliable source of information than survey-collected data. It does not require a respondent to be present and understand question wording, and it is not limited by the respondent’s ability to recall events retrospectively. This paper compares selected survey data, such as demographic variables, from the Longitudinal and International Study of Adults (LISA) to various administrative sources for which LISA has linkage agreements in place. The agreement between data sources, and some factors that might affect it, are analyzed for various aspects of the survey.
Release date: 2016-03-24 - Articles and reports: 11-522-X201700014722Description:
The U.S. Census Bureau is researching ways to incorporate administrative data in decennial census and survey operations. Critical to this work is an understanding of the coverage of the population by administrative records. Using federal and third party administrative data linked to the American Community Survey (ACS), we evaluate the extent to which administrative records provide data on foreign-born individuals in the ACS and employ multinomial logistic regression techniques to evaluate characteristics of those who are in administrative records relative to those who are not. We find that overall, administrative records provide high coverage of foreign-born individuals in our sample for whom a match can be determined. The odds of being in administrative records are found to be tied to the processes of immigrant assimilation – naturalization, higher English proficiency, educational attainment, and full-time employment are associated with greater odds of being in administrative records. These findings suggest that as immigrants adapt and integrate into U.S. society, they are more likely to be involved in government and commercial processes and programs for which we are including data. We further explore administrative records coverage for the two largest race/ethnic groups in our sample – Hispanic and non-Hispanic single-race Asian foreign born, finding again that characteristics related to assimilation are associated with administrative records coverage for both groups. However, we observe that neighborhood context impacts Hispanics and Asians differently.
Release date: 2016-03-24 - 4. Validation of the 10-item Kessler Psychological Distress Scale (K10) in the 2012 Aboriginal Peoples Survey ArchivedArticles and reports: 82-003-X201600114307Description:
Using the 2012 Aboriginal Peoples Survey, this study examined the psychometric properties of the 10-item Kessler Psychological Distress Scale (a short measure of non-specific psychological distress) for First Nations people living off reserve, Métis, and Inuit aged 15 or older.
Release date: 2016-01-20 - 5. Validation of cognitive functioning categories in the Canadian Community Health Survey -Healthy Aging ArchivedArticles and reports: 82-003-X201000411391Geography: CanadaDescription:
This analysis uses data from the Cognition Module of the 2009 Canadian Community Health Survey - Healthy Aging to validate a categorization of levels of cognitive functioning in the household population aged 45 or older.
Release date: 2010-12-15 - Articles and reports: 11-522-X200800010955Description:
Survey managers are still discovering the usefulness of digital audio recording for monitoring and managing field staff. Its value so far has been for confirming the authenticity of interviews, detecting curbstoning, offering a concrete basis for feedback on interviewing performance and giving data collection managers an intimate view of in-person interviews. In addition, computer audio-recorded interviewing (CARI) can improve other aspects of survey data quality, offering corroboration or correction of response coding by field staff. Audio recordings may replace or supplement in-field verbatim transcription of free responses, and speech-to-text technology might make this technique more efficient in the future.
Release date: 2009-12-03 - Articles and reports: 11-522-X20050019461Description:
We propose a generalization of the usual coefficient of variation (CV) to address some of the known problems when used in measuring quality of estimates. Some of the problems associated with CV include interpretation when the estimate is near zero, and the inconsistency in the interpretation about precision when computed for different one-to-one monotonic transformations.
Release date: 2007-03-02 - Articles and reports: 11-522-X20030017694Description:
This paper evaluates the performance of diagnostics for seasonal adjustment as they relate to the performance of X-12-ARIMA and SEATS on a large sample of real and simulated economic time series.
Release date: 2005-01-26 - Surveys and statistical programs – Documentation: 11-522-X20010016225Description:
The European Union Labour Forces Survey (LFS) is based on national surveys that were originally very different. For the past decade, under pressure from increasingly demanding users (particularly with respect to timeliness, comparability and flexibility), the LFS has been subjected to a constant process of quality improvement.
The following topics are presented in this paper:A. the quality improvement process, which comprises screening national survey methods, target structure, legal foundations, quality reports, more accurate and more explicit definitions of components, etc.;B. expected or achieved results, which include an ongoing survey producing quarterly results within reasonable time frames, comparable employment and unemployment rates over time and space in more than 25 countries, specific information on current political topics, etc.;C. continuing shortcomings, such as implementation delays in certain countries, possibilities of longitudinal analysis, public access to microdata, etc.; D. future tasks envisioned, such as adaptation of the list of ISCO and ISCED variables and nomenclatures (to take into account evolution in employment and teaching methods), differential treatment of structural variables and increased recourse to administrative files (to limit respondent burden), harmonization of questionnaires, etc.
Release date: 2002-09-12 - 10. Ensuring the quality of the labor force estimates from the Current Population Survey : a total survey perspective ArchivedArticles and reports: 11-522-X20010016228Description:
The Current Population Survey is the primary source of labour force data for the United States. Throughout any survey process, it is critical that data quality be ensured. This paper discusses how quality issues are addressed during all steps of the survey process, including the development of the sample frame, sampling operations, sample control, data collection, editing, imputation, estimation, questionnaire development. It also reviews the quality evaluations that are built into the survey process. The paper concludes with a discussion of current research and possible future improvements to the survey.
Release date: 2002-09-12
Data (0)
Data (0) (0 results)
No content available at this time.
Analysis (10)
Analysis (10) ((10 results))
- Articles and reports: 11-522-X201700014722Description:
The U.S. Census Bureau is researching ways to incorporate administrative data in decennial census and survey operations. Critical to this work is an understanding of the coverage of the population by administrative records. Using federal and third party administrative data linked to the American Community Survey (ACS), we evaluate the extent to which administrative records provide data on foreign-born individuals in the ACS and employ multinomial logistic regression techniques to evaluate characteristics of those who are in administrative records relative to those who are not. We find that overall, administrative records provide high coverage of foreign-born individuals in our sample for whom a match can be determined. The odds of being in administrative records are found to be tied to the processes of immigrant assimilation – naturalization, higher English proficiency, educational attainment, and full-time employment are associated with greater odds of being in administrative records. These findings suggest that as immigrants adapt and integrate into U.S. society, they are more likely to be involved in government and commercial processes and programs for which we are including data. We further explore administrative records coverage for the two largest race/ethnic groups in our sample – Hispanic and non-Hispanic single-race Asian foreign born, finding again that characteristics related to assimilation are associated with administrative records coverage for both groups. However, we observe that neighborhood context impacts Hispanics and Asians differently.
Release date: 2016-03-24 - 2. Validation of the 10-item Kessler Psychological Distress Scale (K10) in the 2012 Aboriginal Peoples Survey ArchivedArticles and reports: 82-003-X201600114307Description:
Using the 2012 Aboriginal Peoples Survey, this study examined the psychometric properties of the 10-item Kessler Psychological Distress Scale (a short measure of non-specific psychological distress) for First Nations people living off reserve, Métis, and Inuit aged 15 or older.
Release date: 2016-01-20 - 3. Validation of cognitive functioning categories in the Canadian Community Health Survey -Healthy Aging ArchivedArticles and reports: 82-003-X201000411391Geography: CanadaDescription:
This analysis uses data from the Cognition Module of the 2009 Canadian Community Health Survey - Healthy Aging to validate a categorization of levels of cognitive functioning in the household population aged 45 or older.
Release date: 2010-12-15 - Articles and reports: 11-522-X200800010955Description:
Survey managers are still discovering the usefulness of digital audio recording for monitoring and managing field staff. Its value so far has been for confirming the authenticity of interviews, detecting curbstoning, offering a concrete basis for feedback on interviewing performance and giving data collection managers an intimate view of in-person interviews. In addition, computer audio-recorded interviewing (CARI) can improve other aspects of survey data quality, offering corroboration or correction of response coding by field staff. Audio recordings may replace or supplement in-field verbatim transcription of free responses, and speech-to-text technology might make this technique more efficient in the future.
Release date: 2009-12-03 - Articles and reports: 11-522-X20050019461Description:
We propose a generalization of the usual coefficient of variation (CV) to address some of the known problems when used in measuring quality of estimates. Some of the problems associated with CV include interpretation when the estimate is near zero, and the inconsistency in the interpretation about precision when computed for different one-to-one monotonic transformations.
Release date: 2007-03-02 - Articles and reports: 11-522-X20030017694Description:
This paper evaluates the performance of diagnostics for seasonal adjustment as they relate to the performance of X-12-ARIMA and SEATS on a large sample of real and simulated economic time series.
Release date: 2005-01-26 - Articles and reports: 11-522-X20010016228Description:
The Current Population Survey is the primary source of labour force data for the United States. Throughout any survey process, it is critical that data quality be ensured. This paper discusses how quality issues are addressed during all steps of the survey process, including the development of the sample frame, sampling operations, sample control, data collection, editing, imputation, estimation, questionnaire development. It also reviews the quality evaluations that are built into the survey process. The paper concludes with a discussion of current research and possible future improvements to the survey.
Release date: 2002-09-12 - Articles and reports: 11-522-X20010016270Description:
This paper discusses in detail issues dealing with the technical aspects of designing and conducting surveys. It is intended for an audience of survey methodologists.
Following the last three censuses in Britain, survey non-response on major government household surveys has been investigated by linking addresses sampled for surveys taking place around the time of the census to individual census records for the same addresses. This paper outlines the design of the 2001 British Census-linked Study of Survey Nonresponse. The study involves 10 surveys that vary significantly in design and response rates. The key feature of the study is the extensive use of auxiliary data and multilevel modelling to identify interviewer, household and area level effects.
Release date: 2002-09-12 - Articles and reports: 11-522-X20010016281Description:
This paper discusses in detail issues dealing with the technical aspects of designing and conducting surveys. It is intended for an audience of survey methodologists.
Methodology for estimating the sampling error of the non-seasonally adjusted estimate of level of the Index of Production (IoP) has previously been developed using Taylor linearization and parametric bootstrap methods, with both producing comparable results. From the study, it was considered that the parametric bootstrap approach would be more practical to implement. This paper describes the methodology that is being developed to estimate the sampling error of the non-seasonally adjusted IoP change using the parametric bootstrap method, along with the data that are needed from the contributing surveys, the assumptions made, and the practical problems encountered during development.
Release date: 2002-09-12 - Articles and reports: 11-522-X20010016298Description:
This paper discusses in detail issues dealing with the technical aspects of designing and conducting surveys. It is intended for an audience of survey methodologists.
This paper discusses the Office for National Statistics' (ONS) approach to developing systematic quality measurements and reporting methods. It is presented against the background of European developments and the growing demand for quality measurement. Measuring the quality of statistics presents considerable practical and methodological challenges. The paper describes the main building blocks to be used for the new quality measure program, and includes specific examples. Working with other national statistical institutions; and developing an enhanced measurement framework, output measurements, and reporting procedures, are all vital ingredients in achieving recognition of the ONS as a quality organization.
Release date: 2002-09-12
Reference (8)
Reference (8) ((8 results))
- 1. Challenges and results in using Audit trail data to monitor Labour Force Survey data quality ArchivedSurveys and statistical programs – Documentation: 11-522-X201700014707Description:
The Labour Force Survey (LFS) is a monthly household survey of about 56,000 households that provides information on the Canadian labour market. Audit Trail is a Blaise programming option, for surveys like LFS with Computer Assisted Interviewing (CAI), which creates files containing every keystroke and edit and timestamp of every data collection attempt on all households. Combining such a large survey with such a complete source of paradata opens the door to in-depth data quality analysis but also quickly leads to Big Data challenges. How can meaningful information be extracted from this large set of keystrokes and timestamps? How can it help assess the quality of LFS data collection? The presentation will describe some of the challenges that were encountered, solutions that were used to address them, and results of the analysis on data quality.
Release date: 2016-03-24 - Surveys and statistical programs – Documentation: 11-522-X201700014716Description:
Administrative data, depending on its source and original purpose, can be considered a more reliable source of information than survey-collected data. It does not require a respondent to be present and understand question wording, and it is not limited by the respondent’s ability to recall events retrospectively. This paper compares selected survey data, such as demographic variables, from the Longitudinal and International Study of Adults (LISA) to various administrative sources for which LISA has linkage agreements in place. The agreement between data sources, and some factors that might affect it, are analyzed for various aspects of the survey.
Release date: 2016-03-24 - Surveys and statistical programs – Documentation: 11-522-X20010016225Description:
The European Union Labour Forces Survey (LFS) is based on national surveys that were originally very different. For the past decade, under pressure from increasingly demanding users (particularly with respect to timeliness, comparability and flexibility), the LFS has been subjected to a constant process of quality improvement.
The following topics are presented in this paper:A. the quality improvement process, which comprises screening national survey methods, target structure, legal foundations, quality reports, more accurate and more explicit definitions of components, etc.;B. expected or achieved results, which include an ongoing survey producing quarterly results within reasonable time frames, comparable employment and unemployment rates over time and space in more than 25 countries, specific information on current political topics, etc.;C. continuing shortcomings, such as implementation delays in certain countries, possibilities of longitudinal analysis, public access to microdata, etc.; D. future tasks envisioned, such as adaptation of the list of ISCO and ISCED variables and nomenclatures (to take into account evolution in employment and teaching methods), differential treatment of structural variables and increased recourse to administrative files (to limit respondent burden), harmonization of questionnaires, etc.
Release date: 2002-09-12 - Surveys and statistical programs – Documentation: 62F0026M2001001Description:
This report describes the quality indicators produced for the 1998 Survey of Household Spending. It covers the usual quality indicators that help users interpret data, such as coefficients of variation, nonresponse rates, imputation rates and the impact of imputed data on the estimates. Added to these are various less often used indicators such as slippage rates and measures of the representativity of the sample for particular characteristics that are useful for evaluating the survey methodology.
Release date: 2001-10-15 - 5. Creation of an occupational surveillance system in Canada: Combining data for a unique Canadian study ArchivedSurveys and statistical programs – Documentation: 11-522-X19990015652Description:
Objective: To create an occupational surveillance system by collecting, linking, evaluating and disseminating data relating to occupation and mortality with the ultimate aim of reducing or preventing excess risk among workers and the general population.
Release date: 2000-03-02 - 6. An evaluation of data fusion techniques ArchivedSurveys and statistical programs – Documentation: 11-522-X19990015666Description:
The fusion sample obtained by a statistical matching process can be considered a sample out of an artificial population. The distribution of this artificial population is derived. If the correlation between specific variables is the only focus the strong demand for conditional independence can be weakened. In a simulation study the effects of violations of some assumptions leading to the distribution of the artificial population are examined. Finally some ideas concerning the establishing of the claimed conditional independence by latent class analysis are presented.
Release date: 2000-03-02 - Surveys and statistical programs – Documentation: 11-522-X19990015688Description:
The geographical and temporal relationship between outdoor air pollution and asthma was examined by linking together data from multiple sources. These included the administrative records of 59 general practices widely dispersed across England and Wales for half a million patients and all their consultations for asthma, supplemented by a socio-economic interview survey. Postcode enabled linkage with: (i) computed local road density; (ii) emission estimates of sulphur dioxide and nitrogen dioxides, (iii) measured/interpolated concentration of black smoke, sulphur dioxide, nitrogen dioxide and other pollutants at practice level. Parallel Poisson time series analysis took into account between-practice variations to examine daily correlations in practices close to air quality monitoring stations. Preliminary analyses show small and generally non-significant geographical associations between consultation rates and pollution markers. The methodological issues relevant to combining such data, and the interpretation of these results will be discussed.
Release date: 2000-03-02 - Surveys and statistical programs – Documentation: 11-522-X19980015035Description:
In a longitudinal survey conducted for k periods some units may be observed for less than k of the periods. Examples include, surveys designed with partially overlapping subsamples, a pure panel survey with nonresponse, and a panel survey supplemented with additional samples for some of the time periods. Estimators of the regression type are exhibited for such surveys. An application to special studies associated with the National Resources Inventory is discussed.
Release date: 1999-10-22
- Date modified: