Appendix A Concepts and methods

View the most recent version.

Objectives

A wealth of information on workers' outcomes regarding wages and wage inequality, job stability and layoffs, training, job creation and unemployment can be gathered from a wide variety of surveys. In general, researchers have a good understanding of employee and employer outcomes, but a link between these two levels of analysis using micro-data simply does not exist. The WES is the only source of data in the country that allows people to investigate changes that occur among employees and link the changes to events taking place in firms and vice versa. Indeed, such a connection is necessary if one hopes to understand the association between labour market changes and pressures stemming from global competition, organizational and technological changes, and the drive to improve human capital.

Hence, the primary gal of the WES is to establish a link between events occurring in workplaces and the outcomes for workers. The second goal of the survey is to develop a better understanding of the forces shaping companies in this era of substantial change.

The WES is, by design, composed of two components:

• a workplace survey covering subjects such as the adoption of technologies, organizational change, training and other human resource practices, business strategies, and labour market dynamics, to name a few; and
• a survey of employees within these same workplaces covering wages, hours of work, job type, human capital, use of technologies and training.

The diagram below presents the link between workplace and employee characteristics and workplace and employee outcomes.

Figure 1: The workplace and employee survey conceptual framework

Description

Sample sizes and response rates

The WES was conducted for the first time during the summer (employer survey portion) and fall (employee survey portion) of 1999. Overall, 6,351 workplaces and 24,597 employees responded to the survey, representing response rates of 94% and 83%, respectively. The sampled locations are followed over time, with the periodic addition of new locations to the sample to maintain a representative cross section. Employees are followed for two years; a fresh sample of employees is drawn on every second survey occasion (i.e., first, third, fifth). This longitudinal aspect allows researchers to study both employer and employee outcomes over time in the evolving workplace. For 2005, Table A.1 provides sample sizes and estimated population and Table A.2 provides the achieved response rates.

Target population

The target population for the employer component is defined as all business locations operating in Canada that have paid employees, with the following exceptions:

• employers in Yukon and Northwest Territories
• employers operating in crop production and animal production; fishing, hunting and trapping; religious organizations; private households; and public administration.

The target population for the employee component is all employees working in the targeted workplaces who receive a Canada Revenue Agency T-4 Supplementary form.

Survey population

The survey population is the collection of all units for which the survey can realistically provide information. The survey population may differ from the target population because of operational difficulties in identifying all the units that belong to the target population.

The WES draws its sample from the Business Register maintained by Statistics Canada, and from lists of employees provided by the surveyed employers. The Business Register, a list of all businesses in Canada, is updated each month using data from various surveys, business profiles and administrative sources.

Reference period

The reference period for the WES was mainly the 12-month period ending in March of the survey year. Some questions in the workplace portion covered the last pay period of March of the survey year.

Sample design

Two frames are used in the WES. The survey frame, a list of all relevant units, is used for sample design and selection; ultimately, it provides contact information for the selected units.

Workplace component of survey

The survey frame for the workplace component of the WES was created from the information available on Statistics Canada's Business Register.

Prior to sample selection, the business locations on the frame were separated into relatively homogeneous groups called strata, which were then used for sample allocation and selection. The WES frame was stratified by industry (14) and region (6), as well as size (3), which was defined using estimated employment. The size stratum boundaries were typically different for each industry and region combination. The cut-off points defining a stratum of a particular size were computed using a model-based approach. The sample was selected using Neyman allocation.

All sampled units were assigned a sampling weight (a raising factor attached to each sampled unit to obtain estimates for the population from a sample). For example, if 2 units were selected at random and with equal probability out of a population of 10 units, then each selected unit would represent 5 units in the population, and it would have a sampling weight of 5.

The first WES survey collected data from 6,351 out of the 9,144 sampled employers. The remaining employers were a combination of workplaces determined to be out of business, seasonally inactive, holding companies, or out of scope. The majority of non-respondents were owner–operators who had no paid help and were in possession of a payroll deduction account.

Employee component of survey

The frame for the employee component of the WES was based on lists of employees made available to interviewers by the selected workplaces. A maximum of 24 employees were sampled using a probability mechanism. In workplaces with three or four employees, all employees were selected.

Data collection

Data collection, data capture, preliminary editing and follow-up of non-respondents were all done in Statistics Canada regional offices. Interviewers collected the workplace and employee survey data through computer-assisted telephone interviews.

Statistical edit and imputation

In the WES, great care is taken to prevent errors or incorrectly recorded values during the data collection process. This is accomplished via extended input editing in the computer questionnaire application. Following collection, the data are analysed extensively and ratio editing is used to determine outlying observations based on robust outlier detection programs.

Respondents who opt not to participate in the survey—represented by total non-response—are removed and the weights of the remaining units are adjusted upward to keep the sample representative. For respondents who do not provide all required fields—represented by item non-response—a statistical technique called imputation is used to fill in the missing values for both employers and employees. Four imputation methods are used: weighted hot deck, trend, ratio and deterministic.

Estimation

The reported (or imputed) values for each workplace and employee in the sample are multiplied by the weight for that workplace or employee; these weighted values are summed up to produce estimates. An initial weight equal to the inverse of the original probability of selection is assigned to each unit. The initial survey weights are calibrated to agree with known population totals. These adjusted weights are then used in forming estimates of means or totals of variables collected by the survey.

Variables for which population totals are known are called auxiliary variables. They are used to calibrate survey estimates to increase their precision. Each business location is calibrated to known population totals at the industry and region level. The auxiliary variable used for the WES is total employment obtained from the Survey of Employment, Payrolls and Hours.

Data quality

Any survey is subject to errors. Whereas considerable effort is made to ensure a high standard throughout all survey operations, the resulting estimates are inevitably subject to a certain degree of error. Errors can arise because of the use of a sample instead of a complete census, from mistakes made by respondents or interviewers during the collection of data, from errors made in keying in the data, from imputation of a consistent but not necessarily correct value, or from other sources.

Sampling errors

The true sampling error is unknown; however, it can be estimated from the sample itself by using a statistical measure called the standard error. When the standard error is expressed as a percentage of the estimate, it is known as the relative standard error or coefficient of variation.

Non-sampling errors

Some non-sampling errors will cancel out over many observations, but systematically occurring errors (those that do not tend to cancel) will contribute to a bias in the estimates. For example, if respondents consistently tend to underestimate their sales, then the resulting estimate of the total sales will be below the true population total. Such a bias is not reflected in the estimates of standard error. As the sample size increases, the sampling error decreases. However, this is not necessarily true for the non-sampling error.

Coverage errors

Coverage errors arise when the survey frame does not adequately cover the target population. As a result, certain units belonging to the target population are either excluded (under-coverage), or counted more than once (over-coverage). In addition, out-of-scope units may be present on the survey frame (over-coverage).

Response errors

Response errors occur when a respondent provides incorrect information because of misinterpretation of the survey questions or lack of correct information, gives wrong information by mistake, or is reluctant to disclose the correct information. Gross response errors are likely to be caught during editing, but others may simply go through undetected.

Non-response errors

Non-response errors can occur when a respondent does not respond at all (total non-response) or responds only to some questions (partial non-response). These errors can have a serious impact on estimates if the non-respondents are systematically different from the respondents in survey characteristics or the non-response rate is high or both.

Processing errors

Errors that occur during the processing of data represent another component of the non-sampling error. Processing errors can arise during data capture, coding, editing, imputation, outlier treatment and other types of data handling. A coding error occurs when a field is coded erroneously because of misinterpretation of coding procedures or bad judgement. A data capture error occurs when data are misinterpreted or keyed in incorrectly.

Joint interpretation of measures of error

The measure of non-response error and the coefficient of variation must be considered jointly to assess the quality of the estimates. The lower the coefficient of variation and the higher the response fraction, the better the published estimate will be.

Confidentiality

The information presented in this publication has been reviewed to ensure that the confidentiality of individual responses is respected. Any estimate that could reveal the identity of a specific respondent is declared confidential, and consequently not published.

Response and non-response

The response rate includes all units that responded by providing usable information during the collection phase.

The refusal rate includes those units that were contacted but refused to participate in the survey.