Participation and Activity Limitation Survey 2006: Technical and Methodological Report

Warning View the most recent version.

Archived Content

Information identified as archived is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please "contact us" to request a format other than those available.

Survey description
Objectives
Background
Definition of disability
Overview of methodology
Sampling
Data sources
Error detection
Imputation
Estimation
Quality evaluation
Disclosure control
Data accuracy
Severity scale

Survey description

The 2006 Participation and Activity Limitation Survey (PALS) is a post-censal survey of adults and children whose everyday activities are limited because of a condition or health problem. A sample of those persons who answered "Yes" to the 2006 Census disability filter questions were chosen to participate in PALS. Approximately 39,000 adults and 9,000 children living in private, and some collective, households in the 10 provinces and 3 territories were selected to participate in the survey.  PALS focuses on the relationship between functional status, daily living activities and social participation by collecting data on the nature and severity of the activity limitations, and on the needs for assistive technology, social support and accommodation in all spheres of life.

The data were collected in the fall of 2006 and winter of 2007. The survey was last conducted in 2001. This report presents some basic information about the survey and an overview of the methodological and content changes between the 2001 and 2006 PALS. The major difference involves a change in coverage resulting from the inclusion of a number of Aboriginal communities, the addition of the three territories, and the modification to the definition of collective dwellings. There are also some changes to content.

As in the Health Activity Limitations Survey (HALS 1991) and the Participation Activity Limitation Survey (PALS 2001), census information is also used in conjunction with the PALS information to provide socio-economic details on the respondents. These variables are used to aid in painting a broader picture of the people in the PALS survey.

Objectives

The objective of PALS is to develop a comprehensive database on persons with disabilities in order to:

  1. assist social policy development by governments of all levels
  2. support research in the area of disability

PALS conceptualizes disability as activity limitations and participation restrictions associated with long-term physical or mental conditions or health-related conditions. 

These objectives are obtained by designing PALS to identify:

  • Canadians with an activity limitation;
  • the type and severity of activity limitations that they experience;
  • the difficulties and barriers that they may face;
  • costs incurred for assistive technology, human aids or medication due to the disability;
  • the degree to which persons with disabilities are able to obtain help of physical accommodations they need at home, at work, at school or in recreational activities;
  • aids and assistive technology that they need but do not have; and
  • whether they have faced discrimination due to their disability.

Background

In February 1981, (The International Year of the Disabled), The Special Committee on the Disabled and Handicapped published its report entitled "Obstacles". This report made a number of recommendations to various areas of the Federal Government. One of the recommendations was for Statistics Canada to produce data on persons with disabilities in Canada using survey and program data. In order to respond to the recommendation, the government requested Statistics Canada to develop a survey on Canadians with disabilities.

In 1986, Statistics Canada conducted the first Health and Activity Limitation Survey. A second survey took place in 1991. The survey was designed to identify Canadians with disabilities and to determine what limitations they experienced and barriers they faced.  In 2001, HALS was renamed as the Participation Activity and Limitations Survey (PALS). The PALS survey program builds on the groundwork laid by the 1986 and 1991 HALS. As with HALS, PALS is a joint effort by Human Resources and Social Development Canada (HRSDC) and Statistics Canada.

PALS data is used by disability and social policy analysts at all levels of government, as well as by associations for persons with disabilities and researchers working in the field of disability policy and programs. The federal, provincial and territorial governments released their common disability framework in 1998, calling for the promotion of greater inclusion of persons with disabilities in all aspects of Canadian society. Their 1998 report noted the importance of developing a reliable statistical database on disability and underlined the key role PALS would play in supporting policy development and research in this area.

Governments use the PALS data to plan programs and services for persons with disabilities in their jurisdictions and to predict likely rates of program take-up based on different eligibility criteria. Personal outcome indicators in the areas of education, employment and income are also key data requirements for the development and evaluation of their social, disability and income-support policies. Comparisons of these outcome indicators for persons with and without disabilities form an important aspect of policy analysis undertaken at the federal, provincial and territorial levels.

Definition of disability

Disability is an activity limitation or participation restriction associated with a physical or mental condition or health problem.

PALS uses the World Health Organization's (WHO) framework of disability provided by the International Classification of Functioning (ICF). This framework defines disability as the relationship between body structures and functions, daily activities and social participation, while recognizing the role of environmental factors.

The ICF is a multi-dimensional classification, encompassing both a medical and a social model of disability. The ICF is intended to have a number of applications as a statistical tool, a research tool, a clinical tool, a social policy tool, and as an educational tool.

For the purpose of PALS, persons with disabilities are those who reported difficulties with daily living activities, or who indicated that a physical or mental condition or health problem reduced the kind or amount of activities they could do. The respondents' answers to the disability questions represent their perception of the situation and are therefore subjective.

Overview of methodology

Target population

The PALS target population consists of all persons, adults and children, who have an activity limitation or a participation restriction associated with a physical or mental condition or health problem and who were living in Canada at the time of the Census.

This population included persons living in private and some collective households in the 10 provinces and the three territories. However, for operational reasons, the populations living on First Nations reserves, the residents of institutional and some non-institutional collectives were excluded. More precisely, the non-institutional collective dwellings excluded were military bases, Canadian Armed Forces vessels, merchant vessels and coast guard vessels, as well as campgrounds and parks.
 
In order for PALS to reach its target population, all persons who reported "yes" to either of the two disability filter questions on the 2006 Census of Population questionnaire were included in the survey frame. The Census filter questions are as follows:

1. Do you have any difficulty hearing, seeing, communicating, walking, climbing stairs, bending, learning or doing any similar activities?

  1. Yes, sometimes
  2. Yes, often
  3. No

2a. Does a physical condition or mental condition or health problem reduce the amount or the kind of activity you can do at home?

  1. Yes, sometimes
  2. Yes, often
  3. No

2b. Does a physical condition or mental condition or health problem reduce the amount or the kind of activity you can do at work or at school?

  1. Yes, sometimes
  2. Yes, often
  3. No

2c. Does a physical condition or mental condition or health problem reduce the amount or the kind of activity you can do in other activities, for example, transportation or leisure?

  1. Yes, sometimes
  2. Yes, often
  3. No

From this frame, a sample of individuals was selected for the PALS interview. The subset of the surveyed population that also reports disabilities in PALS is considered as the target population. In other words, according to PALS, a person with a disability is defined as a respondent who answers:

YES to a disability filter questions on Census, and

YES to disability filter questions in PALS, or
YES to detailed questions on activity limitations in PALS

Changes to PALS target population

The target population in 2006 differed slightly from that in 2001. In 2006, the territories were included in the target population. In addition, in 2001, the population living in Aboriginal communities was covered by the Aboriginal Peoples Survey (APS) and was thus excluded from the 2001 PALS target population. In 2006, these Aboriginal communities were included in the PALS target population.

Furthermore, the method of collecting information in the senior citizen residences that are non-institutional collective dwellings was modified slightly in the 2006 Census. Prior to this, people living in these residences received only short forms for the Census. Since then, modifications to this process have been made and now one in every five households in these senior residences receives a Census long form; a rate comparable to regular private dwellings. Consequently, these collective dwellings are now included as part of the PALS target population.

It should be noted that since comparisons between 2001 PALS and 2006 PALS results were a key objective in 2006, Statistics Canada derived a historic variable based on the 2001 target population. PALS users will now have the option to directly compare results between 2001 and 2006 but also study the new 2006 target population.

In total, these changes in the target population represent an increase of 1.2% in the number of people included in PALS 2006 (see Table 1). This increase differs slightly from one province to another due to the addition of the Aboriginal communities and the concentration of Aboriginal Peoples in some provinces.

Table 1 People covered by the 2006 Participation and Activity Limitation Survey and percentage increase resulting from the changes between the two populations by province. Opens a new browser window.

Table 1 People covered by the 2006 Participation and Activity Limitation Survey and percentage increase resulting from the changes between the two populations by province

Instrument design

Many different stakeholders were consulted during the development of the adult and children questionnaires for the 2006 PALS. A review of the questionnaires from both the 2001 PALS and the 1991 Health and Activity Limitation Survey (HALS) were conducted. Furthermore, consultation with the client HRSDC, federal and provincial governments and community associations were also held to obtain input for the 2006 survey.

Along with these consultations, several rounds of qualitative testing were conducted in order to test the content of the two questionnaires. This testing took place between 2004 and 2006. Additionally, a pilot test, in both official languages, was conducted in the spring of 2006. This allowed PALS staff to make final changes to the survey content as well as test different aspects of the data collection.

Sampling

Sample and stratification design

The sample design used for PALS 2006 was a two-phase stratified design based on the 2006 Census. In Phase 1, the census itself, the long form was systematically distributed to approximately every fifth household across Canada. Phase 2 involved the selection of individuals who reported an activity limitation during Phase 1 based on various characteristics defining the strata.

The strata were defined in order to ensure large enough samples in the domain estimates and to optimize the sample allocation. Therefore, since one of the survey objectives was to allow for statistical profile dissemination of individuals with a disability by province/territory and of various age groups in the population, these domain estimates were considered in the development of the strata. For the provinces, the domain estimates considered were made through the intersection of the province and the following age groups:

  • younger than 15 years old
  • 15 to 24 years old
  • 25 to 44 years old
  • 45 to 64 years old
  • 65 to 74 years old
  • 75 years and over

The domain estimates for children and adults in the territories were made differently. The estimates for the children were obtained by combining the three territories whereas the estimates for the adults were obtained by separating the three territories.

Furthermore, for a more optimal sample allocation, the severity of the respondent's disability was also included as a stratification variable. Individuals who are severely limited answered "yes, often" at least once to the filter questions in the Census. Mildly limited individuals answered "yes, sometimes" at least once to the filter questions in the Census but never answered "yes, often." The final variable considered in the construction of the strata was probability of selection in Phase 1.  Including this variable in the stratification therefore made it possible to minimize the variability of the initial weight of individuals selected in the same domain and therefore optimizing the sample allocation.

Sample allocation method

Sample distribution was performed in a way that, for each domain, a minimum proportion is guaranteed with a maximum coefficient of variation (CV) of 16.5%. At Statistics Canada, 16.5% corresponds to the upper limit of a CV in order to be able to effectively qualify the corresponding estimate. Among children aged 0 to 14 years, the minimum proportion to estimate was set to 8%. Among young adults (15 to 24), this proportion was set to 9%. Among adults aged 25 to 44 years and 45 to 64 years, this proportion was set to 7.5%. Finally, the proportion was set to 11% for adults aged 65 to 74 years and 75 years and older.

Changes to PALS sample design and sample allocation

The sample design used for the 2006 PALS differed slightly from the 2001 PALS. Due to improvements in the processing of Census data, 2006 data was available in electronic format earlier than it was in 2001. This earlier access made it possible to directly select PALS 2006 respondents from the Census database. This was an advantage that was not available during the 2001 survey. In 2001, respondents were selected directly from the questionnaire boxes making it necessary for the design and allocation to reflect this constraint.

The 2001 PALS sample design was a two-stage stratified sample. At the first stage, Census enumeration area (EA) was selected using probability proportional to size sampling. The second stage involved the selection of individuals according to their characteristics (stratum formed by the province and the age group) It is clear that the efficiency of the 2006 PALS sample design exceeds that of the 2001 PALS allowing for a superior geographic distribution and allocation of the sample.

Nevertheless, it is important to point out that despite the changes made to the PALS sample design and allocation, the comparability of the two surveys is not affected.  In fact, the only impact of these changes affects the probability of the selection of each individual in PALS and therefore, the respondents' weight. When using weighted estimates, the changes made have no impact on the results of the survey.

Sample size

The total size of the PALS 2006 sample was 47,793: 8,954 children (persons under 15 years of age) and 38,839 adults (15 years of age and over). In 2001, 43,276 individuals were selected for PALS. The increase in the 2006 PALS sample size will facilitate the in-depth analysis of issues concerning people with disabilities in the territories and provinces.

Data sources

PALS data collection took place between October 30 2006 and February 28 2007. The interviews were conducted by telephone with the interviewers completing a computer-assisted questionnaire. Because of the numerous advantages it offers, PALS was conducted for the first time by computer assisted interviewing (CAI). It has been shown through other surveys that Computer assisted telephone interview (CATI) improves accuracy of data collected in telephone surveys. It allows interviewers to more easily follow the path in complex questionnaires by taking the interviewers to the next appropriate question based on answers provided by the respondents. Furthermore, the answers given by the respondents are directly captured during the interview making them available electronically immediately. In addition, procedures of quality control can be incorporated directly into the application.

For the adult interviews, the respondent targeted was the selected person and for the child interviews, it was the parent or guardian of the child. However, proxy interviews were allowed in those situations where an adult respondent was unable to answer the questions over the telephone. Further details surrounding the issue of proxy interviews are provided in the following section.

Error detection

The first phase of error detection was carried out during data collection. Edit rules were incorporated in the CATI system to ensure that data capture errors and inconsistencies were reduced. Secondly, the interviewing supervisors observed interviews and reviewed the completed questionnaires in order to identify inconsistencies. Any inconsistencies were discussed with the interviewer who had conducted the interview and the respondents were called back if required.

The second phase of error detection was conducted during data processing and is referred to as the editing phase. Edit rules were developed to validate that respondents followed the right path in the questionnaires and to identify and correct inconsistencies between responses within each section of the questionnaires. For the majority of cases with inconsistencies, an automated correction was specified. This is discussed in greater detail in the Imputation Section. Once this step was completed, a macro verification was conducted by analyzing frequency distributions to identify anomalies such as missing categories or unusually large frequencies.

Imputation

For PALS, a valid response was deterministically imputed for the missing responses if sufficient information was available in the related questions. Otherwise, it was coded to "Not asked". In addition, the questions that were not to be asked were coded to "Valid skip". If a question with a missing answer (coded to "Not asked") should have been used to determine if subsequent questions were to be asked, these subsequent questions were set to "Not stated", because it was not possible to determine whether the question should have been asked.

A non-response is not permitted for demographic information that is required for weighting. The information on age and sex of the respondent is important for later data analysis. In addition, this information assisted in the verification that the correct person had been reached for the interview. These two variables were imputed from the census if the data was missing or invalid. In particular, an age was considered invalid if it was not consistent with the questionnaire that had been administered.

Estimation

In a sample survey, each respondent represents not only himself or herself, but also other people that were not sampled. Thus, a weight is assigned to each respondent creating a representative for a number of people. In order to maintain consistency, this weight must be used for all estimations.

The weight is calculated in a three-stage process. The FIRST stage involves the assignment of an initial weight based on the sampling design. The initial weight is the inverse of the inclusion probability. For the 2006 PALS, the initial weight is the product of two components: the census weight and the subsampling weight (the inverse of the sampling fraction in the second phase). Following this calculation, appropriate weight adjustments are applied.

The SECOND stage of the weighting process includes the adjustment for non-response. More specifically, two adjustments are made at this point. It should be noted that non-respondents can be classified into one of two main categories with very different characteristics: the people not contacted and the people contacted but who did not respond.

The weights are adjusted first for non-contact and then for non-response. As the adjustment method is the same for both types of non-respondents, it is described here only for non-response. The non-response adjustment is done by forming non-response adjustment classes in such a way that the records in each class have similar response probabilities. The estimated response probabilities are obtained by developing a logistic regression model to predict the response probability using explanatory variables.

Many explanatory variables can be used in order to model the probability of response. Given that PALS is a post-censal survey, all long form Census information is available for each respondent and non-respondent. Furthermore, other important variables to predict response are available. Such data collection information as the number of attempts to contact a household and the time and day of the attempts can be used as well. Separate models are used for children and adults. Different classes are then created based on response probabilities and a minimum number of respondents by class. The inverse of the weighted response rate in a class is used as the weighting adjustment factor. The initial weights of the respondents within the class are adjusted accordingly.

The THIRD stage of the weighting adjustment is the post-stratification. This adjustment ensures that the sum of the final weights for the respondents is equal to the population counts obtained from the census. This adjustment is made for groups (post-strata) defined by the combinations of different variables for which this adjustment is important. Examples include province, sex, age group and severity of the limitation reported in the census. The weights corrected for non-response are then adjusted using the ratio of the census count to the sample count for each post-stratum.

Since estimates are obtained from a sample as opposed to a census, estimates will vary from sample to sample referred to as sampling error. The bootstrap method is used to provide estimates of sampling error for statistics with PALS data. This resampling method selects 1,000 subsamples (with replacement) from the main sample. Each subsample is then weighted by calculating the initial weights and applying the same adjustments applied to the main sample weights, i.e. adjustments for non-response and post-stratification. The sampling error is measured and estimated by the bootstrap variance which is the empirical variance of the desired statistic calculated from the main sample and the 1,000 bootstrap subsamples.

Quality evaluation

There are two types of errors that occur in surveys; sampling and non-sampling errors. Unlike sampling errors, non-sampling errors are not explained by sample-to-sample variability and can not be quantified. These errors may occur at any step of the survey process. The various measures outlined below were adopted to minimize these errors in PALS 2006.

As mentioned previously, a pilot test was conducted seven months before the survey. During this test, all survey processes were evaluated, including the questionnaire content, the computer assisted questionnaire and the data processing method.

High response rates are essential for quality data. To reduce the number of non-response cases, the interviewers were provided training by experienced Statistics Canada training staff. In conjunction with the training, detailed interviewer manuals were provided as a reference. Furthermore, all of the interviewers were under the direction of interviewer supervisors. When the need arose, refusals were followed up by senior interviewers to encourage respondents to participate in the survey.

In addition, measures were taken in order to identify and correct errors that could result from misinterpretation of a question by the respondent or a wrong flow in the questionnaire. The questionnaires were first reviewed by the interviewer supervisor. A detailed set of edit rules were then used during data processing to identify and correct any inconsistencies between the responses provided. These edit rules were exhaustively tested before being applied to the data.

Disclosure control

Statistics Canada is prohibited by law from releasing any data that would divulge information obtained under the Statistics Act that relates to any identifiable person, business or organization without the prior knowledge or the consent in writing of that person, business or organization. Various confidentiality rules are applied to all data that are released or published to prevent the publication or disclosure of any information deemed confidential. If necessary, data are suppressed to prevent direct or residual disclosure of identifiable data.

Data accuracy

The data accuracy measure used for each table produced is the estimated standard error of the estimate (sampling error measure), which is the square root of the estimated sampling variance of the estimate. However, the estimated standard error is usually expressed relative to the estimate to which it pertains, and the resulting measure is the estimated coefficient of variation (CV).

The estimated CV is obtained by dividing the estimated standard error of the estimate by the estimate itself and is expressed as a percentage of the estimate. For PALS, all estimated CVs will be obtained using the bootstrap method described in the ESTIMATION Section.

At Statistics Canada, we quantify the accuracy of an estimate by the CV. A small CV implies a small variability in the sample and, consequently, better quality of the estimate. PALS uses the following measurements:

(i) When the CV is greater than 33.3%, the estimate is considered "unacceptable";

(ii) When the CV is greater than 16.5% and less than or equal to 33.3%, the estimate is considered "poor" and must be used with caution;

(iii) when the CV is 16.5% or less, the estimate is considered "acceptable" and can be use without any restrictions.

The ideal case would be to produce estimates with an associated CV of 16.5 or less.

Severity scale

A disability severity index was developed using questions for each type of disability in the PALS questionnaires. At first, a standardized score for each type of disability was calculated based on severity, the maximum score given for someone who reports being completely disabled for a given disability. Questions on intensity and frequency of the limitation were used in order to determine the severity of the disability. For example, a maximum score was given in a situation where someone reported being completely unable to take part in an activity because of a disability and this difficulty was always present.

Next, an overall score of severity was calculated taking the average of all standardized severity scores calculated for each type of disability. Due to the strong relationship between learning difficulties and developmental disability, only the score given to the developmental disability was taken into account in the overall score for respondents reporting both disabilities.

Finally, after discussion with data users, it was decided that the severity scale should be divided into four severity classes. These were created by examining the distribution of the global severity score. In the first step, an attempt was made to identify a "natural cut-off" point in the scale. This cut-off point corresponds to the 70th percentile and is close to a score of 1/9 for the adults and 1/8 for the children. Since these particular scores correspond to the score of someone with a maximum score for one type of disability, it was decided to subdivide the scale into two parts. The two groups were then subdivided again into two parts consisting of four other classes. These two new cut-off points are equivalent to half and twice the maximum score obtained for one disability.

These classes are defined as:

Class 1: Respondents with a score equivalent to less than half the maximum score for one disability.

Class 2: Respondents with an equivalent score between half and the maximum score for one disability.

Class 3: Respondents with an equivalent score between one and twice the maximum score for one disability.

Class 4: Respondents with a score equivalent to more than twice the maximum score for one disability.

In light of the relatively subjective nature of this classification and in order to avoid any misinterpretation, it is preferable not to use specific terms to characterize the classes. The interpretation of the measurement tool is as follows: persons in Class 4 have a more severe disability than persons in Class 3, who in turn have a more severe disability than persons in Class 2, and so forth. However, for practical purposes, names of "mild," "moderate," "severe" and "very severe" were assigned to the classes 1 through 4. It should be noted that there is no judgment associated with the use of this terminology.

Because questions differ according to a child's age, two different scales were created, one for children aged 0 to 4 and another for children aged 5 to 14. Taking into account there are only 4 types of disabilities measured for children aged 0 to 4, only two severity classes were created. The first was labelled as "mild to moderate" and the second was labelled "severe to very severe".

Further technical details regarding how severity scales were derived can be found in appendix A and B.


You need to use the free Adobe Reader to view PDF documents. To view (open) these files, simply click on the link. To download (save) them, right-click on the link. Note that if you are using Internet Explorer or AOL, PDF documents sometimes do not open properly. See Troubleshooting PDFs. PDF documents may not be accessible by some devices. For more information, visit the Adobe website or contact us for assistance.