1.0 Description

The Canadian Perspectives Survey Series (CPSS) is a set of short, online surveys beginning in March 2020 that will be used to collect information on the knowledge and behaviours of residents of the 10 Canadian provinces. All surveys in the series will be asked of Statistics Canada's probability panel. The probability panel for the CPSS is a new pilot project initiated in 2019. An important goal of the CPSS is to directly collect data from Canadians in a timely manner in order to inform policy makers and be responsive to emerging data needs. The CPSS is designed to produce data at a national level (excluding the territories).

The survey program is sponsored by Statistics Canada. Each survey in the CPSS is cross sectional. Participating in the probability panel and the subsequent surveys of the CPSS is voluntary.

The fourth survey of the CPSS is CPSS4 – Information Sources Consulted During the Pandemic. It was administered from July 20, 2020 until July 26, 2020.

Any questions about the survey, the survey series, the data or its use should be directed to:

Statistics Canada

Client Services
Centre for Social Data Integration and Development
Telephone: 613-951-3321 or call toll-free 1-800-461-9050
Fax: 613-951-4527
E-mail: statcan.csdidclientservice-ciddsservicealaclientele.statcan@statcan.gc.ca

2.0 Survey methodology

Target and survey population

The target population for the Canadian Perspectives Survey Series (CPSS) is residents of the 10 Canadian provinces 15 years of age or older.

The frame for surveys of the CPSS is Statistics Canada's pilot probability panel. The probability panel was created by randomly selecting a subset of the Labour Force Survey (LFS) respondents. Therefore the survey population is that of the LFS, with the exception that full-time members of the Canadian Armed Forces are included. Excluded from the survey's coverage are: persons living on reserves and other Aboriginal settlements in the provinces; the institutionalized population, and households in extremely remote areas with very low population density. These groups together represent an exclusion of less than 2% of the Canadian population aged 15 and over.

The LFS sample is drawn from an area frame and is based on a stratified, multi-stage design that uses probability sampling. The LFS uses a rotating panel sample design. In the provinces, selected dwellings remain in the LFS sample for six consecutive months. Each month about one-sixth of the LFS sampled dwellings are in their first month of the survey, one-sixth are in their second month of the survey, and so on. These six independent samples are called rotation groups.

For the probability panel used for the CPSS, four rotation groups from the LFS were used from the provinces: the rotation groups answering the LFS for the last time in April, May, June and July of 2019. From these households, one person aged 15+ was selected at random to participate in the CPSS - Sign-Up. These individuals were invited to Sign-Up for the CPSS. Those agreeing to join the CPSS were asked to provide an email address. Participants from the Sign-Up that provided valid email addresses formed the probability panel. The participation rate for the panel was approximately 23%. The survey population for all surveys of the CPSS is the probability panel participants. Participants of the panel are 15 years or older as of July 31, 2019.

Sample Design and Size

The sample design for surveys of the CPSS is based on the sample design of the CPSS – Sign-Up, the method used to create the pilot probability panel. The raw sample for the CPSS – Sign-Up had 31,896 randomly selected people aged 15+ from responding LFS households completing their last interview of the LFS in April to July of 2019. Of these people, 31,626 were in-scope at the time of collection for the CPSS - Sign-Up in January to March 2020. Of people agreeing to participate in the CPSS, that is, those joining the panel, 7,242 had a valid email address. All panel participants are invited to complete the surveys of the CPSS.

3.0 Data collection

CPSS – Sign-Up

The CPSS- Sign-Up survey used to create Statistics Canada's probability panel was conducted from January 15th, 2020 until March 15th, 2020. Initial contact was made through a mailed letter to the selected sample. The letter explained the purpose of the CPSS and invited respondents to go online, using their Secure Access Code to complete the Sign-Up form. Respondents opting out of joining the panel were asked their main reason for not participating. Those joining the panel were asked to verify basic demographic information and to provide a valid email address. Nonresponse follow-up for the CPSS-Sign-Up had a mixed mode approach. Additional mailed reminders were sent to encourage sampled people to respond. As well, email reminders (where an email address was available) and Computer Assisted Telephone Interview (CATI) nonresponse follow-up was conducted.

The application included a standard set of response codes to identify all possible outcomes. The application was tested prior to use to ensure that only valid question responses could be entered and that all question flows would be correctly followed. These measures ensured that the response data were already "clean" at the end of the collection process.

Interviewers followed a standard approach used for many StatCan surveys in order to introduce the agency.  Selected persons were told that their participation in the survey was voluntary, and that their information would remain strictly confidential.

CPSS4 – Information Sources Consulted During the Pandemic

All participants of the pilot panel for the CPSS, minus those who opted out after previous iterations of CPSS, were sent an email invitation with a link to the CPSS4 and a Secure Access Code to complete the survey online. Collection for the survey began on July 20th, 2020. Reminder emails were sent on July 21st, July 23rd and July 25th. The application remained open until July 26th, 2020.

3.1 Disclosure control

Statistics Canada is prohibited by law from releasing any data which would divulge information obtained under the Statistics Act that relates to any identifiable person, business or organization without the prior knowledge or the consent in writing of that person, business or organization. Various confidentiality rules are applied to all data that are released or published to prevent the publication or disclosure of any information deemed confidential. If necessary, data are suppressed to prevent direct or residual disclosure of identifiable data.

4.0 Data quality

Survey errors come from a variety of different sources. They can be classified into two main categories: non-sampling errors and sampling errors.

4.1 Non-sampling errors

Non-sampling errors can be defined as errors arising during the course of virtually all survey activities, apart from sampling. They are present in both sample surveys and censuses (unlike sampling error, which is only present in sample surveys). Non-sampling errors arise primarily from the following sources: nonresponse, coverage, measurement and processing.

4.1.1 Nonresponse

Nonresponse errors result from a failure to collect complete information on all units in the selected sample.

Nonresponse produces errors in the survey estimates in two ways. Firstly, non-respondents often have different characteristics from respondents, which can result in biased survey estimates if nonresponse bias is not fully corrected through weighting. Secondly, it reduces the effective size of the sample, since fewer units than expected answered the survey. As a result, the sampling variance increases and the precision of the estimate decreases. The response rate is calculated as follows:

[ Responding units / (Selected units – out-of-scope units) ] x 100%

The following table summarizes the response rates experienced for the CPSS4 – Information Sources Consulted During the Pandemic. Response rates are broken down into two stages. Table 4.1.1a shows the take-up rates to the panel in the CPSS- Sign-Up and Table 4.1.1b shows the collection response rates for the survey CPSS4 – Information Sources Consulted During the Pandemic.

As shown in Table 4.1.1b, the collection response rate for the CPSS4 – Information Sources Consulted During the Pandemic is 58.2%. However, when nonparticipation in the panel is factored in, the cumulative response rate to the survey is 13.3%. This cumulative response rate is lower than the typical response rates observed in social surveys conducted at Statistics Canada. This is due to the two stages of nonresponse (or participation) and other factors such as the single mode used for surveys of the CPSS (emailed survey invitations with a link to the survey for online self-completion), respondent fatigue from prior LFS response, the inability of the offline population to participate, etc.,.

Given the additional nonresponse experienced in the CPSS4 – Information Sources Consulted During the Pandemic there is an increased risk of bias due to respondents being different than nonrespondents. For this reason, a small bias study was conducted. Please see Section 6.0 for the results of this validation.

4.1.2 Coverage errors

Coverage errors consist of omissions, erroneous inclusions, duplications and misclassifications of units in the survey frame. Since they affect every estimate produced by the survey, they are one of the most important types of error; in the case of a census they may be the main source of error. Coverage errors may cause a bias in the estimates and the effect can vary for different sub-groups of the population. This is a very difficult error to measure or quantify accurately.

For the CPSS, the population covered are those aged 15+ as of July 31, 2019. Since collection of the CPSS4 – Information Sources Consulted During the Pandemic was conducted from July 20th-26th, 2020, there is an undercoverage of residents of the 10 provinces that turned 15 since July 31, 2019. There is also undercoverage of those without internet access. This undercoverage is greater amongst those age 65 years and older.

4.1.3 Measurement errors

Measurement errors (sometimes referred to as response errors) occur when the response provided differs from the real value; such errors may be attributable to the respondent, the questionnaire, the collection method or the respondent's record-keeping system. Such errors may be random or they may result in a systematic bias if they are not random. It is very costly to accurately measure the level of response error and very few surveys conduct a post-survey evaluation.

4.1.4 Processing errors

Processing errors are the errors associated with activities conducted once survey responses have been received. It includes all data handling activities after collection and prior to estimation. Like all other errors, they can be random in nature, and inflate the variance of the survey's estimates, or systematic, and introduce bias. It is difficult to obtain direct measures of processing errors and their impact on data quality especially since they are mixed in with other types of errors (nonresponse, measurement and coverage).

4.2 Sampling errors

Sampling errors are defined as the errors that result from estimating a population characteristic by measuring a portion of the population rather than the entire population. For probability sample surveys, methods exist to calculate sampling error. These methods derive directly from the sample design and method of estimation used by the survey.

The most commonly used measure to quantify sampling error is sampling variance. Sampling variance measures the extent to which the estimate of a characteristic from different possible samples of the same size and the same design differ from one another. For sample designs that use probability sampling, the magnitude of an estimate's sampling variance can be estimated.

Factors affecting the magnitude of the sampling variance for a given sample size include:

1. The variability of the characteristic of interest in the population: the more variable the characteristic in the population, the larger the sampling variance.
2. The size of the population: in general, the size of the population only has an impact on the sampling variance for small to moderate sized populations.
3. The response rate: the sampling variance increases as the sample size decreases. Since non-respondents effectively decrease the size of the sample, nonresponse increases the sampling variance.
4. The sample design and method of estimation: some sample designs are more efficient than others in the sense that, for the same sample size and method of estimation, one design can lead to smaller sampling variance than another.

The standard error of an estimator is the square root of its sampling variance. This measure is easier to interpret since it provides an indication of sampling error using the same scale as the estimate whereas the variance is based on squared differences.

The coefficient of variation (CV) is a relative measure of the sampling error. It is defined as the estimate of the standard error divided by the estimate itself, usually expressed as a percentage (10% instead of 0.1). It is very useful for measuring and comparing the sampling error of quantitative variables with large positive values. However, it is not recommended for estimates such as proportions, estimates of change or differences, and variables that can have negative values.

It is considered a best practice at Statistics Canada to report the sampling error of an estimate through its 95% confidence interval. The 95% confidence interval of an estimate means that if the survey were repeated over and over again, then 95% of the time (or 19 times out of 20), the confidence interval would cover the true population value.

5.0 Weighting

The principle behind estimation in a probability sample such as those of the CPSS, is that each person in the sample "represents", besides himself or herself, several other persons not in the sample. For example, in a simple random 2% sample of the population, each person in the sample represents 50 persons in the population. In the terminology used here, it can be said that each person has a weight of 50.

The weighting phase is a step that calculates, for each person, his or her associated sampling weight. This weight appears on the microdata file, and must be used to derive estimates representative of the target population from the survey. For example, if the number of individuals who smoke daily is to be estimated, it is done by selecting the records referring to those individuals in the sample having that characteristic and summing the weights entered on those records. The weighting phase is a step which calculates, for each record, what this number is. This section provides the details of the method used to calculate sampling weights for the CPSS4 – Information Sources Consulted During the Pandemic.

The weighting of the sample for the CPSS4 – Information Sources Consulted During the Pandemic has multiple stages to reflect the stages of sampling, participation and response to get the final set of respondents. The following sections cover the weighting steps to first create the panel weights, then the weighting steps to create the survey weights for CPSS4 – Information Sources Consulted During the Pandemic.

5.1 Creating the Panel Weights

Four consecutive rotate-out samples of households from the LFS were the starting point to form the panel sample of the CPSS. Since households selected from the LFS samples are the starting point, the household weights from the LFS are the first step to calculating the panel weights.

5.1.1 Household weights

Calculation of the Household Design Weights – HHLD_W0, HHLD_W1

The initial panel weights are the LFS subweights (SUBWT). These are the LFS design weights adjusted for nonresponse but not yet calibrated to population control totals. These weights form the household design weight for the panel survey (HHLD_W0).

Since only four rotate-outs were used, instead of the six used in a complete LFS sample, these weights were adjusted by a factor of 6/4 to be representative. The weights after this adjustment were called HHLD_W1.

Calibration of the Household Weights – HHLD_W2

Calibration is a step to ensure that the sum of weights within a certain domain match projected demographic totals. The SUBWT from the LFS are not calibrated, thus HHLD_W1 are also not calibrated. The next step is to make sure the household weights add up to the control totals by household size. Calibration was performed on HHLD_W1 to match control totals by province and household size using the size groupings of 1, 2, or 3+.

5.1.2 Person Panel weights

Calculate Person Design Weights – PERS_W0

One person aged 15 or older per household was selected for the CPSS – Sign-Up, the survey used to create the probability panel. The design person weight is obtained by multiplying HHLD_W2 by the number of eligible people in the dwelling (i.e. number of people aged 15 years and over).

Removal of Out of Scope Units – PERS_W1

Some units were identified as being out-of-scope during the CPSS – Sign-Up. These units were given a weight of PERS_W1 = 0. For all other units, PERS_W1 = PERS_W0. Persons with a weight of 0 are subsequently removed from future weight adjustments.

Nonresponse/Nonparticipation Adjustment – PERS_W2

During collection of the CPSS – Sign-Up, a certain proportion of sampled units inevitably resulted in nonresponse or nonparticipation in the panel. Weights of the nonresponding/nonparticipating units were redistributed to participating units. Units that did not participate in the panel had their weights redistributed to the participating units with similar characteristics within response homogeneity groups (RHGs).

Many variables from the LFS were available to build the RHG (such as employment status, education level, household composition) as well as information from the LFS collection process itself. The model was specified by province, as the variables chosen in the model could differ from one province to the other.

The following variables were kept in the final logistic regression model: education_lvl (education level variable with 10 categories), nameissueflag (a flag created to identify respondents not providing a valid name), elg_hhldsize (number of eligible people for selection in the household), age_grp (age group of the selected person), sex, kidsinhhld (an indicator to flag whether or not children are present in the household), marstat (marital status with 6 categories), cntrybth (an indicator if the respondent was born in Canada or not), lfsstat (labour force status of respondent with 3 categories), nocs1 (the first digit of National Occupational Classification code of the respondent if employed, with 10 categories), and dwelrent (an indicator of whether the respondent dwelling is owned or rented). RHGs were formed within provinces. An adjustment factor was calculated within each response group as follows:

$\frac{\text{Sum of weights of respondents and nonrespondents}}{\text{Sum of weights of respondents}}$

The weights of the respondents were multiplied by this factor to produce the PERS_W2 weights, adjusted for panel nonparticipation. The nonparticipating units were dropped from the panel.

5.2 Creating the CPSS4 weights

Surveys of the CPSS start with the sample created from the panel participants. The panel is comprised of 7,242 individuals, each with the nonresponse adjusted weight of PERS_W2.

Calculation of the Design Weights – WT_DSGN

The design weight is the person weight adjusted for nonresponse calculated for the panel participants (PERS_W2). No out-of-scope units were identified during the survey collection of CPSS4 – Information Sources Consulted During the Pandemic.  Since all units were in-scope, WT_DSGN =PERS_W2 and no units were dropped.

Nonresponse Adjustment – WT_NRA

Given that the sample for CPSS was formed by people having agreed to participate in a web panel, the response rates to the survey were relatively high. Additionally, the panel was designed to produce estimates at a national level, so sample sizes by province were not overly large. As a result, nonresponse was fairly uniform in many provinces. The RHGs were formed by some combination of age group, sex, education level, rental status, LFS status, whether or not children are present in the household, eligible household size, and the first digit of the National Occupational Classification (NOC) code for respondents who are employed. An adjustment factor was calculated within each response group as follows:

$\frac{\text{Sum of weights of respondents and nonrespondents}}{\text{Sum of weights of respondents}}$

The weights of the respondents were multiplied by this factor to produce the WT_NRA weights, adjusted for survey response. The nonresponding units were dropped from the survey.

Calibration of Person-Level Weights – WT_FINL

Control totals were computed using LFS demography projection data. During calibration, an adjustment factor is calculated and applied to the survey weights. This adjustment is made such that the weighted sums match the control totals. Most social surveys calibrate the person level weights to control totals by sex, age group and province. For CPSS4, calibration by province was not possible, since there were very few respondents in some categories in the Atlantic and Prairie Provinces. In addition, there were very small counts for male respondents aged 15 to 24 in the Atlantic Provinces, male respondents aged 15 to 24 in British Columbia and female respondents aged 15 to 24 in British Columbia. For this reason, the control totals used for CPSS4 – Information Sources Consulted During the Pandemic were by age group and sex by geographic region, where the youngest age group for males in the Atlantic region, males in British Columbia and females in British Columbia were collapsed with the second youngest age group. The next section will include recommendations for analysis by geographic region and age group.

5.3  Bootstrap Weights

Bootstrap weights were created for the panel and the CPSS4 – Information Sources Consulted During the Pandemic survey respondents. The LFS bootstrap weights were the initial weights and all weight adjustments applied to the survey weights were also applied to the bootstrap weights.

6.0 Quality of the CPSS and Survey Verifications

The probability panel created for the CPSS is a pilot project started in 2019 by Statistics Canada. While the panel offers the ability to collect data quickly, by leveraging a set of respondents that have previously agreed to participate in multiple short online surveys, and for whom an email address is available to expedite survey collection, some aspects of the CPSS design put the resulting data at a greater risk of bias. The participation rate for the panel is lower than typically experienced in social surveys conducted by Statistics Canada which increases the potential nonresponse bias. Furthermore, since the surveys of the CPSS are all self-complete online surveys, people without internet access do not have the means to participate in the CPSS and therefore are not covered.

When the unweighted panel was compared to the original sample targeted to join the panel, in particular there was an underrepresentation of those aged 15-24, those aged 65 and older, and those with less than a high school degree. These differences were expected due to the nature of the panel and the experience of international examples of probability panels. Using LFS responding households as the frame for the panel was by design in order to leverage the available LFS information to correct for the underrepresentation and overrepresentation experienced in the panel. The nonresponse adjustments performed in the weighting adjustments of the panel and the survey respondents utilised the available information to ensure the weights of nonresponding/nonparticipating units went to similar responding units. Furthermore, calibration to age and sex totals helped to adjust for the underrepresentation by age group.

Table 6.1 shows the slippage rates by certain domains post-calibration of CPSS4 – Information Sources Consulted During the Pandemic.  The slippage rate is calculated by comparing the sum of weights in the domain to that of the control total based off of demographic projections. A positive slippage rate means the sample has an over-count for that domain. A negative slippage rate means the survey has an under-count for that domain. Based on the results shown in Tables 6.1 and 6.2, it is recommended to only use the data at the geographical levels and age groups where there is no slippage. That is nationally, by geographic region (Maritime Provinces, Quebec, Ontario, Prairie Provinces, and British Columbia), and by the four oldest age groups.

After the collection of CPSS4 – Information Sources Consulted During the Pandemic, a small study was conducted to assess the potential bias due to the lower response rates and the undercoverage of the population not online. The LFS data was used to produce weighted estimates for the in-scope sample targeted to join the probability panel (using the weights and sample from PERS_W1). The same data was used to produce weighted estimates based on the set of respondents from the CPSS4 survey and the weights WT_FINL. The two set of estimates were compared and are shown in Table 6.3. The significant differences are highlighted.

While many estimates do not show significant change, the significant differences show that some bias remains in the CPSS4 – Information Sources Consulted During the Pandemic. There is an underrepresentation of those where there were three or more eligible participants for the panel, of people with less than a high school diploma, and of people working in NOC8. And there is an overrepresentation of those with a post-secondary certification, of people born in Canada, households with two persons in total, and those where there were two eligible participants for the panel. These small differences should be kept in mind when using the CPSS4 – Information Sources Consulted During the Pandemic survey data. Investigation about differences in estimates is ongoing, and as evidence of differences are identified, strategies are being tested to improve the methodology from one wave of the survey to the next.

Meeting summary: Key points and action items

External stakeholders

• Annette Hester, the Hester View, Canadian Statistics Advisory Council
• Allan Fogwill, Canadian Energy Research Institute
• Bradford Griffin, Canadian Energy and Emissions Data Centre
• Louis Beaumier, Institut de l'énergie Trottier
• Ben Brunnen and Krista Nelson, Canadian Association of Petroleum Producers

Purpose

Meeting with potential participants of the Canadian Centre for Energy Information (CCEI) External Advisory Body to discuss CCEI program objectives and the role of a supporting advisory body in meeting those goals.

Theme 1: Governance/External Advisory Committee role

Participants were generally comfortable with the proposed CCEI External Advisory Committee Terms of Reference, but wanted to have collective discussion on the mechanics of the advisory committee (e.g. how to work as a team, outputs/deliverables, how to reach consensus, engagement with external parties).

Action item

• Include a discussion on governance/committee responsibilities during the inaugural July 2020 advisory committee meeting.

Theme 2: Trust/Transparency and importance of communication/Engagement

Participants expressed need to manage expectations through open and transparent engagement with stakeholders and the public. This included being transparent on advice received from the Advisory Committee, decision making/priority setting by the FPT Deputy-level Steering Committee, documenting how the CCEI is responding, and consider how to share information with the public (i.e. 'Trust Centre', CCEI website or other tools).

Participants also noted the importance of maintaining CCEI as 'policy neutral', both real and perceived, to ensure trust and integrity of the program.

Action items

• Present engagement plan and communications plan at July 2020 External Advisory Committee meeting to discuss approach to obtain feedback from stakeholders and communicating with the public on program 'plans' (e.g. publishing the 5-year plan on CCEI Website to share with the public, ensuring effective communications on initial website content and direction of the program, etc…)
• As the program moves forward and obtains advice from the committee and stakeholders informing CCEI's decision making – consider how to ensure transparency regarding advice received (high-level) and actions/decisions taken by the Centre with the general public.
• Engage with Communications Team at StatsCan on sharing advice received systematically, and having representative from Communications join meetings moving forward to explore.
• Ensuring/maintaining robust conversation on priority setting and data gaps as a key function of the committee (see Theme 4)

Theme 3: Data from external sources – CCEI website

Participants were pleased with the extensive availability of data from federal government, provinces, territories, academia, international institutions, etc… that the initial website would be offering later this summer.

They advised StatsCan provide 'caveats' on data 'ownership' regarding data on the website from other sources (e.g. footnote that data was from external source – to protect integrity of StatsCan and CCEI for data sources were quality has not been confirmed).

Action Item

• Explore with IT and dissemination team possibility of some type of caveat and/or clarity on source of data obtained through CCEI search engine – to flag those coming from external sources.

Theme 4: Initial priority data gaps participants raised

Participants identified the following preliminary 'data priorities'/'issues' as part of the conversation – as well as need for CCEI to maintain 'policy neutrality' – again within the context of protecting the integrity of the program:

• More timely data
• Innovation and commercialization of new technologies (e.g. carbon capture and storage, battery storage technology, etc…)
• Cost of energy and implications for interprovincial/international trade
• Retail and wholesale energy costs and impact on energy poverty/affordability
• Evolution of energy demand
• Establishing consistency in data – through established and agreed upon protocols of data quality

These will be added to other initial data gaps/priorities CCEI has been gathering from stakeholders in previous engagement to date.

Action item

• CCEI team to compile a preliminary list of priorities based on previous engagement with stakeholders and early feedback from advisory body to initiate an early discussion at the July 2020 meeting (and follow-up discussion in September 2020).

Forward agenda: Inaugural CCEI External Advisory Committee meeting in late July

• Finalizing Terms of Reference and Discussion on Committee Roles/Deliverables
• CCEI Presentation and discussion on Engagement and Communications Strategies
• CCEI Presentation and discussion on Early Priorities (as we see them) and feedback from Committee

Release date: October 5, 2020

Status

This standard was approved as a recommended standard on May 26, 2020.

CRDC 2020 Version 1.0

The Canadian Research and Development Classification (CRDC) was developed conjointly by the Social Sciences and Humanities Research Council of Canada (SSHRC), the Natural Sciences and Engineering Research Council of Canada (NSERC), the Canada Foundation for Innovation (CFI), the Canadian Institutes of Health Research (CIHR), and Statistics Canada which is the custodian. This shared standard classification, inspired by the Frascati Model 2015 of the Organisation for Economic Co-operation and Development (OECD), will be used by the federal granting agencies and Statistics Canada to collect and disseminate data related to research and development in Canada. The CRDC first official version is the 2020 version 1.0 and it is composed of 3 main pieces: the type of activity or TOA (with 3 categories), the field of research or FOR (with 1663 fields at the lowest level) and socioeconomic objective or SEO (with 85 main groups at the lowest level).

HTML format

• Introduction
• Type of Activity (TOA) - CRDC 2020 Version 1.0
• Field of Research (FOR) - CRDC 2020 Version 1.0
• Socioeconomic Objective (SEO) - CRDC 2020 Version 1.0

CSV format

• Type of Activity (TOR) - Classification structure (CSV, 843 bytes)
• Field of Research (FOR) - Classification structure (CSV, 292.63 KB)
• Field of Research (FOR) - Elements (CSV, 173.18 KB)
• Socioeconomic Objective (SEO) - Classification structure (CSV, 35.37 KB)
• Socioeconomic Objective (SEO) - Elements (CSV, 128.4 KB)

PDF format

• CRDC 2020 Version 1.0 (PDF, 972.72 KB) (Catalogue number: 89260004)

Introduction

There are a number of indicators that are useful for monitoring the evolution and the impact of a pandemic like COVID-19 in terms of fatalities. Excess mortality is considered a better indicator for monitoring the scale of the pandemic and making comparisons.^{Footnote 1Footnote 2} Excess mortality refers to the "mortality above what would be expected based on the non-crisis mortality rate in the population of interest."^{Footnote 3} Excess mortality also encompasses collateral impacts of the pandemic, such as deaths occurring because of the overwhelming of the health care system, or deaths avoided due to decreased air pollution or traffic.^{Footnote 4Footnote 5Footnote 6}

Estimating excess deaths

There are a number of challenges associated with measures of excess deaths. The most important challenge is to properly estimate some level of expected deaths that would occur in non-Covid19 context as a comparison basis for the current counts of deaths.^{Footnote 1} Indeed, death is a statistically rare event, and important variations may be observed from year to year in the annual counts of deaths, in particular in the less populated provinces and in the territories. Moreover, yearly counts of deaths may be affected by changes in the composition of the population, in regard to age more particularly, and changes in mortality rates (e.g. improvement of mortality).

A second challenge is the difficulty to collect timely counts of deaths. In Canada, death data are collected by the provincial and territorial vital statistical offices. The capacity to provide death data to Statistics Canada in a timely manner varies greatly.^{Footnote 7} Moreover, it is possible that the pandemic imposes a burden on health care and other institutions that disturb the data collection process, although it could instead add pressure for accelerated collection. The incomplete coverage of the numbers of deaths makes it difficult to draw any conclusions on the extent of excess deaths in Canada that could be caused by the COVID-19 pandemic.

Beginning on May 13, 2020, Statistics Canada has been releasing provisional counts of excess deaths for 2020.^{Footnote 8} Although the data were published for transparency and as information to be tracked and updated regularly, the uncertainty associated with the baseline expected death counts and the incomplete coverage of the numbers of deaths made it difficult to draw conclusions on the extent of excess deaths in Canada that could be caused by the COVID-19 pandemic. Statistical models are used to obtain estimated death counts adjusted for incompleteness and to estimate baseline non-Covid mortality. Estimates of excess deaths are obtained by comparing adjusted counts with modeled baseline mortality for all weeks in 2020 up to July 4. A description of the models is provided in the next section.

Methodology

This section describes the distinct models used for estimation of baseline mortality and adjustment of observed death counts.

Estimating expected mortality

The model used to estimate the expected number of deaths is based on a quasi-Poisson regression model fit to weekly death count data. Adapted from an infectious disease detection algorithm developed by Farrington et al.,^{Footnote 9} which has been largely utilized in the context of mortality surveillance in recent years^{Footnote 10}. Later modifications to the algorithm, originally implemented by Noufaily et al.^{Footnote 11} and further expanded by Salmon et al.,^{Footnote 12} that aim at addressing certain limitations of the model were also adopted in this implementation.

The model was implemented in the R programming language with the use of the surveillance package,^{Footnote 12} and was applied to weekly^{Footnote 13} death counts (all-cause) spanning a selected reference period of approximately four years (2016-2019). Historical counts are a combination of published death data from the Canadian Vital Statistics Death Database (2016-2018) and provisional death counts (2019) coming from the National Routing System (NRS). Estimates of expected deaths are produced for all weeks of 2020 up until the week ending July 4, 2020.

An overdispersed Poisson generalized linear model with a linear time trend and a seasonal factor is fit to the data. The seasonal component aims to represent the expected pattern across weeks that repeats from year-to-year, and consists of a zero-order spline term with 11 knots, representing 10 distinct periods within a given year.^{Footnote 14} The 10 periods are split between a single 7-week period corresponding to the current week being estimated and the 3 preceding and subsequent weeks, and 9 other 5-week periods corresponding to the rest of the year.

The model can be expressed using the following log-linear configuration:

$\text{log} {\mu }_t=\alpha +\beta t+{\gamma }_{c\left(t\right)}$

where ${\mu }_t$ is the expectation of the count in week t, $\beta$ is the coefficient corresponding to the linear time trend, and ${\gamma }_{c\left(t\right)}$ the seasonal factor for week t, with c(t) indicating the period in the year that week t belongs to. ^{Footnote 12}

The quasi-Poisson model relaxes the Poisson assumption that the variance must equal the mean. Instead, $E\left(y_t\right)={\mu }_t$, and $Var\left(y_t\right)={\varphi }_t{\mu }_t$, where the overdispersion parameter $\varphi$ is estimated from the model using the formula:

$\hat{\varphi }=\text{max}\left\{\frac{1}{n-p}\sum _{i=1}^n{w}_i\frac{(y_i-{{\displaystyle \hat{\mu }}}_i{)}^2}{{\hat{\mu }}_i},1\right\}$

where n is the number of weeks used in the baseline, and p the number of parameters in the model. A value of $\varphi =1$ implies no overdispersion (regular Poisson model), and $\varphi <1$ implies underdispersion (a rare occurrence, hence the condition on $\hat{\varphi }$). A weight w is assigned to each of the historical observations, based on the value of its standard deviation in an unweighted model. This reduces the influence of potential outliers on the estimation of the expected counts and corresponding prediction interval.

Finally, a 95% prediction interval is computed for the expected count in week t by assuming that the count follows a negative binomial distribution with mean ${\mu }_t$ and size parameter set to ${\mu }_t⁄({\varphi }_t-1)$.

Adjusting death counts for incompleteness

Analysis of death by date (or week) of death is inevitably distorted by delays in reporting. This necessitates appropriate correction of the observed data to estimate the number of death that have occurred but not yet reported. The data received by Statistics Canada via the NRS contain information of day of death, date of report and some demographic information (e.g. age and sex).

Reporting delays are susceptible to change over time, and this is all the more true in a time of a pandemic. For this reason, the model estimates adjustment factors that are based on recent data, and uses different period for weeks that are during the pandemic and those preceding it. Weekly counts of deaths that occurred between December 29, 2019 and March 22, 2020 were adjusted based on the distribution of reporting delays estimated from death records received prior to March 22, 2020. Deaths counts for weeks between March 22 and July 4 were adjusted based on reporting delays observed between March 22 and August 7. In some jurisdictions, the level of data completeness of death records can be very low for the most recent weeks. Weekly adjusted counts are provided only for weeks where the estimated coverage rates satisfy a minimum threshold.^{Footnote 15}

The method used for adjusting observed death counts was originally developed by Brookmeyer and Damiano^{Footnote 16} to model daily counts. It was adapted here to work on a weekly scale. The model was implemented in R programming language using code sent by the authors.^{Footnote 17} The number of deaths occurring on week t and reported in week t+d (i.e. with a delay of d weeks), $Y_{td}$, is modeled using the following Poisson regression model:

$\text{LOG}\left(E\right(Y_{td}\left)\right)={\alpha }_t+{\beta }_d$

where ${\alpha }_t$ represents the log-transformed preliminary reported count in week t, and ${\beta }_d$ is the term representing the adjustment for underreporting. Note that the right side of the equation is in a log scale so the underreporting adjustment can be seen as a multiplicative adjustment in the original scale. The adjusted number of deaths occurring in week t is then the observed deaths count divided by the estimated probability that the lag on the death being reported is less or equal to a maximum of x weeks, with x+t being the last observable week to report deaths, i.e. x is the maximum delay time in the dataset minus the week of the death t:

Adjusted number of deaths (t)=$\frac{{\sum }_{d=0}^{d_{m{ax}^{-t}}}{Y}_{td}}{P(\text{delay}\le d_{\max }-t|\text{time}=t, \text{delay}\le d_{\max })}$

Estimating excess mortality

To calculate the weekly number of excess deaths, the baseline number of deaths in the absence of the pathogen (COVID-19) is subtracted from the observed (and adjusted for reporting delay) number of deaths for the period of interest. The method involves the following steps:

• Quasi Poisson models are fit to the weekly death counts at the provincial and territorial level from January 1st 2016 to January 1st 2020 to obtain a baseline measure of the expected mortality.
• Baseline deaths are projected for year 2020 until July 4.
• Adjust for reporting delays the weekly counts of deaths that occurred from December 29, 2019 to March 22, 2020 based on the distribution of reporting delays estimated from death records received prior to March 22, 2020.
• Adjust for reporting delays the weekly counts of deaths that occurred from the week ending March 22 to July 4 based on reporting delays observed between March 22 and August 7.
• Apply an additional correction to the counts and prediction intervals for the period for March 22 to July 4. This correction factor is the ratio of the adjusted count for the week of March 22 based on the distribution of reporting delays estimated from death records received prior to March 22, 2020 to the unadjusted count for the same week.
• Excess mortality is defined as the adjusted observed mortality minus the baseline for the period of interest.

The 95% prediction intervals surrounding estimates of excess deaths were computed by combining the variances from the two models. An empirical distribution of excess deaths is calculated by randomly pairing 10,000 estimates (replicates) from each model, as per the bootstrap method. The bounds of the prediction intervals represent quantiles of the empirical distribution. The method assumes independence between the two processes that are weekly mortality and collection of death records, but makes no assumption about the statistical distribution of excess deaths.

Validation

The computation of excess mortality requires the estimation of two greatly uncertain processes: how many deaths there should be in a given week, and how many deaths occurred that were not yet recorded at the time of estimation. The use of modelling for estimation of excess deaths aims at improving estimation but also, importantly, to reflect the uncertainty of these processes.

Validation of the models tend to show that they perform well in many regards. Expected counts tend to mimic the seasonal patterns typically observed during a year and follow the increase observed in past years (mainly due to population growth, particularly at old ages). However, because they captured over periods comprising several weeks, these seasonal patterns tend to be smoothed to some extent. For example, the application of time series models to weekly counts tend to produce more defined peaks, in particular in the month of January (likely due to influenza outbreaks). Another limitation has to do with the way prediction intervals are computed. In the model for estimating expected counts, the death counts are assumed to follow a negative binomial distribution, which is well adapted for modelling discrete counts data susceptible to present overdispersion. However, the bounds of the prediction intervals are defined as the quantiles of the negative binomial distribution, and thus do not reflect the variance due to parameter estimation. A better statistical representation would also account for uncertainty in parameter estimation.

The model for adjusting death counts was designed mainly for its capacity to capture recent trends in reporting delays. Experimentation with different time periods suggest that indeed, there have been changes in the pace at which deaths are registered in the provincial and territorial vital statistics database, at least in some provinces. However, the model assumes that there are no changes within the reference period considered. This is not guaranteed, in particular in a time of pandemic. Another limitation is that with the reference period is too short for capturing adequately potential seasonal patterns. The application of times series models to the data reveals the presence of some seasonal patterns in the coverage rates for some lags (number of days between date of death and report date). It is assumed that biases due to changes in reporting patterns are more important than those due to seasonality. Likewise, potential patterns of underreporting related to some specific days of the week, such as Sundays or holidays, were not considered.

Statistics Canada will continue to refine the methodology in an effort to better inform Canadians of the effects of the COVID-19 pandemic.