Throughout the pandemic, Statistics Canada has responded rapidly and with agility to the data needs of various users by developing many new products relevant to COVID-19 and its impacts on Canadians. While this response was agency-wide, the evaluation focused on two types of products that were developed in the context of the pandemic: 1) products derived from crowdsourcing and probability panels, and 2) specific COVID-19 related health statistics.
This evaluation was conducted by Statistics Canada in accordance with the Treasury Board Policy on Results and Statistics Canada’s Risk-based Audit and Evaluation Plan (2020/2021 to 2024/2025). The objective of the evaluation was to provide a neutral, evidence-based assessment of Statistics Canada’s data response to COVID-19. The evaluation aimed at providing valuable information about the timeliness, the relevance and the impact of these products. It also looked at some of the related challenges and opportunities faced by the divisions responsible for these products to inform future direction.
In reviewing the organizational changes that were implemented to develop COVID-19 health and social statistical products, the evaluation found that Statistics Canada took rapid action by assessing and delivering products that shed light on issues of importance during the pandemic. Delivery of COVID-19 statistical products was facilitated by an organizational shift towards business agility, which included innovative approaches to manage resources and analyze data. The dedication of staff was instrumental in the successful release of these products; however, the sheer volume and push for expediency impacted staff well-being.
To inform Statistics Canada on the effectiveness of its response related to social statistical data needs during COVID-19, the evaluation assessed the extent to which the products released addressed the needs of key users. The majority of users considered the products released to be relevant, useful, timely, accessible and unique for providing a national perspective. Although some challenges were noted, users were also satisfied with the agency’s level of engagement and the methodological information provided. Finally, users wanted more granular data, particularly regarding at-risk and marginalized populations.
Finally, the evaluation assessed the extent to which there is an ongoing need for COVID-19 social and health statistical products and the factors that must be taken into account going forward to deliver both those products and regular programs. Users indicated an ongoing need for health statistics related to the pandemic as well as for crowdsourcing and probability panel products. As the pandemic evolves, more disaggregated data on the differential impacts of COVID-19 on different populations will be needed. Forward approach needs to balance demands arising from emerging data needs, such as disaggregated data, ongoing pandemic information requirements and ongoing program delivery. Prioritization of demands, human resources, ongoing innovation and data sources, user engagement, and internal collaboration were identified as key enablers. Opportunities remain to fully explore the lessons learned from the pandemic, which could be leveraged to help the organization further improve its COVID-19 data response and core activities. The lessons learned could also be shared across the agency to foster innovation and continuous improvement in support of the agency-wide response to COVID-19.
The evaluation proposed two recommendations:
Recommendation 1:
The Assistant Chief Statistician (ACS), Social, Health and Labour Statistics (Field 8), should ensure that a comprehensive strategy is developed to identify, prioritize and respond to emerging data needs (i.e., disaggregated data, ongoing pandemic information requirements and ongoing program delivery).
The strategy should consider the key enablers identified (i.e., prioritization of demands, human resources, ongoing innovation and data sources, user engagement, and internal collaboration), and include approaches or plans that set out how:
- emerging data needs will be identified (e.g., stakeholder engagement approaches);
- data gaps will be addressed;
- data needs will be prioritized based on intended outcomes and resource capacity; and
- ongoing monitoring of data needs will be carried out.
Recommendation 2:
The ACS, Social, Health and Labour Statistics (Field 8), should ensure that a lessons learned exercise from the COVID-19 pandemic is conducted to identify approaches that could be applied to respond to COVID-19 data needs and improve core activities going forward.
- The lessons learned should be presented to a Tier 1 committee that will provide direction on their potential implementation across the agency, as applicable.
- They should also be shared across the agency to foster innovation and continuous improvement in support of the agency-wide response to COVID-19.