Evaluation of the Environmental Accounts and Statistics Program
(2012/2013 to 2016/2017)

Evaluation Report

December 2018

The report in short

The Environmental Accounts and Statistics Program (EASP) collects, compiles, analyzes and publishes environmental data. It provides users with consistent, comprehensive, timely and relevant statistics that allow them to better understand the relationship between human activity and the environment. The program includes nine surveys and produces four sets of accounts as well as an annual report spread across a wide range of environmental topics, including water accounts, ecosystems and emissions.

The evaluation of EASP was conducted by Statistics Canada in accordance with the Treasury Board of Canada’s Policy on Results (2016) and Statistics Canada’s Risk-Based Audit and Evaluation Plan (2016/2017 to 2020/2021). This is the first-ever evaluation of the program, and it covers the period from 2012/2013 to 2016/2017. The main objective of the evaluation was to provide a neutral, evidence-based assessment of the relevance and performance of the program in support of decision making, accountability and improvement.

The evaluation methodology consisted of a document and literature review, data reviews, key informant interviews, a survey of users, a bibliometric and webometric assessment, and an international review. The triangulation of these data collection methods was used to arrive at the overall evaluation findings.

Key findings

EASP is aligned with federal roles, responsibilities and commitments. Statistical information from the program is used by federal partners to meet requirements of various acts, including the Canadian Environmental Protection Act and the Federal Sustainable Development Act. The program actively collaborates with partners on projects such as those under the Pan-Canadian Framework on Clean Growth and Climate Change, and it is active internationally with organizations such as the United Nations (UN) and the Organisation for Economic Co-operation and Development.

While the need for environmental statistical data is in constant evolution and growing steadily, EASP data users are generally satisfied with the information available. Some gaps were identified in areas such as ecosystems and water accounts; however, EASP is aware of them and is taking steps to address them. In general, users were very satisfied with the quality of the information produced by the program. However, concerns were raised regarding the timeliness and periodicity of the information.

EASP communicates regularly with data users and partners, mostly through informal mechanisms and some outreach activities. There is evidence, however, that opportunities exist to improve these activities. For example, while EASP developed an outreach plan in 2015/2016, it was only partially implemented. External interviewees noted that greater efforts could be made to promote the information produced by the program.

Although some areas of collaboration between EASP, the Transportation Statistics Program and the Energy Statistics Program were identified, there remain unexplored opportunities that could increase the overall efficiency of the Environment, Energy and Transportation Statistics Division (EETSD). For example, EASP staff could benefit from assistance or support with dissemination activities.


Recommendation 1

It is recommended that EASP enhance its outreach strategy to ensure a systematic and structured approach to consultations and promotion activities. The strategy should be inclusive and cover all key stakeholders, and follow-up communications should take place. Internal discussions should take place with staff to foster engagement and the use of corporate tools to measure success and client satisfaction should be explored.

Recommendation 2

It is recommended that EASP, along with the energy and transportation programs within EETSD, identify additional areas for collaboration and implement formal mechanisms to support them. In addition, to ensure future efforts continue, EETSD should create a forum where ideas for collaboration can be discussed.

Recommendation 3

It is recommended that EASP revisit its current approach to tracking and managing client requests. The revised approach should be viewed divisionally to take into consideration the systems used by the Energy Statistics Program and the Transportation Statistics Program.

Recommendation 4

It is recommended that EASP explore opportunities to increase the frequency of its surveys and to address the identified gaps. In addition, the program should take steps to improve the timeliness of its releases in order to meet the established timeliness target.

1. What is covered

This evaluation was conducted in accordance with the Treasury Board of Canada’s Policy on Results (2016) and Statistics Canada’s Risk-Based Audit and Evaluation Plan (2016/2017 to 2020/2021). The main objective of the evaluation was to provide a neutral, evidence-based assessment of the relevance and performance of the Environmental Accounts and Statistics Program (EASP) in support of decision making, accountability and improvement. This is the first time the program has been evaluated; the evaluation covered EASP activities from 2012/2013 to 2016/2017.

The program

The mandate of EASP is to collect, develop, compile, analyze and publish environmental data, emphasizing their integration with socioeconomic data. The information produced aims to provide users in government, the private sector and the public sector with consistent, comprehensive, timely and relevant statistics that allow them to better understand the relationship between human activity and the environment. The statistics and accounts are intended to assist and support decision makers, to sustain environmental research and to report to the Canadian public about the environment. Total EASP expenditures over the five-year period were approximately $35 million. EASP manages nine surveys and produces four sets of accounts as well as an annual report:

  • Industrial Water Survey (IWS, biennial)
  • Survey of Drinking Water Plants (SDWP, biennial)
  • Agricultural Water Survey (AWS, occasional)
  • Farm Management Survey (FMS, every five years, cost-recovery)
  • Survey of Environmental Protection Expenditures (SEPE, biennial)
  • Waste Management Industry Survey (WMIS, biennial)—business and government sectors
  • Survey of Environmental Goods and Services (SEGS, biennial)Footnote 1
  • Households and the Environment Survey (HES, biennial)
  • Survey of Household Energy Use (SHEU, conducted every four years, as a supplement to the HES)
  • Physical Flow Accounts (PFA, annual)
  • Natural Resource Asset Accounts (NRAA, annual and quarterly)
  • Ecosystem Accounts (irregular)
  • Environmental and Clean Technology Products Economic Account (ECTPEA, annual, cost-recovery)
  • Report on Energy Supply and Demand (RESD, annual, cost-recovery).

In addition, EASP produces technical and analytical publications, such as the Environment Accounts and Statistics Analytical and Technical Paper Series (irregular), EnviroStats (irregular), and Human Activity and the Environment (HAE, annual). As is the case for other Statistics Canada programs, EASP disseminates various products derived from its surveys and accounts, such as Canadian Socioeconomic Information Management System (CANSIM) tables, summary tables, custom tabulations, Daily articles, geospatial analysis, and data layers. A more detailed description of EASP can be found in Appendix 1.

The evaluation

Following discussions with EASP and with senior management, the following nine areas were identified for review in the evaluation:

  • Relevance and responsiveness to needs
    1. Alignment with federal roles and responsibilities, government priorities, and commitments
    2. Continued need for the program and alignment with the needs of users
    3. Program consultations
  • Performance: Effectiveness
    1. Quality of information
    2. Innovative methods and releases
    3. Program reach
    4. Use of information
  • Performance: Efficiency
    1. Utilization of resources
    2. Factors impacting efficiency

Focus – International analysis

To provide additional context and information, the Evaluation Division conducted an international analysis of environmental statistics programs. The objective was to highlight good practices in environmental statistics programs abroad. The organizations selected are comparable to Canada in broad terms and have mature environmental statistics programs, where good innovative practices and models could be identified. The five programs selected were in Australia, Germany, the Netherlands, the European Environment Agency (EEA) and the United Kingdom (UK). The themes selected for the analysis were general success factors, satisfaction of user needs and measurement of success, stakeholder outreach practices, and innovation. The international analysis was planned with the collaboration of EASP officials.

Guided by a utilization-focused evaluation approach, the following quantitative and qualitative collection methods were used:Footnote 2

Administrative and financial reviews

Review of EASP administrative data on activities, outputs and results.

Review of EASP financial data.

Literature review

Review of published reports, peer-reviewed journal articles, grey literature, websites and periodicals, including evidence from similar programs in other jurisdictions (e.g., Organisation for Economic Co-operation and Development [OECD] countries) relating to design and delivery lessons learned and best practices.

Survey n=40

Online survey with respondents from a population of environmental accounts and statistics users, from February 7 to March 8, 2017. The response rate was 14%.

Bibliometric and webometric assessment

A bibliometric and webometric assessment of uptake of EASP releases, as well as a bibliometric assessment of EASP outputs in the peer-reviewed scientific literature.

Document review

Review of internal agency strategic documents.

Key informant interviews n=42

Semi-structured individual interviews of

  • subject-matter experts (n=4)
  • public sector users (n=8)
  • other users (n=7)
  • federal partners (n=9)
  • international informants (n=5)
  • EASP and other Statistics Canada officials (n=9).

International analysis

International analysis of other environmental accounts and statistics programs, which included semi-structured interviews (n=9) and a document review.
The selected programs were in

  • Australia
  • Germany
  • the Netherlands
  • the EEA
  • the UK.

The selected themes were

  • general success factors
  • satisfaction of user needs and measurement of success
  • stakeholder outreach practices
  • innovation.

2. What we learned

2.1 Alignment with federal roles and responsibilities, government priorities, and commitments

Evaluation question

To what extent did the activities of EASP align with federal roles and responsibilities, priorities, and international commitments?

In Canada, the environment is a shared responsibility between the federal government and the provincial and territorial (P/T) governments. The environment was not explicitly defined in sections 91 and 92 of the Constitution Act in 1867, as it was not identified at the time, or even considered, as something requiring governmental protection. Responsibilities for the environment are generally assigned according to the subjects that best describe the core substance of the regulation. For example, public property, criminal law, fisheries and sea coasts, navigation and shipping, and "lands reserved for Indians" are under federal responsibility. Consequently, environmental issues related to these fall under federal jurisdiction. Taking into account its domain of jurisdiction, the federal government enacts laws and regulations such as the Canadian Environmental Protection Act and the Federal Sustainable Development Act.

EASP's statistical information, particularly data from the SEPE, is used by Environment and Climate Change Canada (ECCC) to meet the information requirements of the Canadian Environmental Protection Act. Data from four other EASP surveysFootnote 3 contribute to the Canadian Environmental Sustainability Indicators (CESI) initiative led by ECCC. The CESI is the main instrument to measure the progress of the federal sustainability strategy governed by the Federal Sustainable Development Act.

As stipulated in the Statistics Act, Statistics Canada is required to collect and produce statistical information pertaining to the social, economic and general activities and condition of the people of Canada.Footnote 4 This includes statistical information on the environment. In addition to its collection and production role, Statistics Canada notably has a duty "to collaborate with departments of government in the collection, compilation and publication of statistical information, including statistics derived from the activities of those departments"Footnote 5 and "to promote the avoidance of duplication in the information collected by departments of government."Footnote 6

The Pan-Canadian Framework on Clean Growth and Climate Change is one example of collaborative work between levels of government. The Pan-Canadian Framework was developed in accordance with Canada's commitment at the United Nations Climate Change Conference (COP21), during which the federal government agreed to reduce greenhouse gas (GHG) emissions and support the transition to a global low-carbon economy. Budget 2016 noted that clean technologies will play a critical role in Canada's transformation to a low-carbon economy. Accordingly, it proposed an investment of $2.1 million over two years to enhance clean technology data, a collaborative effort between Natural Resources Canada (NRCan), Statistics Canada, and Innovation, Science and Economic Development Canada (ISED). The objective is to have comprehensive, regular up-to-date data to monitor the contribution of the clean technology sector to the Canadian economy. To respond to this new need, EASP expanded the scope of the SEGS and undertook the development of a clean technology satellite account, as well as a feasibility study on how to improve estimates of the amount of renewable energy produced in Canada.Footnote 7 The satellite account (the Environmental and Clean Technology Products Economic Account) captures all economy-wide transactions in the clean technology sector, such as clean energy and scrap metals.Footnote 8 It is currently being developed in conjunction with the National Economic Accounts Division of the Macroeconomic Accounts Branch. The data are important given the federal government's priority to expand the clean technology sector. Statistics Canada's pathfinder project—Transition to a Low-carbon Economy—addresses information needs related to the Carbon Measurement Strategy (e.g., clean technology). This work was further funded in Budget 2017—funding until 2020/2021 via cost-recovery arrangements with NRCan and ISED.

Canada is an active member of international organizations, including the UN and the OECD, which regularly require the submission of environmental statistics. For example, under the requirements of the UN Framework Convention on Climate Change, Canada must report its GHG emissions and removals.Footnote 9 The EASP contributes to this international GHG reporting by developing the annual Report on Energy Supply and Demand, a key data source for the National Inventory Report on GHG emissions produced by ECCC. By providing data on energy, environmental protection, and natural resources, EASP allows Canada to respect its commitments and international responsibilities.Footnote 10

Overall, although there are no specific Canadian regulations mandating which environmental statistics and accounts must be produced, the evidence suggests that EASP contributes to the achievement of Canada's commitments to Canadians, as identified in Budget 2017 ("to ensure that future generations of Canadians can live in a clean and safe world") and in Budget 2016 ("leaving future generations of Canadians a sustainable and prosperous country").

Focus – International analysis

Regulatory frameworks

The international analysis found that environmental statistics programs within the European Union (EU) are subject to regulations that require mandatory environmental accounts.Footnote 11 International interviewees had mixed feelings with regard to the benefits of having such regulations. Some informants felt that their environmental accounts program would not be as well-developed without the regulations, as they provided funding to develop and test concepts and methods prior to the promulgation of the regulations, and core budget allocations for the compilation and maintenance of the accounts. On the other hand, as budgets are not unlimited, resources spent on mandatory accounts cannot be spent elsewhere (e.g., to address emerging issues and needs of national data users). Having a regulatory framework on environmental accounts is not necessarily a success factor for environmental statistics programs. Australia, which is not subject to regulations, does not appear disadvantaged by their absence: Australia's environmental statistics program, on the whole, is as successful as those of Europe.

2.2 Continued need for the program and alignment with the needs of users

Evaluation question

Does the EASP model (scope and themes of statistical information) respond to the needs of users? Are there gaps?

The statistical information produced by EASP is driven by two major guidance documents: the Framework for Environmental Statistics based on the concept of natural capital and the United Nations System of Environmental Economic Accounting (UN-SEEA).Footnote 12 The Framework was developed in 2012 by EASP following broad consultations with data users and partners. It is based on the concept of natural capital, which essentially consists of the theory that the stocks of environmental assets provide flows of goods and services that contribute to human well-being. Natural capital includes natural resources, water, land and ecosystems. The goods and services provided by natural capital do not have to be traded within the economy to be considered important to society. An example of natural capital measurement is EASP's NRAA, which captures key commercial natural resources (i.e., metallic and non-metallic minerals, fossil fuels, timber, and land) as well as the annual changes in these assets resulting from natural processes and human activity.Footnote 13 The UN-SEEA is an international guideline designed to link environmental information to the economic data storehouse that is the System of National Accounts (SNA). The UN-SEEA promotes international consistency, supports understanding of the interactions between environmental and economic issues, and adheres to the concept of natural capital.

Interviewees noted that EASP integrates major accounts from the UN-SEEA. Some key informants indicated, however, that there are possibilities for increasing the relevance of some of the accounts produced. For example, it was noted that, while protection expenditures focus on businesses, they could provide a more complete picture if governmental and household expenditures were also included. Also, data on environmental goods and services focus only on some providers, and expanding the range of providers would provide a more complete picture. It was also stated that the addition of environmental activity accounts would provide an indication of whether money is being spent to remediate environmental damage.

Over the last few years, EASP has taken steps to produce a broader set of data on environmental goods and services using funds provided through Budget 2016 and Budget 2017. By 2016/2017, work was under way to pilot an expanded version of SEGS (released in October and December 2017). EASP began developing environmental expenditure accounts that included government spending by working with the Public Sector Statistics Division on the environment component of the Classification of Functions of Government (released in January 2018Footnote 14).

Three recurrent data gaps were found in all lines of evidence:Footnote 15

  • the level of disaggregation of the data (e.g., P/T, regional, municipal, and industry or facility level)
  • data on ecosystems, including ecosystem goods and services
  • water accounts (e.g., water quality, water use, wastewater, water management and groundwater).

EASP has taken steps to address these three areas. For example, to address the need for statistics on waste water, EASP reached out to ECCC to discuss the use of its Wastewater Systems Effluent Regulations administrative data for the production of wastewater statistics, held regular discussions with the Canadian Water Network to identify options for the availability of water and wastewater data related to financing for the development of policy tools for municipalities, and is developing jointly with the University of Toronto a feasibility study to investigate EASP's capacity to produce more detailed cross-tabulations on industrial water use while adhering to confidentiality requirements. To facilitate data access and interpretability, EASP included water data in the 2016 HAE, which provided an atlas for each of Canada's drainage regions.Footnote 16

Regarding the disaggregation of data, EASP responded to provincial requests from Ontario, Alberta and Quebec to obtain provincial-level results for the PFA on energy use and GHG emissions. This led to the development of a methodology and preliminary data, which were provided to interested provinces via their focal points. Data at small geographical areas were also made available for other data products.  For instance, the HAE on land cover, land use and ecosystems made data available for each census metropolitan area. Variables for the HES were also made available for census metropolitan areas and census agglomeration areas, with CANSIM tables developed or extended in the last few years to facilitate access to these data.

On ecosystems, EASP is producing experimental ecosystem accounts and is active internationally, as these are a shared emerging priority worldwide. EASP participated in the first meetings of the intergovernmental Group on Earth Observations (GEO)Footnote 17 to promote the knowledge developed in Canada about the use of remotely sensed data for measuring ecosystem services. EASP is a member of the steering committee of the United States National Capital Accounting initiative (Powell Center, United States Geological Survey) to support the creation of land, water and ecosystem accounts at the national and sub-national scales and is a part of the steering committee of the U.S.–GEO EO4EA initiative. Funding for 2016/2017 and 2017/2018 was received through the Statistics Canada Big Ideas Conference II to develop a proposal for the Census of the Environment—an interdepartmental project that would quantify all of Canada's ecosystem assets. Although other departments expressed an interest, no additional funding was obtained to further develop this initiative.

A number of other data gaps were also identified: air emissions, the impact of climate change or vulnerability to change, the green economy, waste accounts, biodiversity, and spatial analysis. Overall, however, all lines of evidence suggested that stakeholders are generally satisfied with the current availability of environmental accounts and statistics.

Over the last five years, EASP has made efforts to identify and prioritize data gaps and is now producing, or in the process of producing, statistical information on the majority of needs expressed by stakeholders: clean technology, ecosystem accounts, physical accounts, household-related data, natural resources, environmental protection, waste management, water, land use, spatial analysis and agriculture. The development of accounts for ecosystems and renewable water stocks and the implementation of the Framework for Environmental Statistics, based on the concept of natural capital and the use of ecosystems as a measurement focus, have been divisional priorities for the last three fiscal years (2014/2015 to 2016/2017). This work has continued, with EASP collaborating with the Standards Division to publish the eco-region geography as an official Statistics Canada geography.Footnote 18

Finally, while EASP is aware of existing data gaps and is adapting its priorities accordingly, it does not have a formal mechanism to measure user satisfaction. Currently, this is largely dependent on informal discussions taking place via the staff's professional networks. Given that corporate efforts are now under way at Statistics Canada to measure user satisfaction, EASP should explore how these efforts could align with their needs.

Focus – International analysis

Adapting to changes in user needs

Informants pointed out that it is important to be aware of changes in the policy landscape so that data can be adapted accordingly. This requires carefully balancing policy relevance against stability, given that policy needs change quickly and statistics usually require long-time series in order to be useful. Informants working on cutting-edge methodologies noted that the field is still developing. It remains important to maintain regular contact with policy departments to ensure that data produced are relevant to policy.

The joint project between the UK's Office for National Statistics (ONS) and Department for Environment, Food and Rural Affairs (DEFRA) provides a good example of a formal process for determining changes in user needs and setting out a plan to adapt to those changes. The project is split into phases, and, at the end of each phase, a review is conducted by relevant stakeholders, and a new roadmap is developed.

Measuring user satisfaction

Organizations use a mix of formal and informal methods to measure user satisfaction. The EEA, for one, has a strong formal method for measuring user satisfaction. The agency is evaluated by the European Parliament every five years. In addition, the EEA performs a review of the European Environment Information and Observation Network (EionetFootnote 19) and reports on its performance in its annual activity reports. Performance indicators used include satisfaction of member countries, relevance of output to user countries, and contributions of stakeholders beyond Eionet.

Some informants noted that, unlike the identification of user needs, the measurement of user satisfaction needs to be done at a lower level of granularity. Environmental accounts and statistics programs need to know whether specific users are satisfied with the data being produced. However, environmental accounts and statistics programs may not know all data users and, as a result, are not able to carry out comprehensive user satisfaction surveys.

2.3 Program consultations

Evaluation question

To what extent are consultation activities carried out in an effective way, aimed at supporting the development and responsiveness of EASP?

In 2011/2012, an external advisory committee—the Environment Statistics Advisory Committee (ESAC)—was created, mainly to inform EASP on the prioritization of the implementation of the Framework on Environmental Statistics. The ESAC met only a few times—it has not been active since 2012. Respondents involved in the ESAC described this approach as useful to gather different external views to help guide the statistical program. The committee was subsequently disbanded; however, the rationale for dismantling the ESAC does not seem to have been communicated to external members.

Evidence suggests that EASP opted for a different approach for gathering advice from external experts in environmental statistics: communities of practice. The communities of practice would provide a forum to share knowledge and experience for improving data quality and focus on priorities. EASP's outreach plan in 2015/2016 suggested communities of practice in several areas: environmental activities, physical flows, ecosystem goods and services, application of remote sensing to land cover change analysis, and water assets accounting. There is evidence that some of these communities were initiated by EASP through the form of ongoing consultations in areas such as clean technology, solid waste, Census of the Environment, and energy supply and demand.

The evaluation found that EASP has a close relationship with data users through formal and informal mechanisms. In 2013/2014, a formal workshop was held to obtain strategic information for the implementation of the Framework for Environmental Statistics. In 2015/2016 and 2016/2017, EASP carried out 35 consultation activities in person, by phone and by email. It consulted different stakeholders from the federal government (e.g., NRCan, ECCC, Global Affairs Canada [GAC], and Agriculture and Agri-Food Canada [AAFC]), municipal and P/T governments (e.g., the Recycling Association of Alberta, the City of Toronto, the Nova Scotia Department of the Environment and Energy, the Ontario Ministry of Finance, Alberta Innovation and Advanced Education, and the Canadian Council of Ministers of the Environment [CCME]), the business community (e.g., ECO Canada, the Electronic Products Recycling Association, Enbridge Gas Distribution Inc., Econometrica, the Canadian Water and Wastewater Association, and the Canadian Association of Petroleum Producers), media (e.g., StrategyCorp and Canadian Broadcasting Corporation Montréal), non-governmental organizations (NGOs) (e.g., the GLOBE Foundation and the Natural Step), and academia.Footnote 20 Although the consultation activities were identified as being carried out only once or occasionally in 2015/2016, meetings with groups such as federal departments (e.g., ECCC, NRCan) and provincial departments (e.g., CCME) were held more frequently in 2016/2017.

A number of suggestions to improve consultation activities, such as the need to increase inclusiveness and communication, were offered by interviewees. In terms of inclusiveness, some interviewees felt that important stakeholders such as researchers, the Federation of Canadian Municipalities (FCM) and P/T stakeholders, were not systematically included in EASP's consultation processes. In terms of communication, the importance of following up with stakeholders after consultations have concluded (e.g., outcomes of the consultation activities, decisions made) was mentioned as an area of improvement.

Focus – International analysis

Stakeholder identification and engagement

Stakeholder consultations are seen as priorities in several of the organizations reviewed. In particular, the Australian Bureau of Statistics (ABS), the ONS, DEFRA and Statistics Netherlands all have major, organization-wide initiatives aimed at improving stakeholder outreach. In spite of this, stakeholder consultation practices remain largely informal, except for the EEA with its Eionet mechanism. Two main reasons were identified for adopting informal approaches: formal approaches implemented in the past did not yield positive results, and the time and effort required to manage more formal processes are too significant given the limited resources organizations have.

Project reference (or advisory) groups are used by some organizations as an approach to stakeholder engagement. Such groups are created only in some instances, however. So, while the groups themselves are formal, their application is informal.

The EEA has the most established and formal stakeholder engagement process of those reviewed, spending a total of 24 days annually on stakeholder engagement through Eionet, which allows for the creation of reference groups around specific themes and topics; however, a reference group for environmental accounts does not currently exist. The 2013 evaluation of the EEA found that the stakeholder engagement process was functioning well, but could be improved through more active engagement with member countries and national focal points. Reference groups are also used in other organizations to ensure input and support from a variety of stakeholders and allow users to take ownership of projects.

DEFRA's user-engagement strategy lists seven tools for engaging with stakeholders: user groups, user events, direct consultations, web or information technology tools, social media, newsletters and partnerships.

A number of informants emphasized the importance of informal networks and ad hoc activities, such as attendance at conferences. Some environmental accounts and statistics programs organize their own conferences.

2.4 Quality of information

Evaluation question

To what extent do Canadians and the public and private sectors have access to quality statistical information and to associated support services?

Within and outside Statistics Canada, the level of satisfaction with the quality (timeliness, accuracy, interpretability, coherence and accessibility) of EASP products is high. The level of satisfaction regarding quality is the same for the base statistical information produced and the cost-recovery products and services. The evidence suggests that there are no major issues with the quality of the outputs produced by EASP. This may be in part attributable to the use of the Framework for Environmental Statistics and the UN-SEEA, which ensure a certain level of international consistency and a stronger analytical capacity to develop EASP's outputs. Issues were identified, however, in terms of timeliness, frequency and accessibility.

Timeliness and frequency of surveys were two issues raised by many key informants from different sectors. Timeliness of statistical information refers to the delay between the end of the reference period and the date on which the information becomes available. In several cases, data were released two years after the reference period (e.g., 2015 data released towards the end of 2017). For many informants, this was problematic.

The timeliness indicator for the program was significantly below the target (90%) in both 2015/2016 (36%) and 2016/2017 (63%), although there was improvement.Footnote 21 There is evidence that EASP is somewhat addressing this issue. Meetings with provincial representatives through the CCME have led to an earlier collection date for the biennial WMIS as well as a staged release strategy that allows for earlier release of the most popular variables.

Frequency refers to how often a survey is administered. Although the HAE is published annually and future SEGS will be administered annually, most EASP surveys are biennial.Footnote 22 It was noted by interviewees that EASP's biennial surveys are unique in a negative way in comparison with other Statistics Canada statistical programs.

The issues around timeliness, coupled with the biennial nature of most of the surveys, mean that some data are quite dated, thus impacting the relevance of the information.

One of the vehicles that EASP used to disseminate its data was survey publications. More recently, however, given resource limitations, EASP moved away from survey publications to CANSIM tables, Daily articles and fact sheets. This move was viewed negatively by a number of users who noted that the publications were useful.

In terms of accessibility, EASP's dissemination approach was considered to be "middle of the pack." Interviewees were generally dissatisfied with Statistics Canada's website, which was found to be one of the main problems (e.g., "hard to navigate" and "not intuitive"). Although this is a known corporate issue being addressed through the New Dissemination Model, it nonetheless has a negative impact on the accessibility of EASP products. Statistics Canada's Policy on Privacy and Confidentiality also has an impact on the accessibility of EASP products: provincial and municipal data, which are important for some users, are often unavailable because of confidentiality requirements. Finally, the research data centres were generally viewed favourably by the handful of interviewees that used them. At the time of the evaluation, only data from the HES were available through these centres while data from the Survey of Drinking Water Treatment Plants were available via the Canadian Centre for Data Development and Economic Research.

While the most prominent issue pertaining to accessibility is corporate in nature (the website), initiatives are nonetheless being implemented by EASP to improve it (e.g., development of a product catalogue and redirection of users to the information).

2.5 Innovative methods and releases

Evaluation questions

To what extent does EASP use innovative methods or products to enhance the quality and the use of statistical information?

Are there innovative practices from other countries (particularly for innovative products and accessibility of statistical information and services)?

Survey respondents described several EASP products and services as innovative; however, there was relatively little overlap among responses, as individuals tended to name a few products with which they were familiar. Two possible approaches to dissemination were mentioned more than once by internal and external interviewees. The first approach relates to the integration of environmental and economic data. Statistics Canada is viewed as a leader in this gradual transition. Linking economic and environmental data allows users to get a bigger picture and understand the impacts of human activity on the environment.

The second approach is the publication of datasets on the Federal Geospatial Platform (FGP), a collaborative online environment consisting of geospatial data, authoritative services and applications, all deployed on a common infrastructure.Footnote 23 The FGP was considered by internal and external respondents as innovative. The FGP is not managed by EASP or Statistics Canada; rather, it is managed by NRCan. EASP contributes to it by producing data that are included in the platform. Dissemination through the FGP arises from EASP efforts to modernize its dissemination approach for the HAE. As of April 2017, EASP had published two datasets on the FGP: "Average annual runoff in Canada (1971 to 2013)" and "Drainage regions of Canada." In addition, EASP posted similar geospatial information on the Statistics Canada Website and Open Maps, the public counterpart of the FGP.

Focus – International analysis


General support for innovation and innovation in dissemination are agency-wide priorities in a number of organizations reviewed in the international analysis, especially in Australia, the Netherlands and the UK. The statistical offices in all these countries recognize the importance of innovation to their survival and ongoing relevance, and they have put in place major policies and initiatives to encourage it. Nonetheless, it is important to note that innovation in environmental statistics programs tends to be internally driven rather than agency-driven.

A good example of program innovation is the work on ecosystem accounting taking place in Australia, at the EEA, and in the UK. All of this work is at the leading edge of environmental account development, and it is being driven largely from within the environmental statistics programs themselves. A noteworthy feature of the development of ecosystem accounts, in comparison with the development of earlier environmental accounts (such as material flow accounts), is that it is being done much more in concert with the policy community. This is particularly the case in Australia, where policy interest in the concept of ecosystem goods and services is strong, especially at the state level.

Another example is the approach taken on the dissemination of water accounts data by the Australian Bureau of Meteorology, which makes use of web tools to present the data in a way that is visually engaging and informative. Other organizations also make good use of innovation in the area of dissemination. The program in Statistics Netherlands, for example, makes use of the functionality of the agency's website to present data to users in a variety of formats (tables, charts, maps and graphics) with a high degree of visual appeal and interactivity.

2.6 Program reach

Evaluation questions

Does EASP have an outreach plan or strategy to reach potential key players and potential users?

Are there good practices from other countries in terms of outreach activities?

During the evaluation period, EASP raised awareness of its products by conducting a number of activities, such as workshops, conferences (e.g., "Talking Stats: A Discussion Series with StatCan" with the topic of "Environmental statistics: Measuring natural capital for the long-term well-being of Canadians" (October 2014); conference on energy data and carbon pricing policy (October 2017)), presentations, training courses, teachers' kits (e.g., Environmental Statistics: Teacher's Kits and Educational Resources [16-507-X], and "Households and the Environment Survey – Elementary level teacher's kit" [16-507-x2015001]), social media campaigns through the official Statistics Canada Facebook and Twitter accounts (e.g., HAE and EnviroStats), Daily articles, other publications (e.g., EnviroStats) and email communications. These activities targeted mostly known users (e.g., workshops, conferences, training courses, social media, Daily articles and email communications), academic institutions (e.g., teachers' kits) and the public (e.g., social media and Daily articles). While evidence of the various activities was found, the information lacked specific details, such as information on the effectiveness of the activities.

EASP developed an outreach plan in 2015/2016 with the objective of improving data quality and access to its products and increasing awareness. According to the plan, enhanced awareness and access would be achieved by using marketing and communication techniques. Four areas were identified as priorities: liaison with stakeholders, communities of practice, The Daily and other publications, and participation in conferences and expert meetings.

For its implementation, EASP's outreach plan proposed two phases with various activities. Phase 1 was dedicated to the initiation of the plan (e.g., preparation of materials for outreach). Phase 2 proposed activities for the ongoing implementation of the plan. In order to secure success, it also recommended that a full-time senior employee be dedicated to the implementation of the outreach plan.

Many respondents indicated that there are gaps in outreach and that promotion activities could be improved. The suggestions included improving the communication of EASP's products and services (including changes over time) within the federal government; meeting directly with users and potential users; sending more emails about publications; promoting publications through social media; doing more workshops, presentations, webinars and conferences; and extending awareness activities to municipalities, students, academics, researchers, and other organizations that are not aware of EASP's products. Gaps in EASP's promotion activities were noted by all groups interviewed, including federal partners, the public sector and NGOs. Promotion channels, such as regular newsletters, social media and webinars, were identified as having the most reach potential.

Focus – International analysis

Promotion of statistics and accounts

Promotion of data beyond standard approaches, such as press releases and product launches, is not given high priority in the statistical programs reviewed. As for stakeholder consultation processes, promotion activities are largely informal. However, a number of activities to promote statistics and accounts are carried out across the organizations studied, including media releases, conference attendance and regular newsletters. Social media is also a popular mechanism for promoting statistics, particularly Twitter and YouTube. More specifically, the ONS's website—Visual.ONS—is devoted to the promotion of its statistics through short, simple releases that tie statistics to issues of current interest in national affairs. A recent release used ONS statistics on vehicles to address the country's readiness to make the switch to electric vehicles in response to the government's announcement that it would ban gasoline and diesel vehicles by 2040. As well, rather than creating standalone events to promote its data, the EEA launches its reports at events organized for other purposes. This helps reach new audiences and save resources.

Evidence suggests that outreach activities could be improved. In terms of awareness, there are opportunities to adopt a more systematic approach to promotion by using social media or other forms of regular communication. In addition, there is no evidence that an integrated approach was taken for the implementation of the 2015/2016 outreach plan. The plan was not implemented by a dedicated full-time senior employee, as recommended, but rather all staff members were deemed responsible individually for its implementation. Evidence suggests that the plan ended up being a guidance document, and that outreach activities took place informally through the staff's professional networks.

The Environment, Energy and Transportation Statistics Division (EETSD) is currently developing a revised outreach plan, which is divided into the three programs (environment, energy and transportation). The EASP component is being designed to ensure program relevance, broaden the client base, manage cost-recovery projects, and address Canada's international obligations. According to the draft outreach plan, future outreach activities will target federal departments, P/T departments, the international community, the business community and NGOs. At the time of the evaluation, no information was available on how the plan will be implemented (e.g., by dedicated employees or by all, as the previous plan) or how its success will be measured.

Although interviewees stressed the necessity to improve EASP's approach to outreach, there are mixed feelings internally on the upcoming revised outreach plan. While senior managers underlined the utility it would have in gathering users' needs and promoting EASP's products, others mentioned that the plan would have little impact on their work.

Given that the previous outreach plan was not fully implemented as originally envisioned and that stakeholders identified opportunities for improvement, efforts should be dedicated to a more systematic, integrated approach, including processes to measure success.

2.7 Use of information

Evaluation question

To what extent do Canadians and the public and private sectors use statistical information to inform public debate, research and decision making?

All lines of evidence suggest that EASP's statistical information is used by the private and public sectors for informing debate, research and analysis, as well as for decision making and developing policy. The evaluation found more evidence of use of EASP statistical information by EASP's federal partners than by any other user.

ECCC, a key stakeholder, uses multiple datasets from EASP for specific purposes. Some examples include survey results for the CESI (a key component of the Federal Sustainable Development Strategy [FSDS] required under the Federal Sustainable Development Act), surveys and accounts to support the FSDS, the SEPE to meet the information requirements of the Environmental Protection Act, waste management surveys to inform policy decisions, the Hazardous Waste Management Industry SurveyFootnote 24 to obtain information on the quantity of hazardous waste processed by the hazardous waste management industry, and the mineral resource account in the 2010-to-2013 FSDS to show the economic importance of mineral resources to the Canadian economy.

In addition to ECCC, EASP has three other major cost-recovery clients: NRCan, ISED and AAFC. ECCC and NRCan fund the Report on Energy Supply and Demand. More recently ISED joined NRCan in providing cost-recovery funds for clean technology statistics, and AAFC funds the Farm Management Survey. Some examples of uses by these clients are detailed information on the use of home heating equipment and fuels and on energy-using equipment in the household collected in the HES, and information on livestock and crop operations collected by the FMS to measure environmental performance in the agricultural sector for AAFC. As ecosystems and clean technologies are perceived as emerging priorities at the international and national levels, interest was also found in EASP statistical information on the impacts of carbon pricing on the Canadian economy.

Statistics Canada is also the lead on Target 17Footnote 25 of Canada's 2020 targets aimed at responding to the Convention on Biological Diversity. The objective of the target is to ensure the opportunity for the diverse values of biodiversity, its contributions to maintaining ecosystem services and opportunities derived from its conservation and sustainable use, to be fully reflected in all relevant public and private decision-making frameworks.Footnote 26 EASP reports on this indicator through the HAE and EnviroStats publications.

Other examples of use of EASP data found during the evaluation include the following:

EASP data found during the evaluation
EASP component Federal department General purpose of use
Physical flow accounts Finance Canada Direct policy relevance
Energy use and GHG emissions accounts Finance Canada Analytical research in support of policy
Environmental accounts Finance Canada Environmental impact assessments
Environmental protection account Commissioner of the Environment and Sustainable Development Reporting
Physical flow accounts (GHG emissions data) Foreign Affairs and International Trade Canada Environmental impact assessments
Source: Internal documents.

Evidence also points out that EASP statistical information was used by internal partners, in some instances to validate and consolidate their own information. For example, EASP shares data from the NRAA with the Macroeconomic Accounts Branch for inclusion in the National Balance Sheet Accounts (NBSA). The integration of natural resource data into the NBSA allows for a more complete picture of Canadian wealth and facilitates the balancing of the NBSA.

EASP's statistical information is also used by provincial governments (e.g., the Quebec government used EASP's urban sprawl measures to complement its Geographic Information System, provincial accounts on GHG emissions were developed with the help of IWS data, and provincial natural capital accounts were produced by using the SEGS), municipal governments (e.g., the City of Toronto uses data from the HES), research and education organizations (e.g., the University of Toronto uses data from the IWS), NGOs (e.g., the David Suzuki Foundation published an analysis of the expansion of settlements on Canada's best agricultural land using EASP data, and natural resource stock accounts data and related indicators were used by Genuine Progress Index Atlantic and the Centre for the Study of Living Standards to analyze environment-economy linkages), international organizations (e.g., the UN-SEEA and OECD), the private sector (e.g., water accounts used for the determination of a business model's sustainability), the Canadian public and the media.

The bibliometric and webometric assessment conducted for this evaluation provided evidence that EASP products are being referenced on the Internet and in international scientific journals. The webometric assessment found 943 hyperlinks on the Internet to products released by EASP, and the bibliometric assessment found 371 bibliographic entries (or citations) in the databaseFootnote 27 citing EASP documents. The academic sector accounted for the majority of the bibliometric entries (mostly Canadian researchers), followed by the public sector (mostly AAFC and ECCC). Hyperlinks to EASP products were most often found on private sector websites (41%), followed by private non-profit websites (37%), public sector websites (20%) and academic websites (2%).

While there is evidence that environmental statistical information is being used, EASP does not have a mechanism in place to measure the impacts of such use. This was also found internationally in the other environmental statistics programs reviewed.

Focus – International analysis

Measuring the impact of data use

Most organizations have no formal method for measuring the impact of data use. Informants in several organizations indicated that tracking web statistics is a simple way to measure the impact of data use. The EEA began to monitor mentions of the organization in publications released by the European Commission, the European Council, the European Parliament, EU agencies and certain interest groups. The collection of this information provides an understanding of how EEA products are being used by policy makers and other important stakeholders.

Some informants noted that there needs to be a distinction between "impact" and "data use." Environmental accounts and statistics programs should be held to the same standard when demonstrating impact as other programs. Interest in environmental accounts and statistics is also variable; when there are droughts, interest in water accounts is high, and when oil prices drop, energy accounts become of greater interest.

2.8 Utilization of resources

Evaluation question

Have EASP's resources been used as planned?

Overall program spending over the period was approximately $35 million (see table below) (annual average of approximately $7 million).Footnote 28 Expenditures increased steadily during the five years, except in 2014/2015, with the highest expenditures being in 2016/2017, approximately $8.9 million.

EASP had an average of 67 full-time equivalents (FTEs) annually. The number of FTEs dropped from 71 in 2013/2014 to 55 in 2014/2015. However, the number of FTEs increased in the subsequent two years.

EASP was affected by budget reductions stemming from Budget 2012. For example, the three-year Measuring Ecosystem Goods and Services (MEGS) project ended one year early, and a number of experienced staff members left. An effort was made to maintain as much of the core program as possible.Footnote 29 This could explain in part the decrease in expenditures and in FTEs in 2014/2015.

The evaluation did not find other significant fluctuations in program spending, and there was no indication of over- or underspending, or an excess in resources. When asked about the adequacy of financial resources, internal interviewees agreed that they are able to meet key user needs. Nonetheless, stakeholders have noted various data needs, and, given that EASP cannot address all of them with its current resources, it has to prioritize them. It is important to note that some data needs may be attributable to the fact that environmental statistics are relatively new compared with other more established fields, such as economic or social statistics, and that the need for environmental data has been growing steadily for the last few years, in part because of international commitments and an increased policy focus on topics such as climate change. Interviewees also agreed that EASP's work on clean technology could not be carried out without the funds provided in Budgets 2016 and 2017.

Environmental Accounts and Statistics Program expenditures ($), including actual full-time equivalents, from 2012/2013 to 2016/2017
Categories Fiscal years Total
2012/2013 2013/2014 2014/2015 2015/2016 2016/2017
Salary ($) 3,923,405 4,881,003 4,285,930 5,679,607 6,450,902 25,220,847
Non-salary ($) 207,636 380,736 195,869 304,587 331,678 1,420,506
Employee benefits and pension ($) 784,681 976,201 857,186 1,135,921 1,290,180 5,044,169
Accommodation ($) 510,043 634,530 557,171 738,349 838,617 3,278,710
Total ($) 5,425,765 6,872,470 5,896,156 7,858,464 8,911,377 34,964,232
Full-time equivalents 50 71 55 76 84  
Source: Statistics Canada administrative financial databases, 2012/2013 to 2016/2017, and computed information.

Focus – International analysis

Financial resources

All organizations reviewed, with the exception of Germany, reported constraints on the human and financial resources available for their programs. Resources decreased substantially in Australia and the Netherlands because of broader cuts in the public service that impacted the statistical offices. In the UK and the EEA, program resources have been more or less stable, though workloads are increasing, which amounts to an effective cut in resources. In Germany, resources have increased because of additional work on sustainable development indicators. In the Netherlands, an increase in the amount of cost-recovery work has helped offset declines in core budgets, but this comes at the expense of program flexibility, as the direction is dictated by the demands of the clients paying for the cost-recovery work.

Evaluation question

Are EASP's activities sufficiently integrated and coordinated to support the achievement of results?

In 2014/2015, a decision was made to join the environment, energy and transportation statistics programs because of the links that exist between the programs. The environment and energy programs had previously each been separate divisions. The creation of EETSD was intended, in part, to increase the level of coordination between the programs. The evidence suggests that efforts were dedicated to enhancing the collaboration and integration between the programs, but that there remain opportunities.

Senior management believes that the creation of EETSD has already simplified the work of the team and will continue to do so because it allows an easier integration of data coming from different sources and a better understanding of the work of each program by the others. In addition, there is evidence that cross-cutting analysis has taken place. For example, three analytical articles were produced that looked at issues that combined the energy, transportation, and environmental perspectives. One such article examined the oil and gas sectors through the lens of the three programs. Coherence analysis and data validation have also been conducted, for instance using household energy data collected through EASP to validate electricity data collected through the Energy Statistics Program. The Transportation Statistics Program team also helped to review the HAE on freshwater.

Another example is the teamwork between EASP and the Energy Statistics Program on the integration of data on energy supply and demand into the PFA and the RESD. These two components were consolidated into EASP to build from its expertise in account compilation to benefit the development of the RESD.

While efforts have taken place to foster collaborative opportunities in terms of data production, internal interviewees were not convinced that the new division has fully taken advantage of collaboration opportunities. Suggestions for integration included opportunities on the technical aspect on their work (e.g., production of statistics and related publications) and also on the administrative aspect. For example, most EASP surveys are administered biennially, while the Energy Statistics Program and the Transportation Statistics Program publish data annually, quarterly and monthly.Footnote 30 As EASP staff undertake the dissemination process infrequently, the process can be time-consuming, as they need to relearn things, and this limits their capacity to conduct other activities. A process to share knowledge in dissemination, or even a central dissemination team within EETSD, was identified as a possible area of collaboration that would increase efficiency.

Enhanced collaboration within EETSD could support EASP in addressing the recommendations identified in the evaluation. Information exchange and sharing of best practices could help EASP administer its surveys more effectively. Increased collaboration could also strengthen EASP's outreach strategy. A coordinated outreach approach, for example, with a dedicated employee overseeing division-wide outreach activities, could secure the success of such activities. Although the energy and transportation statistical programs were not part of this evaluation, increased collaboration could benefit them as well. For example, continued collaborations in spatial analysis. Similarly, discussions about cross-cutting needs could result in the production of analysis and publications meeting the needs of multiple stakeholders simultaneously.

Although integration is one of the topics discussed at EETSD bi-weekly management meetings, no evidence was found of a dedicated divisional-level forum where these types of opportunities for greater efficiency could be discussed in an in-depth manner.

There is evidence of collaboration with subject-matter areas outside the EETSD, including a corporate approach to geospatial analysis with Agriculture Division and the Statistical Registers and Geography Division, and the Environmental Statistics Committee, which involves several different subject-matter and service provision partners. EASP also maintains a relationship with the Macroeconomic Accounts Branch; during the period of the evaluation, quarterly estimates of the Natural Resource Asset Accounts were developed and published. These estimates are directly integrated into the quarterly National Balance Sheet.

Finally, there appears to be two different systems used within EASP to manage client requests. The first system, the Client Relationship Management System (CRMS),Footnote 31 is used by some to track and address requests. The second system, the "Environ" email box, is used by others to track and manage requests. One person is responsible for monitoring the email box and dispatching the requests within EASP, and another person manages the CRMS, which also creates a record in the email box. Not all request information from the email box is recorded in CRMS. While there is no indication that requests were being mishandled, there is a risk that requests could be lost and that duplication of work may occur, particularly as requests are growing steadily. In addition, the benefits of having a corporate-level system, such as a comprehensive client view, are lost. EASP should consider revising its approach to tracking and managing client requests.

2.9 Factors impacting efficiency

Evaluation question

Is governance of EASP working effectively and are partnerships being fostered and maintained?

EASP collaborates and maintains partnerships with its key stakeholders, both external and internal. Key partnerships have been established with federal departments, including ECCC and NRCan. For example, EASP is a member of the CESI Steering Committee along with ECCC and Health Canada, and of the FSDS Steering Committee with ECCC and multiple other federal departments. EASP is also part of the trilateral Climate Change Committee at the assistant deputy minister (ADM) level,Footnote 32 which includes NRCan and ECCC. Finally, along with NRCan, ISED, GAC, ECCC, AAFC, the Department of Fisheries and Oceans (DFO), Transport Canada and the Privy Council Office, EASP is part of the Clean Technology Working Group.

EASP has data sharing agreements with different organizations at the federal and provincial levels (such as ECCC, NRCan, AAFC, the DFO, the Ministère des Affaires municipales et de l'Occupation du territoire [Quebec], and the Ministry of Agriculture and Forestry [Alberta]) and Interdepartmental Letters of Co-operation with several federal organizations. For example, EASP has an agreement with a number of departments that governs the use of microdata from the Census of Mines administered by NRCan. The data are central for the PFA and the natural resource stock accounts. In 2016/2017, efforts were made to increase data sharing agreements, including Section 11 and Section 12 agreements for the SEGS and the WMIS.

Many key informants had positive comments on the effectiveness of EASP's involvement in partnerships. These included being responsive, open and willing to work with stakeholders.

Interviewees also noted that governance in environmental data and statistics could be made more inclusive, and some suggested that informal partnerships should be formalized. Some respondents identified the Canadian Centre for Justice Statistics or the CCME as a model (or, in the latter case, as a potential platform) for incorporating the views and expertise of organizations and bodies that do not, at present, play a significant role in governance. Other suggestions were to develop a mechanism that would operate in parallel to the mechanisms in place to support government collaboration in activities relating to biodiversity, or to establish a senior steering group that would incorporate representatives at the deputy minister or ADM level.

EASP is currently not leading any of the interdepartmental committees or working groups, but rather is a member. In that sense, relatively few stakeholders were positioned to comment on the effectiveness of EASP's current governance mechanisms, although several reported that EASP is currently developing or refining mechanisms to manage the generation of environmental statistical information, including the Census of the Environment and a "community of practice" emerging from the governance structure implemented for the MEGS initiative ("MEGS-Plus").

EASP participates in international organizations, such as the London Group, which provides a forum in which national and international expertise can be shared. By encouraging the adoption of statistical standards and best practices, the London Group contributes to the constant improvement of EASP. EASP is an active member of the OECD's Working Party on Environmental Information, whose purpose is "to exchange information, to further the development of high-quality and internationally comparable data on environmental conditions and trends, and to further the development of indicators relating to the environment and sustainable development."Footnote 33 Finally, staff at various levels participate in events to help develop and implement the UN-SEEA, including assistant chief statistician level participation in the UN Committee of Experts on Environmental Accounting.

Some key informants at the international level noted a lack of coordination, guidance, coherence and collaboration globally. This could affect the coherence of environmental statistical information available. Another challenge mentioned by some key informants was the limited capacity of various organizations in environmental accounting, since the international community of experts in this area is relatively small. According to some interviewees, global coordination and collaboration are essential to help the practice of environmental accounting at the international level. Canada's international collaboration and partnerships may help foster this. EASP's international collaboration and partnerships were viewed positively by key informants. No key informants viewed them as ineffective.

Focus – International analysis


Informants from several organizations emphasized the need to align the development of environmental accounts and statistics with the needs of policy departments. Not only is it important for environmental accounts and statistics to help inform policy development, it is also important to garner broader acceptance of environmental accounts and statistics, particularly new and experimental accounts such as ecosystem accounts. Having a close relationship to policy makers has been a key success factor in the development of environmental accounts in Australia, especially at the subnational level where environmental accounts are being developed within policy departments.

In the UK, the partnership between DEFRA and ONS in the development of ecosystem accounts is another example of strong interdepartmental support. The project steering committee includes senior managers from both ONS and DEFRA, as well as from other policy departments and the UK's devolved administrations. Developing environmental accounts along with policy departments ensures that the accounts will be relevant for policy; that is, it ensures that there is both an "information push" and a "policy pull." Resource constraints are a common issue confronting the development of environmental accounts and statistics. Working jointly on projects allows for resources to be pooled. Interdisciplinary knowledge has been pointed to as an important success factor by a number of informants. Working jointly ensures that there is a wide array of expertise available.

In the Netherlands, other government departments partnered with Statistics Netherlands in the past primarily to get access to data. In recent years, Statistics Netherlands has made a conscious effort to increase its value as a partner by developing new work units staffed with specialists in research. These units focus on projects that are fully cost-recoverable, allowing Statistics Netherlands to market itself to potential clients as more than just a data provider.

Partnerships (beyond those with other government departments) contribute to the success of environmental accounts and statistics in a number of ways. Informants noted that working on a project with a large number of partners gives political weight to the results and ensures they will be published and noticed by policy makers and the public. Partnerships can also help with the dissemination of environmental accounts and statistics products, as the networks maintained by partner organizations reach a broader spectrum of stakeholders than statistical organizations themselves. Partnerships with organizations outside government are important for the access to expertise they provide. Many environmental accounts and statistics programs work with academics and research institutes, including Statistics Netherlands' partnership with Wageningen University on its environmental data compendium.Footnote 34 The ONS in the UK has an important relationship with the private company Ricardo-AEA, which produces data on GHG emissions and energy and has extensive expertise in the field.

International partnerships are another element of success. For example, the ABS works with partners at the UN Statistics Division and the World Bank. The EEA similarly works with the OECD, the International Energy Agency, the UN (and its agencies) and the World Meteorological Organization.

3. How to improve the program

Evidence suggests that EASP is aligned with federal roles, responsibilities and commitments, and that the information produced by the program is of high quality. Given that the production of many parts of EASP is relatively recent, that the needs related to environmental accounts and statistics are in constant evolution, and that environmental concerns have been growing steadily for the last few years, there remain statistical gaps in the program. Because of its close relationship with key users and partners, EASP is aware of these gaps and is taking steps where possible to address them.


Given the rapid evolution taking place in the domain of environmental statistics, it is vital that EASP maintain strong relationships with users and actively promote its statistical information. The outreach plan developed by EASP in 2015/2016 was implemented only in part. Although several activities took place during the period, evidence suggests that the plan ended up being a guidance document, and that outreach activities took place informally through the staff's professional networks.

Evidence of the outcomes of consultations is documented in the end-of-year reports, showing that changes to the program took place as a result of various consultations. However, external interviewees noted that the inclusiveness of stakeholders and the communication approach, could be improved.

Recommendation 1

It is recommended that EASP enhance its outreach strategy to ensure a systematic and structured approach to consultations and promotion activities. The strategy should be inclusive and cover all key stakeholders, and follow-up communications should take place. Internal discussions should take place with staff to foster engagement and the use of corporate tools to measure success and client satisfaction should be explored.


Evidence suggests that EASP has started to take advantage of the integration of the environment, energy and transportation programs. There still remains however untapped opportunities for integration. These collaboration opportunities were identified on both the technical level (e.g., production of statistics and related publications) and the operational level (e.g., dissemination). Collaboration could also support a more integrated, strategic approach to outreach.

Recommendation 2

It is recommended that EASP, along with the energy and transportation programs within EETSD, identify additional areas for collaboration and implement formal mechanisms to support them. In addition, to ensure future efforts continue, EETSD should create a forum where ideas for collaboration can be discussed.

Management of client requests

Evidence suggests that EASP is using two different systems to manage client requests: CRMS and the "Environ" email box. While there are no indications of mishandled requests, the use of two systems, managed by two different employees, poses a risk of requests being forgotten and work duplicated. The risk will grow, as requests for environmental statistical data have been increasing steadily.

Recommendation 3

It is recommended that EASP revisit its current approach to tracking and managing client requests. The revised approach should be viewed divisionally to take into consideration the systems used by the Energy Statistics Program and the Transportation Statistics Program.

Timeliness and scope of the program

Overall, while EASP data users are satisfied with the environmental statistical information available and its quality; they noted several data gaps, highlighted the infrequent nature of many of the surveys, and commented on the late release times. All these factors impact the relevance and usefulness of the information.

Recommendation 4

It is recommended that EASP explore opportunities to increase the frequency of its surveys and to address the identified gaps. In addition, the program should take steps to improve the timeliness of its releases in order to meet the established timeliness target.

Focus – International analysis

The international analysis showed that environmental statistics programs abroad are also evolving. Support from government departments, key strategic partnerships, and sufficient levels of human and financial resources were identified as some of the key success factors. Although broad initiatives were being implemented with regard to the identification of users' needs, the measurement of satisfaction, innovation and dissemination, as well as the implementation of outreach activities, the programs reviewed mainly adopted informal approaches in these areas. Nonetheless, the evidence suggests that all programs reviewed were being proactive in their quest to address growing demands in terms of environmental statistical information. To have a leading international role in the domain, EASP needs to stay abreast of new developments and emerging trends. Its continued effective international involvement, combined with an improved outreach strategy and capitalization on synergy opportunities within EETSD, will support this leading role.

4. Management response and action plan

Recommendation #1

It is recommended that EASP enhance its outreach strategy to ensure a systematic and structured approach to consultations and promotion activities. The strategy should be inclusive and cover all key stakeholders, and follow-up communications should take place. Internal discussions should take place with staff to foster engagement and the use of corporate tools to measure success and client satisfaction should be explored.

Statement of agreement or disagreement

EETSD agrees with the recommendation.

Management response

EETSD will develop an outreach plan for 2018 that will take a systematic and divisional approach to consulting on data gaps and data needs. The plan will focus on key stakeholder groups such as provinces, territories, industry associations and not-for-profit organizations.

Strategic audiences for the outreach plan include:

  • Provincial and territorial governments - establish and where existing, strengthen the relationship
  • Municipal and regional governments
  • Partnerships in key academic networks
  • Business and not-for profit sectors - regional workshops to identify data gaps and provide updates on modernization initiatives.

The expanded plan will also include participating in outreach efforts related to initiatives that have a strong environmental component.

From an ongoing perspective, a divisional approach to outreach will be developed that will balance international and domestic connections. Once available, this work will be led through outreach funds that will be made available to EETSD in 2020/2021.

A basic suite of standard corporate tools to measure success and client satisfaction will be developed and will supplement those already used for each significant EASP release, such as the Physical Flow Accounts, the Human Activity and the Environment publication, the Industrial Water Survey, and the Households and the Environment Survey. An emphasis will be place on using corporate indicators and tools. This will include working with the Financial Reporting Division to set 2018/2019 targets on Departmental Plan and Departmental Results Indicators, which include an indicator on percent of users satisfied with statistical information (R3, indicator 3).

Specific work objectives related to outreach will be developed for all EASP employees, starting in the 2018/2019 fiscal year, and will form part of mid-year and end of year reviews.

Recommendation 1 deliverables
Timeline Deliverable(s) Responsible party
First quarter of 2018/2019 Hold workshops with some key data user types – such as academia and environmental not-for-profit organizations. Director EETSD
Include specific work objectives on outreach in the work objectives of all EASP staff. Director EETSD
Second quarter of 2018/2019 Hold meetings with representatives of the provincial governments to discuss strategic data partnerships and the use of data from environmental-economic accounts. Director EETSD
Develop a plan for systematically contacting other key data users and partners as detailed above. Director EETSD
Work with the Financial Reporting Division to collect data on corporate indicators related to client satisfaction. Director EETSD
Third quarter 2018/2019 Report on consultation outcomes through the Divisional Consultation Outcomes report in December 2018. Director EETSD

Recommendation #2

It is recommended that EASP, along with the energy and transportation programs within EETSD, identify additional areas for collaboration and implement formal mechanisms to support them. In addition, to ensure future efforts continue, EETSD should create a forum where ideas for collaboration can be discussed.

Statement of agreement or disagreement

EETSD agrees with the recommendation.

Management response

EETSD will create additional opportunities for divisional staff to learn about all three subject matter areas in EETSD by reviving its divisional seminar series.

EETSD will also regularly convene its management team to continue to develop ideas on integrating the three programs, both from an operational and a subject-matter perspective.

A number of planned EETSD initiatives will continue existing work to integrate analytical and data work from the transportation, environment and energy areas:

  • Proposed improvements to the accessibility of divisional data. This will include possible new data hubs on energy, transitioning to a low-carbon economy and on the relationships between human activity and the environment. Care would be taken to create useful links between these and the existing transportation data hub. These initiatives will also feature the participation of key data partners, such as ECCC and NRCan.
Recommendation 2 deliverables
Timeline Deliverable(s) Responsible party
First quarter of 2018/2019 Begin work scoping out the energy data hub and one for Human Activity and the Environment (completion data is dependent on capacity outside EETSD, so a completion date cannot be provided). Director EETSD (working with the Communications Division)
Third quarter of 2018/2019 Relaunch divisional seminar series. Director EETSD
Fourth quarter of 2018/2019 Short report documenting implemented, and proposed divisional integration activities. Director EETSD

Recommendation #3

It is recommended that EASP revisit its current approach to tracking and managing client requests. The revised approach should be viewed divisionally to take into consideration the systems used by the Energy Statistics Program and the Transportation Statistics Program.

Statement of agreement or disagreement

EETSD agrees with the recommendation.

Management response

An alternative approach will be developed to integrate the EASP's tracking and management of client requests. This approach will use corporate tools.

Recommendation 3 deliverables
Timeline Deliverable(s) Responsible party
Third quarter of 2018/2019 Modified process to tracking and managing client requests. Director EETSD

Recommendation #4

It is recommended that EASP explore opportunities to increase the frequency of its surveys and to address the identified gaps. In addition, the program should take steps to improve the timeliness of its releases in order to meet the established timeliness target.

Statement of agreement or disagreement

EETSD agrees with the recommendation.

Management response

EETSD will examine the need for any improvements to its program through the consultation exercise noted in Recommendation 1.

Regarding timeliness, EETSD will continue its current efforts to make improvements in this area.  Also, subject to budget, it will examine new methods to do so such as the use of modelling to create more timely estimates.

Recommendation 4 deliverables
Timeline Deliverable(s) Responsible party
Third quarter of 2018/2019 Incorporate elements of improving timeliness into the AEETS modernization plan. Director EETSD
Fourth quarter of 2018/2019 Presentation to senior management on data gaps as identified through the consultation process noted in Recommendation 1, as well as proposals for filling these data gaps using modern methods. Director EETSD

Appendix 1: Program description

The environmental statistics program at Statistics Canada began in the early 1970s under the direction of the Senior Advisor on Integration. The main product was Human Activity and the Environment, the first edition of which was released in 1978. A year later, Statistics Canada released the publication Towards a Comprehensive Framework for Environmental Statistics: A Stress-Response Approach.Footnote 35 The framework has gone on to form the basis of environment statistics in the UN (Framework for the Development of Environment Statistics), the OECD and several national statistical agencies.

The EETSD, within the Agriculture, Energy, Environment and Transportation Statistics Branch (Economic Statistics Field), is responsible for EASP. EASP's mandate is to collect, develop, compile, analyze and publish environmental data, emphasizing their integration with socioeconomic data. The objective is to provide users in government, business and the public at large with consistent, comprehensive, timely and relevant statistics with which to study the relationship between the environment and human activity. To fulfill EASP's mandate and objectives, four main activities are carried out:

  • environmental accounts: the integration of environmental data (those collected both within Statistics Canada and outside Statistics Canada) with socioeconomic data in the form of consistent, comprehensive databases that employ a variety of organizational frameworks
  • environmental statistics: the collection of environmental data directly from businesses, households and governments through ongoing and occasional surveys, as well as the integration of administrative data from other sources
  • spatial analysis: the research and development of environmental statistics (e.g., ecosystem stocks, and flows of ecosystem goods and services) for geographic areas that reflect features of the land rather than political or administrative boundaries
  • dissemination: dissemination of environmental statistics through a variety of catalogued products presenting descriptive analysis and statistics in electronic format.

EASP is under the responsibility of the Assistant Chief Statistician for the Economic Statistics Field, and its activities are overseen by the Director General of the Agriculture, Energy, Environment and Transportation Statistics Branch and the Director of EETSD. The operational management of EASP is led by an assistant director who is supported by five chiefs. The statistical program comprises three major elements around A-based and cost-recovery programming:

  • Environmental accounts:
    • Natural Resource Asset Accounts (NRAA)
    • Physical Flow Accounts (PFA)
    • Environmental and Clean Technology Products Economic Account (ECTPEA)
    • Ecosystem Accounts.
  • Environmental surveys (between 2011 and 2018):
    • Waste Management Industry Survey (WMIS)
    • Survey of Environmental Goods and Services (SEGS)
    • Survey of Environmental Protection Expenditures (SEPE)
    • Industrial Water Survey (IWS)
    • Agricultural Water Survey (AWS)
    • Survey of Drinking Water Plants (SDWP)
    • Households and the Environment Survey (HES)
    • Survey of Household Energy Use (SHEU)
    • Farm Management Survey (FMS)
    • Report on Energy Supply and Demand (RESD).
  • Technical and analytical publications:
    • The Environment Accounts and Statistics Analytical and Technical Paper Series
    • EnviroStats
    • Human Activity and the Environment (HAE).

Expected outcomes

EASP is part of the Economic and Environmental Statistics Program Activity (Sub-program 1.4, Environmental Statistics) in Statistics Canada's Program Alignment Architecture. It contributes to Statistics Canada's Strategic Outcome 1: Canadians have access to timely, relevant and quality statistical information on Canada's changing economy and society for informed debate, research and decision making on social and economic issues. To achieve EASP's objectives, EETSD identified the following key strategic priorities:Footnote 36

  • maintain ongoing production of existing environment statistics products
  • continue moving business environmental surveys to the Integrated Business Statistics Program
  • implement the Framework for Environmental Statistics in the identified priority areas
  • complete long-term planning, continuity and quality maintenance, and self-funded projects.

The following logic model describes EASP's key activities, outputs and outcomes. It is based on Statistics Canada's Quality Assurance Framework,Footnote 37 which describes the six dimensions for quality management (accessibility, timeliness, accuracy, interpretability, coherence and relevanceFootnote 38) and which is applied to all statistical data and products.

Figure 1 - Environmental Accounts and Statistics Program Logic Model

Figure 1.	Environmental Accounts and Statistics Program Logic Model
Description for Figure 1 - Environmental Accounts and Statistics Program Logic Model

This figure depicts the Logic Model for the Environmental Accounts and Statistics Program. It is divided into six layers and shows how each layer contributes upwards to achieve the ultimate outcome. The six layers are from top to bottom: ultimate outcome, intermediate outcome, immediate outcome, outputs, activities, and inputs.

The ultimate outcome is that “Canadians, public and private sectors use environmental statistical information to inform public debate, research and decision-making”. It is at the top of the model.

The intermediate outcome is that “Canadians, public and private sectors information needs are met by environmental statistical information”.

The immediate outcome is that “Canadians, public and private sectors have access to quality environmental statistical information and to associated support services”.

There are two outputs: statistical products which includes datasets, analytical products, research, advice and expertise; and, customized products and services.

There are two activities: program specific core activities according to the GSBPM which includes specify needs, design, build, collect, process, analyse, disseminate and evaluate; and, cost-recovery activities according to GSBPM.

The inputs are program specific data and infrastructure inputs. It includes subject matter input, tools, IT infrastructure, resources, standards, methodology principles and guidelines. The inputs are the lowest layer.

GSBPM referred to under activities is the Generic Statistical Business Process Model.

For quality as is referred to under the immediate outcome, Statistics Canada defines the quality of information in terms of its fitness for use. This is a multidimensional concept embracing both the relevance of information to users' needs, and characteristics of the information such as accuracy, timeliness, accessibility, interpretability and coherence that affect how it can be used.

Program resources

The expenditures for EASP over the last five fiscal years totalled $34,964,232.

Appendix 2: Evaluation limitations and mitigation strategies

Three main limitations were identified, and a set of mitigation strategies was employed.


  1. Given the scope of the evaluation, it was not possible to assess in detail specific surveys or accounts, nor was it possible to assess quality dimensions.
  2. Because of the large number of partners and users covering a wide variety of areas, not all stakeholders could be included for interviews.
  3. EASP conducted limited ongoing data collection on EASP client satisfaction.

Mitigation Strategies:

  1. The use of a general inductive approach allowed the evaluation team to conduct an assessment at a broader level, while keeping the possibility of conducting in-depth analysis if specific issues arose.
  2. The use of other lines of evidence, such as a survey of key users, allowed the evaluation team to validate interview findings.
  3. The evaluation collected data on client satisfaction through key informant interviews and the survey of key users. Data were limited particularly for satisfaction with older products (e.g., more than three years).
Date modified: