The Consumer Price Index and the CPI Enhancement Initiative 2008-2009 to 2012-2013

Relevance

The CPI is an important measurement used by the Government of Canada and other key stakeholders to inform economic and monetary policy. It is used in the development of the GDP measure. The Bank of Canada uses the CPI to measure rates of inflation; Finance Canada uses it to help determine transfer payments to the provinces and territories. Internationally, the CPI is one of the most widely-used statistics for measuring price changes and economic performance; it is the most visible and credible measure to anchor inflation expectations. Consequently, there is a continued need for the CPI; it remains highly relevant.

The CPI is aligned with federal and provincial government priorities, and is used by all levels of government. The production of the CPI is an ongoing priority for Statistics Canada. The Government of Canada has a legitimate role and responsibility to produce the CPI. Statistics Canada, given its ability to provide objective and non-partisan statistics, should be the agency to do so.

Design and Implementation of the CPI-EI

The evaluation found evidence that the CPI-EI was necessary to address known weaknesses of the CPI, and to ensure that the CPI continues to be a high-quality, reliable index. Evidence suggests that the design of the CPI-EI is sound, and that the initiatives underway are the right ones to improve the CPI. Furthermore, there is evidence that the CPI-EI is being implemented as planned; some elements are being implemented ahead of schedule.

The evaluation found CPI-EI governance arrangements to be appropriate and sufficient to support the initiative and the ongoing management of associated risks. However, there is some concern about the clarity of roles and responsibilities within CPD, and about governance arrangements and mechanisms understanding for managing cross-cutting issues at lower levels in the division. Another concern is the potential erosion of subject-matter expertise — either through turnover or retirement — and the potential risks and impacts on the CPI.

Achieving Expected Outcomes

Good progress has been made toward realizing CPI and CPI-EI outcomes at the immediate and intermediate level. CPI is timely, relevant and, despite some weaknesses in availability of information and limitations of the index, credible. The CPI is an excellent index when used for its intended purposes. The evaluation's findings confirm that, as a result of the CPI-EI, biases in the CPI will be reduced, leading to a more reliable and credible price index with greater internal capacity. As for the ultimate outcome, that "Canadians have accurate, timely, relevant set of consumer price indexes meant to support a variety of key policy programs," the CPI program is well positioned. More specifically:

Economy and Efficiency

A review of program financial data indicates that, from a financial perspective, the CPI-EI project is being implemented according to plan. Expenditures on the CPI and the CPI-EI appear to be in accordance with the original budget. The CPI seems to use an efficiency-based approach in its practices. Some evidence shows that the CPI has efficient administrative and operational procedures.

There is room to improve efficiency by implementing a performance measurement system, including harmonizing processes used in the CPI with those used in the System of National Accounts and the Survey of Household Spending Household Survey. Some alternative approaches and considerations for the CPI were identified, such as moving the CPI to a one-year update cycle; making use of alternative data sources and data collection methods; examining the benefits of introducing a superlative index and allowing for revisions; and using seasonal adjustments.

Recommendations emerge from the evaluation findings that advocate strategic planning for sustainability after CPI-EI complete implementation, enhanced governance at the sub-project level, improved communication with stakeholders, accessibility of CPI products, and strengthened capacity for systematic performance measurement to demonstrate the achievement of results.

2.1.1 The CPI-EI

In 2009/2010, the Government of Canada introduced this five-year, $45-million initiative to improve the CPI based on economic theory and best practices, in order to estimate price changes most accurately and to use the CPI's resources most cost-effectively. Statistics Canada received new funding to permit improvement to Canada's CPI in these areas:

• Sample size activity aims to increase the number of prices collected. The number of observations taken relative to the size or characteristics of a population can affect the quality of the inferences made about the population being measured. As the sample sizes of some products in Canada's CPI are considered too small to be representative, the CPI-EI aims to increase the number of prices collected of goods and services purchased by Canadians. The objective is to double the number of prices collected (from 60,000 to 120,000) within the next five years — approximately 5,000 additional price quotes in each of the first three years, and 15,000 in the fourth year and 30,000 in the fifth.
• Sample design activityaims toimprovethe selection of stores and products to be sampled. These changes should better reflect what Canadians are actually purchasing, as well as shifts in consumer product and store preferences. They should improve timeliness in the introduction of new outlets and products, as well as avoidance of oversampling of less significant products and/or undersampling of popular items. The CPI-EI objective is to improve the sample design by using more scientific sampling methodologies and better targeted, more efficient collection. These changes should be fully operational by year 5.
• Basket updates activity aims to ensurethat the weights of the products in the CPI basket are updated more frequently and processed more quickly. Prior to the CPI-EI, the CPI basket was updated every four years: each update was based on data 18 months previous to the year the expenditure data were gathered. This led to the basket becoming gradually outdated and not being reflective of the recent changes in consumers' buying patterns. CPI-EI committed to providing a biannual update of CPI basket weight by 2013/2014 and to reducing the time to incorporate a new basket into the CPI to 12 months to make updates a better and earlier reflection of consumer spending.
• Quality adjustment activity aims to develop new methodologies to better distinguish pure price movement from that resulting from changes in product quality or product characteristics. Currently, about one-quarter of products in the CPI are quality adjusted. The CPI-EI targeted the development of better methodologies for an additional 75 products by 2014/2015, bringing the total number of adjustment to 225 products. Coverage will be extended to appliances, furniture and electronics.

These areas are supported by two key IT infrastructure projects that are critical enablers to the CPI-EI.

• Data collection infrastructure improvements aim to increase management capabilities to control collection activities in the field. This will better align collection activities to the CPI's data needs. The intention is to ensure that products and stores are better targeted and that more information is collected for improved quality assurance. Hand-held devices used in the field for data collection will be replaced and their software upgraded.
• Data processing changes aim to improve the CPI's data processing and index estimation infrastructure. The index estimation system (MPS), which constructs indexes from the raw data, is built on technology and systems design principles that are over 20 years old. It is also a complex system not designed to efficiently accept frequent changes to basket methodology. The objective is to modernize the technology used in the index estimation and improve its documentation, resulting in a system that is easier to use and maintain.

To support these four key activities and two IT infrastructure initiatives, Statistics Canada developed a recruitment strategy to hire new employees and a dedicated training program to shorten learning curves.

In 2010/2011, Statistics Canada started to receive funding to undertake the CPI-EI. In 2012/2013, the CPI-EI began its third year of implementation. In this fiscal year, the initiative has focused on developing, piloting and implementing new business processes, IT systems and sample changes.

2.2.1 The CPI

The CPI's objective is to reflect changes in consumer prices experienced by Canadians. This could be achieved through a diverse set of activities, such as collecting prices, applying stringent quality control measures to produce error-free and quality adjusted prices, providing representative weights at timely intervals, periodically re-evaluating and improving methods in line with international standards, as well as analyzing and disseminating data.

The immediate outcomes from these activities are to produce trusted data, put information products in the public domain and ensure that released data rest on a robust IT infrastructure. The intermediate outcomes include supporting key programs; providing high quality service to users; enhancing the design of the program (i.e., concepts, methods and data sources) and its IT infrastructure. The long-term outcomes are to ensure that Canadians have access to accurate, timely and relevant set of consumer price indexes meant to support a variety of key policy programs (monetary, fiscal and social), as well as meeting System of National Accounts (SNA) requirements.

2.2.2 The CPI-EI

The CPI-EI's objective is to produce a better measure of price movement in Canada based on sound and internationally respected methodologies and supported by a more efficient and stable technical infrastructure.

Through the four key activities and two support initiatives mentioned in the section above, the anticipated CPI outcomes include: more accurate (representative) product prices (averages); better reflection of what Canadians are buying, leading to a more representative CPI; better understanding of changing habits of Canadians; and better estimates of pure price change (removing noise in the movement of prices arising from changes in product characteristics). Like the CPI, the CPI-EI supports the long-term outcome: "improved quality of policy decision making from using an enhanced CPI."

The enhancement initiative is embedded in the CPI. CPI-EI's activities are meant to improve the CPI's ability to achieve stated outcomes. For further details, Appendix A presents logic models related to the CPI- and the CPI-EI.

2.3.1 Program Management

CPD, a member of the Business and Trade Statistics Field at Statistics Canada, is responsible for the overall management of the CPI and the CPI-EI. The division's mandate is to produce the CPI based on timely and relevant data and analysis of consumer price change temporally (over time) or spatially (geographically).

CPD comprises the production sub-division and the development sub-division, which are involved in the CPI and CPI-EI.

Production sub-division is responsible for producing a wide range of temporal (base funded) and spatial (cost recovery funded) price indexes monthly. It is also responsible for coordinating releases, responding to information requests and developing and integrating a divisional information management framework.

Development sub-division is chiefly concerned with developing methodologically and conceptually sound processes to ensure ongoing and sustainable CPI relevance and quality. It also produces purchasing power parities and spatial program outputs. This work is conducted as part of sectional operations and projects that are part of the CPI-EI. This sub-division hosts the Cygnus development section. This group is responsible for redesigning the index estimation system, and for building subject-matter expertise on price index estimation.

2.3.2 Governance and Project Management Structure

A number of organizational changes were made to the CPI to support the introduction of the CPI-EI and the Departmental Project Management Framework Guidelines in April, 2012. CPD distinguishes the governance structure from the project management structure. The CPI-EI governance consists of two tiers of steering committees (decision-making bodies): a director general-level steering committee for the CPI-EI, responsible for oversight of the whole CPI-EI; and three director-level steering committees, chaired by the director of CPD, which oversee components of the CPI-EI.

The project management structure consists of boards and committees that are responsible for the strategic aspect of the program, including the management board, the CPI-EI Management group and the sub-project groups.

More details on governance and the CPI structure are presented in the project governance organization structure chart in Appendix B.

2.3.3 CPI Key Stakeholders

CPD has several internal and external key stakeholders that provide inputs, such as raw data or services, or who benefit from or use the outputs, such as price indexes.

Figure 1. Consumer Prices Division and its stakeholders

2.3.4 CPI Program Resources

While Statistics Canada receives ongoing funding for the Consumer Price Index, since fiscal year 2010/2011 the CPI program has also received supplemental A-base funding to implement the CPI-EI for a total of $17,238,157 in 2012/2013 (Table 1). According to internal data, this new funding was allocated to all activities related to the CPI to support new activities. Total funding for the CPI and the CPI-EI is detailed in Table 2.

3.3.1 Preliminary Consultation

Prior to undertaking the evaluation, preliminary interview consultations were done with CPI leaders. This led to the validation of the logic model and the development of an evaluation work plan, including the methodological approach.

3.3.2 Lines of Evidence

Data for this evaluation were collected from multiple sources incorporating four lines of evidence: a literature review, document and jurisdictional reviews and interviews with key informants. Logical and statistical analysis was used and findings were correlated to identify trends and patterns. The evaluation team synthesized and analyzed the preliminary findings related to each line of evidence into a summary matrix.

Literature review: The review covered 21 pieces of literature, and examined national and international academic sources, as well as studies produced by stakeholder organizations. This review explored the need for a CPI; some of the key issues and challenges for CPIs; and general trends and recommendations for addressing key CPI challenges and weaknesses.

Documentation review: This review included two categories of documents: broader government-wide documents, including appropriate acts; and Statistics Canada's program-specific documents. The review of the program-specific documents provided information on the rationale and continued need for the CPI and CPI-EI and alignment with overriding government priorities and Statistics Canada's legislative mandate; basic financial information on the CPI and the CPI-EI; as well monitoring and reporting documents in relation to the overall CPI-EI plan. The review of these documents informed evaluation issues concerning overall performance of the CPI-EI.

Jurisdictional and benchmarking review: As part of this review, a high-level comparison of the CPI approaches of six other countries—Australia, the United States, France, Netherlands, New Zealand, and the United Kingdom—to that of Canada was done pre- and post-CPI-EI. This review included comparison of different CPI baskets (e.g., number of stores, frequency of updates), sample size, distribution of sample, quality adjustments and revision approach.

Key informant interviews: 28 stakeholders from various key informant groups were interviewed. They were:

• program representatives involved with CPI production and release, including senior leaders, chiefs and subject-matter experts from CPD, those leading system upgrades, methodology and collection changes, as well as representatives from Statistical Information Systems Division and the Survey of Household Spending — 15 individuals were interviewed in 12 meetings
• end users of the CPI, including representatives of the Bank of Canada, Finance Canada and other interested federal departments; representatives from provincial finance ministries (eight including one internal end user); and a national association with an interest in the CPI
• five Canadian and non-Canadian internationally recognized experts.

3.3.3 Methodological Limitations

Below are some of the key challenges and limitations in relation to the evaluation of the CPI.

4.1.1 A Continued Need for the CPI

A review of program documents and the literature, as well as consultations with key informants, showed that the CPI continues to address a demonstrable need. The CPI is a critical index that is used around the world to track and measure price changes.

The CPI is widely used as an indicator of the change in the general level of consumer prices or the rate of inflation. It is used in three ways in Canada:

• SNA deflation—to transform nominal variables into real variables, for example, to distinguish real changes in GDP from those due to inflation
• indexation—to compensate individuals for price changes, for example, to adjust the value of government transfers to individuals
• inflation target—to inform monetary policy made by the Bank of Canada, the CPI feeds into the calculation of inflation and interest rate.^{Footnote 1}

Internal key informants and documents attested to the fact that the price-level information in the CPI is vital to a wide range of personal, government, and business decision makers.

Within Statistics Canada, the CPI is important, even central to other agency activities. For instance, the CPI is a key input for macro-economic programs to monitor current economic conditions, such as a deflator for the GDP index for the System of National Accounts.

Externally, the CPI is an important index used by the Bank of Canada to help set interest rates. The Bank also uses a Core CPI published by Statistics Canada.^{Footnote 2} The Bank monitors core inflation to help inform inflation targeting, not as a replacement for the CPI.^{Footnote 3} The Bank of Canada also maintains an inflation calculator, which presents the impact of inflation on purchasing power, based on monthly CPI figures dating back to 1914.^{Footnote 4} As a Bank of Canada report mentions, the CPI is used extensively for these purposes because it is available on a monthly basis, it has a short publication lag, and it relies heavily on retail transaction prices rather than imputed prices.^{Footnote 5}

The CPI is also used by Finance Canada to determine transfer payments to provinces and territories and by other government departments as an escalator for federal programs. Key informants also noted that the CPI is used by government representatives as a good reading on the pace of inflation in Canada and to inform various policy and budget decision-making advice and recommendations related to tax policy, labour market decisions as well as structural efficiency and inefficiency. The CPI is also used by governments and other sectors to adjust pensions, income taxes and contracts, such as wages, rents, and leases, as well as by universities for research.

The uses of the CPI affect all Canadians at some point in their life. For example, the CPI is used to index the Old Age Security, which impacts more than 4 million people every three months and the Canada Pension Plan, which impacts more than 5 million people every 12 months.

Internationally, the CPI is one of the most widely-used statistics for measuring price changes and economic performance. For example, the International Monetary Fund (IMF), of which Canada is a member, uses the CPI as a tool to assess countries' economic and financial development and to ensure that countries pursue policies that are conducive to orderly economic growth and reasonable price stability. According to internal key informant interviewees, CPI is also used by the OECD, in spatial indexes, and by the United Nations.

5.3.1 Governance

A CPI-EI governance document^{Footnote 29} details the governance structure that supports the CPI-EI (Appendix B). Documentation provided and consultations with internal key informants show that the CPI-EI governance arrangements are reasonable: they are also adequate to support the implementation of CPI-EI and to manage associated risks. Interviews with key internal informants and selected external experts confirmed that the CPI-EI governance structures are in place and working well.

In addition to its governance structures, CPD has an advisory body, the PMAC. At its annual meetings, PMAC plays the critical role of bringing international expertise and best practices to inform Statistics Canada's approach to the CPI; acting as a sounding board for new ideas; and performing a challenge function. According to some key informants, however, there is room for the PMAC to improve, "by focusing on the setting of the agenda, asking the right questions, and narrowing down priorities and trade-offs."

5.3.2 Project Management

Being a highly specialized division, CPD had significant statistical expertise but needed "more planning expertise to deliver on time and budget." At the beginning of the CPI-EI, the practice in the division was more reactive than strategic (e.g., "to do more acting than planning.") CPD invested in project management training, in particular the Project Management Body of Knowledge (PMBOK) principles for CPD managers. This investment was viewed as taking time to develop capacity, but most internal informants believe that it is working and that the program management practices have improved: the PMBOK principles provide rigor and objective measures for monitoring project status, focusing on effort expended (time, budget, and resources).^{Footnote 30} This, internal informants attest, has helped build a strong project management capacity and culture. Examples of relevant management practices are presented in Appendix E.

Program documents show that, as of 2012, CPD has adopted the Departmental Project Management Framework (DPMF) and has developed several tools to facilitate the planning process and to provide inputs into the DPMF templates.

5.4.1 Governance Challenges

Interviews with internal key informants provided evidence of some challenges in setting up the CPI-EI governance structure and ongoing project management. The primary reason raised was that, operationally, the CPI-EI was set up and funded as a single project; however, the project actually consists of six inter-related yet independent sub-project groups.^{Footnote 31} Further challenges were noted:

• Governance arrangements do not seem to be well understood by all CPD staff, particularly those at lower levels. This is also related to the need to further clarify areas of responsibilities and hand-offs in areas such as sample design, QA and basket updates.
• A few internal informants noted that there were no effective mechanisms for bringing cross-cutting issues to the table for discussion at lower levels in the organization. They made reference to the previously existing Issues Committee, composed of economists, where areas of interest to the CPI such as research, and findings and recommendations used to be presented to management. These informants suggested that there continues to be a need within CPD for a horizontal mechanism at the chief, director, senior researcher and economist levels to discuss cross-cutting issues. This may have been addressed with the recently established practice of weekly integration meetings.

5.4.2 Capacity-building and Continuity-planning Challenges

Given the level of complexity of the CPI and the CPI-EI, internal key informants noted that CPD requires an expanded body of professional expertise in the theory and practice of price index numbers. This skill set is relatively scarce; few universities have graduate programs to address it. At the launch of the initiative, CPD needed to recruit additional personnel from within and outside Statistics Canada, and then develop these resources to meet the project needs. The hiring process and subsequent learning and development initiatives took investment of time and effort during the first two years: this investment is expected to yield substantial returns in subsequent year of the project. Program progress reports indicate that these activities have succeeded in building capacity and have not slowed project implementation.

However, several internal informants indicated concern that the CPD has lost significant subject-matter expertise, primarily at the chief level and above, as a result of retirement and organizational turnover. There is concern that an erosion of subject-matter expertise could put CPI-EI implementation and CPI modernization at risk, especially with respect to succession and continuity planning.

6.2.1 Web-based Accessibility

The documentation reviewed showed that Statistics Canada's CPI products, research and publications are made public in an open and regular manner on its website. Table 5 shows that users with Internet access could choose from a number of products since fiscal year 1997/1998.^{Footnote 37} However, between fiscal years 2008/2009 and 2012/2013, only two new publications and four information documents for analysts and researchers were publicly released. This confirms some key informants' perceptions that Statistics Canada was not releasing enough updates and recent publications on the CPI.

The main tool to disseminate and inform Canadians on the CPI is The Daily. This document is a record of the first official release of new datasets and products, and is also used to inform the public of the significant and noteworthy findings contained in the released CPI datasets and analytical products. To date, The Daily is the most efficient way to meet IMF standards for data releases. Published in conjunction with The Daily, the Consumer Price Index module is amonthly publication, geared toward educated laypeople and expert users on CPI data. It offers a more detailed monthly analysis for each CPI component, with accompanying charts, tables and briefing notes.

According to internal data, during fiscal year 2011/2012 the CPI module in the Daily was the most viewed part of the Statistics Canada site, with a total of 515,958 viewers.^{Footnote 39} End users consulted indicated that more information could be made available through The Daily such as monthly variation and rolling averages across different time periods.

Users can also access detailed CPI data from Summary tables, CANSIM and ESTAT. CPI data are loaded on CANSIM, a database that is updated daily. Until 2012, E-STAT was largely used to access CPI data. However, since February 1, 2012, all Statistics Canada's self-serve standard products, including CANSIM data, became available free of charge in its website, rendering E-STAT no longer relevant. E-STAT is no longer updated and will be discontinued July 1, 2013.

The availability of free data and the dissemination strategy of Statistics Canada's communications and dissemination group increased the access and availability to statistical data. Table 6 shows that publication views grew 215 percent and the viewed Summary tables grew 36 percent over the period 2006/2007 to 2011/2012. CANSIM information has accounted for more traffic since it became freely available in February 2012. The CPI data are the most popular table downloaded by CANSIM distributors for their clients.

A few external end users raised issues about the accessibility of information because of the user interface of Statistics Canada's website. To find the right information, the user has to know their way around the Statistics Canada website in relation to the CPI. Some of the external end users consulted found searching for CPI-related information to be time-consuming. But the general view of all the experts and most end users consulted was that basic CPI information and products are well disseminated and communicated by Statistics Canada through its website.

Social media are also used to disseminate information on CPI. Registered followers can receive updates on CPI from Twitter (since January 2010), RSS feeds and mobile web (since March 2011), as well as Facebook (since February 2012). One user indicated that the RSS feeds increase the timeliness of the CPI data release.

The Statistics Canada CPI publications are frequently reported in the national media. According to an internal reporting document, the print, television, Internet and radio media use CPI data extensively — so extensively that it is difficult to fully assess media usage. In 2008/2009 fiscal year, there were 340 media mentions of the CPI in the 45 main news sources monitored in Newsbriefs. For fiscal year 2011/2012, the number of media mentions rose to 485 and as of March 26, 2013, the number of mention in the media for fiscal year 2012/2013 was 332 for fiscal year 2012/2013.^{Footnote 40}

6.3.1 More detailed information expected

For particular needs, end users consulted expressed appreciation that Statistics Canada now offers greater access to the CPI microdata:^{Footnote 41} this access enables them to run a deeper level of analysis. Internal key informants noted that if CPI information related to the above-mentioned access is released to the public, the underlying confidential data are not made public.

Key informants consulted indicated that the CPD could do a better job of communicating CPI information. External informants indicated a need for Statistics Canada to explain the CPI in simpler language that the average person would understand and, ultimately, for end users to explain the CPI to various stakeholder groups. The educational approach of the Bank of Canada website was raised by an expert as a good example to follow. Specific areas identified include, what does the CPI measure; how is it developed; what does it mean to Canadians; and how does Canada's CPI compare to CPIs of other jurisdictions. Better, simpler information on the CPI would generate efficiencies by reducing or eliminating the need for CPD and other government end users to spend time explaining the CPI to those who enquire.

More frequent updates on the CPI and the CPI-EI are also needed. According to key informants, little is known publicly about the CPI-EI to the point that other than CPD's primary clients — other end users knew little about the CPI-EI or its progress. Most users interviewed suggested that updates on CPI-EI should be shared via the website. The CPI reference paper and updates on the methodology used is also an issue for end users. A few key informants noted that the reference paper is dated and would require frequent updates to ensure it reflects changes in calculations and current practices at Statistics Canada.

Informants suggested that more open communication between CPD and the public may help reduce mixed messages and misinterpretation on the validity of the methodological approach, and may help key stakeholders defend Statistics Canada data and methodology.

6.3.2 Quality of Service Support

External end users reported that Statistics Canada staff is competent and have the knowledge needed to respond to specific requests. However, it can be difficult to identify the most appropriate expert at CPD who can respond to a given request or discuss CPI questions and issues. While calls to request help on the CPI are responded to, end users indicated that it can take up to two days for Statistics Canada to follow up. These timelines are not helpful to federal representatives or officials in provincial/territorial finance departments trying to answer an urgent question about a CPI release (e.g. minister's briefing notes).

Experts and end users indicated that it would be advantageous for Statistics Canada to publish a list of officials to contact for more information about specific technical topics such as basket composition, how the CPI is calculated, or sub-index composition and weighting. They suggested that such a list of officials who are expert in specific areas would facilitate getting CPI questions answered more promptly.

According to a 2003 IMF report,^{Footnote 42} Statistics Canada's CPI has generally followed internationally accepted guidelines and good practices, and has met the international standards^{Footnote 43} (see Table 7) for several elements: prerequisites to quality, integrity, methodological soundness, accuracy and reliability, serviceability, as well as accessibility. Of these, the elements that directly pertain to research and development are methodological soundness and accuracy and reliability.^{Footnote 44 , Footnote 45}

A wealth of detailed information regarding the CPI is available in the public domain. The CPI reference paper, various research documents on the CPI, and the CPI sub-indexes are public. CPI data are public, as well as the CPI basket, basket updates, and the weighting of the elements in the basket. This open approach of Statistics Canada enables public and independent parties to review and challenge the CPI, if need be: thus, it adds to CPI credibility.

According to internal key informants, research and publications on various topics related to the production of the CPI are published periodically. In addition, Statistics Canada officials participate regularly in international expert meetings with their counterparts in other countries. This exchange of information among international experts is seen by the some of the key informants as an opportunity to advance thinking and enable different jurisdictions to exchange views on statistical matters: this, in turn, makes it possible for CPD to benefit from practices in other jurisdictions. Along those lines, key informants acknowledge PMAC's role in providing independent expert advice and a challenge function. This demonstrates Statistics Canada's openness to outside scrutiny and dedication to meeting high quality standards, thus, contributing to the reputation of Canada's CPI.

Overall, key informants agree that the CPI can withstand the test of expert and public scrutiny. Nevertheless, they feel that there is always room for improvement, and recommend continual research and development effort to ensure that CPD keeps abreast of emerging theories and practices. Experts and end users identified areas that could benefit from further research such as continuous evaluation of price changes and their impact. Examples also include owner-occupied housing, price movements of rents, seasonal sales and automobiles. However, there was no consensus among external interviewees on which areas are of greatest priority.

Evaluation findings previously presented indirectly support the observation that CPD's research and development in recent years, particularly in relation to the CPI-EI, has led to improvements in the CPI.

More specifically,

• Issue 4, Design and Implementation of the CPI-EI: evidence was presented that the design of the CPI-EI is sound, the initiatives underway are the right ones to improve the CPI, and the CPI-EI is being implemented as planned and is progressing well.
• Evidence that recent changes introduced with the CPI-EI are well aligned to produce more trusted and relevant data were found regarding data production (Section 6.1)

Recent program reports confirm that, under the CPI-EI, the program focuses its research and development capacity in assuring that the CPI is accurate (e.g., represents what it seeks to measure), coherent (its component parts are integrated harmoniously), and renewable (e.g., it can perpetuate and reinvent itself over time).

For example, to increase the representativeness of the sample, the CPI-EI improved the program's 'bricks and mortar'^{Footnote 47} frame, to ensure the sample is "not probabilistic but rather a result of a judgmental sampling, applied in a much more sophisticated way than before." The improvements support the use of alternative inputs (such as Census, Business Register, Quarterly Retail Commodity Survey, other surveys, and third-party outlet or product databases); the use of different collection modes (including scanner data, web scraping, and internet) and the introduction of some pure Internet elements.

The coherence of the CPI has been improved as a result of reviews of key methodologies; alignments applied to CPI classifications and the aggregates' formulas; and coordinated or simultaneous updates of upper- and lower-level indexes. In the same vein, the program documents testify about stronger links and better communication with the analysis unit and the field collection staff, and about increased capacity and know-how on processing larger volumes of price quotes.

Internal interviewees specifically mentioned the research conducted to assess the impact of changing the sample size in order to broaden the sample to account for rural areas. Those consulted also describe the benefits of Cygnus, the new powerful price index estimation system, for research. The advanced IT tools of Cygnus will allow internal users and researchers to run simulations, which will help improve understanding, inform future improvements and quality adjustments. Internal key informants believe that Cygnus will open up research and analysis and will increase the accuracy of the CPI.

All categories of key informants agree that CPI-EI is on the right track to achieve its outcomes, including: broader range of prices selected, a more representative sample, more timely reflection of changing consumer spending patterns and advanced techniques to implement quality adjustment.

Expert informants acknowledge the contribution of Statistics Canada's CPI research and publications related to CPI, and suggest some areas of additional research that could generate future benefits:

• Mini-reviews of the basket every year would allow CPD to tweak it, and are needed because the rapid change in consumer experience could not be accurately predicted;
• Use of scanner data and web-scraping data. In this regard, experts clarify that the use of scanner data for improving the sampling is different from the use of scanner data as a data source to compile CPI, thus, different research goals and challenges are associated with each;
• A comprehensive owner-occupied housing index;
• Alternative data sources. In this regard, the experts reported that other countries (not only Canada) struggle with many challenges linked to the use of alternative data sources (such as issues, expenses, and problems of cleaning the data), and that so far no simple and affordable solution exists.

7.1.1 Relevance

The CPI is an important measurement used by the Canadian government and other key stakeholders to inform economic and monetary policy, such as developing the GDP, measuring rates of inflation for the Bank of Canada and determining transfer payments to provinces and territories by Finance Canada. Internationally, the CPI is one of the most widely used statistics for measuring price changes and economic performance; it is the most visible and credible measure to anchor inflation expectations. Consequently, the need for the CPI continues, and the CPI remains highly relevant.

The CPI is aligned with federal and provincial government priorities. The production of the CPI is one of the ongoing priorities of Statistics Canada. It is used by all levels of government. It also responds to the priority of transparency by providing objective and non-partisan statistics.

The Government of Canada has a legitimate role and responsibility in producing a CPI, and a responsibility for it to be produced by Statistics Canada. Strong evidence shows that CPI activities are congruent with Statistics Canada's legislated role and the federal government's jurisdiction. As a member of the International Monetary Fund, the federal government must produce a CPI to meet legislative and international reporting requirements. Evaluation evidence indicates that Statistics Canada is best positioned to produce Canada's CPI, given its status as an independent agency and its expertise and capacity. This is in line with the international practice — in all jurisdictions reviewed, the CPI is produced on a monthly basis by a national independent statistical agency.

Although evidence showed that the CPI is relevant, it also revealed unmet needs, and an interest in a deeper level of data than what is now released by Statistics Canada. Consultations with end users indicated an interest in the CPD developing and issuing CPI sub-indexes — particularly in the areas of seniors and those living close to the poverty line (a basic necessities index). In fact, the need to better reflect the actual cost of living for some age cohorts was also raised in several academic publications.^{Footnote 52} CPD has recognized that a single CPI cannot serve all users equally well and has been examining publishing 'families' of indices.

7.1.2 Design and Implementation of CPI-EI

The CPI-EI was necessary to address known weaknesses of the CPI and ensure that the index continues to be high quality and reliable. Evidence suggests that the design of the CPI-EI is sound, and that the initiatives underway are the right ones to improve the CPI. Furthermore, evidence showed that the CPI-EI is being implemented as planned; some facets are being implemented ahead of schedule.

The CPI-EI governance arrangements are viewed as appropriate and sufficient to support the initiative and the ongoing management of associated risks. Senior CPD management showed an ability to address emerging issues and challenges by strengthening overall project management and governance, by enhancing CPD human resource capacity, and by making other organizational changes. The evaluation revealed that structures and processes are in place to monitor implementation progress of the CPI-EI, and that the initiative has contributed to building a project management capacity and culture. Some challenges related to implementation were identified, mainly: better clarification of roles and responsibilities, conveying governance arrangements more clearly to the lower levels in CPD, and more horizontal discussions of cross-cutting issues. A concern was also raised with respect to the significant subject-matter expertise lost recently in the division. Regardless of some successful capacity-building initiatives in CPD, a potential erosion of subject-matter expertise is a risk factor for the implementation of CPI-EI, particularly in relation to succession and continuity planning.

7.1.3 Performance — Achieving Expected Outcomes