Monday, September 25, 2006

Concurrent workshops

• Session A-1 Non technological innovation
• Session A-2 Multidisciplinary science, technology and innovation
• Session A-3 Global knowledge flows - Human resources and science and technology

Session A-2 Multidisciplinary science, technology and innovation

• A framework to measure the impact of investments in health research
• Towards a nanotechnology statistical framework
• Indicators for benchmarking biotechnology innovation policies
• Biotechnology impact indicators: From measures of activities, linkages and outcomes to impact indicators

A Framework to Measure the Impact of Investments in Health Research

Biographical Note

Alan Bernstein is the inaugural President of the Canadian Institutes of Health Research, Canada 's lead agency for health research. An internationally respected researcher and scientific leader, he has made key contributions to our understanding of embryonic development, hematopoiesis and cancer.

Abstract

Today, more so than ever before, citizens and politicians around the world want evidence that public funds are being invested wisely and that this investment is producing positive results. Quantifying impact, or return on investment, is a particular measurement challenge in the area of research – something that countries around the world are grappling with. Demonstrating direct and objective links between particular research investments and immediate, short-term or long-term, identifiable and measurable, outcomes is difficult. This presentation will describe the approach taken by a national health research funding agency in Canada, the Canadian Institutes of Health Research, to identify an appropriate means to measure the value of its investments in health research. Specifically, this initiative, undertaken in 2005, sought to identify methods to measure the impacts of its health research investments and the appropriate measures that can be used to establish benchmarks and gauge progress in realizing the value of health research.

As the Government of Canada's agency for health research, the Canadian Institutes of Health Research (CIHR) receives $700M annually to fund researchers at universities and teaching hospitals across Canada. In 2005, w hile the agency routinely conducted evaluations of its programs and had anecdotal information which demonstrated the impact of individual CIHR-funded research projects, there existed little systematic and comprehensive data to demonstrate the overall value of these investments. Hence, the need to address this gap through the identification of an Impact of Health Research conceptual framework and associated measures to provide a global perspective of health research returns.

In early 2005, CIHR convened a group of international and Canadian experts to review the present state of knowledge about measuring impact in health research and to provide advice on the creation of the conceptual framework. Among the international agencies reviewed, the panel found consistency regarding the main objectives of health research funding. These objectives include: fostering excellence in research; creating a strong community of researchers; and translating the results of research to provide benefits for the health sector and the larger society in which it operates. The panel also agreed on the challenges of measuring impact and made recommendations on a preferred approach to measure impact. The draft conceptual model which was subsequently identified was then further refined at a workshop of key decision-makers.

The conceptual framework is based on the five-dimensional categorization of the Buxton-Hanney Payback model (developed by Dr. Martin Buxton and colleagues at Brunel University). As adapted, the categories are: knowledge production; research targeting and capacity building; informing policy; health and health sector benefits; and economic benefits.

Through continued refinements to the framework and ongoing analytical work, including indicator development, the Impact of Health Research framework will provide a global perspective of health research returns and increase CIHR's accountability to Canadians. It will also be useful for guiding future planning and evaluation activities.

Towards a Nanotechnology Statistical Framework

Top of Page

Biographical Note

Kevin Fitzgibbons is the Executive Director of the Office of the National Science Advisor to the Prime Minister, a position that was created in April 2004. From 1991 to 2004 Kevin worked as a strategic planning and policy analyst at the National Research Council of Canada. He has a Master's degree in Political Economics from l'Université de Montréal.

Chuck McNiven is the Unit Head for the Life Science Unit of the Science, Innovation and Electronic Information Division at Statistics Canada. Prior to joining Statistics Canada, Mr. McNiven lectured at the University of Western Ontario while completing his post graduate studies. He is responsible for surveys on biotechnology use and development, bioproducts and, functional foods/nutracueticals and for development work on nanotechnologies. His experience includes development, analysis and reporting of Canadian statistics on biotechnology and has participated in international activities on the development of biotechnology statistics.

Abstract

Context

In the theory of innovation and technological change, considerable debate has focused on the development, commercialization and socio-economic impacts of emerging technologies such as micro-electronics and information technologies in the late 1970s, biotechnology and genomics in the 1980s, and the Internet in the 1990s.

Over the past decade, developments in nanoscience and nanotechnology have drawn the attention of governments, industry, academia and the public to the potential industrial benefits, estimated by some (Lux, 2005) to reach billions of dollars in the coming ten years, as well as to concerns for health, environmental, ethical, legal and social risks (ETC, 2004).

As is the case with other emerging technologies, nanotechnology presents important conceptual and practical issues associated with statistical measurements:

• They are emerging, therefore rapidly evolving and not well defined;
• There are few examples of commercial applications and therefore have no baseline outcome statistics such as products, processes, and services developed and adopted in the market place;
• They are highly interdisciplinary and therefore not easily classified;
• They are enabling in that they have a wide range of potential applications in many sectors;
• They are potentially disruptive, representing both significant benefits and risks in industry as well as for health and environmental applications; and
• They are prone to hyperbole resulting from highly speculative claims from many quarters and improper and misleading use of the terminology.

There is no internationally recognized, formal definition or statistical framework for nanotechnology. Initiatives are currently under way in international fora (e.g. ISO TC229, OECD-CSTP-NESTI, International Risk Governance Council, UNESCO, etc.) that are looking into the public policy implications of nanotechnologies. These initiatives will require commonly recognized definitions, statistical protocols and frameworks to inform discussions and help guide decision making.

Objectives:

Building off the experience of biotechnology the proposed paper will:

• Address the theoretical underpinnings of statistics in emerging technologies;
• Provide an overview of lessons learned in emerging technology indicators;
• Review the state of play of statistical indicators in nanotechnology in Canada ;
• Identify the core issues associated with the development of a nanotechnology statistical framework in Canada ;
• Recommend a process and system for the collection, validation, storing and accessing of R&D expenditures, adoption and diffusion rates in Canada.

Indicators for benchmarking biotechnology innovation policies

Top of Page

Biographical Note

Thomas Reiss is Head of the department emerging technologies at Fraunhofer ISI, which is one of the leading institutes in the field of innovation dynamics and innovation policy in Europe.

Iciar Dominquez-Lacasa is a researcher at this department.

Abstract

Since the 1980s science and technology policy recognises that scientists and innovators are part of a larger network of organisations and institutions. Accordingly, there is an increasing demand for empirical tools to benchmark national science and technology policies from a systems perspective. This contribution proposes a benchmarking approach to biotechnology policies.

The approach combines descriptive portfolios of national policies with a set of quantitative indicators assessing performance in each country. The national policy portfolios consider the range of policies that theoretically can be applied to strengthen a national biotechnology innovation system. The set of quantitative indicators aims at measuring effectiveness of the biotechnology policy portfolio. To develop policy portfolios and performance indicators four sub-areas of the biotechnology innovation system were identified:

(1) generation and maintenance of a suitable knowledge base;
(2) transmission of knowledge to possible applications;
(3) integration of biotechnology into economic sectors;
(4) industrial development of the biotechnology sector.

Within these sub-areas, 14 policy goals for creating or strengthening a biotechnology innovation system were identified. National biotechnology policy portfolios were mapped using a questionnaire for policy-makers. For assessing the policy portfolios and thus, the achievement of policy goals a set of 13 performance indicators was elaborated which relates directly to the four sub-systems of the innovation system listed above. The comparison between the performance of countries in biotechnology and the respective policy settings in the past allows discussing the effectiveness of various policy approaches. The empirical results relate to policies implemented in 14 EU Member States, the USA and Canada in the mid 1990s and their performance in the period 1994-2002.

The analysis indicates that a strong financial commitment to supporting a biotechnology knowledge base is an important but not sufficient precondition for effective policies. Similar important is the relation between biotechnology-specific and generic policies. Specific policies for biotechnology pay off in early stages of the sector. The evaluation of policies supporting knowledge transmission indicates that well-performing countries have implemented a mix of generic and biotechnology-specific measures. Concerning policies to improve market access for biotechnologies most well-performing countries had fiscal instruments in place which aimed at supporting innovative activities of large firms. Such an approach contributes to generating a large domestic "technology market", where large firms demand technologies and services provided by biotechnology companies.

The analysis provides a proof of concept of the benchmarking approach and shows that the suggested performance indicators provide meaningful information on the achievement of certain policy goals set in the past. The comparison between the performance of countries in biotechnology and the respective policy settings allows discussing the effectiveness of various policy approaches. In this context it should be noted that simple correlations between policy input and national performance are not adequate, because policy is only one among several factors having an impact on performance.

Biotechnology impact indicators: From measures of activities, linkages and outcomes to impact indicators

Top of Page

Biographical Note

Antoine Rose built the biotechnology statistics program in Statistics Canada and led the development of statistical standards for biotechnology at the OECD. He holds a degree in Science, Technology and Society and a Master degree in Economics from the University of Québec in Montréal.

Chuck McNiven is the Unit Head for the Life Science Unit of the Science, Innovation and Electronic Information Division at Statistics Canada. Prior to joining Statistics Canada, Mr. McNiven lectured at the University of Western Ontario while completing his post graduate studies. He is responsible for surveys on biotechnology use and development, bioproducts and, functional foods/nutracueticals and for development work on nanotechnologies. His experience includes development, analysis and reporting of Canadian statistics on biotechnology and has participated in international activities on the development of biotechnology statistics.

Abstract

A large positive rhetoric exists on future benefits of biotechnology. In short, this rhetoric could be summarized as: biotechnology is the next technology wave that will deeply transform the economy and society by providing products and processes that will solve health problems, feed the world with new agricultural products, cure the environment and provide sustainability. Beyond the rhetoric, indicators of biotechnology activity show an emerging phenomenon that is still relatively small. Many actors from the private sector, governments and universities are betting on future success of biotechnology. A recurring policy question is: what evidence provides a solid basis or argument for government to invest in biotechnology.

The last decade saw the emergence of a wealth of data and statistics attempting to portray the development of biotechnology. This work is largely summarized in the OECD Framework for Biotechnology Statistics (2005), which provide guidance and standards for the measurement of biotechnology activities. It should be noted that most of the data currently collected focuses on activities, linkages and outcomes of firms engaged in innovative activities through the use of biotechnology.

Activity indicators focuses on data collected on actors (who) performing an activity (what) in a location (where) to fulfill an objective (why). Linkage indicators collect data on how much resources were committed and how actors are connected to other social or economic organizations and institutions. Outcome indicators provide measures of results achieved. Most of these indicators are available in Canada through the wealth of surveys on the Use and Development of Biotechnology conducted by Statistics Canada.

Biotechnology specific impact indicators should measure changes induced in socio-economic indicators. Typically, an impact indicator is most of time a ratio and is tracking a change. Examples: X% of the vaccine market is composed of product developed using biotechnology compared to Y% in previous time period; X% of total R&D is made using biotechnology; occurrence of a disease, expressed as a percentage of population, diminished by Y% after the introduction of a genetic therapy...

Typical problems associated with impact indicators related to an emerging technology are twofold. When compared to long-term trends observed in major socioeconomic impact indicators, the magnitude of biotechnology related indicators appears relatively small, if not negligible. This type of impact indicators capture a net wealth effect, that is, the difference between new economic activities generated minus economic activities that were discontinued as a result of the emergence of the new technology. This paper argues that an important part of the impact of biotechnology is the result of substitution effects such as changes in productions (process innovations) and changes in supply chains. Measuring these impacts remains a challenge for statisticians.

This paper focus on impact indicators that would be relevant to policy analysts and decision makers and discuss what transformations or additions are required to existing measures of activities, linkages and outcomes to allow for the construction of impact indicators of biotechnology activities.