Tuesday, September 26, 2006

Concurrent workshops

• Session B–1 The role of public sector actors in innovation
• Session B–2 Specialised surveys: Developing countries, remote regions, special topics
• Session B–3 Global knowledge flows – The diffusion and location of knowledge

Session B–1 The role of public sector actors in innovation

• University research in an innovation society
• Developing internationally comparable indicators for the commercialisation of publicly–funded research
• Micro–level indicators of knowledge production: The AQUAMETH project on European universities
• Benefits from investment in the Canadian federal government

Biographical Note

Richard Hawkins holds a Ph D from the science Policy research Unit (Sussex). He has recently moved from the TNO in the Netherlands to take up the Canada Research Chair in science and Innovation Policy at the University of Calgary.

Cooper H. Langford coordinates the Science, Technology, and Society Program at the University of Calgary . He is the former Vice President (Research) and a former Director of Physical and Mathematical sciences at Science and Engineering Canada (NSERC).

Kiranpal S. Sidhu is studying M.Sc. (Interdisciplinary Graduate Program) at the University of Calgary under the supervision of Dr. Cooper H. Langford. His work explores innovation, effects of governmental policies, and statistical analysis of science and engineering indicators.

Abstract

This paper will argue for a more 'olistic' approach to measuring the social and economic impacts of university research. Current indicators focus primarily on inputs and/or impacts of a highly selective group of knowledge 'products' based almost entirely in the natural and applied sciences. Most indicators focus on the production of scientific discoveries and technological inventions and on their subsequent rates of commercialization. But the results overall indicate rather inconsistent, haphazard or disappointing knowledge transfer outcomes that often inject ambiguity into the public policy context.

We will argue that current indicators give a false impression of the scale, scope and quality of university–industry knowledge exchange, a consequence of an overwhelmingly 'supply–side' orientation. At present we measure the production of a narrow range of total university research output mostly under demand assumptions that either are exogenous or otherwise poorly characterized and contextualized. Efforts to avoid the simplistic 'demand pull' models characteristic of production function approaches to explaining innovation have resulted too often in consigning the knowledge demand issue to a black box.

Our overarching hypothesis is that effective knowledge transfer depends upon reflexive relationships between factors of production and demand in a more broadly defined socio–economic context — 'society' rather than just 'industry' as the context within which innovation occurs. In order to gauge the full extent of university research impact, indicators will be needed that represent supply–demand relationships in this context. We approach this goal in three steps.

The first step is a critique of the present indicator profile, especially as it deals with conventionally designated 'technology transfer' outcomes through the measurement of licensing, royalties, and spin–off formation. We examine the literature on major pathways of knowledge exchange (collaborative R&D, licensing and spin–off, technical assistance, information exchange, and hiring) between universities and the broader economy. A focus on 'technology' transfer deals with only a fraction of knowledge exchange. This is risky for policy formation as captured by the expression; "you get what you measure". More formally the problems of institutional isomorphism (DeMaggio and Powell, 1983) are the source of risk. This critique leads to the first conclusion that the present set of indicators still merits extension. The first recommendations are modestly aimed at input correlates of the other four pathways based on refined metrics of university–industry interactions that can be readily implemented in the Canadian data context.

The second step of the analysis draws on selected case studies to achieve a more intimate picture of representative involvements of university research in innovation. The case studies expand the view beyond science and technology to include the full range of innovation envisioned in Schumpeter's original five types. Analysis of the cases also underlines the complex, path dependent, character of innovation. Metrics need to be appropriate to complex adaptive systems.

The final step proposes preliminary demonstrations with user sensitive metrics appropriate to characterization of an innovative society that accommodates the complexity.

Developing internationally comparable indicators for the commercialisation of publicly–funded research

Top of Page

Biographical Note

Anthony Arundel is a Senior Researcher at the Maastricht Economic Research Institute on Innovation and Technology (UNU–MERIT) of the University of Maastricht. His research focuses on innovation surveys and the use of survey data to develop indicators for policy analysis.

Catalina Bordoy is a researcher at UNU–MERIT, which is a research institute affiliated with the University of Maastricht in the Netherlands. Her research interests focus on knowledge flows, including between firms and between universities and private firms.

Abstract

Over the past decade, European policy has stressed the need to improve the commercialization of research results from public universities and research institutions. This is in response to a perception that Europe has failed to benefit very much from its substantial investments in public research, in contrast to the American experience where universities such as Stanford, Columbia and the University of Florida earn substantial licensing income from patented inventions. However, it is hazardous to compare European performance against a limited number of highly successful American universities. What is required is internationally comparable indicators of the technology transfer activities of all universities and research institutions. In 2001–2002 the OECD ran a multi–country survey of the technology transfer activities of universities, but the results were rarely comparable across countries, due to a lack of comparable denominators. Results such as the number of patents per technology transfer office (TTO) are not comparable because of large differences in the number of researchers or research expenditures affiliated with the TTO.

In this paper we use the results of a recent survey of the members of the Association of Science and Technology Professionals (ASTP), drawn from TTOs representing European universities and public research institutions, to develop comparable indicators for commercialization activities. The survey was conducted by MERIT between January and March 2006 and covers technology transfer activities in 2004 and 2005. We use the results to develop several indicators for the number of spin–offs established, patents, and earnings from licensing revenues. We compare the ASTP results with the survey results of the American University and Technology Managers (AUTM), after developing indicators based on outputs per million US dollars (in purchasing power parities or PPP) of research expenditures.

The interim results find that the American AUTM institutions perform better on almost all indicators when using indicators per institution (or TTO). However, after adjusting for research expenditures, we find the American AUTM affiliated institutions perform better that the European ASTP institutions on only two of five indicators: patent applications and licenses executed per million PPP$ of research expenditures. Conversely, the performance of the European ASTP members is better for three indicators: invention disclosures, patent grants, and the number of established start–ups. Features of the two patent regimes could explain part of the differences in patenting performance. There are also substantial differences within Europe in the performance of universities and public research institutions.

The paper discusses some of the problems with the data, such as the fact that both the AUTM and ASTP surveys are limited to a self–selected group that do not represent all universities and public research institutions in each country. In order to meet European policy needs, similar types of indicators would need to be obtained through a larger survey that can overcome self–selection bias.

Top of Page

Biographical Note

Andrea Bonaccorsi, professor of Economics and Management, University of Pisa. Member of the High Level Expert Group (HLEG) of the European Commission–DG Research on Maximising the Benefits from basic research (2005) and of the HLEG on Key Research Actors Scenario for 2020 (2006).

Cinzia Daraio, post–doc research fellow at the IIT–CNR (Italy ). She is an active member of the PRIME working group on Public Sector Research and is involved in several international research projects. Her works have been published in several international refereed journals.

Abstract

The paper is based on the recent construction of a large dataset of microdata on European universities in six countries (UK, Spain, Italy, Portugal, Norway, Switzerland), covering 271 institutions. For each university we have available the data presented in Table 1 for the period 1995–2003 (with some breaks).

The AQUAMETH project, developed under the PRIME Network of Excellence, has produced a number of studies on the University system in these countries, addressing also some policy–relevant transversal issues (see Bonaccorsi and Daraio, 2006).

In this paper we develop new indicators and use efficiency measures in order to characterize the way in which universities use their inputs (academic and non academic staff, funding) in the effort to position themselves in the space of output (undergraduate teaching, postgraduate education, fundamental research, contract research, third mission), while keeping efficiency under control.

AQUAMETH database structure

While these data allow a fine–grained descriptive analysis, we also use a full range of newly developed robust nonparametric techniques (Daraio and Simar, 2006) in order to characterize efficiency while taking into account small size biases, the presence of noise/outliers and in general the main limitations of traditional nonparametric efficiency methods, such as Data Envelopment Analysis. Furthermore, we use conditional efficiency analysis (Bonaccorsi and Daraio, 2004; Daraio and Simar 2005a; 2005b, 2006) in order to shed light on the complex trade–offs between inputs and between outputs.

We find evidence of significant specialization going on in European universities:

• in the relation between undergraduate and postgraduate education
• in research intensity
• in the rate of growth of student enrolments.

At the same time we find that specialization is strongly subject to institutional constraints, associated to the definition of university mission (identity) and to the structure of funding. Universities that try to differentiate their profile look for additional resources, but the extent to which they can leverage on student tuitions and on industry contract research is severely limited.

The European university system is largely based on public, government–funded, universally accessible institutions, formally of equal value. In the European institutional landscape, there is no intrinsic pressure towards specialization, rather the opposite is true.

We suggest a number of policy conclusions based on the evidence produced.

Top of Page

Biographical Note

Pierre Therrien is a senior research economist at Industry Canada. Prior to joining Industry Canada, he worked for Human Resources Development Canada, and for the Centre for Interuniversity Research and Analysis on Organizations (CIRANO). His primary research interest concerns best practices in the innovation process and the development of S&T indicators.

Abstract

Canada has a decentralized federal science and technology (S&T) system where each department and agency is responsible for its science activities. As a result, each department is accountable for reporting its science expenditures and personnel, and more recently, for providing indicators of the expected and realized benefits of their activities.

At the federal level, input data (expenditures and personnel) are aggregated annually to get a comprehensive overview of national S&T activities. Some output data (only those with a commercialization profile such as patents and intellectual property incomes) are reported and aggregated at the national level. To date, however, no systematic exercise has been performed to compile efforts made by departments to monitor the expected and realized benefits of departmental scientific activities.

This paper seeks to address this gap and will provide new insights on the effort of departments and agencies to monitor their R&D investment. Two indicators from all departments and agencies will be collected and compiled using the information already available through administrative documents. Interviews with major R&D program managers will be performed to complement this information. The indicators will then be analyzed and aggregated by the different roles the federal government plays in the national innovation system. Such roles (R&D performer, S&T policy maker and regulator, R&D supporter and facilitator, high–skill worker trainer) and the associated benefits are already well defined in the literature (e.g. KPMG, 2004, Salter–Martin, 2001). Sets of indicators will be associated with each role, and when possible, linked to R&D investment.

Currently, official Canadian S&T data are not specifically designed to provide information by such roles. The only data currently available about the rationale of the funding is categorized by socio–economic objectives (as defined by the Frascati Manual). While this variable may be highly informative of the field benefiting the investment, it says nothing on why this investment is publicly funded (creation of knowledge, departmental mission–oriented, policy or regulation oriented, etc). Moreover, as departments usually fulfill more than just one role, collecting data at the program level (dealing with usually only one issue) might strengthen our understanding of the relationship between R&D investment and accurate result indicators. Suggestions will be made on how the major findings from the study could be incorporated in official S&T data.