Wednesday, September 27, 2006

Concurrent workshops

• Session C-1 Understanding the impacts of innovation
• Session C-2 New indicators for science and technology policies
• Session C-3 Global knowledge flows – the spatial dimension

Session C-2 New indicators for science and technology policies

• Indicator for complex innovation systems: A scale-independent view
• Constructing a multi-level Scientometric Indicators System
• Indicators of science & technology catch-up by East Asian nations: A composite analysis combining scientific publications and patenting data
• Innovation systems' based indicators: Relationships between innovation, human capital, and information and communication technologies

Indicator for complex innovation systems: A scale-independent view

Biographical Note

Sylvan Katz's research focuses on applying knowledge gained from the study of complex systems and networks to developing indicators that reflect the complex character of science and innovation systems.

Abstract

Performance indicators such as national wealth (GDP per capita), R&D intensity (GERD/GDP) and scientific impact (citations/paper) are used to compare innovation systems. These indicators are derived from the ratio of primary measures such as population, GDP, GERD and papers. Frequently they are used to rank members of an innovation system in order to inform decision makers. This is illustrated by the European Research Area S&T indicators scoreboard used to compare the performance of member states.

It is assumed that a performance indicator derived from the ratio of two primary measures is normalized by the denominator. In other words, if the denominator is a measure of size (e.g. GDP, population and papers) then the ratio is assumed to be normalized for size.

Katz (2000; 2005 showed that the amount of scientific recognition received by groups measured using citations, c, exhibits a power law correlation with the sizes of the groups measured using papers, p. Between 1981 and 1994 every time a group doubled the number of scientific papers they published the amount of recognition they received increased about 2.4 (2 1.27) times, that is c a p 1.27. From the rules of power laws we know that c/p a p n-1 therefore c/p a p 0.27 hence scientific impact (citations/papers) scales with the size of the group (papers) too. Scientific impact is not normalized for size. It should not be used to compare groups of different sizes.

Two other common measures, R&D intensity and national wealth have been shown to scale with the sizes of European countries and Canadian provinces (Katz 2005b). Power law correlations exist between GERD & GDP and GDP & population (a) across time and (b) at points in time across members of an innovation system. National wealth and R&D intensity performance indicators just like scientific impact are also not adjusted for size.

This paper argues that if innovation systems are complex systems then they are expected to exhibit scaling correlations between primary measures (Amaral; Ottino 2004; Newman 2003). These scaling relationships can be used to build scale-independent performance indicators that have been normalized (scale adjusted) for the scaling correlation between the numerator and denominator. Scale-independent performance indicators can be used to compare groups of vastly different sizes.

• The rankings of provinces and nations in the Canadian and European innovation systems yielded using conventional measures of national wealth will be compared to the rankings yielded using scale adjusted measures of national wealth. It will be shown that systemic overview of an innovation system given by the two ranking methods is subtle and profoundly different. The difference can impact decision makers charged with developing public policies that impact innovation systems.

Constructing a multi-level Scientometric Indicators System

Top of Page

Biographical Note

Hiroyuki Tomizawa is director of Research Unit for Science and Technology Analysis and Review, National Institute of Science and Technology Policy (NISTEP), Ministry of Education, Culture, Sports, Science and Technology [MEXT].

Takayuki Hayashi is affiliated fellow, Second Theory-Oriented Research Group, National Institute of Science and Technology Policy [NISTEP], and associate professor, National Institution for Academic Degrees and University Evaluation [NIAD].

Abstract

Bibliometric indicators have been used as helpful indicators to measure output of scientific research quantitatively. Although they indicate only some aspects of the activities that are referred to as scientific enterprise, they have no equal in terms of the wealth of information they contain.

However, macro-bibliometric indicators have conventionally been used in science policy only as indicators of the size of research output, and their abundant potential has not been fully utilized. One factor that prevents more diverse application of bibliometric indicators is the simple assumption that they can indicate nothing more than the size of the research output. In other words, bibliometric indicators are regarded as encouraging a one-sided way of thinking that places emphasis on the number of publications produced (the more, the better).

In scientometrics, the study of quantitative analysis of science, bibliometrics has played a central role and explored various possibilities mainly on a micro level. Micro-bibliometrics has been focused not only in scientific productivity but also in interdependency and dynamism, which are the features of scientific enterprise. Is it possible to integrate macro-bibliometrics that covers the overall scientific research of one country, with micro-bibliometrics that reflects various aspects of scientific enterprise? We have been working to construct a Multi-level Structural Analysis Database in which macro data on the situation in Japan as a whole are linked with micro data on contributions to scientific research by all institutes and organizations located in Japan. In this database, bibliometric data on a macro, meso, and micro level are linked together horizontally. This is also unique for its integrity, since the sum of micro data equals macro data. The database will make it possible to analyze not only external trends of production of scientific papers in Japan but also structural changes in output of scientific research in Japan.

We have also been proceeding with inclusion of individual statistical data on R&D in this database. Through this process, we can establish a Scientometric Indicators System that provides multi-level data and also achieves the link between research output and input. For instance, by using this system, we can make an analysis on the institutes to which research resources were concentrated in the past ten years and the results of such a concentration of research resources as reflected in research output.

Constructing this database might be, in a sense, no more than a reorganization of existing data. However, by developing a new method for linking data on various levels, we aim to reflect interdependency, the essential feature of scientific enterprise, in quantitative data, and through such efforts, we will be increasingly likely to be able to carry out the structural analysis for a national research system that is indispensable to science and technology policy.

Indicators of science & technology catch-up by East Asian nations: A composite analysis combining scientific publications and patenting data

Top of Page

Biographical Note

Poh Kam Wong is Associate Professor, Business School and Director, Entrepreneurship Centre, National University of Singapore. He obtained his Bachelor's, MSc and Ph.D. degree from MIT. He has published and consulted widely on innovation strategy, S&T policy and technology entrepreneurship.

Yuen-Ping HO

Abstract

The growing competitiveness of East Asian economies in science and technology has received increasing attention. Using data on scientific publications covered by SCI and SSCI, King(2004) and National Science Foundation(NSF)(2005) highlighted the growing share of selected East Asian economies not only in total outputs of scientific publications, but also in citation counts of journal articles as well as in the share of articles in highly cited journals. In this paper, we extend these prior analyses in three ways. Firstly, we apply similar metrics to USPTO patenting data to develop international comparative indicators on national technological output quantity and quality, and examined the sensitivity of the quality indicator to variations in the definition of high citations (from the top 1% to the top 10%). Second, we combine both science and technology measures to derive a number of composite ratios that indicates the relative emphasis of nations on science vs. technology. Thirdly, we extend the analysis to a number of specific scientific and technological fields where the selected East Asian nations have made the most rapid catch-up.

By integrating data on both scientific and technological output indicators, and by examining both quantity as well as quality indicators, our composite analysis extends earlier findings in several ways. In particular, we found systematic variations in the dynamics of S&T development among the three distinctive groups of Asian economies: the three East Asian Newly Industrialized Economies (NIEs) (Taiwan, Korea and Singapore); the two large giants of China and India ; and the ASEAN economies minus Singapore ("ASEAN-S"). While all the three groups increased their share in the world total of both scientific and technological outputs over time, the NIEs achieved the fastest growth. Secondly, while all three groups emphasized quantity growth first, followed by improvement in quality later on, the NIEs again exhibited the fastest rate of improvement in quality in both science and technology outputs. Thus, in terms of both quantity and quality, there has been a growing S&T divide between the NIEs and the ASEAN-S economies, with the former catching up (in terms of intensities, i.e. output per capita) to the level of the OECD average, while the latter remains far behind. Thirdly, while China and India had achieved much faster scientific output growth than technological output growth, the NIEs had achieved faster growth in the latter than the former, both in quantity and quality. Overall, in comparison with the group of advanced OECD economies along the two dimensions of scientific and technological strengths, the East Asian NIEs are found to have relatively higher emphasis on technology than science, while for China and India (as well as many other developing nations), it was the reverse. Despite some possible size effect bias in the use of USPTO patent data as international comparative indicator for national technological output, this finding is consistent with prior qualitative literature on the national innovation system characteristics of the East Asian NIEs that suggest early public policy emphasis on technological applications. Finally, we found evidence that the above distinctive features of S&T development of the East Asian NIEs to be even more highly accentuated in the specific science and technology sectors that they had made the most advance – electronics engineering and information and communications science/technology.

We believe that a number of normative policy implications can be derived from our composite analysis. Firstly, as developing nations embark on S&T catch-up with more advanced countries, they need to shift emphasis from quantity to quality. Second, the experience of the East Asian NIEs suggest that policy makers in developing nations should pay greater attention to developing technological innovation capabilities that are relevant to competing in international markets, rather than just focusing on S&T manpower developments. Thirdly, even among advanced nations, it may be useful to use our composite indicators on relative emphasis on science vs. technology and quality vs. quantity as benchmark indicators for comparing and explaining relative S&T policy focus across countries.

Innovation systems' based indicators: Relationships between innovation, human capital, and information and communication technologies

Top of Page

Biographical Note

Monica Salazar is a PhD candidate at Simon Fraser University. Ms. Salazar has a B.Sc. in Economics from Rosario University (Bogota, Colombia) and a M.Sc. in Technical Change and Industrial Strategy from PREST at the University of Manchester (U.K.). She worked in the past for the Colombian Institute for the Development of Science and Technology and the Department of National Planning.

Abstract

A lot of experience has been accumulated, both in developed and developing countries, which explain innovation processes and how the environment affects innovative performance. Although all that knowledge has not permeated the public policy sphere; policy-makers and government agencies should take advantage of this, and improve the design of innovation surveys, indicators and policies. Without doubt the innovation systems approach has provided useful insights to a better understanding of innovation process. Here are some of the conceptual underpinnings of this approach:

• Firms do not innovate alone, they create innovation networks and rely on various supporting organizations and institutions.
• Learning is interactive and cumulative, and it is a crucial factor in innovation processes.
• Interaction is central to the process of innovation.
• Evolutionary processes play an important role.
• Innovation occurs in institutional, political and social contexts.
• Innovation is embedded in social relationships.
• Innovation capabilities are sustained through local communities that share a common knowledge base and a common set of rules, conventions and norms.

The systemic nature of innovation tells us different things. For instance, it talks about systemic interaction and performance, as McKelvey explains it: "the concept of NIS encompasses an idea of systematic interactions, which cannot be reduced simply to the actions of specific firms, or to the existing R&D system, or to competitions among firms or institutions" (McKelvey, 1991: 136-137). The question to ask is whether innovation surveys are helping to characterize current complex innovation systems or not, and if they account for the variety of interactions among diverse actors.

It seems that performance indicators, such as TPP innovation – the main emphasis of innovation surveys? which are oriented to compare and benchmark firms, sectors, regions and countries are not enough to characterize the innovation process. How can we appraise innovation capabilities, innovativeness (attitude and potentiality), and innovation efforts? We need to refine indicators that measure firms? capacities to innovate, and the impact that economic, social and cultural conditions and the environment have upon these capabilities.

The objective of this document is to find commonalities between innovation indicators, knowledge or information society indicators, and clusters indicators. The core of the proposal is to propose new indicators to measure the role and impact (incidence) that the human capital has in the knowledge society/economy.

The document will be organized as follows, I will start explaining the logic model, followed by some comments regarding the biases and missing aspects in the measurement of innovation. The third section will deal with the topic of human capital, especially looking at how various human resource management practices are closely related to ICTs adoption and innovative performance. The final section will deal with the need to focus more on systemic innovation policies (Guy & Nauwelaers, 2003; Smits & Kuhlmann, 2003) and how innovation indicators should respond to the challenge that this focus impose.