Main article

Warning View the most recent version.

Archived Content

Information identified as archived is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please "contact us" to request a format other than those available.

  1. Introduction
  2. Measuring user innovation
  3. Measurement issues and lessons learned
  4. Conclusions
  5. References

1   Introduction

The purpose of this paper is to provide some metrics for the measurement of user innovation. It will explain what is meant by user innovation and provide background on its measurement at Statistics Canada, drawing attention to some more influential work. Challenges to the measurement of user innovation will be presented. An exploration of user innovation through Statistics Canada's first and recently conducted survey of user innovation will include details on the survey methodology and survey findings. Measurement issues and some lessons learned from the survey will be discussed. The paper will conclude by presenting contributions of this study to understanding user innovation.

Innovation is a vital part of economic growth, development, and competitiveness. The measurement of innovation is therefore important as it provides an indicator of a firm's ability to compete, both nationally and in the global economy, and the health of a country's economy. In order to develop effective policies to support innovation, an understanding of the innovation process is critical. Surveys of innovation have been carried out on an occasional basis in Canada since 1993. These surveys have provided innovation indicators for policy development. There are many innovation activities carried out by firms. Technology acquisition 1  is a specific innovation activity that has long been of interest to policy analysts and academics. The acquisition of newer, more advanced, technologies, can allow firms to increase their production capabilities, improve their productivity, and expand their lines of goods and services.

The acquisition of technology is an indicator of innovation diffusion that is important to measure and analyze. The dynamics of knowledge and technology transfer including the source and the relative importance of these sources are necessary elements to providing an understanding of linkages in the innovation process. The mapping of linkages and knowledge flows has strong innovation policy relevance. This allows analysts to identify actors and activities that should receive policy focus in order to foster desired innovation outcomes. As a result, how firms acquire new technology has implications for the innovation system and the economy. User innovation is one way firms can acquire technologies.

1.1  What is user innovation and how is it measured at Statistics Canada?

Users are firms or individual consumers that expect to benefit from using a good or service whereas producers expect to benefit from selling a good or service (von Hippel, 2005). Users can be owners, operators or employees in a firm or individuals in a household. This paper uses business data where users are the people in a firm carrying out various activities. One such activity is technology acquisition.

How a firm acquires technologies is an important aspect of the innovation system. Statistics Canada conducted its first survey of technology use and planned use in 1987 and there were similar surveys in the U.S., Australia and some other countries (Ducharme and Gault, 1992). The finding that Canada did not perform as well as its US counterparts for technology adoption stimulated policy debate and generated interest in the Organization for Economic Cooperation and Development (OECD). The 1993 Survey of Innovation and Advanced Technology included questions on innovation and advanced technology on the same questionnaire. Subsequently, distinct questionnaires evolved with a 1996 Survey of Innovation and a 1998 Survey of Advanced Technology in Canadian Manufacturing. The focus in Canada in recent years has been on surveys of innovation, rising from the 2001 federal government's Innovation Strategy (Industry Canada, 2001) which drew heavily from results of Statistics Canada's 1999 Survey of Innovation. Although Statistics Canada carried out surveys of innovation in 2003 and 2005 it was not until 2007 that a survey of advanced technology use would be carried out again. 2 

Manufacturers generate revenues and expect benefits from selling a product (good or service) to users who expect to benefit from its use. Typically, product innovations are viewed as having been developed by manufacturers for commercialization in the marketplace, an assumption that has influenced innovation-related research and activities ranging from how firms organize their research and development to how governments measure innovation and approach policy development for innovative activity. In his book The Sources of Innovation (von Hippel, 1988), von Hippel presents an alternate approach to the manufacturer-centric view, demonstrating that innovation occurs in a variety of places in different industries. Through a series of studies he has shown that end-users, material suppliers, and others are the typical sources of innovation in some fields. Innovation occurs as a response to a need in processing activities that would not otherwise be met by existing products.

It is important to understand how user innovation is approached using traditional innovation metrics. Users were included in the list of sources of information for innovation in the first edition of the OECD Oslo Manual (OECD 1992:31) and in subsequent editions (OECD/Eurostat 1997:71 and 2005:81). The third edition elaborates on what is referred to as "inbound diffusion" (OECD/Eurostat 2005:78). As innovations are spread through market and non-market channels during the process of innovation diffusion, they are changed by users and feedback can be supplied to the original innovator. Users have been identified as an important source of information for innovation (Survey of Innovation, 2005) signalling the importance of users in the innovation process.

Innovation, the acquisition and use of advanced technologies and the role of the technology user have been the subject of study for decades (von Hippel, 1988). A firm must have the internal capabilities to successfully acquire new technologies or risk failure (Montgomery and Levine, 1996) and how firms acquire advanced technologies has been found to be as indicator of these technological capabilities (Arundel and Sonntag, 2001).

Technology can be acquired in three ways: by purchasing existing technology; by modifying an existing technology, or by creating a new technology. In their study of advanced manufacturing technologies, Arundel and Sonntag (2001) found that half of advanced manufacturing technology users only purchased their advanced technologies off the shelf (46.1%), one quarter customized or significantly modified existing technologies (25.7%) and one quarter (28.2%) developed new technologies. These technological advancements play a strong role in innovation process.

1.2  The measurement challenge

In the Oslo Manual innovation is defined as:

"…the implementation of a new or significantly improved product (good or service), or process, a new marketing method, or a new organisational method in business practices, workplace organisation or external relations." (OECD/Eurostat, 2005)

To be considered as an innovation, a product must be introduced onto the market and a process, marketing method, or organizational method must have been used in the firm's actual operations. The measurement of the incidence of innovation among firms relies on the identification of this activity. This paper limits its scope to technological innovations which include product and process innovations. Identification of product innovations is clear. A product must be introduced onto the market for it to be considered as a product innovation. Process innovations on the other hand may not be as obvious introducing a degree of nuance. As part of their Innovation Strategy (OECD, 2007), the OECD has begun to explore the issue of "hidden innovation", that is innovation that is not always fully reflected by traditional indicators of innovation. The National Endowment for Science, Technology and the Arts (NESTA) has looked at the gap between perceived innovation and conventional metrics (NESTA, 2007). Drawing from their work, innovation may be "hidden" when firms combine or modify existing technologies and processes in a new way to create innovative solutions. This activity may not be obvious in the firm, occurring as on the job solutions by users in response to production issues. In the absence of R&D, there is a risk that this may not be recognized as an innovation activity in the firm. Capturing this activity may require an approach that complements the Oslo Manual or elaborates on existing guidelines that explore linkages in the innovation process and the actors involved.

One aspect of measuring linkages in the innovation process and the actors involved includes identifying the developers of innovations. The Oslo Manual (OECD/Eurostat, 2005) recommends three options, that:

  1. The innovations were mainly developed by the firm itself.
  2. The innovations were developed by the firm in conjunction with other firms or institutions.
  3. The innovations were mainly developed by other firms or institutions and were adopted by the firm as part of a diffusion process (OECD, 2005).

Questions aimed at identifying the developers of innovations have been included in recent innovation surveys including Eurostat's CIS3 and CIS4, Statistics Canada's Survey of Innovation 2003 and Survey of Innovation 2005 and the Survey of Advanced Technology 2007. Among the half of manufacturing plants who were process innovators during years 2002 to 2004 (Survey of Innovation, 2005), almost two thirds (64.2%) indicated that the development of their process innovations were mainly by their plant or their firm.

The diffusion of innovations results in benefits for other firms or individuals who make use of the innovation. It is not clear that the role of the technology user would be captured given the options provided for identifying developers of innovations. Identifying the actors in the innovation process, the nature of interactions between actors and how knowledge and technology flows in the system is a challenge that is necessary for a better understanding of the diffusion of innovations. Although traditional innovation surveys collect information on sources of information for innovation, information on the linkages between the developer of innovations and the sources of information would provide insights into how knowledge flows in this system. If users are important actors in the innovation process this has direct relevance to innovation policy and supporting user innovation.

2   Measuring user innovation

2.1  Analytic approach

This paper uses data from the Survey of Advanced Technology 2007 and the Follow-up to the Survey of Advanced Technology 2007. The Survey of Advanced Technology 2007 was sent to a stratified random sample of 9,441 manufacturing and 373 logging statistical establishments. The questionnaire substituted the more familiar term "business unit" which is hereafter referred to as plants in this paper. In order to be considered for sample selection the plant had to have at least 20 employees and have at least $250,000 in revenues. 3 

In the spring of 2008, the former Science, Innovation, and Electronic Information Division of Statistics Canada, in conjunction with subject matter experts, designed a questionnaire, the Follow-up to the Survey of Advanced Technology 2007, to probe the activities of manufacturing firms that indicated they had modified existing technologies or developed new technologies as a means of integrating advanced technologies into their firm.

The analytic approach used in this paper is based on that of Arundel and Sonntag (1999) which categorizes technology adoption by highest level of method used. This approach considers that the degree of effort required for each method increases as the complexity of the introduction method, and resulting need for internal capabilities, increases. These range from minimal for purchasing off-the-shelf technologies to extensive in the case of developing new technologies. All responses to Question 4 from the Survey of Advanced Technology 2007 were assigned to one of three classes depending on the highest level of required internal capabilities resulting in a taxonomy of type of plant: 4 

  1. Technology purchaser: These plants either purchase technologies off-the-shelf, lease technologies off-the-shelf, or license new technologies;
  2. Technology modifier: These plants customize or significantly modify existing technologies; or
  3. Technology developer: These plants develop new technologies (either alone or in conjunction with others).

For example, a plant that indicated it both customized or significantly modified existing technologies and that it developed new technologies was classified as a technology developer. A plant classified as a technology purchaser used only this method and neither customized nor developed new technologies. Plants that modify existing technologies or develop new ones provide an indicator of user-innovation and will be the focus of the analysis in this paper.

The Survey of Advanced Technology 2007 shows that manufacturing plants that introduce advanced technologies are most likely to be technology purchasers with more than one half of plants (57.1%) choosing this method. Four in ten (42.5%) of the plants that introduced advanced technologies were user innovators. These plants are not relying on manufacturers to simply supply them with technologies that they require. Rather, they are active participants in innovation, either modifying existing technologies or developing new ones. One in five (20.7%) manufacturing plants that introduced advanced technologies was a technology modifier and an equal proportion (21.8%) of manufacturing plants were technology developers. 5 

The Follow-up to the Survey of Advanced Technology 2007 6  was designed to explore activities in user-innovative plants. It was sent to a quota sample of manufacturing firms classified as technology modifiers or technology developers according to responses to the Survey of Advanced Technology 2007. 7  In March 2008, 1,750 questionnaires were mailed, 915 to technology modifiers and 869 to technology developers. The response rate was 73.0% with 72.1% of modifiers and 73.9% of technology developers returning questionnaires.

The following sections will present results based predominantly on the Follow-up to the Survey of Advanced Technology 2007 with some findings from the Survey of Advanced Technology 2007. The existence of "hidden" innovations will be explored. Results from the Follow-up to the Survey of Advanced Technology 2007 will be presented, first for user innovators overall and then in more detail. Characteristics of technology modifiers and of technology developers will be presented individually and then compared for these two types of user innovators. Given that the survey used a quota sample approach, results are expressed as a percentage of the total number of plants that responded to the question being analysed rather than being representative of the population. Response rate is provided as an indicator of data quality in Appendix A, Table A.

2.2  Identifying "hidden" innovations

Do traditional innovation surveys adequately capture innovative activities? Results of the Survey of Advanced Technology 2007 raise some questions but are not conclusive. Bearing in mind that user innovation occurs in response to production issues it would be expected that the percentage of process innovators would be larger than the percentage of user innovators and that user innovators would be a sub-set of process innovators. User innovations that are not a sub-set of process innovations can be described as "hidden" (NESTA, 2007) and would show that traditional innovation survey questions are not completely capturing innovative activities.

Results of the Survey of Advanced Technology 2007 show that the incidence of process innovation among advanced technology users (51.3%) is greater than the incidence of user innovation (42.5%) (Table 1).

However, among advanced technology users, one in six (16.5%) plants were identified as a user innovator but did not indicate they had a process innovation. In other words, almost four of every ten (38.8%) user innovators did not indicate that they were process innovators. This apparent disconnect agrees with findings of von Hippel and de Jong (2008). Despite this, no conclusions can be drawn as the data have considerable caveats. The Survey of Advanced Technology, 2007 reference period for process innovations was 2004-2007. Any process innovations occurring outside this three year time frame were not captured. There was no reference period for the method of acquisition or integration of advanced technologies. It is not known if the technology was acquired prior to 2004. As a result, any user innovation occurring prior to 2004 would not be a subset of the process innovations from 2004 to 2007. Using the existing data it is not possible to isolate these. Ideally, only user innovations that occurred from 2004 to 2007 should be considered in an examination of their relationship to process innovation rates.

Despite these limitations, the data show that half (50.6%) of process innovators that used advanced technologies had a user innovation and 61.2% of user innovators were process innovators. This highlights the importance of this activity in the innovation process. Understanding the activities of user innovators may help in the formulation of questions that can be used in conjunction with more traditional innovation survey questions to better capture this type of innovative activity and can help identify areas for policy focus for innovation stimulation. Results of the Follow-up to the Survey of Advanced Technology 2007 provide some insights to user innovation activities.

2.3  Activities of user innovators

User innovators are more likely to modify technologies or develop new technologies occasionally than continuously (Chart 1). Responding user innovator plants were two and a half times more likely to carry out technology modification or new technology development as part of an informal rather than as part of a formal program of technology adoption.

Further, survey results show that almost half (48.0%) of user innovators indicated that they developed new technologies or modified an existing technology occasionally and as part of an informal program of technology adoption (Chart 2).

User innovator respondents that indicated they carried out user innovation as part of a formal program of technology adoption were twice as likely to carry out these activities on a continuous rather than on an occasional basis (Chart 3). Similarly, user innovators that carried out technology modification activities as part of an informal program of technology adoption were twice as likely to carry out these activities occasionally rather than continuously. It would be interesting to explore whether the frequency of user innovation has an impact on innovation overall; whether plants that continuously carry out user innovation have more innovations than those that carry out user innovation occasionally. If so, this would emphasize the importance of formal programs of technology adoption for increasing innovation rates.

The vast majority (98.4%) of user innovator respondents indicated they fund the development of new technologies or modification of existing technologies internally (Chart 4). Customers and suppliers were indicated as a source of funding for about one in ten user innovators each.

The majority (77.9%) of user innovator respondents indicated one source of funding. Few (5.8%) had more than two (Chart 5).

Further, three quarters (76.4%) of user innovators indicated internal funding as the sole source of funding for their user innovation projects.

Almost half (46.3%) of responding user innovators indicated that they have a dedicated budget to fund the modification of technologies or development of new technologies (Chart 6). Responding user innovators were just as likely to fund the development of new technologies or modification of existing ones from their R&D budget as from the maintenance budget. Only one in ten plants (9.9%) responded that it was part of the innovation budget.

Almost two thirds (63.7%) of responding user innovators indicated they used only one budget to fund their user innovation projects (Chart 7). One in four (26.3%) used two.

More than half of responding user innovators (58.9%) indicated that they did not know of any other firms that had developed similar new technologies to the ones they had developed or had carried out similar modifications. This suggests that most user innovators are producing novel innovations. Conversely, four in ten (41.1%) responding plants knew of other firms that had.

Cooperation is common among user innovators. Six out of ten (60.2%) user innovator respondents indicated they cooperated with other plants, firms or institutions to develop new technologies or modify existing technologies. Cooperation for user innovation is more prevalent than for innovation overall where two in ten (21.5%) innovative plants cooperated for innovation (Survey of Innovation 2005). Given that a proportion of user innovators are contributing to this, the contrast should be greater if the calculation for innovation overall did not include user innovators.

The most common type of cooperative partner for user innovators was suppliers (Chart 8). Half (54.3%) cooperated with other plants in the firm. About four in ten cooperated with clients or customers (43.6%) and consultants (40.9%). Almost one quarter (22.2%) of user innovators indicated they collaborated with universities.

Four out of five plants (82.5%) identified one to four types of cooperation partners (Chart 9). One quarter (24.3%) identified two partners and one quarter (26.9%), three partners.

User innovations are being diffused. About one in five (18.0%) user innovators that responded indicated that they shared their new technologies or technology modifications with other firms or institutions (Chart 10). When plants did share, six out of ten (60.7%) responding firms indicated they did so at no charge.

More than half (53.7%) of the user innovators who shared their user innovations indicated they chose to share to allow a supplier to build a more suitable final product. These linkages between plants promote the diffusion of innovations and are an important component of innovation overall. Clearly these firms are part of the innovation process by contributing ideas. However it is not known whether the supplier would recognize or acknowledge these firms as a source of information for innovation when responding to more traditional innovation survey questionnaires. User innovators see benefit from sharing their innovations. Enhancing reputation and gaining expertise were indicated as reasons for sharing by at least four of ten plants that shared (Chart 11). Contractual obligation was a motivator for only about one quarter (22.0%) of user innovative plants. Almost one third (31.3%) indicated they have nothing to lose from sharing.

More than half (53.3%) of the user innovators who responded indicated that they use some method to protect the intellectual property resulting from the modification of technologies. This contrasts to the recent findings of von Hippel and de Jong (2008) who find that only 9% of all user innovations in manufacturing SMEs (less than 100 employees) are protected and suggests that plant size should be considered in the analysis. Confidentiality agreements were used by most user innovators to protect their intellectual property (Chart 12). Patents were used by more than half (57.3%) of user innovator respondents who protected their intellectual property.

There is debate over the impact of intellectual property protection on fostering innovation. Two thirds (66.7%) of user innovators who shared their user innovations used some method to protect their intellectual property. However, among user innovators that did not share, half (50.3%) still used some method to protect their intellectual property. This suggests that plants that use intellectual property protection are more likely to share their user innovations than those that do not protect their intellectual property and warrants further investigation. What is not known is whether the user innovation that was shared was protected.

More than half of user innovators used one or two methods to protect the intellectual property resulting for the modification of technologies or development of new technologies (Chart 13). One in five (20.9%) used three methods.

There are no large differences in the methods used to protect intellectual property by plants that share and those that do not share their user innovations (Chart 14). Whether or not user innovations are shared does not seem to influence how the intellectual property of these user innovations is protected. A statistical sample would permit the construction of confidence intervals and it could be determined if estimates were statistically significantly different from one another.

User innovations are being diffused making user innovators part of the innovation process. One quarter (26.3%) of user innovator respondents indicated that the new technologies (or parts of them) developed by their plant were adopted by another manufacturing firm to produce and supply the new technology or that the modifications were adopted by the supplier of the original technology. These suppliers may be producing and supplying these product innovations to other users. It is not clear whether this would be indicated as a process innovation by the user innovator and a product innovation by the original supplier or both. However it is clear that user innovators are part of the innovation process. In addition, one quarter (25.3%) of user innovation respondents indicated that the new technologies they developed were adopted by other firms to use the new technology or that their modifications were adopted by other firms that use the original technology.

In order to facilitate response to questions on cost of new technology development or modification of technologies, respondents were directed to respond in reference to their most recently developed new technology or most recently modified technology. The average estimated total cost of labour for the most recent user innovation was $326,177 with a median value of $45,000. The average estimated total cost of machinery, equipment and/or materials required for the most recent user innovation was $484,595 with a median value of $50,000. Considering labour cost and cost of machinery, equipment and materials together, the average total cost of the most recent user innovation was $815,580 with a median value of $125,000. It should be noted that during cognitive testing of the questionnaire, some respondents found it easier to recall a larger project rather than the most recent. This introduces some degree up upward bias in the cost estimates with a few very large projects skewing the average costs and can help to explain why the average costs are about ten times larger than the median costs.

Two thirds (65.2%) of user innovators that responded indicated that it took from 2 months to less than two years to complete the development of their most recently developed new technology or modification of their most recently modified technology (Chart 15).

2.4  Activities of technology modifiers and technology developers

Response rates by question for technology modifiers and technology developers differ by no more than 5.7% (Appendix A, Table A). Given comparable response rates and response rates of at least 84% for all questions, response bias is not considered an issue and characteristics of technology modifiers can be compared to technology developers.

Responses provided by technology modifiers indicate that they are most likely to carry out technology modification on an occasional basis, whereas technology developers have an equal likelihood of carrying out user innovation occasionally or continuously (Chart 16). Technology modifiers are more likely than technology developers to carry out user innovation on an occasional basis. Conversely, technology developers are more likely than technology modifiers to carry out user innovation continuously.

Technology modifiers and technology developers are most likely to carry out user innovation as part of an informal program of technology adoption (Chart 17). Technology modifiers are four times more likely to carry out technology modification as part of an informal rather than as part of a formal program of technology adoption. Technology developers are more than one and a half times as likely to develop new technologies as part of an informal program as opposed to a formal program of technology adoption. When technology developers are compared to technology modifiers, technology developers are much more likely than modifiers to have a formal program whereas technology modifiers are more likely than developers to have an informal program of technology adoption.

Both technology modifiers and technology developers are most likely to carry out user innovation occasionally and as part of an informal program of technology adoption (Chart 18). Technology modifiers and technology developers have the same likelihood of carrying out user innovation continuously and having an informal program of technology adoption. They also have the same likelihood of carrying out user innovation occasionally and having a formal program of technology adoption. Technology developers distinguish themselves from technology modifiers in that they are more than twice as likely to carry out user innovation continuously and to have a formal program of technology adoption. Technology modifiers, on the other hand, are more likely than technology developers to carry out user innovation occasionally and have an informal program of technology adoption.

Responses from technology modifiers and developers suggests that plants with formal programs of technology adoption carry out user innovation more frequently (i.e. continuously) than those with informal programs. Among technology modifier respondents that carried out technology modification activities as part of an informal program of technology adoption, modification was more than twice as likely to be carried out occasionally. Among technology modifier respondents that carried out technology modifications as part of a formal program of technology adoption there was a slightly greater likelihood of carrying out these activities on a continuous as opposed to an occasional basis (Chart 19).

These contrasts are even sharper for technology developers (Chart 20). Among technology developers that developed new technologies as part of a formal program of technology adoption, plants were more than two and half times more likely to continuously develop new technologies and those that developed new technologies as part of an informal program of technology adoption were more than one and a half times as likely to occasionally develop new technologies. This suggests that having a formal program of technology adoption promotes continuous innovation.

Almost all technology modifiers and technology developers indicated that they funded their modification or development of technologies internally (Chart 21). Customers followed closely by suppliers were the next most commonly indicated sources of funding for both technology modifiers and technology developers. However, technology developers are more than twice as likely as modifiers to have these sources of funding, highlighting that technology developers are more likely than technology modifiers to have external sources of funding for their user innovation projects. Customers are a source of funding for one in six (16.8%) technology developers. This linkage between actors in the innovation system is something that should be explored further.

Although most likely to have only one source of funding for their user innovation projects, technology developers are twice as likely as technology modifiers to have more than one source of funding (Chart 22). Three in ten (29.3%) technology developers had multiple funding sources compared to fewer than two in ten (15.1%) technology modifiers. Technology modifiers (84.9%) are more likely to have only one source of funding than technology developers (70.7%). Technology developers (20.3%) are almost twice as likely as technology modifiers (12.5%) to have two sources of funding.

Technology modifiers are more likely than technology developers to exclusively fund user innovation internally. Almost seven in ten (69.3%) technology developers indicated only internal funding for their user innovation compared to more than eight in ten technology modifiers (83.3%). This suggests that technology developers have more linkages in the innovation system than technology modifiers.

Technology developers are most likely to fund the development of new technologies as part of the R&D budget (48.6%) followed closely by having a dedicated budget for each project (43.7%) (Chart 23). Technology modifiers, on the other hand, are just as likely to have a dedicated budget for each project (48.8%) as they are to fund the modification of technology as part of the maintenance budget (49.2%). Three in ten (30.5%) technology modifiers indicated they funded technology modification from their R&D budget. Only one in ten technology modifiers (10.2%) and one in ten technology developers (9.5%) responded that they funded their user innovations through an innovation budget. It is interesting to note that only about half of the technology modifier and technology developer respondents indicated that they had a dedicated budget for each modification or development project or program. This is not unusual considering that user innovation projects are most likely to be carried out occasionally and as part of an informal program of technology adoption for both technology developers and modifiers in which case a dedicated budget would not be expected.

Both technology modifiers and technology developers are most likely to use only one budget for their user innovation projects (Chart 24). Responses indicate that technology modifiers and developers have similar behaviour for the number of budgets used.

The majority of both technology modifiers (62.0%) and technology developers (55.8%) indicated that they did not know of any other firms that had created similar user innovations. It would be interesting to examine how these respondents would report the novelty (world, country or plant first) of their innovations. A high incidence of world or country-first innovation novelty would be expected amongst each type of user innovator.

Although a common activity for both technology modifiers and technology developers, cooperation on user innovation projects is more likely among technology developers. More than half (55.5%) of responding technology modifiers indicated they cooperated with other plants, firms or institutions to modify existing technologies compared to almost two thirds (65.1%) of technology developers that cooperated to develop new technologies. The linkages created through cooperation characterize technology developers as having a higher propensity to be linked than technology modifiers.

More than four out of five technology developers and modifiers that responded they co-operated on their user innovation projects indicated they cooperated with suppliers, making them the most likely partner for both types of plants (Chart 25). Technology developers are next most likely to cooperate with clients or other business units in the firm, followed by consultants. Technology modifiers are next most likely to cooperate with other business units in the firm, followed by clients and then consultants. Technology developers are more likely than technology modifiers to cooperate with clients or customers, commercial labs, universities, colleges, and federal government labs whereas technology modifiers are more likely than developers to cooperate with other plants in the firm.

There is less than 5% difference between the percentage of technology modifiers and developers cooperating with industrial associations, private non-profit research institutes, provincial or territorial labs or research institutes, consultants, competitors, and suppliers. As a result, there is no greater likelihood that a technology developer or modifier will choose one of these partners.

Among user innovators that cooperate, technology developers are likely to have a larger number of types of cooperation partners than technology modifiers (Chart 26). Technology modifiers are more likely than developers to have one or two types of partners whereas technology developers are more likely than modifiers to have four or more types of partners. Both are just as likely to have three.

Responses of technology modifiers and technology developers indicate that although they are just as likely to share their user innovations it is not a common activity for either type of user innovator with fewer than one in five plants doing so (17.2% modifiers and 18.8% developers). In spite of this, it is important to note that innovations are being diffused in the economy and there is opportunity to increase this diffusion rate. The sharing of user innovations creates linkages as part of the innovation process. Clearly these plants are contributors to innovation, however it is not known whether the beneficiary of the sharing would recognize or acknowledge the plants as the source of the innovation when responding to questions on traditional innovation surveys.

Responses of technology modifiers and technology developers show that the most common way to share among both types of user innovators is at no charge (Chart 27). That said, technology modifiers are more likely than developers to share at no charge, with three quarters (75.8%) of technology modifiers who responded indicating they share at no charge compared to less than one half (47.3%) of technology developers. On the other hand, technology developers are three times more likely than technology modifiers to share for a fee and are more likely to share in exchange for something of value.

Among the technology modifiers and developers who indicated they shared the technologies they modified or developed, the most commonly indicated reason for sharing was to allow a supplier to build a more suitable final product (Chart 28). The next most indicated reasons, gaining feedback and expertise and enhancing reputation, were indicated more frequently by technology developers than technology modifiers. Responses provided suggest that a larger percentage of technology developers recognize the advantages gained by sharing.

One in four (26.8%) technology developers who responded indicated they had nothing to lose by sharing compared to one in three (36.3%) technology modifiers. This suggests that a larger percentage of technology modifiers may be less concerned with competition than technology developers. Technology developer respondents were almost twice as likely as technology modifiers to indicate they shared as a result of a contractual obligation so although they did share, it is not known whether sharing would have occurred without a legal motivator.

Six in ten (60.3%) technology developers that responded indicated they protected the intellectual property resulting from their user innovations compared to less than half (46.4%) of technology modifiers. Although von Hippel and de Jong (2008) found a lower incidence of use of intellectual property protection than this study they too found that technology developers protected their intellectual property more often than technology modifiers.

Among plants that protected intellectual property, confidentiality agreements were by far the most frequently indicated method used by both technology developers and technology modifiers (Chart 29). Although the relative use of each method among technology developers and modifiers follows the same hierarchical order, technology developers have a consistently larger percentage of plants using each method compared to technology modifiers.

Presumably, plants that are developing new technologies would have more novel innovations than those modifying existing technologies. It would be expected that plants with more novel innovations would have a greater propensity to use intellectual property protection methods than plants without novel innovations. This remains to be explored.

Technology developers are more likely to use a greater number of methods to protect the intellectual property of their user innovations than technology modifiers (Chart 30). Technology developers are most likely to use two methods whereas technology modifiers are most likely to use one.

There is debate on the impact of intellectual property protection on the stimulation or hindrance of innovation. More than three in four (76.1%) technology developers and more than half (56.6%) of technology modifiers who responded that they shared the new technology developments also responded that they protected the intellectual property resulting from their user innovations (Chart 31). Although not as common, still more than half (56.6%) of technology developers and 44.3% of technology modifiers who responded they do not share their user innovations indicated they protected the resulting intellectual property. Did the use of intellectual property protection methods increase the likelihood of sharing? This cannot be answered with existing data; However it should be explored. What can be said is that technology developers and technology modifiers who share their user innovations are more likely to protect the intellectual property of their user innovations than plants that do not share.

The taxonomy of plant type considers the degree of effort required for each method of technology adoption increases as the complexity of the introduction method. As the complexity of the introduction method increases so does the need for internal capabilities. As a result, it would be expected that it would be more costly and time consuming to develop new technologies than it would take to modify an existing technology. Responses provided by technology modifiers and developers on costs and time spent on their most recent user innovation project agree with this hypothesis. The average total cost of labour to develop the most recently developed new technology was almost twice the average cost to modify the most recently modified existing technology and median costs almost four times as large (Table 2). The average cost of machinery, equipment and materials was 40% higher and median cost almost double for technology developers than for technology modifiers. The total overall cost for the most recent technology development project was 59.7% higher than the cost of the most recent technology modification project.

Not surprisingly, projects for the development of new technologies take longer than modification of existing ones. Technology developers who responded were more likely than technology modifiers to take upwards of 6 months in elapsed time to complete their most recently developed or modified technology (Chart 32). Conversely, technology modifiers who responded were more likely than technology developers to take 6 months or less.

3   Measurement issues and lessons learned

Statistics Canada's first survey of user innovation has successfully shown that respondents are able and willing to complete questions on user innovation activities allowing the production of some metrics. Cognitive testing of the questionnaire with potential respondents and incorporation of respondent feedback into the final version of the questionnaire is considered as a key factor in attaining the high overall response rate (73%), achieved with the mailing of one reminder card and no telephone follow-up. Further, this response rate is consistent for both the sample of technology modifiers (72.1%) and technology developers (73.9%). Among the completed questionnaires, individual question non response was highest for the quantitative questions (response rates were lowest for these questions). These included total cost of labour for technology modification or development with response rates of 88.0% and 86.9% respectively and total cost of machinery and equipment for modification or development with response rates of 87.1% and 84.9% respectively. If imputation had been used for item non response, the imputation rate per question would have only reached a maximum of 15.1%. It should be noted that the sample for the Follow-up to the Survey of Advanced Technology 2007 was taken from respondents to the Survey of Advanced Technology 2007. These respondents had already indicated a willingness to respond to Statistics Canada surveys. A stand-alone survey may not yield as high a response rate.

Analysis of survey results has allowed description of some innovation activities of user innovators. The survey has shown that the activities of technology modifiers and developers have some marked differences. When possible, attempts should be made to distinguish these two sub-populations to facilitate detailed study.

Respondent-supplied information where respondents specified a response other than the options provided will help improve future survey questionnaires. For example, among respondents who specified a funding source other than the three options provided, a large percentage specified that the development or modification of technologies was government-funded; some even provided the name of the program such as the SR&ED tax credit program, NRC-IRAP, and grants.

von Hippel (2005) describes user innovation as development or modification of technologies for use within a plant to enhance operations when specific process needs cannot be met by existing market solutions. Regardless of whether this technology ultimately becomes a product that is sold, the original intention was to have it used by the plant itself. These differ from technologies created with the intention of marketing and selling them as a product. The formulation of a simple description allowing respondents to make this distinction proved problematic. During cognitive testing of the questionnaire attempts were made to qualify the activity of technology modification or development to restrict it to user innovation; However, despite several attempts, a simple question could not be formulated. Businesses communicated that if their modification or development projects were not going to be used as part of the firm's operations then they would not carry them out. They considered product sales as part of the plant's operations, being used by the plant to enhance revenues. This measurement issue remains unresolved.

Cognitive testing revealed that the identification of technology modification by respondents was not always clear. This was most apparent in cases of use of design software. Some manufacturing plants interpreted that specification of input such as that required in a CAD program was modification of the software by the user to meet their needs. The distinction between the application of software and the modification of software's source code could not be conveyed in a manner where it was understood the same way as was intended. Ultimately, this affected 1.5% of technology modifiers who contacted Statistics Canada for clarification during data collection. This proportion reflects a lower limit as it would be expected that not all respondents would necessarily seek this clarification. Future surveys will need to consider this in questionnaire design.

The development of indicators for the cost of technology modifications or the development of new technologies also proved problematic during questionnaire testing in three general areas. First, respondents communicated that projects could be diverse making it difficult to provide an annual cost figure. They were able to describe individual projects. The questionnaire was modified to direct reporting on the "most recently developed new technology" or "most recent modification", as appropriate although bias towards larger projects in the respondents' recollection has been recognized.

Second, the target respondent was the CEO and he did not necessarily have the detailed information to provide cost information from an accounting perspective. Respondents were not sure how to report. Coupled with this was a reluctance to provide detailed cost figures. As a result, respondents were asked to provide information for three indicators of the cost of user innovation: cost of labour time; average cost of machinery, equipment and materials; and time required to complete the most recent project.

Third, during cognitive testing of the questionnaire several respondents explained that personnel may be involved in one or more projects with varying degrees of time commitments. This is reflected in the responses to the Follow-up to the Survey of Advanced Technology 2007 which show that almost two-thirds of technology modification projects and one half of technology development projects were carried out on an occasional basis which can help explain why reporting cost information is difficult. The use of units of measure such as person hours made responding too difficult during questionnaire testing. To resolve this, the qualification that the time required to complete modification or development projects be "elapsed time" was included in response to respondent feedback that this was an easier number to derive. An indicator of commitment of personnel was measured by cost of total labour. Finally, a question on the cost of machinery, equipment, and materials required to develop or modify the technology was included to complete the picture. High response rates by technology modifiers and developers (Appendix A, Table A) show that respondents were willing and able to provide this information.

Surveys of innovation ask questions on new or significantly improved processes. The qualifier "significant" is open to interpretation and it can be argued that some innovative activities are being overlooked or discounted. We have learned that one third of technology modifiers who responded to the Follow-up to the Survey of Advanced Technology and more than one half of technology developers carried out these activities as part of the maintenance budget. This raises the question of whether a traditional innovation survey question on process innovation would capture activities that fall under a heading of "maintenance". Adding to this, 15.3% of technology developers and 12.3% of technology modifiers specified a budget not included in the supplied options. A preliminary analysis of these responses indicates a high percentage specified that user innovation was funded as part of the operating budget or as part of capital expenditures, raising concerns that these activities could be overlooked when responding to a traditional survey of innovation.

4   Conclusions

User innovation is a common activity among Canadian manufacturing plants that use advanced technologies. In 2007, one in five (20.7%) manufacturing plants that had introduced advanced technologies were technology modifiers and an equal proportion (21.8%) were technology developers. Taken together, this provides an indicator for user-innovation with one in ten (42.5%) manufacturing plants that introduced advanced technologies having carried out user-innovation in 2007.

Questions still remain unresolved as to whether traditional innovation survey questions are completely capturing innovation activities. Although 42.5% of plants that introduced advanced technologies were user innovators, only 61.2% of these plants were found to be process innovators during the years 2004 to 2007. Did the remaining 38.8% of user innovators carry out their process innovation outside the reference period? Without a reference period for the user innovation activity no firm conclusions can be drawn regarding the efficiency of the traditional innovation question on process innovation. What can be concluded is that half (50.6%) of process innovators that used advanced technologies also had a user innovation highlighting the importance of this activity to innovation overall.

User innovators are an important contributor to, and source of information for, innovation with their activities creating linkages with other actors in the innovation system. Survey results have shown that about one in six technology modifiers shared the technologies they modified with other firms or institutions and about one in five technology developers shared the new technologies they developed. Further, user innovators have a high incidence of cooperation on their projects. Six out of ten user innovative manufacturing plants indicated they cooperated for the modification or development of new technologies (55.5% of technology modifiers and 65.1% of technology developers). This is three times the incidence of cooperation for innovation activities in general. 8  Clearly user innovators play a role in the innovation system.

User innovations are being shared, most commonly at no charge. About one in five (18.0%) user innovators shared their user innovations with other firms or institutions with more than half (53.7%) of these plants choosing to share in order to allow a supplier to build a more suitable final product. The most common way to share for both technology modifiers and technology developers is at no charge. Gault and von Hippel (2009) develop the intellectual property policy implications of these findings.

User innovations are being diffused in the Canadian economy. One in four (26.3%) user innovators indicated that their user innovations were adopted by another manufacturing firm to produce and supply the new or modified technology and one in four (25.3%) indicated their user innovations were adopted by other firms. This evidence of innovation diffusion makes user innovators an important actor in the innovation system.

Lack of funds within the plant or firm was one of two obstacles to innovation 9  most frequently identified as having high importance (Survey of Innovation 2005). Government support programs such as the R&D tax credit program are aimed at increasing innovation. Half of user innovators had a dedicated budget for each project. Four in ten indicated they used their R&D budget and four in ten their maintenance budget. Only one in ten user innovators indicated they used their innovation budget to fund their user innovations. One explanation could be that in plants where user innovation is not part of a formal program of technology adoption and/or is carried out occasionally there is no dedicated innovation budget in the plant. Understanding how plants approach user innovation in their activities and treat it in their accounting can help government design programs to promote and support user innovation.

The analysis in this paper has been on the manufacturing industry overall however recent work by von Hippel and de Jong (2008) has shown that incidence of user innovation varies by industry and by firm size. It would be interesting to explore whether industrial differentiation exists at a finer level of detail within manufacturing as well as by size class. In addition, the statistical measurement of incidence of user innovation in a broad range of industries of varying size classes would provide a better indicator of the prevalence of this activity and contribute to a better understanding of innovation in Canadian industry. Firm size remains to be explored in analysis of user innovation.

Future work on user innovation should explore the linkages, including the flow of information, between user innovators and other actors in the innovation system. This study has made use of data from a pilot survey. A statistical survey program of innovation including questions on user innovation would allow better indicators of this activity in the Canadian economy.

5   References

Arundel, Anthony and Viki Sonntag (2000), Patterns of advanced manufacturing technology (AMT) use in Canadian manufacturing: 1998 AMT survey results. Catalogue no. 88F0017MIE, no. 12, Ottawa: Statistics Canada.

Ducharme, Louis Marc and Fred Gault (1992), Surveys of manufacturing technology, Science and Public Policy, 19, 393-399.

Gault, Fred and Eric von Hippel (2009), The prevalence of user innovation and free innovation transfers: Implications for statistical indicators and innovation policy. MIT Sloan School of Management Working Paper 4722-09.

Industry Canada (2001), Achieving excellence: Investing in people, knowledge and opportunity, Canada's innovation strategy.

NESTA (2007), Hidden innovation, how innovation happens in six 'low innovation' sectors.

OECD (2007) Innovation and growth: Rationale for an innovation strategy.

OECD/Eurostat (2005). Oslo Manual, Guidelines for Collecting and Interpreting Innovation Data (3rd edition), Paris and Luxembourg: OECD and Eurostat.

von Hippel, Eric (1998), The Sources of Innovation, New York: Oxford University Press.

von Hippel, Eric (2005), Democratizing Innovation, MIT Press.

von Hippel, Eric and Jeroen de Jong (2008). User innovation in SMEs: Incidence and transfer to producers. Scientific Analysis of Entrepreneurship and SMEs (SCALES), working paper H200814.