Measure for Measure
Tech Transfer
December 2011 / January 2012
Volume 9 Number 6
Saving lives, kilowatt hours of electricity, gallons of gasoline and barrels of oil. Increasing output of a chemical process. Decreasing the cost to manufacture a product or the time through technology development to the market. These are outcomes of technology transfer that we can all recognize as desirable, worthy measures of success. Quantifying these benefits can, however, be a somewhat speculative task for a technology that is new to market. The stories are real, they involve real people, real companies, real benefits and real impacts. But they are difficult to quantify in any meaningful way across a laboratory’s portfolio of technologies.
On October 28, the White House issued a memorandum to department and agency heads that includes the directive to establish performance goals, metrics and evaluation methods to improve the returns from the federal R&D investments and to track the progress made in achieving those goals. The DOE laboratories, along with other federal labs, are required to develop a five-year plan to evaluate this progress.
Through the memo, the Department of Commerce is charged with improving and expanding its collection of metrics for an annual technology transfer summary report that covers the 11 departments and agencies that conduct technology transfer at federal laboratories. What new measures or methods will ultimately be developed and selected remains to be seen. The memo itself lists several types of legal transactions that are already measured by the labs. It also suggests that new products and successful self-sustaining spinoff companies related to those products may be useful measures. Some gathering of that data already occurs, but formalizing the collection of that data may be informative.
The difficulty of measuring effective technology transfer has been discussed and lamented since technology transfer was formalized following enactment of Bayh-Dole. The action of capturing a single year’s metrics provides but a snapshot in time. Even comparing one year to the next doesn’t provide sufficient information to draw realistic conclusions. Technology transfer occurs over time and with multiple points at which to invest in its success, leaving also multiple failure points. Technology transfer doesn’t have a single date in which it occurs, like a birth date. Yet we have been trying to measure its success as if it did. These points along the tech transfer pathway can be viewed like a chemical experiment, with starting materials, intermediates and product outcomes. And there is a predictive pattern: each successive step can be expected to yield a smaller quantity than the preceding step.
We need to know whether we have sufficient starting material to feed the process—are we providing sufficient opportunity and incentive for our inventors to invent and to interact with industry partners who will be interested in working with the inventor or licensing the technology? It is only by having sufficient amounts of these starting materials can we expect to generate an optimal number of invention disclosures, industry partnerships and legal transactions to measure as the intermediate in our process of technology transfer. Licenses and industry partnerships of course are not the end goal. Only some of these in the intermediary step will mature into a product, part of a product or as a commercial process. We need also to measure these ultimate outcomes.
To some extent, metrics from each of the stages are already gathered, although the value of providing sufficient opportunity and incentive for the scientist and the potential industry partner to come together is not yet recognized in the metrics. In our efforts, we increasingly show an understanding of this critical starting point, but without the metric, we cannot measure its performance or drive behavior sufficiently in its direction. And in evaluating the metrics, we have done little to acknowledge the relationship of one stage to the other. This is a critical point. Without an understanding of what feeds a system, we cannot hope to improve it.
To begin to evaluate the effectiveness of our efforts in technology transfer and to drive behavior toward improving it, our metrics must recognize that one stage feeds into the next. This can be done by looking at ratios—types or amounts of incentives to the number of invention disclosures; number or types of interaction opportunities to number of partnerships entered; type and number of transactions to type and number of products, startups or other outcome desired. It is then that we can have more confidence that we know what is needed and drive numbers and behavior in the direction we wish.
Since technology transfer happens over time, often several years, such a relationship or ratio is only meaningful if we acknowledge the time lag between stages. For example, when we provide funding for a basic research project, can we expect technology transfer during that same year? That’s highly unlikely. More often, it occurs three to five years downstream, sometimes much longer. To evaluate the return on an investment of R& D dollars in 2007, for example, we would be better served to look in the aggregate to 2010 invention reporting and 2012 licensing of technology. And since the technology development and transfer does not follow a rigorous timetable, we may draw an even more informative picture of effectiveness by aggregating three years of data for each part of the ratio, comparing, for example, R&D dollars invested in 2006-2008 with invention reporting in 2009-2011.
Recognizing the lag time from stage to stage and from initial Federal R&D investment to outcome is more critical now than ever. As R&D budgets are strained, our measures must increasingly reflect the reality of technology transfer if we are to evaluate its effectiveness and seek ways to improve its performance to achieve returns on investment. Using a method beyond gathering the snapshot of a year’s data or even comparing one snapshot to another requires resources, mostly time, at a point when all are already being asked to do more with less. But if we truly desire to understand how to drive ever more effective efforts, we are obliged to evolve evaluation methods that increasingly provide a better reflection of the actual drivers in the process. We have an opportunity and an urgency now to engage in that discussion. For the outcome to achieve its purpose, however, we cannot see it solely as a matter for technology transfer. The efforts and culture of the research enterprise itself feeds the starting material of the tech transfer process and can serve as a powerful catalyst for accelerating tech transfer from the lab to market.
Rochelle Blaustein is senior advisor in the DOE Office of Technology Transfer.
More
Copyright © 2012 | Innovation America