It’s always helpful to hear from people who have struggled with and overcome a problem you’re facing. With this post, we’re excited to launch the Practitioner’s Dispatch, a new series featuring innovation practitioners’ battle-tested advice on common problems. First up is Stewart Witzeman, who has more than 30 years of experience in innovation management. We asked him to give you his best advice on measurement for breakthrough (see also: disruptive, Horizon 3, etc.) projects. His bio follows the post. Thanks, Stewart!
“You can’t manage what you can’t measure.” (Peter Drucker)
“There are three kinds of lies: lies, damned lies, and innovation metrics.” (with apologies to Mark Twain)
When I work with companies on various aspects of their innovation processes the subject of metrics inevitably comes up. Long gone are the days when the Innovation Leader could say “give us money and good things will happen.” Furthermore, as is captured in the above quote by Drucker, a measurement system is essential to management and improvement of any process or activity, including innovation. With this stated we need to be cognizant of the assumptions, limitations, use and potential misuse of any measurement system, hence my appropriation of Mark Twain’s well-known quote.
Choosing metrics for different types of innovation projects is far from straightforward.
Innovation by its very nature is fraught with uncertainty, making any measurement system far from straightforward. Further complicating this picture is that appropriate metrics will vary by type and stage of project.
To understand this situation let’s consider two extremes on the innovation continuum:
- For innovation projects related to translation of core technology to well-understood markets the associated metrics will have uncertainty associated with technology development, market adoption, and competitive activities. When considering a portfolio of projects of this nature these uncertainties can usually be managed and understood.
- The other end of the continuum is breakthrough projects involving new technology, new markets, new product offerings/business models or some combination of the above. As with projects related to core activities, useful metrics can be developed for these projects, but the nature of the metrics and uncertainties associated with them are decidedly different.
Beware metrics that convey certainty where none exists.
The danger comes if one attempts to compare or equate the metrics associated with these very different types of projects. With the first kind, core translation projects, standard outcome metrics such as Net Present Value (NPV), Estimated Sales Opportunity (ESO), Revenue, Margin and Market Share can be used with great success. More certainty is obtained as these projects progress, which informs project and portfolio decisions.
For breakthrough projects, use of the above listed value metrics implies a level of certainty where no such certainty exists. More appropriate metrics for these types of projects include market potential or “size of the prize,” potential for market growth, and measures of strategic alignment. Reviews of projects of this nature should thus be focused on learnings, the next steps needed to reduce uncertainty, as well as continued evaluation of the market/business potential. The more ambiguous nature of these metrics makes project and portfolio decisions decidedly more difficult due to the more subjective criteria involved. To help with this ambiguity many companies emulate the venture capital investment community, where funding is provided to the next milestone and further funding is contingent on continued project advancement against milestones.
Measuring strategic alignment can help maintain balance.
Choosing metrics for a portfolio of different types of projects leads to a significant management challenge. If one uses discounted cash flow or similar approaches, then growth/breakthrough projects will be discounted so highly as to make their value effectively zero. In the absence of other criteria this leads to an inherent bias away from new/breakthrough areas. On the other hand, focusing exclusively on the higher-level metrics such as market size and growth potential can turn into an exercise in “chasing shiny objects,” resulting in lack of continuity and neglect of core platforms.
The answer to this dilemma lies in measuring strategic alignment—calibrating against a clear and concise strategy that sets forth both core and emerging areas of emphasis. This enables targeted allocation of resources by area and type of project and, equally as important, the discipline to adhere to these established allocations. This discipline will prevent the temptation to pull resources from growth/breakthrough innovation for shorter term projects that are closer to the core.
For more detail on ensuring balance in your measurement system, Stewart recommends reading up on the Technology Value Pyramid. See here, pages 42-45 of our Dashboard Survey Report, and:
James W. Tipping, Eugene Zeffren & Alan R. Fusfeld (1995) “Assessing the Value of Your Technology,” Research-Technology Management, 38:5, 22-39
Lawrence Schwartz, Roger Miller, Daniel Plummer & Alan R. Fusfeld (2011) “Measuring the Effectiveness of R&D,” Research-Technology Management, 54:5, 29-36
You can also check out Commodore’s framework, here. It’s organized around the 12 questions your metrics need to answer.
About the author: J. Stewart Witzeman spent more than 30 years in the chemical industry working for Eastman Chemical Company. During his tenure at Eastman he held a wide variety of positions managing various Research and Development organizations, including a final adventure establishing the Eastman Innovation Center on the North Carolina State University Centennial Campus. Since leaving the corporate world he has worked as an independent consultant where he specializes in various aspects of innovation and university – industry partnerships as well as working with local non-profits and in initiatives associated with STEM education.
Stewart is a former board chair of the Industrial Research Institute (IRI, now known as the Innovation Research Interchange) where he gained insight on best practices across a wide range of industries and technologies.
Stewart holds a PhD in Organic Chemistry from the University of California at Santa Barbara, a BS in Chemistry and Environmental Science from Northern Arizona University and was a post-doctoral fellow at the University of Chicago.
