A lot has been written about measuring innovation in recent years. I’ve been reviewing this literature as we build our service offering for Commodore Innovation. (For anyone interested, we have a live list of resources related to measurement of innovation here). Many of the articles, blog posts, surveys of industry practice, etc. suggest frameworks for measuring innovation. That is, they propose a way of thinking about what companies should measure when it comes to tracking innovation. Done well, such frameworks become tools that enable companies to quickly review and enhance their innovation measurement systems or establish entirely new systems. The existing frameworks available fail to achieve that objective, for reasons I describe below.
Existing frameworks for measuring innovation performance conflate innovation performance and innovation capabilities.
Many frameworks fail because they attempt to describe two quite different things, without drawing the distinction:
- Measuring the performance of an innovation function (as measured by metrics like number of ideas generated, or percent of revenue from new products / services introduced in the last 3 years).
- Assessing a firm’s innovation capabilities (e.g. share of employees have been trained in certain innovation methods, number of incentive schemes supporting innovation)
These are two fundamentally different things. Commodore Innovation is focused on measuring the performance of an innovation function, so that’s where this blog post will focus. I’m not suggesting you should ignore innovation capabilities. But we do believe it’s critical to be able to measure the performance of your innovation function first. Otherwise, how will you know any attempt to improve your capabilities was successful? Sure, you can count the number of design thinking workshops you’ve conducted. Or even measure the adoption of design thinking practices by your team. But did that investment have any positive outcomes for your business?
Some frameworks include categories of innovation metrics that lack clear definition.
Looking at the dozens of categories of metrics defined or recommended in the literature, it’s easy to see how some companies end up with cluttered dashboards. Of these, we think inputs, outputs, and outcomes are critical. Not all sources use these exact labels. Definitions vary and are sometimes confusing. We like the following definitions, based on a report by the National Academy of Science:
- Inputs: measures of tangible quantities put into your innovation process to enable you to achieve your innovation goals.
- Outputs: measures of the things your innovation process has produced – which may be a “finished” product or service ready to go to market and interim outputs such as a new idea, proof of concept prototype, etc.
- Outcomes: measures of results that stem from use of the outputs of your innovation process.
Other common categories (that don’t quite make my cut as “critical”) include the following.
- Impact – measures of the broader socio-economic consequences of an outcome from your innovation process. Measuring impact may be important in certain situations, like for a social impact incubator.
- Activity – Measures of activity (e.g., number of ideas submitted by employees) are generally measures of output or interim output so don’t merit a separate category. Also, it’s common for companies to get stuck measuring only activity.
- Processes – Measures of the course of action to achieve innovation goals (e.g., an effective project review process) seem to be largely about assessing innovation capabilities. As explained above, we think it’s best to focus on performance first.
- Portfolio – e.g., the share of investment by stage of development (such as breakthrough projects versus product line extensions). Looking at the portfolio is extremely valuable, but it is not a separate category of measurement. Instead, it’s one of the levels at which you can drill down if you have a nesting measurement system (more on this in our next blog post).
Many frameworks stop short of recommending what aspects of innovation performance measure.
An input/output/outcome framework is attractive in its simplicity. But at this point in my review, I realized I saw very little guidance on what companies should measure within these large buckets. After all, you could easily satisfy the condition of measuring “outputs” but fail to measure aspects of innovation performance that are critical. Within outputs, for example, a firm could track number of patents granted, but know nothing about the expected commercial value of those patents.
To learn more, I reviewed around 150 innovation metrics used by companies today. Clustering these metrics by theme reveals the questions firms are trying to answer when measuring innovation outputs and outcomes:
- Value: Are the firm’s innovation projects creating value (or future value)? For example, are they creating future new revenue opportunities, de-risking innovation initiatives or enabling cost savings through business process improvements?
- Learning: Are the firm’s innovation projects making progress in a way that is unlocking future value?
- Strategy: Are the firm’s innovative projects aligned with strategy? For example, does the portfolio of innovation activities achieve the target spread across innovation “horizons”?
- Efficiency: Are the resources utilized by the innovation function being efficiently used to create future value?
- Speed: Are the firm’s innovation projects moving quickly enough?
Think these questions aren’t rocket science? Good! I hoped not. But, if you can succinctly answer these questions (with both quantitative metrics and qualitative insight), then you’re providing your organization with the information it needs. If the answers are “good” then you’re well positioned to demonstrate how well you’re performing. If the answers are “bad” then you’ll know where to focus to get to “good.”
The diagram below (click to download a PDF version) maps these questions to the input / output / outcome framing. A good measurement system should answer all ten of the questions here. That said, the metrics themselves and their relative importance will vary depending on your specific context (and over time). The metrics in the diagram are illustrative only. We’ll have more to say on how to choose the right metrics for your organization in the future.
Hopefully this framework provides a more useful resource if you’re trying to establish or review your innovation measurement systems. But keep in mind there’s more to a measurement system than the metrics. You also need a way to measure at different levels of the organization (e.g., project, business unit, portfolio). You need to consider how to communicate your metrics (think: your reporting cadence, adding qualitative insight). Visit our Resources page for further guidance on these topics, including our step-by-step guide to measuring innovation performance at a project level.
Download our Innovation Performance Measurement Framework
Reference:
National Academy of Science, (2005). Thinking Strategically: The Appropriate Use of Metrics for the Climate Change Science Program. The National Academies Press, Washington, D.C. (link)
