Understanding the innovator’s dilemma

Earlier this year, I wrote how using analytics to gain deep customer insight could inhibit innovation, based on Clayton Christensen’s Innovator’s Dilemma. I wanted to dive deeper into the Innovator’s Dilemma, as it is important to all tech companies, from those trying to grow to those trying to disrupt established industries. I will also tie everything together by showing how the free-to-play model disrupted the electronic game industry, destroying companies like Acclaim and Midway and creating billion-dollar companies like King.com and Kabam.

What is the innovator’s dilemma

At its core, the innovator’s dilemma is the apparent contradiction between knowing your customers intimately and optimizing your product for them, versus the fact that innovation will be driven by a different group of customers. Additionally, if you are a successful company the problem is magnified, as the new market may initially be much smaller than your current market and thus not warrant your attention. Finally, as Christensen writes, “blindly following the maxim that good managers should keep close to their customers can sometimes be a fatal mistake.”

Christensen cites many examples in the book, but one of the strongest is how the disc drive industry evolved. Some great companies, IBM, Control Data, Seagate, made all the right textbook moves by innovating based on what their customers wanted and then found themselves outflanked by other companies. In fact, the established 14-inch drive manufacturers were held captive by customers. Mainframe computer manufacturers did not need or want an 8-inch drive. They explicitly did not want it: They wanted drives with increased capacity at a lower cost per megabyte. So the drive manufacturers created products that their customers were demanding, missed the market for PC drives and then the manufacturers of PC drives went upstream and took their core market. This is a cycle that has been repeated in the drive industry buy also in multiple high-tech and low-tech industries.

We experienced it in the game industry. Companies like Playfish, Playdom and Crowdstar created games on Facebook and MySpace. Most “gamers,” people who bought $40 or $50 console games, were not interested in these products as the graphics and depth of gameplay were grossly inferior. But there were millions of new customers who were not looking for the high-end features and flocked to the new games on Facebook and MySpace. That allowed those start-ups to improve their offerings and then eat into the market of traditional game companies, to the point many went out of business and the others suffered a huge loss in profitability.

Innovation is not about knowing your customer but knowing the customer

One of the key elements of the innovator’s dilemma is that disruptive innovation is not created by building something for your existing users but by building something for a new set of customers. Disruptive products tend to be simpler, cheaper, and more reliable and convenient than established products. Attributes that make disruptive products worthless in existing markets often are the most powerful features in emerging markets.

Though I am not a huge fan of experts’ analysis regardless, it is particularly useless when planning a disruptive product. Realistically, it is impossible to predict how disruptive products will be used or how large the market will be (investors also need to keep this in mind). One thing I can say with certainty is that experts’ predictions and estimates will be wrong, and more likely matched by those of untrained monkeys. On a related note, as markets for disruptive technologies are unpredictable, companies’ initial strategies for entering these markets will also usually be incorrect.

Given that you cannot test or survey your customers about what features in a disruptive product would be valuable, since these are not the customers of the disruptive product, you need to find new ways to evaluate market potential. As Christensen writes, ”Markets that do not exist cannot be analyzed: Suppliers and customers must discover them together. Not only are the market applications for disruptive technologies unknown at the time of their development, they are unknowable. The strategies and plans that managers formulate for confronting disruptive technological change, therefore, should be plans for learning and discovery rather than plans for execution.”

Customer needs do not equal market needs

In addition, while your customers will want more features of the existing product, a disruptive product is likely to have fewer features but a different value and usability equation for a different audience. The power and influence of leading customers is a major reason why companies’ product development trajectories often overshoot the demands of mainstream markets.

Successful companies have a practiced capability in taking sustaining technologies to market, routinely giving their customers more and better versions of what they say they want. This is a valued capability for succeeding by sustaining innovation, but it will not facilitate disruptive technologies. If your company stretches or forces a disruptive technology to fit the needs of current, mainstream customers—as we saw when traditional game companies tried to compete in the social gaming space (see Civilization for Facebook)—it is almost sure to fail.

Effectively, when launching a disruptive product you cannot have a defined marketing strategy, but should launch the product and sell to whoever will purchase. The dominant application of your product will surface from this trial and error approach. By watching for small hints of where the product might be difficult or confusing to use, the developers direct their energies toward a progressively simpler, more convenient product that provides adequate, rather than superior, functionality.

Innovation is not about technology

One of the most interesting aspects of Christensen’s analysis of disruptive innovations is that they are technologically straightforward, consisting of off-the-shelf components put together in a product architecture that usually is simpler than prior approaches. They offered less of what customers in established markets wanted and so could rarely be initially released there.

Disruptive technologies do not necessarily involve new technologies but are built around proven technologies and put together in a novel way that offers the customer a set of attributes never before available. Again, if you look at how social games disrupted the game industry, they did not use any particularly innovative technology, but took existing tech and applied it in a new way.

In Christensen’s book, the data shows that established firms confronted with disruptive technology change did not have trouble developing the new technology; prototypes were often in place before the company had to make a decision. Disruptive projects, however, stalled when it came to allocating scarce resources between competing product and technology development proposals. Sustaining projects (improved technology that improved the experience for existing users) addressing the needs of the firms’ most powerful customers, almost always taking resources from disruptive technologies with small markets and poorly defined customer needs.

Disruptive companies did not create novel or cutting-edge technologies but instead disrupted by attacking from value networks below with cost structures set to achieve profitability at lower gross margins. The attackers therefore were able to price their products profitably, while the defending, established firms experienced a severe price war.

It is not crucial for managers pursuing growth and competitive advantage to be leaders in every element of their business. In sustaining technologies, in fact, evidence strongly suggests that companies, which focus on extending the performance of conventional technologies, and choose to be followers in adopting new ones, can remain strong and competitive. This is not the case with disruptive technologies, however. There are enormous returns and significant first-mover advantages associated with early entry into the emerging markets in which disruptive technologies are initially used.

Another key takeaway is that you should not have a blanket technology strategy to be always a leader or always a follower. Companies need to take distinctly different approaches whether they are addressing a disruptive or a sustaining technology. Disruptive innovations entail significant first-mover advantages: Leadership is important. Sustaining situations, however, very often do not.

Beware the small opportunity

I have worked with and spoken to large companies who would not consider any opportunity if the value was below a certain threshold (in both my situations, $1 million). So even if it would be a profitable endeavor, the expected return was not enough to impact the company’s numbers so they would not pursue it. This strategy is a huge impediment for an industry leader to creating a disruptive technology. Using the game industry, a company like Take 2, that makes hundreds of millions of dollars when it launches a version of Grand Theft Auto, might not be interested in a mobile opportunity that could only generate $500,000 in profits. This strategy, however, keeps them from entering markets until others have already build a defendable position. As Christensen writes, “one of the bittersweet rewards of success is, in fact, that as companies become large, they literally lose the capability to enter small emerging markets.”

Disruptive technologies, though they initially can only be used in small markets remote from the mainstream, are disruptive because they subsequently can become fully performance-competitive within the mainstream market against established products. The most powerful protection that small disruptive firms enjoy as they build the emerging markets for disruptive technologies is that they are doing something that it simply does not make sense for the established companies to attempt.

Your business structure can be part of the problem

Traditional successful business have very analytic ways at looking at market opportunities. In many instances, the information required to make large and decisive investments in the face of disruptive technology simply does not exist. They size the market, look at the investment required, compare the return on investment to other opportunities and put resources where they are likely to have the highest return. This strategy works when creating sustaining innovations for existing customers, where you know the customer and how they will react to the technology.

In disruptive innovations, where you know least about the market and there are such strong first-mover advantages, you cannot replicate the level of analysis of evaluating sustaining technology investments. This is another example of the innovator’s dilemma.

Companies whose investment processes require quantification of market sizes and financial returns before they can enter a market get paralyzed or make serious mistakes when faced with disruptive technologies. As discussed earlier, the market data is not available (or is wrong) so demands by finance for this data are counter-productive. Furthermore, judgments based upon financial projections when neither revenues or costs can, in fact, be known are meaningless and misleading.

Related, an organization’s structure and how its groups work together may have facilitated the design of its dominant product, the direction of causality may then reverse itself: The company’s structure and the way its groups learn to work together can then affect the way it can and cannot design new products.

To use the game industry analogy, a social game company may have built its organization around optimizing its distribution and virality on a platform like Facebook. Product decisions would be made by analysts who are adept at manipulating these channels. Then, when the company tries to create a disruptive technology, say mobile games, where these capabilities are no longer valuable, the fact that analysts are still making the product decision inhibits the company’s ability to be disruptive.

As the example above shows, the capabilities of most organizations are far more specialized and context-specific than most leaders believe. This specialization is because capabilities are forged within value networks. Thus, organizations have capabilities to take certain new technologies into certain markets. They have disabilities in taking technology to market in other ways.

Innovating and pivoting

The dominant difference between successful ventures and failed ones, generally, is not the astuteness of their original strategy. As I have said before, the ability to pivot is one of the most valuable skills a business founder can have.

Guessing the correct strategy from day one is not as important as conserving enough resources for a second or third stab at getting it right. If you run out of resources before you can iterate you will fail.

Christensen describes a technique for disruptive technologies: Agnostic marketing, which is marketing under an explicit assumption that no one knows whether, how or in what quantities a disruptive product can or will be used before they have experience using it. With agnostic marketing, you will try to penetrate the right market with the right product and the right strategy from day one, but there is a high probability that a better direction will emerge as the business heads toward its initial target. The important tactic is to pivot when you see where the traction is for your disruptive technology.

Key takeaways

• The innovator’s dilemma is the conundrum companies face between developing technology for their customers, for whom they have built their business around, or for an unknown market.
• Creating a disruptive technology is not contingent on a better technology but is usually a combination of existing technologies that are simpler or that provide a new value to a different market segment.
• You cannot anticipate customer needs for a disruptive technology. Expert opinions and a deep analysis of existing customers, however, will almost always be wrong.