I recently had a conversation with a gaming industry CEO whom I deeply respect that reinforced a MIT Sloan Management Review article, “Embrace Your Ignorance” by Michael Schrage, about how the savviest leaders promote and embrace ignorance. The thesis for both Schrage and the CEO was that you cannot accurately predict what your customers will want, like or need. Thus, you need to embrace this ignorance and run experiments to get the data.
Moneyball and The Innovator’s Dilemma
I have seen many companies where the leadership “felt” they understood the customer and would develop new products for these customers. It leads to project green light meetings very similar to the draft room in Moneyball, where people argue based on their experience which initiatives have the most potential. It is also one of the biggest contributors to the huge number of failed projects, particularly in the gaming space where we typically see more than 8 out of 10 new games fail.
This issue is actually often a bigger problem with executives who have had past successes. Even if they knew their existing or past customers very well, they do not necessarily know what a broader or new market wants. Even their existing data can skew innovation effort, which is the core point of the Innovator’s Dilemma: Companies that have been leap-frogged often create innovations for existing markets rather than new markets.
You already are ignorant—accept it
In Schrage’s article, he discusses how Microsoft’s Ronny Kohavi (a pioneer in online experimentation) challenges tech-savvy audiences when he speaks. Kohavi shows screenshots of actual A/B tests that Microsoft has run for website design. He then asks his audience to predict the outcome of the tests. Although the audience is sophisticated, they almost always fragment with different opinions. Kohavi then advises, “stop debating…it’s easier to get data.”
Companies may be confident they know their customers, but they’re likely to be overconfident. Most executives are not nearly as smart, perceptive or customer centric as they think. Schrage points to research by Daniel Kahneman, the leading academic in the behavioral economics space, that shows “experts” are particularly prone to predictive overconfidence. There is multiple research that shows we are bad at assessing the value of our ideas. In research by Microsoft’s Kohavi, intuition regularly underperformed a coin toss.
Test, don’t guess
Successful leaders have cultivated an unusual way of adapting to their limited powers of prediction. As Schrage writes, “they are humbled.”
Successful innovators have the courage of their curiosity. They do not run tests because they are confident in the results; they run tests because they know they cannot predict the results. They are less interested in validating their ideas than in challenging their assumptions. They want to learn from, and with, their customers (and potential customers).
With so much data now available, patterns and correlations can be used to identify areas for further exploration. Identifying these patters is a new kind of intuition and different from “expert” over-confidence: you are effectively choosing to act humbly.
Key takeaways
- You, and other business leaders, do not know your customer as well as you think.
- Thus, you are often less likely to identify new product or game opportunities than if you chose them randomly.
- Testing innovation ideas is the solution to the problem of relying on intuition but requires the humility to accept your intuition is not perfect.
The Business of Social Games and Casino 