I recently had a conversation with a gaming industry CEO whom I deeply respect that reinforced a MIT Sloan Management Review article, “Embrace Your Ignorance” by Michael Schrage, about how the savviest leaders promote and embrace ignorance. The thesis for both Schrage and the CEO was that you cannot accurately predict what your customers will want, like or need. Thus, you need to embrace this ignorance and run experiments to get the data.
Moneyball and The Innovator’s Dilemma
I have seen many companies where the leadership “felt” they understood the customer and would develop new products for these customers. It leads to project green light meetings very similar to the draft room in Moneyball, where people argue based on their experience which initiatives have the most potential. It is also one of the biggest contributors to the huge number of failed projects, particularly in the gaming space where we typically see more than 8 out of 10 new games fail.
This issue is actually often a bigger problem with executives who have had past successes. Even if they knew their existing or past customers very well, they do not necessarily know what a broader or new market wants. Even their existing data can skew innovation effort, which is the core point of the Innovator’s Dilemma: Companies that have been leap-frogged often create innovations for existing markets rather than new markets.
You already are ignorant—accept it
In Schrage’s article, he discusses how Microsoft’s Ronny Kohavi (a pioneer in online experimentation) challenges tech-savvy audiences when he speaks. Kohavi shows screenshots of actual A/B tests that Microsoft has run for website design. He then asks his audience to predict the outcome of the tests. Although the audience is sophisticated, they almost always fragment with different opinions. Kohavi then advises, “stop debating…it’s easier to get data.” Continue Reading…