I recently had a conversation with a gaming industry CEO whom I deeply respect that reinforced a MIT Sloan Management Review article, “Embrace Your Ignorance” by Michael Schrage, about how the savviest leaders promote and embrace ignorance. The thesis for both Schrage and the CEO was that you cannot accurately predict what your customers will want, like or need. Thus, you need to embrace this ignorance and run experiments to get the data.
Moneyball and The Innovator’s Dilemma
I have seen many companies where the leadership “felt” they understood the customer and would develop new products for these customers. It leads to project green light meetings very similar to the draft room in Moneyball, where people argue based on their experience which initiatives have the most potential. It is also one of the biggest contributors to the huge number of failed projects, particularly in the gaming space where we typically see more than 8 out of 10 new games fail.
This issue is actually often a bigger problem with executives who have had past successes. Even if they knew their existing or past customers very well, they do not necessarily know what a broader or new market wants. Even their existing data can skew innovation effort, which is the core point of the Innovator’s Dilemma: Companies that have been leap-frogged often create innovations for existing markets rather than new markets.
You already are ignorant—accept it
In Schrage’s article, he discusses how Microsoft’s Ronny Kohavi (a pioneer in online experimentation) challenges tech-savvy audiences when he speaks. Kohavi shows screenshots of actual A/B tests that Microsoft has run for website design. He then asks his audience to predict the outcome of the tests. Although the audience is sophisticated, they almost always fragment with different opinions. Kohavi then advises, “stop debating…it’s easier to get data.” Continue reading
I love writing about applying lessons from sports to the tech and game spaces, so an article I saw in the MIT Sloan Management Review, “What Businesses Can Learn from Sports Analytics” by Thomas Davenport, really resonated with me. Davenport is one of the people who have molded strongly my love of analytics, as his book Competing on Analytics initially got me thinking how the game industry could be improved by applying analytics. That the Oakland A’s are again one of the surprise success stories in baseball further reinforced the relationship of analytics and sports. In Davenport’s recent article, he shows how we can apply techniques used successfully in sports to tech or game companies.
Analytics is all the rage in sports. Davenport points out that every professional baseball team has an analyst on staff, while many professional football, soccer and basketball teams also do. Yet, other sports teams are behind many other industries because they are often smaller organizations and typically have old-school executives who do not appreciate the value of analytics. Although not applied universally, Davenport draws several lessons from how sports teams use analytics that are relevant to all businesses. Continue reading
I have been intrigued for years that a huge financial sector has continued to rely on intuition while industry after industry has discovered that using analytics give you a better chance to succeed. Moreover, it is a sector that brags about the fact that it fails 99 percent of the time yet fails to embrace methods to improve those odds. I am talking about venture investing, the venture capital industry.
Moneyball and the venture community
For those who have seen the movie or read the book Moneyball, which I have written about multiple times, one of the most poignant scenes is the Oakland A’s smoke filled draft room where scouts with years of experience determine the best prospects to select based on their gut of what makes a great baseball player. When I first read about it, the parallels to how game company executives select what games to green light were incredibly apparent and I was certain you would see a similar transformation of the game industry. We did, with analytics driven social game companies putting many old school game companies out of business.
The venture capital space has uncanny parallels to the pre-Moneyball baseball industry. You have investors with years of experience sitting in Red Bull filled rooms deciding which investments to pursue based on intuition. The claim that they are investing in the management team is another way of saying they are selecting those leaders who feel like rock stars; who they think look like a star. They are basing it on measurable that they feel are important but have not proven empirically are the keys to success (just as baseball executives undervalued walks and over-valued defense).
Correlation Ventures, the Billy Beane of VC
A recent article in Forbes, “Venture By Numbers,” shows this situation is changing. Correlation Ventures started in 2011 with the philosophy to bring a quant-based approach to venture investing. Their mission was to stockpile 25 years of data on every venture deal consummated, evaluate this data with proprietary algorithms and then pick investments via pattern-matching software. Continue reading
Most of my posts about the reasons and methodology for creating accurate customer lifetime value (LTV) predictions have focused on the numbers and metrics, but a key element to predicting accurately LTV is observational (qualitative) data. It all comes down to more data is better, so predictions with qualitative data are going to be more accurate than those that rely solely on quantitative data. A mistake that is commonly made in the analytics world, and particularly in gaming, is to disregard anything that is not a quantitative value.
Some examples of incorporating effectively qualitative data
The example that had the most impact on me is that Billy Beane and the Oakland A’s, the subject of Moneyball (and multiple blog posts by me), has one of the highest scouting budgets in baseball. Scouts provide data on variables, like mental make-up and desire to win, that are not evident in the historical metrics. So although Beane makes personnel decisions based on metrics, he has also invested large sums in getting qualitative data (scouts watch players and prospects and then report on how they perceive the player’s skills). This approach has proven successful, as Beane’s A’s again surprised people by winning their division last year. What Beane has mastered is finding a way to incorporate the scouting reports with the available quantitative metrics. Continue reading
I have written several times about Moneyball and many times about customer lifetime value (LTV), so I wanted to bring the two together. Moneyball was the Michael Lewis book turned into a successful film about Billy Beane and how he made the Oakland A’s competitive by relying on analytics over intuition (for more detail, please see Lessons from Moneyball for the Social Game Industry and Moneyball Strikes Again). The same principles that help the Oakland A’s compete effectively could help social game companies compete, even against better financed firms. The same phenomenon holds with LTV, in which many of the metrics people focus on do not have maximum impact on long-term success.
Runs = LTV
LTV serves the same role in your business as runs do in baseball. Beane and his analysts realized that the success of a baseball team comes down to scoring more runs than your opponent. They thus reverse-engineered the game and its players into what contributed to scoring runs and what contributed to preventing runs. They then used their resources that maximized the delta between runs scored by the A’s and runs that they allowed. Continue reading
Everyone probably already knows the clear winner of last week’s US election, it was a resounding victory for analytics over “intuition” and “expertise.” New York Time blogger Nate Silver, who uses statistical models to analyze polling and economic data, correctly projected which candidate would win each of the fifty states (and District of Columbia). Conversely, not one expert (often referred to as pundit) came close to predictably as accurately, the election. Moreover, many missed by a huge margin while mocking Silver before the election. This is the second consecutive Presidential election where Silver was uncannily accurate (he predicted 49 states correctly in 2008), showing he was not just lucky. As you may have noticed, I have long been a fan of Silver’s and incorporated his RSS feed into this blog over a year ago.
Silver and Moneyball
What happened in the political arena mirrors the lessons from Michael Lewis’ Moneyball that I have written about several times (my original Moneyball post and my follow-on when the Oakland A’s made the playoffs). Continue reading
Monday night the Oakland A’s gained a playoff berth with a roster that most experts believed at the beginning of the season would not even approach playing .500 baseball. Although most experts in hindsight now see the quality of the A’s players, they are failing to realize their success is again the result of Billy Beane’s (Oakland’s General Manager) ability to use analytics to gain a competitive advantage.
I have hesitated in publishing a “reading list” because often when I see them on other blogs, they are little more than the author’s effort to get some referral income. As many of you know, I do not monetize this blog at all (there is no advertising and I have refused all sponsorship offers) and the links in this post are not tied to any monetization. With that in mind, I wanted to share some books that have made me much more successful and I think will help anyone in the gaming ecosystem (and probably any other business). Given that we all have very limited time, even to read, I have listed the books by how much of an impact they have had for me. Continue reading
Last week I posted about how the book and movie Moneyball provided great lessons for the social game industry (Lessons from Moneyball for the social game industry post). I just came across a post on the Business 2 Community blog (Game Change: Moneyball and the Reality of Social Business)that did a great job of showing how the lessons from Moneyball and the rise of Billy Beane is so similar to what happens in social businesses (including social gaming). It really makes some great points about hiring, arguably your most important task. I definitely recommend you take a look at the post.