Someone I respect recently posted an article from a news source that I also respect, but the article actually highlighted how data can mislead, either intentionally or not. An article on the Guardian.com, Amazon Prime Video’s growth outpaces Netflix in UK, tells the story of how Prime Video is growing at a faster rate than Netflix. The sub-title stresses that, “cross-promotion to Amazon shoppers and new on-demand series rank it top in 2017.”
The article goes on to point out several reasons why Amazon is top in 2017:
- New series of the Grand Tour and Transparent are fueling growth
- Hefty cross promotion of Prime Video to regular Amazon shoppers is also contributing
- Prime Video increased its subscribers to 4.3 million in 2017, representing 41% year-on-year growth
- Netflix only grew 25% in the same period.
If you stopped reading the article there, and who reads an article until the end these days, you would think Amazon is doing a great job in the video market and Netflix should be very worried. If you worked at Amazon and get a similar report from your analytics team, you might high-five the head of Amazon Prime in the UK. If you were at Netflix and got a similar report from your analytics team, you might panic a little and divert resources to the UK.
The problem is that although the data is accurate it is misleading. The key figure is that Netflix added 1.6 million new subscribers in 2017, while Amazon added 1.3 million new subscribers for Prime Video in the UK. Thus Netflix actually extended its lead over Amazon by 300,000 customers in 2017. Netflix is in 8.2 million UK households (at the end of 2017), versus 3 million for Amazon.
How different would the story have been if the headline was Netflix extends lead by another 300,000. How different would the reception be at Amazon and Netflix respective headquarters if their analytics team presented data in this way.
The mistake
The mistake in this case (and I will be generous and assume the Guardian was not click-baiting) is comparing growth rates (or any other rates) while neglecting the size of the relative base. It would be the same in football if you looked at Messi’s goal scoring versus a second year player. The latter may be scoring twice as many goals as he did as a rookie while Messi may have been flat or added a few. Thus the young player is growing his goal scoring 100% while Messi is adding only a few percent to his lifetime numbers. That does not mean that the second year player is either having as good a season as Messi or closing the gap.
The same happens in the mobile game world. Your slot game may be growing 100% month on month while Slotomania is growing 10% (not real numbers), but because their base is so high they are adding millions in revenue while you are still not profitable.
The key is only comparing trends when you are comparing apples to apples. Trends mean something if you are looking at two products or companies of comparable size in the same stage of their lifecycle. Looking at two auto companies who launched an SUV the same year in the same market makes sense, comparing growth rates of two automakers, one who is new and has no dealer network with one that has been around 100 years is worthless.
The answer
You need to look deeper into the numbers. Look at the absolute numbers. Look at the pricing. Look at the target market. Look at percent usage (in the Amazon case, how engaged are Prime users who may have bought it just to get free shipping versus Netflix users). The key to using data effectively is look deeply at the data and understand what is driving the results. You also need to make sure your analytics team does the same. It is very easy to make conclusions based on obvious trends. Avoid superficial analysis and, more importantly, superficial conclusions.
Key takeaways
- A recent article implied Amazon Prime Video was doing better than Netflix in the UK as it grew 41% versus 25% by Netflix.
- The article is misleading as Netflix actually added 300,000 more customers than Amazon. This obfuscation shows how data can mislead if you focus on trends but are not comparing comparable companies or products.
- The key to using data effectively is look deeply at the data and understand what is driving the results.
The Business of Social Games and Casino 