One of the most frequent mistakes in the game industry is making decisions based on a false negative. A false negative is a result of a test that shows as absent something that is actually present. One example of this is drug test that comes back negative but the person being tested is actually taking drugs and has taken masking agent to mislead the test. I have seen false negatives applied to virtually every element of the game industry, from green light to development to international markets.
False negatives in green light
Many games have not even been started because of false negatives. When determining the market potential for a game, one of the key criteria is the performance of similar titles. Frequently, though, people stretch to define “similar titles,” and this leads to projects being rejected. There are two potential problems that can lead to a false negative and thus rejecting a project that could have a positive return:
- The games being compared to are not really comparable. For example, your company is considering creating a business simulation. In reviewing the marketplace, your team compares it to time management games like Diner Dash and decides there is no market on mobile. However, your game would have been more of an economic simulator than a twitchy time management game. But by ignoring the knowledge that these genres have different potential users and focusing on Diner Dash‘s lack of mobile success, you don’t have a true validation of whether your economic sim would have worked.
- You compare your concept to a poorly done game. Again, assuming you are creating an economic simulator. You review other games on the platforms you’re considering, and the best-known economic sim—created a big game publisher—was a complete failure. The false negative would be rejecting your project solely for this reason, when the truth may be big pub’s game failed because it was poorly executed (e.g., Civilization on Facebook failed because of execution, not because people were not interested in builders). Your idea may have had great potential if developed as you intended.
False negatives in development
Another area where developers often draw false negatives is when looking at optimizations and features. With a free-to-play game, more important than the development process is how you manage the live game. This involves putting the right features and changes in after launch to grow the game and improve its metrics. Many times people draw false negatives by looking at features other studios have implemented and if the feature failed to improve metrics, they will not pursue the same feature. The problem is that even if the concept was sound, the feature or optimization was not implemented well. For example, let’s say a company created special Christmas items and they clearly did not monetize. You see this result and decide not to create holiday items. In reality, the Christmas items may not have sold because they were unattractive and had nothing to do with Christmas.
False negatives in international markets
Another area (one that particularly frustrating to me) in which companies draw false negatives is in decisions about pursuing international markets. For example, you have recently launched a game and you advertise in Turkey. The users you obtain do not monetize and retain poorly. Based on these results, you decide not to localize into Turkish. But the problem may have been that your test had poor results because many Turks do not speak English. As a result, they rejected the game because it was in a language they did not understand, not because it did not resonate with them. The localized results would have not correlation with the test of the English language game. Thus, you miss a market that could have been very receptive to your product.
Summing it up
These are three of many areas where you can make a wrong decision if you misinterpret the results of a test or “comparable” situation. If King had cancelled Candy Crush Saga because they believed Bejeweled Blitz did not monetize well, that false negative might have cost them a billion dollars. Be careful of the conclusions you draw.
The Business of Social Games and Casino 
2 thoughts on “Avoid the false negative”