General Social Games Business – The Business of Social Games and Casino

An often-better alternative to AB testing?

While AB testing is an integral element of mobile and social game development (as well development of most digital products), in many situations there is a better option. Several years ago, I had the opportunity to serve as an advisor to a company that had some brilliant people. Their CTO was a strong advocate of using multi-armed bandit testing as a superior alternative to AB testing. Multi-armed bandit testing is not new, there was a popular post in 2012 (http://stevehanov.ca/blog/index.php?id=132), and it is used by Google and other tech giants, but people (especially product managers) still regularly default to traditional ABn testing.

The problem with AB testing is that you leave money and performance on the table. Until the test is over, the poorer performing variant(s) will always get a significant share of your traffic. With the multi-armed bandit approach, you allocate increasingly less traffic to poorly performing variants.

What is multi-armed bandit testing

A multi-armed bandit approach allows you to dynamically allocate traffic to variations that are performing well while allocating less and less traffic to underperforming variations. Instead of two distinct periods of pure exploration and pure exploitation, bandit tests are adaptive, and simultaneously include exploration and exploitation. As Optimizely wrote recently, ” multi-armed bandit optimizations aim to maximize performance of your primary metric across all your variations. They do this by dynamically re-allocating traffic to whichever variation is currently performing best. This will help you extract as much value as possible from the leading variation during the experiment lifecycle, so you avoid the opportunity cost of showing sub-optimal experiences.”

Multi-armed bandit testing is a Bayesian approach to AB testing. As Shawn Lu writes in a post titled Beyond A/B testing, “The foundation of the multi-armed bandit experiment is Bayesian updating. Each treatment (called “arm”) has a probability of success, which is modeled as a Bernoulli process. The probability of success is unknown, and is modeled by a Beta distribution. As the experiment continues, each arm receives user traffic, and the Beta distribution is updated accordingly.”

A recap on ABn testing

To compare bandit testing with ABn testing (AB is with two variants, a test and control, n allows for additional variables), let’s quickly recap how AB testing works. Alex Atkins summarizes it succinctly, writing “in statistical terms, a/b testing consists of a short period of pure exploration, where you’re randomly assigning equal numbers of users to Version A and Version B. It then jumps into a long period of pure exploitation, where you send 100% of your users to the more successful version of your site.”

Benefits of multi-armed bandit testing

Bandit algorithms try to minimize opportunity costs and regret (the difference between your actual return and the return you would have collected had you deployed the optimal options at every opportunity). Rather than letting an AB test run until it is statistically significant, a bandit test moves subjects into the best performing group faster, allowing you to capture more gains. Matt Gershoff writes, ““Some like to call it earning while learning. You need to both learn in order to figure out what works and what doesn’t, but to earn; you take advantage of what you have learned. This is what I really like about the Bandit way of looking at the problem, it highlights that collecting data has a real cost, in terms of opportunities lost.”

A related advantage of multi-armed bandit testing is you make fewer mistakes. An A/B test will always send a significant portion of traffic to the sub-optimal group.

Also, as Shawn Lu writes, “[an] advantage of bandit experiment is that it terminates earlier than A/B test because it requires much smaller sample. In a two-armed experiment with click-through rate 4% and 5%, traditional A/B testing requires 11,165 in each treatment group at 95% significance level. With 100 users a day, the experiment will take 223 days. In the bandit experiment, however, simulation ended after 31 days, at the above termination criterion.” if the treatment group is clearly superior, we still have to spend lots of traffic on the control group, in order to obtain statistical significance.”

Finally, while not mathematically an advantage, bandit testing relieves the pressure to end a test too early. With ABn testing, frequently you will see one option perform better “directionally” and decide, or be forced to decide, to terminate the test and move everyone to the higher performing bucket before you get significant results. Unfortunately, this sometimes leads to picking an option that would be reversed once there is more data.

Why multi-armed bandit is not always the correct approach

The value of bandit testing does not mean you should abandon completely ABn testing. In Lu’s post, he writes “the convenience of smaller sample size comes at a cost of a larger false positive rate.” That is, you end up sometimes gravitating to the sub-optimal solution.

Alex Atkins also writes, “in essence, there shouldn’t be an ‘a/b testing vs. bandit testing, which is better?’ debate, because it’s comparing apples to oranges. These two methodologies serve two different needs.”
A/B testing is a better option when the company has large enough user base, when it’s important to control for type I error (false positives), and when there are few enough variants that we can test each one of them against the control group one at a time.”

The Bandit Option

While multi-armed bandit testing is not always a better option than ABn testing, you should look closely at using bandit testing when possible. It can reduce the opportunity cost of your testing and relieve pressure to terminate tests prematurely.

Key takeaways

While AB testing is the most common method of optimizing between alternatives, in many situations the multi-armed bandit approach is optimal.
A multi-armed bandit approach allows you to dynamically allocate traffic to variations that are performing well while allocating less and less traffic to underperforming variations.
Multi-armed bandit testing reduces regret (the loss pursing multiple options rather than the best option), is faster and lowers the risk of pressure to end the test prematurely.

Confirmation of Confimation Bias

Although I have written many times about different kinds of biases, the one I find most common, even in my decision making, is his confirmation bias. An article I recently came across, Confirmation Bias: Why You Make Terrible Life Decision by Nir Eyal, confirmed to me what I suspected, confirmation bias is more pervasive than most people realize. Confirmation bias is where people pick out anecdotes or facts that support their belief, while neglecting conflicting evidence.

I see if often in the games industry, you want to add a new feature (say a chat system) and you point to three products with chat systems that are highly successful. You do not look at the ten products with chat systems that have failed. Maybe you are looking to build a new product and you want to license an expensive IP. You justify it by pointing to the revenue that Kim Kardashian’s game generating while not including in your calculation the 20 branded games that failed.

This problem is not limited to the game industry. You may believe that social democracy is the best path forward for a country and you use Sweden, Norway and the Netherlands to confirm your belief, while not noticing Cuba, Greece and Spain. Or you believe free markets are the answer and confirm this by looking at Singapore, Taiwan and the US while not noticing Sweden, Norway and the Netherlands. Sometimes people knowingly pick the cases that prove their point, however, confirmation bias is when they sub-consciously accept the evidence that supports their beliefs.

Data is not the panacea

Using data rather than emotions appears as a vaccine against confirmation bias but data can contribute to the problem. Analysts suffer from the same biases as other people and will often look at the data that confirms their hypothesis.

You may have launched a new product that has a strong LTV and is growing rapidly, surpassing your existing product. One analyst who belongs to the team that launched the new product might look at the retention and monetization metrics and compare them with the original product. These KPIs are higher in the new product so the analyst recommends marketing funds shift from the older product to the new product. An analyst on the team for the existing product might identify that shows that total revenue (new plus old product) is flat since the launch of the new product, despite additional marketing spend on the new product. They might argue that the new product is simply cannibalizing the existing product. The better KPIs of the new product are a result of the best users moving to the flashy new toy, not a fundamentally better product.

Depending on the analyst’s initial partiality, they will either investigate and then present the first or second data set. Neither data is incorrect but the conclusion and actions they lead to are very different. Even though both analysts are being honest and believe they are objective, confirmation bias is driving their analysis.

Confirmation bias can impact career decisions

Confirmation bias is not only prevalent in deciding what decisions to make in business but also how to manage your career. You might feel your company is not treating you fairly. Then when two colleagues get large bonuses and you do not, it confirms that you are being treated unfairly. The data you may not be considering is that you are on a higher compensation level already or had received a bonus six months ago.

You also may be considering moving to a competitor. You have met people at a trade show from the other company and they mentioned some of the great perks. You interview and are then offered a position. Before accepting, you see some negative reviews on Glassdoor. You have already decided that you want the new job so you convince yourself the reviews have to be from a different business unit or boss. When you get to the new job, you learn the problems are real.

How do you fight confirmation bias

Given the prevalence of confirmation bias, it is important to create a strategy to combat it. A recent article, Facts Don’t Change People’s Minds, Here is What Does by Ozan Varol, provides some great suggestions:

Do not feel your beliefs create your identity. Do not get defensive when someone questions you or your project. The data is not personal and you are not a better or worse person if your hypothesis is wrong.
Develop better empathy. If someone disagrees with you, it is because they think they are correct. Understand why they are disagreeing and be open to them being potentially correct.
Get out of your echo chamber. As Varol writes, “make a point to befriend people who disagree with you. Expose yourself to environments where your opinions can be challenged, as uncomfortable and awkward as that might be.” Seek out data that disproves your position.

Additionally, I have found pre-mortems a very useful tool to combat confirmation bias. A pre-mortem is a meeting held before a major decision where all those involved in making the decision imagine themselves six or twelve months after the decision was taken, assume it turned into a debacle, and then explore why it was a disaster. This type of meeting forces you to look at contradictory facts and raise potential problems.

It is important to be cognizant of confirmation bias and seek out all the information before making important decisions. Most importantly, look at contradictory information and do not discount it because it does not support your position.

Key takeaways

Confirmation bias is where people pick out examples or facts that support their belief, while neglecting conflicting evidence.
Being data driven does not avoid confirmation bias, as people gravitate to the data that supports their beliefs.
To combat confirmation bias, do not equate your beliefs with your identity, understand why others have different positions, seek out non-confirming data and do a pre-mortem for important decisions.

When to use qualitative research

In the mobile gaming space, being data driven is largely synonymous with using quantitative data, with qualitative data (such as user research) relegated to third class status. Most game companies believe their decisions should be based on user activity in their product. They often act and allocate resources only to analyze their existing data, with design thinking and user research considered old fashioned and not as accurate. These companies, however, are missing an opportunity.

As I have said before, all data should be used to make decisions and optimize your product. Gameplay data is extremely valuable but so are qualitative data sources. Young children learn quite early that when you add a positive number to a number, the new number is larger. Thus, there is no way adding qualitative data to your quantitative data can make your analysis weaker. As long as the qualitative research does not detract from the analysis, it makes you stronger.

There is a great article, Qual vs Quant: when to listen and when to measure by Laura Klein, that explains when qualitative research is most useful. The article explains that you should constantly use both qualitative and quantitative data, but different situations require different focus.

When to focus on qual research

Qualitative research is important to understand why you are experiencing a problem or situation. Quantitative research tells you that you have a problem and what the problem is (i.e. D1 retention is down 20%).

What method in which situation

Klein highlights three different high level scenarios and then discusses whether qualitative or quantitative research would help most. In the first scenario, you are only considering changing one variable. You may be deliberating whether to change the maximum bet on slots in your casino. In this situation, quantitative analysis is optimal. With a change this small, users in a testing session or discussion will not give you any actionable information. Qualitative feedback will not provide failure to compensate for the time and money it takes to set up interviews, talk to users, and analyze the data.

The second scenario is when you are planning a multi-variable or flow change. You may be planning to implement a new feature, like a progression system, that would among other things require you adjust the user interface and journey. While you could launch it and measure performance, you would never know why it failed or succeeded. Qualitative research allows you to learn before launching what elements of the feature are compelling, what is confusing and how you can optimize the feature before you launch the feature.

The third scenario is prioritizing your roadmap. In this situation, a combination of qualitative and quantitative research is optimal. As Klein writes

The key here is that you want to look at what your users are currently doing with your product and what they aren’t doing with it, and you should do that with both qualitative and quantitative data.

Qualitative Approaches:

Watch users with your product on a regular basis. See where they struggle, where they seem disappointed, or where they complain that they can’t do what they want. Those will all give you ideas for iterating on current features or adding new ones.

Talk to people who have stopped using your product. Find out what they thought they’d be getting when they started using it and why they stopped.

Watch new users with your product and ask them what they expected from the first 15 minutes using the product. If this doesn’t match what your product actually delivers, either fix the product or fix the first time user experience so that you’re fulfilling users’ expectations.

Quantitative Approaches:

Look at the features that are currently getting the most use by the highest value customers. Try to figure out if there’s a pattern there and then test other features that fit that pattern.

Try a “fake” test by adding a button or navigation element that represents the feature you’re thinking of adding, and then measure how many people actually click on it. Instead of implementing an entire system for making friends on your site, just add a button that allows people to Add a Friend, and then let them know that the feature isn’t quite ready yet while you tally up the percentage of people who are pressing the button.

What to do

There are some situations where you should rely on the quantative research, other situations where qualitative research is best, but generally a combination of the two is the optimal way to formulate your product strategy. By using both, you are collecting all available information and using this data to drive optimal decisions.

Key takeaways

Qualitative research (survey, user panels, etc.) is often neglected by game companies who prefer to use quantitative data (KPIs) to drive decisions, but this approach neglects that more information is always better.
In situations where you are only impacting one variable, quantitative data is the answer; if you are adjusting flow or multiple variable, lean on qualitative data; and if you are prioritizing your roadmap, use a robust combination.
There are some situations where you should rely on the quants, other situations where qualitative research, but generally a combination of the two is the best way to formulate your product strategy.

It isn’t what it is

One of the most insidious phrases I have heard almost everywhere I have worked is, “it is what it is.” The phrase portends inefficiencies, politics, despair, incompetence or a lack of caring (or a combination thereof). It is critical to recognize this phrase as a problem itself and also not accept the situation it is used in reference to.

When you are likely to hear “it is what it is”

The phrase “it is what it is” generally means that the situation is how the company operates and cannot be changed. There is nothing that can be done about it. It may be that there are certain processes in place that inhibit efficiency or the ability to drive new initiatives.

The phrase will come up when someone sees an opportunity to increase profitability, maybe through a new marketing channel or by changing a software provider, only to be told the company needs to continue doing things the existing way. You may ask why is our marketing budget focuses on MySpace, and the response is we know it probably is not optimal but “it is what it is.”

You also hear the phrase when somebody is in the wrong position in the organization, cannot do their job effectively and it inhibits other people’s performance or initiatives. In American Football, a fast receiver may ask his quarterback for more long-pass attempts as he has his defender beat. The quarterback responds he would love to but the offensive coordinator does not understand the passing game. The receiver asks him to talk to the coordinator about the opportunity but the quarterback responds, “it is what it is.”

Another situation where you hear this destructive phrase is in reference to a company’s systems and software. The company may be using Windows 98. When you talk to IT about getting a newer operating system you are told they only support Windows 98. When asked why, the response is we know there are better systems out there but “it is what is.” You go back to trying to create your marketing strategy using WordPerfect.

Why “It is what it is” is so bad

The most destructive impact of “it is what it is” is not the initiative that may get stalled or the sub-optimal allocation of resources but how it destroys a company’s culture. When someone repeatedly hears “it is what it is” in response to ideas on how to improve the company, they eventually stop proposing ideas. The company then stops innovating and optimizing, which in the hyper-competitive era we live in, is the beginning of a death spiral.

”It is what is” also erodes employee morale and increases churn of your good employees. Most people work not only to make money but also because they want to achieve (most studies find that making money is not one of the key motivators for people). When the culture of it is what it is becomes prevalent, the high achievers become the most dissatisfied. There is very little worse than not pursuing an initiative they know would be good for the company not for a good reason (cash flow, strategic focus elsewhere, etc.) but because “it is what it is.” These employees are likely to find an opportunity where they can strive to be best and grow their company. This also creates a death spiral where the company is left only with people who are content to maintain the status quo.

The Kobayashi Maru

I learned a related lesson in my childhood while watching Star Trek (a show that largely shaped my adult life), a situation called the Kobayashi Maru. Not to be too geeky, but the Kobayashi Maru was a simulation given the officers at Star Fleet Academy(the training academy for people who would fly spaceships) where they had to decide between two awful scenarios, both of which resulted in a great loss of life. The point of the simulation was to show that sometimes there is not a good solution and no Star Fleet cadet ever passed.

One cadet, James Kirk (who even non-geeks probably heard of as Captain Kirk), though, did pass. What Kirk did was go in one night and reprogram the simulation, which was against Star Fleet rules. Even though it was against the rules, because he ended up succeeding where everyone else had failed, his victory was accepted.

The lesson here is that even in a no-win situation, finding a way to succeed by changing the rules. Kirk succeeded because he did not accept that “the Kobayashi Maru” is what it is. He took the initiative to make it something else and make it beatable.

Do not accept doing things sub-optimally

Start by never, ever, saying “it is what it is.” If you catch yourself about to say it, think about the situation and how you can change it. Never accept the status quo unless it really is the base path forward. If it is not, take the initiative to change the way things are done. If there is an incompetent person blocking your path, find another path. Find an ally who will work with you to drive an initiative over the blocker. If there is a better process or software solution, take it on your self to build the business case. It is easy for someone to block you by saying “it is what it is,” it is much harder from them to block you when you have a well reasoned business case in how the change will make the company an extra $500,000.

If you are in a leadership position, train your team not to accept (or say) “it is what it is.” If they are pushing an initiative forward and run into resistance, empower them to find a solution or change the rules. Only in extreme cases should they escalate the situation. It is more important that you foster a culture that your team members find a way around “it is what it is” rather than you doing it in each case (which is almost as bad). Tell your team you do not want to hear what we can’t do or are not allowed to do but want to hear how we are doing it.

When you run into “it is what it is,” rather than accepting it let everyone know the cost of not changing. If you hear it from a colleague or someone on your team, either encourage them to find a way to change things or take it upon yourself. Just as accepting “it is what it is” can put a company in a death spiral, showing it actually is not the case leads to a healthy, growing organization.

Key takeaways

”It is what it is” represents one of the most insidious phrases in the business environment, effectively telling people the status quo cannot be changed.
This phrase stops initiatives that would improve profitability, inhibits new ideas and drives away your best employees.
To combat “it is what it is,” do not accept it and build a business case to over come it.

Lifetime Value Part 28: Why you are probably under allocating resources to Live Services

Last month, I wrote a post about how many game companies do not dedicate enough resources to retention. Another area that most game companies, other than the top mobile gaming companies, are also under allocate resources to is live services. The live services team, however, largely drives success in the gaming space. A strong live services team is a key part to having an LTV that justifies marketing (and thus growth).

During the MAU conference in April, there was an interesting session on why the top social casinos were successful. The speakers were from the leading social casino publishers, Playtika, Zynga and Play Studios. All of them credited live services as the key to their success, with one attributing 80 percent of their success to live services. Also, if you look at the recent surge in Zynga’s stock price, it is largely driven by the performance of recent acquisitions. Zynga’s ability to improve the live services at these studios is the key driver of this positive performance.

What are live services

The textbook description of live services are changes that are made to a game or app that do not require a new build (development work). Live services product managers are focused on optimizing retention or monetization KPIs, rather than the PMs or designers driving new content or features.

There are many types of live services initiatives, ranging from optimizations to full programs. Some that have strong impact include:

In-game events. In many mobile games, you will see daily or weekly (sometimes hourly) special events, such as a one-day race to the top. The event may include a special leaderboard for players who get the most combinations or kill the most of a certain type of enemy in a specified period. These events serve multiple purposes. They create a sense of excitement outside the core game loop. They provide a reason for players to replay a level or a machine. They drive more engagement as players compete to get higher on a leaderboard. They provide variation, a way of introducing new content without having to build more content. Events are one of the strongest drivers of both retention and monetization KPIs in the gaming space.
Economy optimization. A key to keeping a free-to-play game successful long-term is managing the economy, an incredibly complex task. Managing the economy of a social game (or real money poker) is very similar to what Jay Powell, Chair of the Federal Reserve or Mark Carney, Governor of the Bank of England, must manage. Powell and Carney must regularly adjust interest rates, quantitative easing (asset purchases), reserve requirements, borrowing levels, etc. to manage inflation, demand, unemployment and the asset (stock) markets. If Powell or Carney gets it wrong, inflation could get out of control, unemployment could rise or asset prices (stocks and real estate) could plummet.
It is the same in games. The Live Services team must ensure that the in-game economy stays in sync and is a good experience for all players while encouraging spend (keeping up the value of assets). They need to ensure prices for goods or bets in a casino are at a price level that gives players a fair return on their investment and keeps them engaged. They need to build a balance between the main and premium currencies. They need to ensure the spin speed of a slot creates a good experience without draining a player’s wallet. They need to guarantee that a new player has a good experience but elder players are still having fun. All of these initiatives are connected and a failure in one area could create the gaming equivalent of Venezuela.
Purchase package optimization. Related to optimizing the economy, the live services tem must ensure continuously players are getting appropriate value for their purchases. If packages are priced incorrectly, a customer might not get sufficient value when they make a purchase (for example, five minutes of gameplay rather than one hour) and thus become less likely to make future purchases. Conversely, they may get so much value that they never have a need to make additional purchases.
Challenges. One of the most engaging features in games are challenges. Challenges are usually offered on a daily or weekly basis, helping direct gameplay. They are useful for keeping players engaged, encouraging them to test new content or features or play more. The live services team should create effective challenges and structure them (rewards, timing, amount of effort required, etc.) to optimize the impact on KPIs.
Sales and promotions. Just as in the retail space, sales and promotions are a valuable driver of monetization. If not structured properly by the live services team, the sales could end up cannibalizing purchases or negatively impacting the economy. When done well, they encourage higher sustained revenue.
Subscriptions As I have written recently, subscriptions are a great opportunity for game companies. Managing the pricing and options available for the subscription model sits with the live services team.

This list is a subset of some of the projects that are driven by a good live services team. Strong live services product managers will proactively identify other areas of optimization that will improve retention and monetization.

While I previously said that the live services team drives improvements that do not require development work, that is often the case in theory only. Many games are not architectured in a way that these changes can be made without development, especially older products. The need for development work should not be a line in the sand on whether these initiatives are pursued by the live services team, the benefits of launching events or optimizing the economy persist regardless of any need for development. Additionally, most successful live services teams also impact the product development efforts, for example ensuring the pricing of a new feature enhances the existing product.

Where’s the love

Just like it’s cousin retention marketing, many companies do not allocate sufficient resources (financial and people) to live services. Most of the resources end up going to product development instead. This allocation occurs because the development side is on the face of it easier to measure and sexier. A new feature under development might have a projected impact of a 2% lift on revenue after six months of development. That 2 percent uplift would be incorporated in the financial projections and when the feature launches and the revenue accelerates, everyone is taken to dinner. The PM who designed the feature can then create a Powerpoint that they use to brag to their colleagues.

Live services is not as glamorous. Economy improvements might improve revenue 0.5% every month but over the course of the month, not overnight. The live services PM probably won’t be taken out to dinner for the 0.5% increase (especially as it does not happen overnight but gradually over the month) over a given month. Over the same six months that it took to develop the larger feature, however, the economy improvements generate over 3 percent uplift (assuming the improvement is compounded monthly), a 50 percent larger increase than the new feature. Additionally, it does not require the development resources (and costs) that the new feature absorbed.

The other factor inhibiting allocation of sufficient resources to live services is the cross over with marketing. Many of the live services activities described above fall under marketing at some game companies. As marketing is often focused on acquisition, it is not prioritized on the marketing team’s agenda. Even when it is, live services need to be integrated with the core game experience to succeed. The expertise to design and optimize live services is usually more consistent with Product Managers than Marketing Managers.

Live services is critical to managing your LTV

I have written many times about lifetime value (LTV), and as I have said before it is the lifeblood of any app or game. Products are successful when their LTV is greater than the cost of acquiring a new user (CPI), only when this happens can a company afford to market. Without marketing, products eventually wither and die. Most companies, particularly in the social gaming space, are fighting a perpetual battle to find acquisition channels where LTV is great than CPI. Live services drives continuous improvement of LTV, thus allowing products that would otherwise not justify acquisition thrive. It is often the big difference between the game companies that can maintain their franchises (see Words with Friends, Slotomania, Clash of Clans, etc) to the ones that rise and then burn out.

Key takeaways

Live services is the key to the success of the largest mobile game companies (Supercell, Zynga, etc.) but it is an area often over under resourced at other companies.
Live services projects include in-game events, economy and purchase package optimization, sales, challenges and subscriptions.
Live services often get fewer resources than product development because a new feature that takes six months to develop and generates a two percent revenue uplift is sexier and easier to visualize (and put in a P&L) than improvements that add 0.5 percent a month, though the latter leads to a 50 percent bigger impact.

Why you are probably under allocating resources to retention

One issue I see repeatedly with game and gaming companies, and many other industries, is under-indexing their investment in retention versus acquisition and reactivation. While there is no evil plot to neglect retention, it is the most challenging part of the life cycle to determine ROI on investments.

With acquisition marketing, most successful companies have visibility into the CPI (or CPA depending on how you look at your UA efforts) they are paying and the LTV for those players. By comparing CPI and LTV, there is a clear ROI. Companies can then allocate resources to acquisition marketing as long as the ROI is above their needed return on capital. Also, as you are probably spending thousands or even millions of dollars per month on acquisition, optimizing that spend justifies additional resources (analytics, tools, etc.) as you want to get the most out of your spend.

Reactivation is similarly easy to quantify. You look at the cost of getting the player back and then how much they are likely to spend once they are back. Not only is it easy to calculate the ROI on reactivation spend, you can justify the investment because the revenue will be higher than the investment. No executive can argue with spending $1 to get $2. The only caveat with reactivation spend is ensuring you are not spending to bring back customers who would return regardless.

While retention is your most important KPI, it often does not get the same level of love because the ROI is much less obvious. You are not adding a user and revenue. You are not bringing back a lost customer. Retention, however, warrants more investment than every other part of the business combined.

Retention spend avoids spending $1 for $0.40

Every dollar you spend to retain existing players impacts your recurring player base. With acquisition marketing, even the best apps and games are lucky to see a 40 percent D1 rate (40 percent of customers who download the app return the next day). Thus, for every dollar you spend on a new player, only $0.40 or less is contributing to your product’s performance the next day. Given that a normal (though still good) D7 retention rate is 10 percent (1 out of 10 newly acquired players play on the seven days after downloading your game), you effectively only have $0.10 left from that $1 of spend still working for you.

Conversely, virtually every dollar spent on existing players is contributing to your product for days or months. A contest that encourages people to play more touches all your players and could impact their behavior over their lifetime. You are not losing 60 percent or 90 percent even before you can do a thorough analysis.

Retention spend can make the cost of acquisition less painful

The first subject everyone talks about at conferences (and even importantly at the bar during conferences) is how expensive it has become to acquire customers. Yet despite the money and resources spent on acquiring the player, the same effort is not made keeping and delighting the player.

Many marketing teams feel their job is done once the player is in the game (or has made a purchase) and thus their efforts are focused on acquisition. The true value to the company, though, is turning those new customers into a loyal player. While it is not solely marketing’s responsibility, it is a critical part of the equation. If marketing acquires a player for $5, they then spend $7.50 and churn, it may look like (and is) a marketing success as they have achieved a 150% ROAS (return on ad spend). However, if they could use retention marketing to have that player make $7.50 purchases every month for a year, the impact is 12X better than the acquisition spend. Moreover, the cost of getting that player to become a repeat customer is almost certainly not 12 times the acquisition cost, but a fraction of that cost. Thus the ROI is significantly higher.

Retention is not just about the product

At this point, many of my marketing friends are probably saying (or thinking) that retention is the responsibility of the product team, they have already done their job. Product and marketing are no longer two distinct functions. Companies like Facebook and Uber have blitzscaled because their products were also their acquisition channels (hence the term growth hacking), thus these products were the marketing. Conversely, product can only touch players when they are in the product (though they can build systems to bring customers back). Marketing, however, can impact players any time. The player may not have been in the game a week but you can still impact them by email, text, Facebook, Snapchat, television, blimp, etc. Retention is largely about triggers, reminding customers to use a product again, and marketing must work with product to ensure customers stay engaged.

A bird in the hand…

Optimal investment is not only about maximizing return but also managing risk. There is much less risk dealing with a known entity (an existing player) than an unknown. If you do take more risk, you should only do it if you will have higher return. A risk-based approach to allocating your resources also suggests that retention is a neglected investment vehicle. With new user acquisition, the new player is a question mark. They may fit your existing LTV curve or they might over/under perform. There is little information to base your estimate (largely the performance of other players acquired from the same or similar channel).

With an existing player, it is easier to estimate the impact of additional retention marketing. You have data on what they play, how often they play, what incentives impact behavior, etc. Based on this data, you can estimate accurately the return one additional marketing dollar will bring. These estimates will track much closer to actual results than new acquisition, especially with new channels, thus reducing the volatility of your return on marketing spend.

Retailers get it

While game companies, and many tech companies, disproportionally emphasize acquisition, retailers have learned over hundreds of years that their marketing budget is better optimized for retention. Most of the advertising and promotions from retailers are focused on bringing back existing customers, not getting them into the store for the first time. JC Penney or Target or Curry’s are not focusing their marketing on getting new customers, they are focused on driving behavior from existing customers. Sales are designed to optimize repeat purchasers and getting existing customers to spend more, very few are built to bring in new customers.

Amazon Prime is arguably the biggest factor making Amazon one of the most valuable companies in the world. Prime, however, does not get many people to try Amazon. Instead it converts existing customers into ones who spend more and are more loyal. The dynamics of retail are not that different than gaming, it is just that they have learned that there is a higher return in devoting resources to existing customers.

What to do

The value of retention marketing does not diminish the importance of acquisition or reactivation but you should design your structure so it is not neglected.

Build your organization so that retention marketing is on an equal level. The head of retention should not report to acquisition. You should not have an EVP or SVP leading acquisition and a Manager leading retention. On the org chart, they need to be at comparable levels.
Ensure you have KPIs in place measuring retention, what gets measured gets done. Virtually any company is monitoring daily its acquisition spend, CPI (or CPA) and ROAS. You need to be both measuring and reviewing regularly the success and growth that your retention team is driving.
Rebalance your resources to ensure that you are optimizing your retention programs. Not only should you be running programs, but you need sufficient resources to build the campaigns, create the marketing collateral and analytic resources to review and optimize. Do not skimp on these resources for the quick thrill of acquisition or reactivation.
As well as physical resources, you need to deploy sufficient budget to retention for long-term success. Start with a basic smell test. If you are spending $1 million a month on acquisition and your retention budget is $25k, probably something wrong. Try to align all your marketing spend with the understanding that retention drives significant value at lower risk.

If you do not focus on retaining players, any success will be short lived. To become great, your entire company needs to focus on keeping your customers.

Key takeaways

While the primary focus of most game companies is user acquisition, retention marketing is often neglected. Retention marketing, however, is more important to a company’s prolonged success.
For every dollar spent on acquisition marketing, at least $0.60 is lost the next day only 40 percent (at best) of players come back. Conversely, all dollars spent on retention marketing impact customers over their lifetime in your game.
You should ensure your structure is built on optimizing retention as well as acquisition and allocate resources (both people and money) to reflect the immense opportunity with retention marketing.

Subscriptions: The new weapon in the game monetization arsenal

People in the game industry are continually asking about “a new business model” but they usually want new monetization techniques (ie. gatcha mechanic, piggy bank, etc.). Now, however, there is a real opportunity to disrupt the industry with a new model, subscriptions. I have been in the games industry since 1993 and in that time there have only been two new models, try-before-you-buy and free-to-play. Subscriptions may usher in the next era of gaming.

Try-before-you-buy was introduced in the early 2000s and perfected by Big Fish Games, who released via download a game every day that was free for the first hour and then the player would have the option of purchasing the full game. While the model did not have a huge impact on the traditional game companies (who were selling their product for a fixed cost in retail), it was blue ocean as it brought an entirely new demographic into gaming. For the first time, gaming was not dominated by teen age boys playing in their parents’ basements (or 30 year old boys playing in their parents’ basements) but saw an influx of female players, particularly older women.

Early in the 2010s the gaming industry experienced its greatest disruption. Free-to-play gaming gained traction in the US (and Europe) after dominating Asian markets. In this model, games were truly free and over 90 percent of the players would never spend a penny. The games, however, were built to get the most engaged players to spend to improve or speed up their gaming experience, and many of these players would spend tens or even hundreds of thousands of dollars in their favorite games. Social gaming companies, led by Zynga, gained millions of daily players, pulling them from other gaming or entertainment companies.

Free-to-play was truly disruptive. Household names like Atari, Acclaim and THQ (which had earlier reached over $1 billion in sales) went bankrupt. Zynga saw its valuation reach over $10 billion. Disney and Electronic Arts both spent hundreds of thousands of dollars to acquire companies in the space. The concepts behind free-to-play have grown to shape the video game space, even those old-school companies that still monetize with an upfront purchase use in-game monetization to drive their revenue growth.

Given the impact of free-to-play and the millionaires it, everyone has been looking for the next disruptive business model. Based on how other industries are evolving, subscriptions are likely to be the next disruptive model in the game industry,

How subscriptions are changing the world

While Asia provided a clue that free-to-play would disrupt Western video game markets, developments in other industries show the likelihood that subscriptions will emerge as a disruptive force. The largest retailer in the world (by market cap), Amazon, uses its Prime subscription service to lock in customers. Salesforce.com, the most important company in the enterprise software space, eschewed the high fixed fee model for a subscription model that left its established competitors in the dust. Adobe, the largest provider of graphics software, abandoned its old business model to move to a subscription model and is now valued at $135 billion. Netflix, the second most important entertainment company in the world (nobody is beating Disney for a while), gained its position with a subscription model. Even Disney is betting its future on subscriptions with Disney Plus.

Enabling this shift is a change in people’s attitude. Ten years ago, people would not pay for digital content and overall wanted to own things. People did not pay for music (remember Napster). People would buy a DVD or CD, even if they would only experience it once. The examples above (and the hundreds I left out) show that attitudes have shifted. Millions of people are willing to pay Spotify money every month without owning a song. According to the Reuters Institute, in the United States, the proportion of people ages eighteen to twenty-four paying for online news leaped from 4 percent in 2016 to 18 percent in 2017. Attitudes have clearly shifted.

Why subscriptions work

Subscriptions have succeeded because they better align customers with providers than other business models. Rather than the linear model of selling a product to a customer, the subscription model creates a dynamic where the company to please constantly its customers. As Tien Tzuo says in Subscribed, “companies that know what their customers want, and how they want it, will succeed over companies that spend a lot of time and effort creating a product they think is a good idea, then spend equal amounts of time and effort trying to persuade people to buy it.”

Further driving the success of the subscription model are the benefits it has for the provider. Subscriptions allow companies to start the month (or year) with a guaranteed base of business. Rather than having to estimate how many units you will sell, you look at your subscriber base and can accurately forecast your revenue. This stability allows companies to market aggressively, invest in new content, etc., as they can predict cash flow.

The subscription model also aligns companies with their customers. As Tzuo writes, “instead of thinking about reseller margins and unit sales, [companies are] thinking about subscriber bases and engagement rates.” Companies driven by a subscription model have direct ongoing relationships with their customers. They no longer have to segment customers, they now have individual subscribers. With the industry leaders (Amazon, Netflix, etc), every subscriber has their own home page, their own activity history, their own red flags, their own algorithmically derived suggestions, their own unique experiences. And thanks to subscriber IDs, all the boring transactional point-of-sale processes disappeared. As companies can never be too close to their customers, subscriptions create the loop that makes customer intimacy a reality.

Will subscriptions work in the game industry

Now that we agree that subscriptions are a great opportunity overall, will they work in the game industry. First, many game companies already are using this model. According to a great blog post by Google, they have seen global growth in game subscriptions of 70 percent year over year. Second, it is working. According to the post, game companies that have integrated subscriptions experience 20 percent higher retention. They also have seen higher overall monetization. Finally, subscriptions offset risk in developing and launching new content. According to Tzuo, “regardless of whether a show is successful or not, investing in sharp new content helps Netflix to both (a) attract new subscribers and (b) extend the lifetime of its current subscribers. Those shows don’t go away! Together, they’re increasing the overall value of the portfolio. They are instrumental in driving down customer acquisition costs (as more subscribers sign up) and increasing subscriber lifetime value (as more subscribers stick around for longer).”

How you should implement subscriptions in games

While subscriptions are an exciting opportunity, success with the model will come down to execution. Just as hundreds (or thousands) of game companies failed to implement successfully free-to-play, succeeding with subscriptions is more difficult than adding another package to your purchase page. There are several core concepts in building a product that leverages subscriptions.

Subscriptions need to be about access

The biggest challenge, and most common mistake, game companies face is what to provide for the subscription fee. The easy answer is virtual currency, after all it is what customers are willing to pay for with in-app purchases. The easy answer is wrong. As stated in the Google post, “it’s important to move away from the mindset that subscriptions are just an auto-renewal mechanism for discounted IAP. Instead, subscriptions need to be thought of as offering highly-retentive long-term access to content, rather than the one-time situational purchase of content offered by IAP.”

Successful subscriptions are about giving players access to content and special benefits, access that can be gained or lost. In a social casino, it could be access to new slots or unique table games. In a game like Archero, it could be access to special levels or powers. The benefits could also be exclusive tournaments, special avatars or unique in-game events. The key, though, is not limiting (or even relying) on giving players virtual or premium currency but access to a premium experience.

Keep it simple

One of the core principles in creating successful products is to focus on simplicity, which is often very complex to do, and subscriptions are one area where it is easy to fall into the complexity trap. Companies with very successful subscription offerings have very few options.

If you offer customers too many options, it is likely to overwhelm them and preclude them from choosing any of the options. This concept of cognitive load is critical to the success of many products, from games like slots to apps like Uber. Given that the human brain consumers 20 percent of the body’s energy but only is 2 percent of the body’s mass, it is important to understand that people will subconsciously work to reduce the amount of energy the brain is using.

Cognitive load is how much info people are processing at any one time. Cognitive load is tied to working memory, the more information in that short-term memory the higher the cognitive load. As cognitive load increases, consumers are less likely to make a purchasing decision.

With subscriptions, this is directly tied to the offerings. If a player has different options ranging from the term of the subscription, monthly costs, benefits levels, they are likely to choose none. For example, you might offer people a month-to-month, 3-month-, 6-month or 1-year plan, with pricing at $4.99, $9.99, $19.99 and $49.99, each with different benefits. Rather than the player finding the one that optimizes their utility (to use an economist term, or makes them happiest, to use a human term), they are more likely to shut off and just pass on the offerings.

Instead, offer them one or two (at most) options. It can be a regular subscription or a premium one (additional benefits) or a short-term plan and an annual plan. You do not see Netflix offering ten different types of subscriptions. The key is make it very easy for the player to understand the value and choose between the two plans and whether or not to subscribe.

Keep it honest

One of the reasons subscriptions took so long to be commonly accepted is that until recently they were part of a sleazy industry. Companies would trick customers into signing up for a subscription, then make it very difficult to cancel the subscription. They might let you sign up easily, then require you to call them to cancel at a call center open one hour a week every second week. Even then, the agent you spoke to would do everything humanly possible to keep you from cancelling, creating an awful experience. These practices soured people overall on signing up for subscriptions. With social media and sites like TrustPilot, word quickly gets out of deceptive subscription tactics.

Preventing customers from leaving or tricking them into subscribing is not only unethical, it is bad business. One of the fundamental values that subscriptions create for a business is the connection with the customer. It forces the company to ensure every month it is creating value for the customer and that is why the customer renews or maintains the subscription. Everyone on the product team looks at new content and features and judges whether it will help retain customers and bring in new subscribers. While scamming customers may bring short term gain, it is the customer connection that subscriptions create that leads to great companies like Amazon, Netflix, Spotify, etc.

The best companies use subscriptions to improve their underlying business. Tien Tzuo writes in Subscribed that “the smart [companies] realize that if they really want to retain their subscribers, they need to focus on building a great service, without relying on lame tricks like hiding the cancel button.…Make it easy for customers to leave if they want to. You can certainly ask them why they’re leaving, or try to win them back, but don’t get in their way—the digital equivalent of blocking the exit with a hulking security guard. When you build subscriptions into your game, let customer value drive the offering rather than tricks on keeping customers from cancelling.

Build a loop

A successful subscription plan should be tied to engagement in the underlying game. The more a customer plays the game, the higher the value of the subscription. According to the Google post, “in mobile games’ subscriptions design, some offer a booster or bonus points, to reinforce the action of ‘play.’ Some create a durable good, such as a permanent building or character, that levels up as a player remains a subscriber for a longer period of time. In these cases, the desired action is “continue to subscribe.” In other cases, subscribers get bonus premium items, currency or points to reinforce the action of in-app purchases.

Looking outside the game industry, airlines have done a good job of creating a loop around their frequent flier programs. With frequent flier programs, members improve their status by flying more or buying expensive tickets, such as business class. According to the Google post, “the ‘earn’ criteria here — flying or spending — is precisely the desired customer actions that the airlines want to reinforce.”

Evolve benefits

Another important element of a successful subscription program is that benefits evolve. According to Google, “as the players invest more in the game, whether it’s with their time, skills, or other IAP, the subscription benefit also compounds.” Thus, the player can unlock more sophisticated content or new challenges that would not have been relevant for them earlier in their experience.

Celebrate VIPs

VIPs are the core of virtually any social game’s success. Most free-to-play games generate 60-90 percent of their revenue from the top 1 or 2 percent of players. Many product managers have avoided subscription programs because of concern on how it would impact VIPs. If a player can subscribe to a VIP program for a fixed sum, the concern is that would put a cap on how much the VIP would spend in the game.

This concern leads back to the first point on subscription design, that is should be about access, not a replacement for existing purchases. Thus, the subscription plan might give the VIP access to slots they would love to play but not chips to play those slots.

When thinking about your VIPs, do not forget they are already VIPs. If someone is spending significantly in your game, do not try to take another $5 or $10 from them every month. Instead, turn the subscription into a celebration of their VIP status. Give them a free subscription, the goodwill will be worth much more than the short term revenue you would generate from forcing your VIP to purchase a subscription.

Use subscriptions to drive acquisition and convert players

In addition to driving monetization and engagement, subscriptions are a great way of increasing retention they are also a strong acquisition tool and powerful CRM element early in the product life cycle. First, an offer of a one or three month complimentary subscription can entice a potential customer not only to try your game but invest time to learn about your product.

Second, subscriptions can help convert players into customers of in-app purchases. They provide a way to let players see and test the spectrum of in-app offerings. According to Google, “Scopely’s game Wheel of Fortune frames its subscriptions offer as an all-access pass. These subscriptions feature exclusive rewards that a potential buyer would want in addition to a sales discount. Surfaced right after the first-time user experience (FTUE), with benefits such as ‘more energy, this subscription aims to increase these new buyers’ in-game engagement, and cultivate a habit of playing regularly and investing in their future gameplay.”

Third, subscriptions can increase virality, helping your existing users bring in new customers. Campaigns that let your subscribers give free months to their friends, and get free months themselves, are very effective at driving new user acquisition. For example, a promotion where a player can gift a new player three free months, and get a free month for every new player who signs up, helps you acquire players with the only cost being the lost subscription revenue of your advocate.

Making subscriptions a reality

Rather than being a follower, future successful game companies will push forward with subscriptions and help disrupt the industry, not react to the disruption. By focusing on execution and building a strong subscription offering, it is likely we will see the next Netflix or Spotify.

Key takeaways

Many industries are evolving from a discrete purchase model to a subscription model. From retail (Amazon) to music (Spotify) to entertainment (Netflix) to enterprise software (Salesforce.com), the subscription model is redefining winners and losers. The game industry will eventually succumb to the same forces.
To create a successful subscription program, the offering needs to center around providing customers with unique access and benefits, not replicating what they get when making in-app purchases.
Successful subscriptions also need to build an honest relationship with players, provide simple options, create a loop where subscribers enjoy more benefits by playing more, appeals to new potential customers and rewards your VIPs.

Are the glory days of social casino over? Probably!

While revenue in the social casino space continues to increase (a streak that has not been broken since the first days of Zynga Poker and Slotomania), dark clouds on the horizon have started to dampen the enthusiasm. Most social casino companies would not publicly disclose that they are expecting growth to slow (or reverse) but unspoken indicators are bearish.

Stagnant player growth

The greatest threat to the social casino industry is that the user base is not growing. Over the past couple of years industry revenue has continued to increase but active players has remained virtually stagnant. The revenue growth has been driven by certain companies (particularly Playtika) becoming increasingly adept at growing revenue per customer, especially among their VIPs. At some point, however, social casino operators will hit a ceiling as VIPs cannot and will not spend more.

Actions show that the top companies do not believe in the space

While no social casino operator has publicly warned about the challenges they are facing, their actions speak more loudly. Playtika, the largest social casino company, acquired Seriously last month, after acquiring Wooga last year.

Huuuge Games, the biggest success story in the social casino space in the last three years, launched a publishing arm. Critically, it is focused on hypercasual and traditional social games (such as Traffic Puzzle) rather than social casino. Given how involved Huuuge is with social casino, if it expected tremendous growth it would almost certainly be focusing its efforts to further increase market share in this space.

The public markets are also talking

While social casino operators are showing how they look at the industry through their actions, the public markets also show how investors view the opportunities in social casino. The first pure play social casino IPO, SciPlay (the social casino operations of Scientific Gaming), has seen its share price drop from $16 when it went public in May to $10.35, losing over 30 percent of its value.

From 2015 to 2017, virtually every Zynga’s earnings call highlighted its social casino division. Initially, it focused on the growth of its slots products (primarily Hit It Rich!), which was largely the only bright spot for the company. Not only did it tout the success of its slots products, but the big IP licenses it was signing for future products (such as Willy Wonka). The calls then incorporated Zynga’s success with Poker, which experienced a renaissance. Very noticeably, over the last year, Zynga has downplayed or even ignored the role of social casino in its growth projections. In part, this is due to the success it has experienced with recent acquisitions (and the struggles it has experienced recently in the social casino space), but it also highlights that investors are not very receptive to initiatives in the social casino space.

Again, actions speak louder than words. While investors are not perfect (The Big Short, anyone), they are focused on optimizing return and look across a broad spectrum to find the best opportunities. The lack of appetite for social casino shows that investors no longer think social casino is easy money.

The other looming risk

Another cloud dampening the prospects for social casino is real money gaming. The US is disproportionally important for the social casino industry, it derives a much higher percentage of total revenue (over two thirds) than other areas of the video game industry (which derives over 50 percent of revenue outside the US). While there are too many factors to determine causality, the US is the only major social casino market where real money online casino is largely illegal. Thus, many customers who would normally play in a real money environment can only get their online casino experience through social casino.

Eventually, as real money casino play becomes legal in more US states, it represents an existential risk to the social casino industry. This risk is not an immediate one, legislation has to be approved on a state-by-state basis and I do not expect a significant number of states to approve legislation until 2022-2023 at the earliest (sports betting is a very different phenomenon). Investors and companies, however, do look beyond the next few years to determine their best opportunities and real money gaming is an acute part of that equation.

What social casino companies should do

First, innovate. While it sounds trite, innovation is the key to revitalizing the sector. The industry (and many other parts of the game industry) has been driven by copying what is already working, trying to do it a little better and continued optimization. They are thus relying on fewer players to generate more revenue. Coin Master is a great example of how a company can generate hundreds of millions of dollars in the social casino space by breaking the mold and creating a casino product for new customers. Finding Blue Ocean opportunities using casino mechanics can reverse the dynamics threatening the industry.

Second, embrace Real Money gaming. While Real Money is a risk, it is also potentially salvation. Land based casino companies largely resisted social casino for years (one particularly reactionary one still does) only to find that customers who also play social casino have a higher lifetime value. MGM’s relationship with Play Studios (MyVegas) has driven millions of dollars of value to MGM, displayed by MGM’s increasing its support (actions speak louder than words).

While the relationship between social and real money online has not yet been proven, the size of the opportunity should generate more attention from social casino operators. Real Money online gaming, a $54+ billion industry, dwarfs social casino. If social casino companies can learn how to capture a portion of that revenue (and player base), it can exceed greatly the growth rates of the past.

Key takeaways

- Despite growing every year since the first social casino products launched, the industry faces significant risks. There are highlighted by recent growth driven by improved monetization of existing players rather than appealing to new customers.
- Further corroborating this problem is how leading social casino companies are looking outside the space for acquisitions while investors are showing little interest in social casino.
- To combat these trends, social casino companies need to look for Blue Ocean opportunities (use social casino mechanics to appeal to new customers) and leverage the roll out of real money gaming.
Share this:
Facebook
LinkedIn
Like this:
Like Loading...

December 2019
S	M	T	W	T	F	S
« Nov
1	2	3	4	5	6	7
8	9	10	11	12	13	14
15	16	17	18	19	20	21
22	23	24	25	26	27	28
29	30	31

What is multi-armed bandit testing

A recap on ABn testing

Benefits of multi-armed bandit testing

Why multi-armed bandit is not always the correct approach

The Bandit Option

Key takeaways

Share this:

Like this:

Share this:

Like this:

Data is not the panacea

Confirmation bias can impact career decisions

How do you fight confirmation bias

Key takeaways

Share this:

Like this:

When to focus on qual research

What method in which situation

What to do

Key takeaways

Share this:

Like this:

When you are likely to hear “it is what it is”

Why “It is what it is” is so bad

The Kobayashi Maru

Do not accept doing things sub-optimally

Key takeaways

Share this:

Like this:

What are live services

Where’s the love

Live services is critical to managing your LTV

Key takeaways

Share this:

Like this:

Share this:

Like this:

Retention spend avoids spending $1 for $0.40

Retention spend can make the cost of acquisition less painful

Retention is not just about the product

A bird in the hand…

Retailers get it

What to do

Key takeaways

Share this:

Like this:

How subscriptions are changing the world

Why subscriptions work

Will subscriptions work in the game industry

How you should implement subscriptions in games

Subscriptions need to be about access

Keep it simple

Keep it honest

Build a loop

Evolve benefits

Celebrate VIPs

Use subscriptions to drive acquisition and convert players

Making subscriptions a reality

Key takeaways

Share this:

Like this:

Stagnant player growth

Actions show that the top companies do not believe in the space

The public markets are also talking

The other looming risk

What social casino companies should do

Key takeaways

Share this:

Like this: