The Business of Social Games and Casino

Customer experience best practices from the best

Earlier this year, I wrote about how to create a customer experience that improves retention, and thus profits. Given the importance of the customer experience on retention, I have since discovered additional best practices in delivering customer WOW. The New Gold Standard by Joseph Michelli shows how the Ritz-Carlton hotel group delivers fantastic customer service.

Start with attitude

To provide a customer experience that will generate loyalty and engagement and prevent churn you first need to have a message everyone in your company is aligned with. Great customer service begins with a clear and consistent organizational focus. For the Ritz-Carlton group, the phrase “the answer is yes; . . . now what is the question?” crystallizes the attitude throughout the company. This message conveys that if the service Ritz employees offer does not surpass the expectations of guests, then the splendor of the hotel becomes far less meaningful. Michelli quotes Ritz leadership as saying, “the amenity that matters most to our guests is not a fancy chocolate on the pillow but a dedication to service that never wavers.”

Once you have developed a message that conveys the experience you want your team to provide, the approach and behaviors will flow from it. Going back to the Ritz example, by starting with “the answer is yes” everyone knows that there is never any such thing as saying “That’s not my job.” When it comes to providing a service to a guest, the word no is not in the vocabulary of those who work at Ritz-Carlton.

This attitude then guides people not only in their interactions with customers but also with each other. Richelli writes, “this spirit of wanting to serve not only our guests but to lend an extra hand to fellow staffers is how the Ritz-Carlton culture of caring permeates all of our lives.” Thus, the service message becomes an operating principle throughout the organization.

Hire right and set them up to succeed

Once you have aligned your company with a message that conveys the customer service experience you want for your customers, you need to build the team to deliver it. When I wrote about creating a great customer experience, one of the keys I highlighted was the need to align hiring with creating a great experience. You can’t teach emotional intelligence, so it is critical to hire people who can radiate warmth, friendliness, happiness and kindness. Ritz-Carlton takes the same approach, believing that excellences starts with the right raw talent instead of attempting to manage employees to overcome talent deficits.

Having people who can deliver a great customer experience does not end with hiring the right people, though, you also have to dedicate the time and resources to train them properly. Training is important not only to provide your people with the tools to create a great experience but also to ensure consistency. You want the experience someone enjoys in Paris to be as good as the experience they would enjoy with you in Sydney. Even in the online world, you want the experience someone gets when talking to an agent about a purchase issue to be as good as when they are dealing with a VIP host about a sale. With the Ritz, all managers undergo three weeks of training. The first two days cover the typical Ritz-Carlton orientation; day 3 involves the expectations of leadership including how to treat their colleagues. On Day 21, after becoming certified in the operational standards of their positions, staff members are given a forum to discuss openly the positives and negatives they have encountered in their first three weeks. It is this training, coupled with openness and feedback, that ensures all customers enjoy a consistent Ritz experience.

Empower the team

Once you have the right people with the right training, you need to empower them to give a great experience. At the Ritz, they “empower through trust…Ritz-Carlton leadership sends a clear message that every staff member has the full authority to use his or her discretion to produce grand experiences for guests…. Every person, including a member of housekeeping and an employee working in the laundry, is empowered to use judgment, without seeking permission from a supervisor, to spend up to $2,000 on each guest each day!”

Creating a great experience depends on the employees, not the leadership. If a Ritz-Carlton employee sees a problem, they own it and are expected to fix it. The job of leadership is to empower people to create that experience. With Ritz, it’s not only the $2,000 budget but also the mandate that it is up to people on the front line to generate unique and memorable experiences. Leadership at the Ritz-Carlton believes “you just can’t micromanage unique and memorable outcomes.”

Part of empowering your people is giving them the freedom to modify their work to respond to customer needs, not just giving them a budget. If a guest has a problem or needs something special, Ritz-Carlton staff will should break away from their regular duties, address and resolve the issue. This action is only possible by not micro-managing the staff or basing their performance evaluation on hard metrics (i.e. tickets responded).

Empowering your team, and giving them the needed resources, not only leads to Wow experiences but is also cost effective long-term. An analysis of manufacturers helped leadership at Ritz-Carlton appreciate that the longer defects went undetected, the more expensive the defects were to repair. Additionally, the longer a defect remained in place, the more that defect caused other errors. When problems are not resolved satisfactorily, they not only create customer churn (for the engaged Ritz-Carlton customer, this lifetime value can be in excess of $ 1 million) but they also produce people who are vocally negative about the brand.

A Ritz-Carlton leader said, “I’ve come to learn that the least costly solution is the one that happens immediately. The longer and higher a customer complaint lives in an organization, the more it grows. By the time a complaint hits senior leadership, what could have been resolved by getting the guest the amenity he or she requested with a slight enhancement turns into resolutions on a par with an upgraded night on the Club Level (an elevated service experience affording access to a lounge serving multiple daily complimentary food offerings and the ready assistance of concierge staff).”

Deliver WOW

All of these elements should come together to deliver WOW. A Wow experience hinges less on the inherent exhilaration of the product and more on delivering service that appeals to both the thinking and feeling aspects of your customer. Michelli writes “operating from this understanding that customer engagement is linked to the consumers’ wanting ‘to feel a rush, Ritz-Carlton leadership calls this desired memorable and emotional connection a ‘Wow experience’ and encourages staff to personally affect guests to achieve this level of emotional intensity…. Wow starts with a commitment to a culture of extraordinary service. Ingenuity brings it to life…. Extremely satisfied customers emerge through memorable and emotional connections forged between them and a business.”

Some of the greatest opportunities for wowing customers occur when breakdowns happen. Earlier this year I wrote that “mistakes are one of the best things that can happen in the customer experience world. Players remember the way mistakes are handled much more than the mistake and often more than the actual gaming experience. Mistakes provide an opportunity to create a great memory and a connection with your customer.” Breakdowns will occur despite the best intentions to provide flawless service. Empathy, quick attention, and a willingness to go beyond the resolution will salvage a bad situation and turn it into a winning outcome. Although many businesses go out of their way to deny responsibility for guest problems, the staff of Ritz-Carlton typically acts responsibly, without ascribing blame, through targeted corrective action.”

Hold the team to high standards and measure, measure, measure

Once you have the right people and they are trained and empowered, you need to ensure they are delivering on your core message and it is resonating with customers. If people are set up to succeed, then it is also their responsibility to create a great customer experience. You must monitor your team and make sure they are creating Wow experiences for your customers.

Not only should you work with them individually, you should look at how well your customers are responding. Start with monitoring NPS (Net Promoter Score). Net promoter score is very straightforward. It is the answer to one question, on a scale of 1-10: How likely is it that you would recommend the company to a friend? Those who are answer with a 9 or 10 are considered loyal enthusiasts who will keep buying and refer others. Those who answer 7 or 8 are passives, satisfied but unenthusiastic customers who are vulnerable to churning. Those with a score of 0-6 are considered detractors, unhappy customers who can damage your brand and impede growth through negative word-of-mouth. While target NPS scores are dependent on your industry and audience, you can monitor your team’s performance by looking at the trend, whether your NPS is improving or deteriorating.

The Ritz-Carlton takes measurement to the next level and that provides a great way to move from providing excellent customer service to delivering Wow. At Ritz, they focus on customer engagement using a methodology developed by the Gallup organization:

How satisfied are you with Ritz-Carlton?
How likely are you to continue to choose Ritz-Carlton?
How likely are you to recommend Ritz-Carlton to a friend or associate?
Ritz-Carlton is a name I can always trust?
Ritz-Carlton always delivers on what they promise?
Ritz-Carlton always treats me fairly?
If a problem arises, I can always count on Ritz-Carlton to reach a fair and satisfactory resolution?
I feel proud to be a Ritz-Carlton customer?
Ritz-Carlton always treats me with respect. 10. Ritz-Carlton is the perfect hotel for people like me?
I can’t imagine a world without Ritz-Carlton?

By looking at NPS and customer engagement, you can measure how well you are delivering a great customer experience. Most importantly, look at the trends and see if you are maintaining a strong experience, getting better or deteriorating. Once you understand how your customers feel, you can then work with your team to improve. Transparency is the key, at Ritz, once the data is collected, results are posted monthly, and an 18-month rolling average is used to place each hotel in a green, yellow, or red zone. You should use this data to adjust swiftly what you and your team is doing to enhance the emotional bond with your customers.

Treat your staff like you treat your customers

The final key to creating a great customer experience is treating your team the way you want them to treat your customers. Leaders need to lead by example and this includes helping people see how they should treat others. At the Ritz-Carlton, the philosophy is that everyone is as important as everyone else, from the CEO to the housekeepers and clerks. Michelli writes that “by not confusing title with importance, leadership at Ritz-Carlton understands that creating an environment of respect universally results in a respectful service….I knew the guests were very important. But a few months later I realized that the maître d’ I watched every day was just as important because every guest was proud when he talked to them. Why? Because he was a first-class professional. He was somebody special—because of the excellence he created for the guests.”

This attitude also should translate into protecting your team when necessary from customers. If a customer is disrespectful, that is no more acceptable than an employee being rude to a customer. You need to empower your team to fire customers gracefully to reinforce the importance of treating everyone well.

A story to create WOW

Integrating best practices from the Ritz-Carlton with the way Danny Meyer created a restaurant empire, it becomes clear that building a great customer experience is not about one thing (a catchy slogan, a big CS team, etc.) but about a process. First you need to create the appropriate mentality across your organization, then hire the right people, train them consistently and focus on (and measure) delivering what the customer does not even realize they want. With these elements in place, you will create a customer experience that accelerates engagement and retention of your customers and translates into higher profits.

Key takeaways

Ritz-Carlton, one of the most profitable hospitality groups, leverages creating a great customer experience to build its competitive position
To provide an exceptional experience, you need a message that your organization can rally around; for the Ritz-Carlton it is: the answer is yes; . . . now what is the question?
You then need to build a team focused on delivering that experience, from hiring correctly, to training and then empowering.

Beware of equating sector performance with that of a dominant company

One cognitive bias that is often overlooked is equating the performance of an industry or sector with that of its dominant player. When one company represents 60, 70, 80+ percent of a market, its fortunes will drive the growth (or decline) rate of the industry. This growth, however, may be due to mistakes or issues with the dominant company rather than the underlying market and potential of the space. This bias, what I refer to as Dominant Player Bias, risks abstaining from good business opportunities and misallocating resources, missing Blue Oceans and accepting sub-optimal performance. The chart below shows how one company’s poor performance can imply the entire sector is contracting, though the problems are due to the dominant company rather than reduced interest:

There are many examples where the fortunes of the dominant company are equated with the fortunes of the industry. Facebook has nearly 70 percent of the social media market and if people lose interest in Facebook, it drives a decline in social media usage. Roughly, a 20 percent drop by Facebook, would drive down overall social media numbers about percent 15 percent (I am not using real numbers for the purpose of this example). Thus, investors may invest less in social media or other companies may accept declining usage because they feel the industry is contracting significantly. The actual cause though may be Facebook’s customers’ dissatisfaction with privacy or advertising policies. Dominant player bias has caused these investors and competitors to equate Facebook’s performance with the social media landscape.

There are several actual examples of dominant player bias (the Facebook one above is for illustrative purposes only). When Zynga’s performance dropped significantly after its IPO in 2011, many heralded the end of social gaming. The success (and valuation) of companies like King.com, Epic, Supercell, etc., show that the underlying market remained healthy and Zynga was performing due to internal issues (shift to mobile, reliance on the Facebook feed, etc.).

Underestimating the market

The first problem caused by Dominant Player Bias is that it prompts companies to underestimate, and thus under-invest, in a market. To use the Facebook example, if Facebook is facing difficulties but people misinterpret its problems for shrinking demand for social media, they are underestimating the market and thus missing potential opportunities. These opportunities include:

The dominant player under-investing and seeking to diversify. If they do not realize that internal decisions have led to falling performance, they may seek to diversify or even exist a market with high potential because they believe the potential market is smaller than it actually is
Competitors may shy away from trying to build a business in the market because they feel it is not worth the investment. This bias can result in under-investment in new product development, marketing or other avenues that companies could challenge the dominant player. In this case, dominant player bias is actually compounding the problem because if the market is shrinking due to poor performance by the dominant player, it is actually a great opportunity for someone else to come in and try to break its hold on the market.
Investors are less likely to fund companies with ideas that can disrupt the market if they perceive the sector is not attractive. Investors might underestimate the potential of an opportunity if they think the market is shrinking, though the market may only be shrinking because of poor decisions by the dominant player.

While dominant player bias can be either positive or negative, it is not a significant problem when it exaggerates the market potential. If a great company is continually expanding the market, then the great company’s performance is a good proxy for the overall opportunity.

You may miss a Blue Ocean

Another issue generated by Dominant Player Bias is that it might blind companies or investors to Blue Ocean opportunities. I have written several times about Blue Ocean strategy, rather than competing directly find an adjacent market space where there is no competition, and how Blue Oceans over time generate a higher return than competing in red oceans. If you succumb to Dominant Player Bias, however, you may never find the Blue Ocean because you falsely decide it is not worth looking for. This problem is particularly salient in industries with a dominant player, because their dominance may have prevented them from appealing to anyone except their existing customers (who they feel represent the entire market).

A great example of Dominant Player Bias and how a Blue Ocean strategy proved it wrong is the circus industry. Years ago, Ringling Brothers dominated the market. However, they let their product get stale and did not react well to changing tastes. Most observers at the time, 1984, believed that the overall circus market was dying and not worthy of investment. The founders of Cirque de Soleil, however, saw past this bias and realized it was the Ringling Brothers offering, not the circus market, that was causing the decline and launched a reimagined product. Now Cirque de Soleil is orders of magnitude larger than Ringling Brothers ever was because the Cirque founders were not convinced the circus market was dying due to Ringling’s poor performance.

A convenient excuse

Whether you are the major company or one of the small players, dominant player bias can mask under-performance, and thus opportunities to improve your company. Rather than looking inwardly, either intentionally or unintentionally, poorly performing dominant companies will blame the market for poor results. This bias will lead to several problems:

Resources are under-allocated to the market where they have a dominant position. As mentioned earlier, dominant companies normally enjoy higher profit margins. Rather than trying to grow these businesses further (and thus increase a high margin business), internal investment is diverted to other efforts that have a lower ROI because the market potential of the core market is underestimated.
There is unnecessary diversification or divestment. If the company believes the market it dominates is declining due to existential issues, it may diversify or close its business in the sector while it could enjoy a higher return if it resolved the internal problems leading to declining results.
The dominant player misses opportunities to expand the market because they misinterpret declining results for a smaller market.
Allowing poorly performing leaders to remain in critical positions. Rather than realizing deficiencies in leaders, poor results are blamed on overall conditions. This mistake keeps weak leaders in place rather than in competitive markets where the strongest leaders rise to the top and help improve the company’s fortunes.

Concentration is often the cause of an industry’s decline

While most companies dream of dominating their market, the type of concentration that creates Dominant Player Bias also can negatively impact the dominant player. By not facing competition, the company is not forced to innovate or grow the market. Instead, it enjoys monopoly rents, and thus a high profit margin, but is not focused on creating additional value. Without a focus on improvement, these companies usually reach a peak where they consider their huge market share the ceiling.

They do not provide anything to people who might not like their product but would enjoy something somewhat different. They also suffer from the innovator’s dilemma, they are satisfying existing who they know very well but because of their focus on their customers they do not create offerings that appeal outside the existing market. Over time, because others experience Dominant Player Bias, nobody brings new products to market, and the perception that the market is saturated leads to the reality that there are no new customers. Additionally, when a company enjoys a dominant position, the cost to them of acquiring new customers is very high relative to the value (almost everyone knows the dominant player, the low- and mid-hanging fruit has already been picked, etc.) so resources are focused on increasing profitability, generally by cutting costs. Since there is no effective competition, in the short term the company sees higher profits but eventually people are driven to alternatives (there are always alternatives, just potentially not direct alternatives), which creates a declining market (though the market shrinking is not driven by a smaller addressable market but by the dominant player reducing the total net value it is providing). This decreasing market both negatively impacts the dominant player over the long-term and prompts other companies to avoid entering the space.

The antidote for dominant player bias

To counter Dominant Player Bias everyone (the dominant player, competitors, potential market entrants, investors, etc.) should focus on the underlying market dynamics and the value to the customer rather than the short or mid term trends in the market. Going back to the Zynga example, looking in the mirror it is clear how other companies avoided Dominant Player Bias. If rather than looking at Zynga’s troubles and extrapolating it to the social gaming market, you looked at the value they were giving game players with a product that appealed to an untapped market (non-core gamers) and a business model (free-to-play) that made it easy for consumer to test and enjoy products, you would have seen that there was still a tremendous opportunity. That is exactly what happened at King and Supercell and Scopely, who avoided Dominant Player Bias and built billion dollar companies. By focusing on how much value you can deliver, it is much easier to scope the market rather than relying on how much one company has already delivered.

Key takeways

Dominant Player Bias is when you mistakenly think a sector is contracting because the dominant firm is performing poorly, while the underlying cause is mistakes by the dominant firm.
Dominant Player Bias impacts both the dominant company and competitors (and potential competitors), as it prompts them to underinvest in the sector and not seek new, Blue Ocean, ways to expand the market.
To combat Dominant Player Bias, you should look at the underlying value to customers and potential customers, rather than the performance of one company.

The subscription KPIs that matter

I have written several times recently about building and running a successful subscription modelin gaming but I did not address how to measure whether it is successful. To grow any business you need to understand what to measure so you can then optimize against these KPIs. While subscriptions do share some common characteristics with the free-to-play business, driven by the in-app purchase model, there are certain KPIs unique to subscriptions that you should focus on when building your program. As Leandro Faria says in The Essential SaaS Metrics Guide, “data doesn’t do you any good unless you act on it.”

MRR

Monthly recurring revenue (MRR) is generally the first KPI that companies focus on when looking at the health of their subscription program. You calculate MRR by looking at the average revenue per subscriber by the number of subscribers. If you only have one subscription level, then it is simply the monthly subscription cost times the number of subscribers. If you have subscriptions for various terms (monthly, quarterly and annual, for example), you calculate the average monthly revenue from the different subscription (an annual subscription of $144 generates $12 per subscriber in MRR). The formula is

MRR = Average monthly revenue per subscriber (ARPS) * Total number of subscribers

The reason MRR is the first KPI that subscription businesses monitor is because it shows the value of the model. MRR effectively is how much revenue the business can count on every month. The company can then allocate this cash flow to marketing, operations, acquisitions, etc. By having a guaranteed amount of revenue (which you do not have with discrete sales or in-app purchases), you have a clear source of funds to operate your business. Most companies will extend MRR to create an Annual Run Rate (ARR), which is important both for business planning purposes and understanding the value of the subscription component of your business.

MRR Growth

In addition to looking at MRR, you should monitor MRR Growth. To analyze MRR growth, you need to break it into three components. The first is new MRR, revenue brought by newly acquired customers.

The second component of MRR growth is Expansion MRR. Expansion MRR is increases in subscription revenue from existing subscribers. This revenue is driven by up-selling and cross-selling your customers.

The third element of MRR Growth is churn. Churned MRR is the revenue that has been lost from customers cancelling or downgrading their plans.

Taking these three components into account, the overall formula for MRR Growth is:

MMR Growth = New MRR + Expansion MRR-Churned MRR

CURR

I have written before about CURR (current user return rate), and it is as valuable in subscription businesses as it is for other online models. CURR shows how loyal your existing customers are; you should consider CURR the inverse of churn. If your CURR increases, it means you have improved your product’s appeal to existing players or customers, if CURR declines you have made your game worse. CURR is also an excellent way of looking at how your game is performing among different segments, VIPs versus payers versus never-spenders.
To calculate CURR:

Subscribers who were active between t-14 (14 days before today, today minus 14) and t-20 and who used the product between t-7 and t-13, what percentage returned between t-0 and t-6.

Measuring CURR is critical to see how engaged your subscribers are. If CURR trends downward, you are likely to experience increasing customer churn.

ARPS

As mentioned above, Average Revenue Per Subscriber (ARPS) is central for calculating MRR, but it is also an important KPI itself. Increasing ARPS shows that customers are upgrading and most likely seeing high value in your offering, thus they are willing to pay more. Conversely, declining ARPS shows that customers are not experiencing sufficient value and are either down-grading or moving to free plans

You should also monitor ARPS separately for existing players versus new players. As Faria writes, “there is a good practice of measuring the Average Revenue per [subscriber] separately for new customers. So instead of having an [ARPS] metric for all your customers, you’d have two different metrics: Average Revenue per Existing [Subscriber] and Average Revenue per New [Subscriber].” To calculate:

ARPS [existing] = Revenue from existing subscribers / # of existing subscribers
ARPS [new] = Revenue from new subscribers / # of new subscribers

Churn

Churn is the enemy of any business, but is even more troubling for subscription businesses. You never want to lose customers, but with subscription businesses churn means you are losing not simply a sale but an entire revenue stream.

You need to monitor both Customer Churn and Revenue Churn.
Customer Churn is how many players have canceled their subscription while Revenue Churn is how much those lost customers represents in revenue. There are thus three churn KPIs you should closely monitor:

Churn = # of Churned Customer / Last Month # of Customers
MRR Churn = SUM (MRR of Churned Customer)
MRR Churn % = Churned MRR / Last Month’s Ending MRR Negative

CAC

Customer Acquisition Cost (CAC) is the cost to acquire an additional customer, your marketing cost per customer. One way to calculate CAC is to consider the three variables that compose it. This method allows you to go into detail and might give you good insights about your sales process cost and conversions:

CAC = (CPL (cost per lead or cost per install) + Touch cost per customer (cost of your marketing team and any consultants)

I prefer to focus on cost per install (CPI) or cost per subscriber (CPS). As long as your lifetime value is higher than your CPS, you can continue to acquire subscribers and manage your overhead, including your marketing infrastructure.

LTV

Speaking of customer Lifetime Value (LTV), I have written repeatedly how it is the lifeblood of a successful business. LTV is a function that shows the present value of a new customer, how much that customer is worth to your company. While the equation is effectively the same for any business (the total expected value of your customer over their lifetime), it is somewhat simpler to calculate for subscription businesses. To calculate LTV for a subscription business, the following formula captures the core elements:

LTV = ARPS * % Gross Margin / % MRR Churn Rate

The hybrid model

While these KPIs show the health of a subscription business, you need to modify how you use them in most social games as well as iGaming. As subscriptions will only be one element of your revenue stream, in-app purchases will remain a major part of social gaming while gambling revenue will drive the casino industry. The above KPIs will help understand the health of the subscription element of your business, and whether you should invest in growing it, but they need to be incorporated into your other KPIs to understand both the impact on your business and your overall financial health. Subscription revenue will only be one part of your overall LTV calculation while you may want to look separately at CURR of players customers versus those making in-app purchases. There are many different combinations of models but the core subscription KPIs need to be incorporating into your daily review of the health of your business.

Key takeaways

To run a successful business, you have to constantly monitor KPIs and optimize based on this data; the subscription model is no different but the KPIs are not the same as the ones you are used to reviewing
MRR (monthly recurring revenue) is the most important KPI for the subscription model, how much subscription revenue you generate (and can count on) each month.
Other critical metrics for the subscription model are MRR growth average revenue per subscriber (ARPS), current user return rate (CURR), cost to acquire a customer (CAC), customer lifetime value (LTV) and churn.

Creating an experience that retains players

One of the most useful book I read last year had nothing to do with tech or the gaming space but was Danny Meyer’s Setting The Table, about how he created an incredibly successful restaurant empire. Meyer, who is not a chef, has built arguably the most successful restaurant business in the hyper-competitive New York market and is one of the founders of Shake Shack. Meyer built his empire not on creative dishes but creating a fantastic customer experience, which resulted in very high customer retention.

Given the importance of retention to game companies, creating a great customer experience is critical to retaining your players. While the product contributes to the experience, there are many other factors. When you go to a restaurant, the food is important but a key reason whether you return (are retained) is the overall experience. You might have great food, but if the waiter is surly, you have an issue paying the bill or even the cloakroom attendant is rude, you might not come back and will probably leave a bad Yelp or TripAdvisor review. Thus, the restaurant industry provides great lessons on how to create a superior customer experience, and Danny Meyer is probably the best restaurateur at delivering a fantastic experience. By extrapolating Meyer’s philosophy into a more general strategy, you can build a roadmap for improving almost any business.

The Golden Rule

Creating a fantastic customer experience begins with the Golden Rule, “do as you would be done by.” In effect, you should treat your customers the way you would want to be treated (and spoken to). If you do not want to get a call at 5 AM, do not try to call your customers at 5 AM because they are more likely to answer. If you would not want to have to answer ten stupid questions to cash out from a casino, do not ask your customer ten stupid questions. In all situations, put yourself in your customer’s perspective and ask how you would want to be approached or treated.

You are as strong as your weakest link

While your core product offering may be fantastic, customers are going to remember the worst part of their experience. If you are in a hotel, you may have a beautiful room with a very comfortable bed but if when you check out you are charged for a bag of cashews from the minibar that you did not take, that is what you are likely to remember. If you go to the hotel restaurant and the food is bad, that is what you will write about on TripAdvisor. It also does not matter to the customer if the restaurant is owned by the hotel or licensed to a third party; the customer will probably be more irate if you try to blame someone else. The key lesson is if you are spending time and money creating a great game or product, do not neglect all the other ways you interact with your customer or player.

Create a connection

One of the strongest motivators for people is seeking connections. As I wrote last week, after satisfying physiological needs and safety, people focus on needs of belonging and esteem, so if the organization is focused on building connections with customers that focus creates tremendous value. Meyer writes, “business, like life, is about how you make people feel….Service without soul is quickly forgotten.” Creating this connection and sense of affiliation builds trust and leads to repeat business.

To create a connection, the first step is to make your players feel important. They should not feel like a number or one of many players (you are number 800 in the queue, please hold on). Every customer should feel like a VIP, they should feel important and loved by the company. According to Meyer, “everyone goes through life with an invisible sign hanging around his or her neck reading, ‘make me feel important.’” If everyone dealing with customers treats them (and considers them) VIPs, you will build a long-lasting connection that keeps the customer from churning and probably improves engagement.

Customer’s time is money

Many companies fail to realize that a customer’s time is more valuable to them than money. All game and gaming companies at their core are entertainment companies, people are choosing between playing your game online or watching the latest episode of the Witcher, land based casinos have learned the Bellagio is competing not only with the Wynn but also with a trip to Hawaii. Your customer facing team must realize it is as important to save customers’ time as how much money they are spending.

Optimizing your customers’ time is also critical in ensuring their experience is better than their next best option. If you are using your customers’ time, you need to provide value (to them, not you) in exchange. Meyer writes, “what mattered most to me was trying to provide maximum value in exchange not just for the guests ’ money but also for their time. Anything that unnecessarily disrupts a guest’s time with his or her companions or disrupts the enjoyment of the meal undermines hospitality.”

If you have a great game, say an online casino, that they enjoy but have to spend half their time dealing with technical issues or trying to cash out, it effectively reduces their enjoyment 50 percent. Even if they get 10 percent more pleasure in your online casino then they would watching the Witcher, by forcing them to waste 50 percent of their time you make the Witcher, your competitor, a superior option.

Agents over gatekeepers

Creating a great customer experiences requires agents to act as advocates of the customers, not as gate-keepers. In every business, there are employees who are the first point of contact with the customers (attendants at airport gates, receptionists at doctors’ offices, bank tellers, executive assistants). Those people can come across either as agents or as gatekeepers. An agent makes things happen for others. A gatekeeper sets up barriers to keep people out. They need to represent the customer’s interest, fight for the customer and thus understand the customer’s concerns. As Meyer writes, “hospitality is present when something happens for you, it is absent when something happens to you.”

Mistakes are an opportunity

To me, mistakes are one of the best things that can happen in the customer experience world. Players remember the way mistakes are handled much more than the mistake and often more than the actual gaming experience. Mistakes provide an opportunity to create a great memory and a connection with your customer. Meyer writes, “The road to success is paved with mistakes well handled. Business is problem solving. As human beings, we are all fallible. You’ve got to welcome the inevitability of mistakes if you want to succeed in the restaurant business — or in any business. It’s critical for us to accept and embrace our ongoing mistakes as opportunities to learn, grow, and profit.”

Meyer identifies five elements for effectively addressing mistakes, fortunately all start with the letter A:

Awareness. Knowing that a mistake has been made.
Acknowledgment. Admitting that a mistake was made. It is incredibly frustrating having to argue with a company that they made a mistake. I remember a recent business trip to Sydney where the Internet in my room was not working. I had to argue with the front desk, then with a maintenance worker, both assumed that I did not know how to connect my phone to the Internet. Long story short is the wireless access point in the room was broken but their refusal to acknowledge the problem quickly ended up in my cancelling a stay the following month and not staying during visits every quarter. That is the cost of not acknowledging a mistake.
Apology. Saying you are sorry is an important step in turning a mistake into a good experience for your customer.
Action. Fix the mistake. Say what you are going to do to make amends then follow through. Make sure that the issue is resolved to the customer’s satisfaction and that you take care of it, do not put the resolution of the problem on the customer (remember to value their time).
Additional generosity. Do not simply make good for the mistake, provide more than you have to. Turn the bad experience into a great one. If a diner has a bad experience at one of Meyer’s restaurant, not only would they probably not be charged for the meal, but they might get a bottle of wine or champagne.

Another area to leverage mistakes is to turn a customer’s mistake into your mistake. Rather than fighting with a customer, accept responsibility even if it was not your fault. If you stress the customer made a mistake, either they will be mad at you or mad at themselves (if they do not believe you). Either way, they are not having a good experience. Instead, turn the mistake into your mistake and make them happy. That will drive additional engagement.

You need to align hiring with creating a great experience

To create a fantastic experience for your customer or player, the people responsible for dealing with them must have the right mentality. To have the right people, you need to hire the right people. As Meyer stresses, “you can’t teach emotional intelligence.”

You can have scripts and processes for dealing with customers but unless your team members can radiate warmth, friendliness, happiness and kindness, you will not be able to create great experience. Thus, you need to hire warm, empathetic people who have an excellence reflex. The excellence reflex is a natural reaction to fix something that is not right or to improve something that can be better. Not all hiring is perfect, so if you end up with some people who are not empathetic or have an excellence reflex, then you need to find them a new home. Otherwise, you will not be able to create great customer experiences.

You also need to ensure your people can control their moods. We all have bad days, but the customer does not care. You need people with personal mastery, team members aware of their moods and able to keep them in check.

Hiring is the key. As Meyer explained, “Over the years, the most consistent compliment we’ve received and the one I am always proudest to hear, is ‘I love your restaurants and the food is fantastic. But what I really love is how great your people are. ‘ The only way a company can grow, stay true to its soul, and remain consistently successful is to attract, hire, and keep great people.”

Customer experience is the key to success

While it is very challenging to build an organization with great customer experience, it is critical to engaging your players and preventing churn. Meyer’s success using customer experience as the key differentiator in building a restaurant empire in New York City, one of the most competitive and saturated markets, shows how this feature can help companies in other industries (like gaming) stand out and succeed.

Key takeaways

The key to strong retention is creating a great customer experience outside of the actual product, ensuring that customer contact is extraordinary.
To create a great experience for customers, everyone dealing with them needs to treat customers as they would want to be treated.
It is also critical to ensure you have no weak links in your interaction with customers, you create a connection with your player, your people act as agents for the customers and not gatekeepers, you treat mistakes as opportunities and you hire for emotional intelligence.

Taking the subscription model to the next level

Last year, one of my most popular posts was about the subscription model and opportunities for social game companies. I recently read a book, The Membership Economy by Robbie Kellman Baxter, that provided another layer to building and launching a successful subscription model. Baxter’s work provides useful advice for creating a sustainable subscription business.

Look at subscribers as members

Baxter’s book is not solely about subscriptions but about memberships, which can then be applied to the subscription model to make it more powerful. By looking at subscriptions as memberships, you not only generate an ongoing and stable revenue stream but also create value for your customers because membership provides recognition, stability, and convenience while connecting them. This connection is very powerful, as Maslow showed in his hierarchy of needs, after satisfying physiological needs and safety, people focus on needs of belonging and esteem before ultimately moving to self-actualization. Membership as part of a subscription helps people satisfy those needs.

Baxter writes that “membership is an attitude , an emotion. A subscription is a financial arrangement. It’s quite possible for something to be both a subscription and a membership organization…. Members love membership models because they fulfill powerful human drives — like needs for affiliation and prestige…. If you provide individuals with the infrastructure to enable them to connect and help them build behaviors that help themselves and others, there is tremendous potential to enable people to share ideas, content and physical products that otherwise might go underutilized.”

With membership, you are also less likely to churn customers. Members are committed until they cancel, the equivalent of breaking up or getting divorced. That break up is a much more emotional decision than deciding whether they want to make another purchase.

Subscriptions are about access

A key to success with subscriptions, as I discussed in my last post about the model, is to provide access as the core value rather than consumables. As Baxter writes, “too many loyalty programs are commodities, effectively just discounts for volume purchases, and don’t really create authentic loyalty or strengthen membership relationships. Don’t be that kind of loyalty program.” Subscriptions that are about discounts largely cannibalize your existing business and do not appeal to customers looking to build a long-term relationship with your product or game.

Access also takes you beyond the traditional discrete purchase model with customers, even in-app purchases (IAPs). Part of this access is access to a community of like-minded people, hopefully fans of your core offering. Access is so much bigger than ownership, and the subscription model ties customers to organizations in an ongoing relationship with an opportunity for benefits on both sides.

Given that access is probably a different value than you have focused on previously, the question becomes what do you offer subscribers. I cannot answer that question as it is different for different games and products. In the game space there are many types of access that appeals to customers, examples include:

Games or levels (i.e. Loyalty specific slots, hidden levels)
Community
Referral program
Additional bonus and side games
High roller features (i.e. higher MaxBet, higher limits)
Speeding up rewards
Battle Pass type features (avatars and other vanity items that show your expertise)
Online events (training, sneak peeks, webinars, etc)
Physical events for entire tiers (parties, casino visits, et al)
Individual physical events (personalized experiences for top tier players)
Additional customer service channels (phone, live chat, etc)
Charitable contributions

These are some examples but the key is not to offer subscribers a consumable item, you are not trying to replace an individual purchase but grant them access.

Building a strong subscription program

Programs should be tailored to your audience and product, ensuring you are creating something many of your players will enjoy. There are, however, several common steps to creating a successful program for subscribers:

Member/benefit alignment. Start with research and analysis of your existing data so you understand your potential subscribers. If you are not certain a customer would love the offering if he or she knew about it, there is no point in investing in anything other than fixing the offer and figuring out whom to target.
Remove frictionAnything that slows down a user’s ability to engage with the services offered, especially during the sign – up process.
Provide benefits at every stage of the journey, especially immediately. Provide meaningful benefits from the moment a player enrolls (or is automatically enrolled).
Make your trial period great.Rather than trying to limit what a customer gets with an initial or free trial, over deliver so they see all the benefits of subscribing. If you provide a minimal experience or create too many gates for the player, rather than convert they will get annoyed and churn.
Do not overpromise Simplicity is always critical to success, including with subscriptions. If you promise too many things, customers get confused and might not be able to find the benefit that really matters to them Also, if you promise too many things, it is hard to deliver on all of them.
Personalize.People value experiences differently so ensure that you provide a breadth of benefits to appeal to your addressable market. With data and machine learning, you can then surface the benefits specific subscribers would most enjoy so they get the best possible experience from the subscription. You want to personalize so your subscribers feel more connected than non-subscribers.

Baxter writes about how Caesars Casino uses personalization with its loyalty program, “Caesars sees itself not just as a gaming company, but as an entertainment company . It segments users to provide each group with a differentiated set of benefits and offers. It uses data to determine member preferences for things like favorite wines, room locations, and hotel amenities. Management empowers frontline employees to become ‘local hosts’ who use the data to make members’ visits special. This element is critically important because too many loyalty programs don’t build relationships with members at all — they just provide discounts, thus encouraging people to join all programs across an industry just to collect the points.“
Take risks As Howard Schultz, the founder of Starbucks once wrote, “whatever you do, don’t play it safe. Don’t do things the way they’ve always been done. Don’t try to fit the system. If you do what’s expected of you, you’ll never accomplish more than others expect.” Starbucks has made some major advances on traditional loyalty programs with loyalty cards that are great looking. The card is also tied to payments, reducing a major source of friction.
Tie loyalty and payments together.As Starbucks shows, there is tremendous value to integrating loyalty and payment. By tying payments to the account, it eases the purchase process, thus increasing purchases.
Continue to add value Players needs will evolve and your program will also need to evolve. You need to offer additional value to justify the ongoing costs of membership. As Baxter writes, “Amazon Prime is a paid membership program. Amazon’s approach has been to have members pay something so that they are more…. Amazon has systematically and thoughtfully continued to invest in layering benefits over the core offering of free two – day shipping.”

The first time subscriber experience

As discussed above, one of the critical elements in creating a loyalty program is the early user experience. According to Baxter, “onboarding done correctly dramatically reduces the number of people who sign up for trial or become full members and then cancel within the first month or two. It also increases the number of people who become long-term members. Members who participate heavily in the first few months of the membership are much ore likely to become long-term customers.”

To ease on-boarding, you need to keep the offering simple and easy to use. Provide a very straightforward experience and quickly reward your subscribers.

Also, try to turn the subscription into a two way street. In most membership organizations, the members have a responsibility to provide something — membership dues to an association or country club, content as with Snapchat or Twitter, or even just personal information as with Groupon — in exchange for access to the membership benefits. By having your customers contribute, they feel more invested in the offering and are less likely to churn. The contribution also should be very easy, let the subscriber contribute easily and quickly without having to jump through hoops. Then reward them for the contribution, to build a positive reinforcement cycle.

Determining pricing

Once you have designed your subscription program, you need to determine pricing. As with other elements of the program, remember the value of simplicity. While a program’s designer may feel multiple options allow players to find the optimal program, it actually confuses and overwhelms customers (just as retailers understand people buy more tomato sauce when there are 3-5 options rather than 30 options). In addition to simplicity, successful subscription programs are transparent. Customers need to understand clearly the value and benefits, the costs and how to cancel.

According to Baxter, the optimal number of subscription options is three, and this number is consistent with consumer behavior research. Baxter writes, “Most people prefer to have multiple options, that three is the right number of choices, and that the majority prefers the middle option.” You can then layer a la carte services on to these three options (as satellite TV companies frequently do).

These a la carte options allow customers to personalize their experience while not creating confusion (they also can be introduced once the subscriber is comfortable with the core offering). According to Baxter, “À la carte services are out-of-the-ordinary services members do not require on an ongoing basis — for example, a one-time indexing of your content, or an on-boarding fee, or a health audit at the gym. A mistake many organizations make is to put this type of one-time service into a higher-level, ongoing pricing tier.” Ancillary products can also be offered on an a la carte basis, such as headsets for Skype or exercise shorts sold by fitness centers.

Also look at offering different subscription terms. SurveyMonkey found that annual subscribers were more loyal customers than monthly payers. They were committed to the platform and used it in multiple ways. While you may have three tiers, you can also offer monthly and annual subscriptions or fewer tiers but vary the length. The key, though, is keep it simple and easy to understand for subscribers.

Change your thinking

Once your pricing is determined, you then need to change your company’s culture to support a successful subscription business. The subscription model relies on long-term customer retention and thus the entire organization needs to be focused on retention rather than discrete purchases.

This new attitude needs not only to be reflected in the product but also with your marketing and customer support (CS) teams. According to Baxter, “marketing is more than campaigns — it’s about focusing on the market. This is always true, but especially in the Membership Economy, where retention matters more than acquisition. Marketing should ensure that the offerings the organizations create meet the ongoing requirements of the target buyers and that those prospective buyers know about the benefits of the offering, sign up, and become loyal….Good marketing is honest. Sometimes people have the idea that good marketing is about tricking people into buying things they don’t need. Not only is this approach unethical, but in the Membership Economy, with the need for ongoing relationships, it simply doesn’t work.”

In addition to marketing, your customer support team needs to become agents for your subscribers. At its core, CS in a subscription model needs to focus on building loyalty, not reducing anger. Baxter writes, “In an ownership economy business, the support staff’s goal is to minimize customer anger. In the Membership Economy, the goal is to maximize loyalty …Their task is to help callers use the product — more effectively, more efficiently — so that they will be increasingly loyal.” Wwhen you launch your program, ensure all your customer facing teams are focused on retaining your customers.

KPIs, analysis and avoiding churn

The final key to a successful subscription program is measuring the right metrics and using them to prevent churn. There are several elements of churn to monitor:

Passive churn. Passive churn is the number of subscribers who churn by not updating their payment method. This churn is a good indicator of problems with your payment platform. To mitigate this situation, give good customers the benefit of the doubt when there is a payment problem (which requires knowing who your best customers are) and reach out to them (phone or their preferred communication platform) to update the information while not suspending service in the interim.
Active churn. Active churn is when a subscriber proactively decides to cancel. Minimizing active churn requires understanding the reasons someone might want to cancel and creating options and incentives for them to stay (while not forcing the to stay). Below are some areas that Baxter recommends you review to determine the root cause of active churn:
1. Date cohorts. Measure elapsed time from the day they signed up to see if certain things happen after a certain fixed time.
2. Utilization cohorts. Do people tend to cancel after a certain amount of time of low usage or engagement?
3. Original lead source cohorts. Do certain ads or partnerships bring in subscribers who behave differently from established members

Once you measure and understand your churn, there are several ways to reduce it. You need to build a system that makes leaving a difficult decision to make. You do this by ensuring people are using your product regularly and are very engaged. People are more likely to perceive value in memberships that they use frequently and for extended durations, so many subscription companies devotedly track visits and lengths of visits of members.

Subscribers are also more reluctant to cancel memberships when they have achieved status, customized their experience, scheduled regular activities, or built personal spaces. Finally, Baxter points out that if “someone simply wants to stop paying , offering a free subscription — something that allows the member to remain part of the family — is a best practice.”

Key takeaways

Treat your subscription service as a membership service. By looking at subscriptions as memberships, you not only generate an ongoing and stable revenue stream but also create value for your customers because membership provides recognition, stability, and convenience for your players while connecting them.
The core value to subscribers from the subscription should be additional access, features or games or community not available to non-subscribers.
Focus on the early experience. Onboarding done correctly dramatically reduces the number of people who sign up for trial or become full members and then cancel within the first month or two.

An often-better alternative to AB testing?

While AB testing is an integral element of mobile and social game development (as well development of most digital products), in many situations there is a better option. Several years ago, I had the opportunity to serve as an advisor to a company that had some brilliant people. Their CTO was a strong advocate of using multi-armed bandit testing as a superior alternative to AB testing. Multi-armed bandit testing is not new, there was a popular post in 2012 (http://stevehanov.ca/blog/index.php?id=132), and it is used by Google and other tech giants, but people (especially product managers) still regularly default to traditional ABn testing.

The problem with AB testing is that you leave money and performance on the table. Until the test is over, the poorer performing variant(s) will always get a significant share of your traffic. With the multi-armed bandit approach, you allocate increasingly less traffic to poorly performing variants.

What is multi-armed bandit testing

A multi-armed bandit approach allows you to dynamically allocate traffic to variations that are performing well while allocating less and less traffic to underperforming variations. Instead of two distinct periods of pure exploration and pure exploitation, bandit tests are adaptive, and simultaneously include exploration and exploitation. As Optimizely wrote recently, ” multi-armed bandit optimizations aim to maximize performance of your primary metric across all your variations. They do this by dynamically re-allocating traffic to whichever variation is currently performing best. This will help you extract as much value as possible from the leading variation during the experiment lifecycle, so you avoid the opportunity cost of showing sub-optimal experiences.”

Multi-armed bandit testing is a Bayesian approach to AB testing. As Shawn Lu writes in a post titled Beyond A/B testing, “The foundation of the multi-armed bandit experiment is Bayesian updating. Each treatment (called “arm”) has a probability of success, which is modeled as a Bernoulli process. The probability of success is unknown, and is modeled by a Beta distribution. As the experiment continues, each arm receives user traffic, and the Beta distribution is updated accordingly.”

A recap on ABn testing

To compare bandit testing with ABn testing (AB is with two variants, a test and control, n allows for additional variables), let’s quickly recap how AB testing works. Alex Atkins summarizes it succinctly, writing “in statistical terms, a/b testing consists of a short period of pure exploration, where you’re randomly assigning equal numbers of users to Version A and Version B. It then jumps into a long period of pure exploitation, where you send 100% of your users to the more successful version of your site.”

Benefits of multi-armed bandit testing

Bandit algorithms try to minimize opportunity costs and regret (the difference between your actual return and the return you would have collected had you deployed the optimal options at every opportunity). Rather than letting an AB test run until it is statistically significant, a bandit test moves subjects into the best performing group faster, allowing you to capture more gains. Matt Gershoff writes, ““Some like to call it earning while learning. You need to both learn in order to figure out what works and what doesn’t, but to earn; you take advantage of what you have learned. This is what I really like about the Bandit way of looking at the problem, it highlights that collecting data has a real cost, in terms of opportunities lost.”

A related advantage of multi-armed bandit testing is you make fewer mistakes. An A/B test will always send a significant portion of traffic to the sub-optimal group.

Also, as Shawn Lu writes, “[an] advantage of bandit experiment is that it terminates earlier than A/B test because it requires much smaller sample. In a two-armed experiment with click-through rate 4% and 5%, traditional A/B testing requires 11,165 in each treatment group at 95% significance level. With 100 users a day, the experiment will take 223 days. In the bandit experiment, however, simulation ended after 31 days, at the above termination criterion.” if the treatment group is clearly superior, we still have to spend lots of traffic on the control group, in order to obtain statistical significance.”

Finally, while not mathematically an advantage, bandit testing relieves the pressure to end a test too early. With ABn testing, frequently you will see one option perform better “directionally” and decide, or be forced to decide, to terminate the test and move everyone to the higher performing bucket before you get significant results. Unfortunately, this sometimes leads to picking an option that would be reversed once there is more data.

Why multi-armed bandit is not always the correct approach

The value of bandit testing does not mean you should abandon completely ABn testing. In Lu’s post, he writes “the convenience of smaller sample size comes at a cost of a larger false positive rate.” That is, you end up sometimes gravitating to the sub-optimal solution.

Alex Atkins also writes, “in essence, there shouldn’t be an ‘a/b testing vs. bandit testing, which is better?’ debate, because it’s comparing apples to oranges. These two methodologies serve two different needs.”
A/B testing is a better option when the company has large enough user base, when it’s important to control for type I error (false positives), and when there are few enough variants that we can test each one of them against the control group one at a time.”

The Bandit Option

While multi-armed bandit testing is not always a better option than ABn testing, you should look closely at using bandit testing when possible. It can reduce the opportunity cost of your testing and relieve pressure to terminate tests prematurely.

Key takeaways

While AB testing is the most common method of optimizing between alternatives, in many situations the multi-armed bandit approach is optimal.
A multi-armed bandit approach allows you to dynamically allocate traffic to variations that are performing well while allocating less and less traffic to underperforming variations.
Multi-armed bandit testing reduces regret (the loss pursing multiple options rather than the best option), is faster and lowers the risk of pressure to end the test prematurely.

Confirmation of Confimation Bias

Although I have written many times about different kinds of biases, the one I find most common, even in my decision making, is his confirmation bias. An article I recently came across, Confirmation Bias: Why You Make Terrible Life Decision by Nir Eyal, confirmed to me what I suspected, confirmation bias is more pervasive than most people realize. Confirmation bias is where people pick out anecdotes or facts that support their belief, while neglecting conflicting evidence.

I see if often in the games industry, you want to add a new feature (say a chat system) and you point to three products with chat systems that are highly successful. You do not look at the ten products with chat systems that have failed. Maybe you are looking to build a new product and you want to license an expensive IP. You justify it by pointing to the revenue that Kim Kardashian’s game generating while not including in your calculation the 20 branded games that failed.

This problem is not limited to the game industry. You may believe that social democracy is the best path forward for a country and you use Sweden, Norway and the Netherlands to confirm your belief, while not noticing Cuba, Greece and Spain. Or you believe free markets are the answer and confirm this by looking at Singapore, Taiwan and the US while not noticing Sweden, Norway and the Netherlands. Sometimes people knowingly pick the cases that prove their point, however, confirmation bias is when they sub-consciously accept the evidence that supports their beliefs.

Data is not the panacea

Using data rather than emotions appears as a vaccine against confirmation bias but data can contribute to the problem. Analysts suffer from the same biases as other people and will often look at the data that confirms their hypothesis.

You may have launched a new product that has a strong LTV and is growing rapidly, surpassing your existing product. One analyst who belongs to the team that launched the new product might look at the retention and monetization metrics and compare them with the original product. These KPIs are higher in the new product so the analyst recommends marketing funds shift from the older product to the new product. An analyst on the team for the existing product might identify that shows that total revenue (new plus old product) is flat since the launch of the new product, despite additional marketing spend on the new product. They might argue that the new product is simply cannibalizing the existing product. The better KPIs of the new product are a result of the best users moving to the flashy new toy, not a fundamentally better product.

Depending on the analyst’s initial partiality, they will either investigate and then present the first or second data set. Neither data is incorrect but the conclusion and actions they lead to are very different. Even though both analysts are being honest and believe they are objective, confirmation bias is driving their analysis.

Confirmation bias can impact career decisions

Confirmation bias is not only prevalent in deciding what decisions to make in business but also how to manage your career. You might feel your company is not treating you fairly. Then when two colleagues get large bonuses and you do not, it confirms that you are being treated unfairly. The data you may not be considering is that you are on a higher compensation level already or had received a bonus six months ago.

You also may be considering moving to a competitor. You have met people at a trade show from the other company and they mentioned some of the great perks. You interview and are then offered a position. Before accepting, you see some negative reviews on Glassdoor. You have already decided that you want the new job so you convince yourself the reviews have to be from a different business unit or boss. When you get to the new job, you learn the problems are real.

How do you fight confirmation bias

Given the prevalence of confirmation bias, it is important to create a strategy to combat it. A recent article, Facts Don’t Change People’s Minds, Here is What Does by Ozan Varol, provides some great suggestions:

Do not feel your beliefs create your identity. Do not get defensive when someone questions you or your project. The data is not personal and you are not a better or worse person if your hypothesis is wrong.
Develop better empathy. If someone disagrees with you, it is because they think they are correct. Understand why they are disagreeing and be open to them being potentially correct.
Get out of your echo chamber. As Varol writes, “make a point to befriend people who disagree with you. Expose yourself to environments where your opinions can be challenged, as uncomfortable and awkward as that might be.” Seek out data that disproves your position.

Additionally, I have found pre-mortems a very useful tool to combat confirmation bias. A pre-mortem is a meeting held before a major decision where all those involved in making the decision imagine themselves six or twelve months after the decision was taken, assume it turned into a debacle, and then explore why it was a disaster. This type of meeting forces you to look at contradictory facts and raise potential problems.

It is important to be cognizant of confirmation bias and seek out all the information before making important decisions. Most importantly, look at contradictory information and do not discount it because it does not support your position.

Key takeaways

Confirmation bias is where people pick out examples or facts that support their belief, while neglecting conflicting evidence.
Being data driven does not avoid confirmation bias, as people gravitate to the data that supports their beliefs.
To combat confirmation bias, do not equate your beliefs with your identity, understand why others have different positions, seek out non-confirming data and do a pre-mortem for important decisions.

When to use qualitative research

In the mobile gaming space, being data driven is largely synonymous with using quantitative data, with qualitative data (such as user research) relegated to third class status. Most game companies believe their decisions should be based on user activity in their product. They often act and allocate resources only to analyze their existing data, with design thinking and user research considered old fashioned and not as accurate. These companies, however, are missing an opportunity.

As I have said before, all data should be used to make decisions and optimize your product. Gameplay data is extremely valuable but so are qualitative data sources. Young children learn quite early that when you add a positive number to a number, the new number is larger. Thus, there is no way adding qualitative data to your quantitative data can make your analysis weaker. As long as the qualitative research does not detract from the analysis, it makes you stronger.

There is a great article, Qual vs Quant: when to listen and when to measure by Laura Klein, that explains when qualitative research is most useful. The article explains that you should constantly use both qualitative and quantitative data, but different situations require different focus.

When to focus on qual research

Qualitative research is important to understand why you are experiencing a problem or situation. Quantitative research tells you that you have a problem and what the problem is (i.e. D1 retention is down 20%).

What method in which situation

Klein highlights three different high level scenarios and then discusses whether qualitative or quantitative research would help most. In the first scenario, you are only considering changing one variable. You may be deliberating whether to change the maximum bet on slots in your casino. In this situation, quantitative analysis is optimal. With a change this small, users in a testing session or discussion will not give you any actionable information. Qualitative feedback will not provide failure to compensate for the time and money it takes to set up interviews, talk to users, and analyze the data.

The second scenario is when you are planning a multi-variable or flow change. You may be planning to implement a new feature, like a progression system, that would among other things require you adjust the user interface and journey. While you could launch it and measure performance, you would never know why it failed or succeeded. Qualitative research allows you to learn before launching what elements of the feature are compelling, what is confusing and how you can optimize the feature before you launch the feature.

The third scenario is prioritizing your roadmap. In this situation, a combination of qualitative and quantitative research is optimal. As Klein writes

The key here is that you want to look at what your users are currently doing with your product and what they aren’t doing with it, and you should do that with both qualitative and quantitative data.

Qualitative Approaches:

Watch users with your product on a regular basis. See where they struggle, where they seem disappointed, or where they complain that they can’t do what they want. Those will all give you ideas for iterating on current features or adding new ones.

Talk to people who have stopped using your product. Find out what they thought they’d be getting when they started using it and why they stopped.

Watch new users with your product and ask them what they expected from the first 15 minutes using the product. If this doesn’t match what your product actually delivers, either fix the product or fix the first time user experience so that you’re fulfilling users’ expectations.

Quantitative Approaches:

Look at the features that are currently getting the most use by the highest value customers. Try to figure out if there’s a pattern there and then test other features that fit that pattern.

Try a “fake” test by adding a button or navigation element that represents the feature you’re thinking of adding, and then measure how many people actually click on it. Instead of implementing an entire system for making friends on your site, just add a button that allows people to Add a Friend, and then let them know that the feature isn’t quite ready yet while you tally up the percentage of people who are pressing the button.

What to do

There are some situations where you should rely on the quantative research, other situations where qualitative research is best, but generally a combination of the two is the optimal way to formulate your product strategy. By using both, you are collecting all available information and using this data to drive optimal decisions.

Key takeaways

Qualitative research (survey, user panels, etc.) is often neglected by game companies who prefer to use quantitative data (KPIs) to drive decisions, but this approach neglects that more information is always better.
In situations where you are only impacting one variable, quantitative data is the answer; if you are adjusting flow or multiple variable, lean on qualitative data; and if you are prioritizing your roadmap, use a robust combination.
There are some situations where you should rely on the quants, other situations where qualitative research, but generally a combination of the two is the best way to formulate your product strategy.

It isn’t what it is

One of the most insidious phrases I have heard almost everywhere I have worked is, “it is what it is.” The phrase portends inefficiencies, politics, despair, incompetence or a lack of caring (or a combination thereof). It is critical to recognize this phrase as a problem itself and also not accept the situation it is used in reference to.

When you are likely to hear “it is what it is”

The phrase “it is what it is” generally means that the situation is how the company operates and cannot be changed. There is nothing that can be done about it. It may be that there are certain processes in place that inhibit efficiency or the ability to drive new initiatives.

The phrase will come up when someone sees an opportunity to increase profitability, maybe through a new marketing channel or by changing a software provider, only to be told the company needs to continue doing things the existing way. You may ask why is our marketing budget focuses on MySpace, and the response is we know it probably is not optimal but “it is what it is.”

You also hear the phrase when somebody is in the wrong position in the organization, cannot do their job effectively and it inhibits other people’s performance or initiatives. In American Football, a fast receiver may ask his quarterback for more long-pass attempts as he has his defender beat. The quarterback responds he would love to but the offensive coordinator does not understand the passing game. The receiver asks him to talk to the coordinator about the opportunity but the quarterback responds, “it is what it is.”

Another situation where you hear this destructive phrase is in reference to a company’s systems and software. The company may be using Windows 98. When you talk to IT about getting a newer operating system you are told they only support Windows 98. When asked why, the response is we know there are better systems out there but “it is what is.” You go back to trying to create your marketing strategy using WordPerfect.

Why “It is what it is” is so bad

The most destructive impact of “it is what it is” is not the initiative that may get stalled or the sub-optimal allocation of resources but how it destroys a company’s culture. When someone repeatedly hears “it is what it is” in response to ideas on how to improve the company, they eventually stop proposing ideas. The company then stops innovating and optimizing, which in the hyper-competitive era we live in, is the beginning of a death spiral.

”It is what is” also erodes employee morale and increases churn of your good employees. Most people work not only to make money but also because they want to achieve (most studies find that making money is not one of the key motivators for people). When the culture of it is what it is becomes prevalent, the high achievers become the most dissatisfied. There is very little worse than not pursuing an initiative they know would be good for the company not for a good reason (cash flow, strategic focus elsewhere, etc.) but because “it is what it is.” These employees are likely to find an opportunity where they can strive to be best and grow their company. This also creates a death spiral where the company is left only with people who are content to maintain the status quo.

The Kobayashi Maru

I learned a related lesson in my childhood while watching Star Trek (a show that largely shaped my adult life), a situation called the Kobayashi Maru. Not to be too geeky, but the Kobayashi Maru was a simulation given the officers at Star Fleet Academy(the training academy for people who would fly spaceships) where they had to decide between two awful scenarios, both of which resulted in a great loss of life. The point of the simulation was to show that sometimes there is not a good solution and no Star Fleet cadet ever passed.

One cadet, James Kirk (who even non-geeks probably heard of as Captain Kirk), though, did pass. What Kirk did was go in one night and reprogram the simulation, which was against Star Fleet rules. Even though it was against the rules, because he ended up succeeding where everyone else had failed, his victory was accepted.

The lesson here is that even in a no-win situation, finding a way to succeed by changing the rules. Kirk succeeded because he did not accept that “the Kobayashi Maru” is what it is. He took the initiative to make it something else and make it beatable.

Do not accept doing things sub-optimally

Start by never, ever, saying “it is what it is.” If you catch yourself about to say it, think about the situation and how you can change it. Never accept the status quo unless it really is the base path forward. If it is not, take the initiative to change the way things are done. If there is an incompetent person blocking your path, find another path. Find an ally who will work with you to drive an initiative over the blocker. If there is a better process or software solution, take it on your self to build the business case. It is easy for someone to block you by saying “it is what it is,” it is much harder from them to block you when you have a well reasoned business case in how the change will make the company an extra $500,000.

If you are in a leadership position, train your team not to accept (or say) “it is what it is.” If they are pushing an initiative forward and run into resistance, empower them to find a solution or change the rules. Only in extreme cases should they escalate the situation. It is more important that you foster a culture that your team members find a way around “it is what it is” rather than you doing it in each case (which is almost as bad). Tell your team you do not want to hear what we can’t do or are not allowed to do but want to hear how we are doing it.

When you run into “it is what it is,” rather than accepting it let everyone know the cost of not changing. If you hear it from a colleague or someone on your team, either encourage them to find a way to change things or take it upon yourself. Just as accepting “it is what it is” can put a company in a death spiral, showing it actually is not the case leads to a healthy, growing organization.

Key takeaways

”It is what it is” represents one of the most insidious phrases in the business environment, effectively telling people the status quo cannot be changed.
This phrase stops initiatives that would improve profitability, inhibits new ideas and drives away your best employees.
To combat “it is what it is,” do not accept it and build a business case to over come it.

February 2020
S	M	T	W	T	F	S
« Jan
						1
2	3	4	5	6	7	8
9	10	11	12	13	14	15
16	17	18	19	20	21	22
23	24	25	26	27	28	29

Start with attitude

Hire right and set them up to succeed

Empower the team

Deliver WOW

Hold the team to high standards and measure, measure, measure

Treat your staff like you treat your customers

A story to create WOW

Key takeaways

Share this:

Like this:

Underestimating the market

You may miss a Blue Ocean

A convenient excuse

Concentration is often the cause of an industry’s decline

The antidote for dominant player bias

Key takeways

Share this:

Like this:

MRR

MRR Growth

CURR

ARPS

Churn

CAC

The hybrid model

Key takeaways

Share this:

Like this:

The Golden Rule

You are as strong as your weakest link

Create a connection

Customer’s time is money

Agents over gatekeepers

Mistakes are an opportunity

You need to align hiring with creating a great experience

Customer experience is the key to success

Key takeaways

Share this:

Like this:

Look at subscribers as members

Subscriptions are about access

Building a strong subscription program

The first time subscriber experience

Determining pricing

Change your thinking

KPIs, analysis and avoiding churn

Key takeaways

Share this:

Like this:

What is multi-armed bandit testing

A recap on ABn testing

Benefits of multi-armed bandit testing

Why multi-armed bandit is not always the correct approach

The Bandit Option

Key takeaways

Share this:

Like this:

Share this:

Like this:

Data is not the panacea

Confirmation bias can impact career decisions

How do you fight confirmation bias

Key takeaways

Share this:

Like this:

When to focus on qual research

What method in which situation

What to do

Key takeaways

Share this:

Like this:

When you are likely to hear “it is what it is”

Why “It is what it is” is so bad

The Kobayashi Maru

Do not accept doing things sub-optimally

Key takeaways

Share this:

Like this: