General Social Games Business – The Business of Social Games and Casino

Leading in a complex world: building a team of teams

I am very careful to avoid management books that are popular because they are the flavor of the day or driven by the halo effect (looking back and attributing success or failure to an individual based on results though the success was driven by other factors), but once or twice a year I come across a book that is invaluable. I just came across that book for 2020, Stanley McChrystal’s Team of Teams . The book initially appealed to me for several reasons:

Many of the most valuable lessons I have applied to business come from sports or war. McChrystal is one of the most successful military leaders in my lifetime.
The thought of a former General evangelizing less hierarchy, with him coming from one of the most hierarchical organizations in the world, was intriguing.
He does have a strong track record, he’s literally not an armchair general. Halo Effect aside, he is largely responsible for turning around the war effort in Iraq and defeated Al Qaeda there.
McChrystal’s challenges in Iraq are in many ways consistent with the challenges I have experienced in mobile gaming — having to adapt structures built to fight one enemy (the Russians) to fight something entirely different (Al Qaeda).
Reviews on both Amazon and Goodreads were not only positive but pointed to real life applications.

Team of Teams met and exceeded my expectations, providing a great framework for designing your business organization to meet modern day challenges, particularly the type faced by mobile and social game companies. The book shows how to build a fast, resilient and adaptive company. It also shows that building a great company is not about finding brilliant people but about creating the underlying structure. As McChrystal experienced, “although [his] Task Force struggled in Iraq, we could not claim we were mismatched against a world-class team. Honestly assessed, Al Qaeda was not a collection of supermen forged into a devilishly ingenious organization by brilliant masterminds. They were tough, flexible, and resilient, but more often than not they were poorly trained and under resourced. Much like a Silicon Valley garage start-up that rides an idea or product that is well timed rather than uniquely brilliant to an absurd level of wealth, AQI happened to step onto an elevator that was headed. Second, and most critically, these factors were not unique to Iraq, or to warfare. They are affecting almost all of us in our lives and organizations every day. We’re not lazier or less intelligent than our parents or grandparents, but what worked for them simply won’t do the trick for us now.”

As Walter Isaacson (an author I respect) wrote in the foreword, “whether in business or in war, the ability to react quickly and adapt is critical, and it’s becoming even more so as technology and disruptive forces increase the pace of change. That requires new ways to communicate and work together.” McChrystal describes the dynamics that have impacted the world, and business, and then how you can build your company to react to this new environment.

This isn’t our parent’s world

McChrystal highlights how the world has evolved and why companies need to adapt to survive and thrive. While Al Qaeda in Iraq (AQI) initially seemed to be a traditional insurgency, he found that it functioned differently than anything the US military previously faced. Rather than a traditional hierarchy, “it took the form of a dispersed network that proved devastatingly effective against our objectively more qualified force.” This experience is very similar to the new type of competitors many of us face.

AQI’s success was not simply a result of how they used new technology. It was their underlying structure, networked and non-hierarchical, that made them a dangerous enemy. It was not that they knew how to post on YouTube or communicate with messaging apps. McChrystal writes, “in some ways, we had more in common with the plight of a Fortune 500 company trying to fight off a swarm of start-ups than we did with the Allied command battling Nazi Germany in World War II.”

This new and unorthodox structure diverged radically from what the US military ever faced previously as it was “more connected, faster paced, and less predictable than previous eras.” This phrase sounded very familiar to challenges I have faced in gaming. To win, or even survive, McChrystal had to change. The change according to McChrystal, was “less about tactics or new technology than it was about the internal architecture and culture of our force — in other words, our approach to management.”

He had to change, and forget, what the military thought they knew about war and the world. They had to tear down familiar organizational structures they were comfortable with and build completely different lines and flows to deal with the new type of threat.

The key takeaway is that the situation was not unique to the military but is what companies are facing every day. In our world, game development now is entirely different than what game developers faced in the 1990s or 2000s. We also need to tear down our familiar organizational structure and reconstitute it along completely different lines, swapping our robust architecture for organic fluidity.

Complexity vs Complicated

The most important concept I got from the book is the difference between complexity and complicated and how that impacts your organizational structure.
Being complex is different from being complicated. Things that are complicated may have many parts, but those parts are joined, one to the next, in straightforward and simple ways. A complicated machine like an internal combustion engine might be confusing to many people but it can be broken down into a series of neat and tidy deterministic relationships.

Conversely, things that are complex, such as insurgencies or the mobile gaming ecosystem, have a diverse range of connected parts that interact regularly. McChrystal explains, “because of this density of linkages, complex systems fluctuate extremely and exhibit unpredictability. In the case of weather, a small disturbance in one place could trigger a series of responses that build into unexpected and severe outcomes in another place, because of the billions of tiny interactions that link the origin and the outcome.” In a complex system (like mobile games or insurgencies), it is often impossible to tell what events would lead to what results. Moreover, due to these dense interactions, complex systems exhibit nonlinear change .

I often like to bring Chaos Theory into conversations about the gaming space, not only due to the chaotic nature of many game companies, but because it points to the impossibility of predicting how changes in the environment could impact your development. McChrystal also references Chaos Theory, pointing to Lorenz’s butterfly effect (a butterfly in China flapping its wings can cause a hurricane in Europe). McChrystal writes that “the significance of Lorenz’s butterfly effect is not, however, just the nonlinear escalation of a minor input into a major output. There’s uncertainty involved; the amplification of the disturbance is not the product of a single, constant, identifiable magnifying factor—any number of seemingly insignificant inputs might—or might not—result in nonlinear escalation. If every butterfly’s fluttering always led to a hurricane halfway across the world two days later, weather would be predictable (if insane). The butterfly’s fluttering leads to a storm only if thousands of other minor conditions are just right. And those conditions are so precise as to be practically immeasurable, rendering the outcome unpredictable….The reality is that small things in a complex system may have no effect or a massive one, and it is virtually impossible to know which will turn out to be the case.”

The same technologies that help the military with enhanced transportation, communication, and data abilities concurrently fill the operating environment with escalating nonlinearity, complexity, and unpredictability. According to McChrystal, “speed and interdependence together mean that any given action in any given time frame is now linked to vastly more potential outcomes than the same action a century or even a few decades ago: endeavors that were once akin to a two-or three-ball pool problem now involve hundreds of collisions.”

This unpredictability is fundamentally incompatible with reductionist managerial models based around planning and prediction. There are too many events occurring simultaneously that no matter how big your Big Data is, you cannot monitor and process it all. The new environment demands a new approach.

The old approach

The most familiar structure and attitude that needs change is the focus and goal of efficiency. Since the Industrial Revolution, most business have pursued Scientific Management (developed by Frederick Taylor), a system that is excellent for achieving highly efficient execution of known, repeatable processes at scale. Scientific Management revolves around replacing working by common sense to studying work and determining the most efficient way to perform specific tasks. It also entails allocating the work between managers and workers so that the managers spend their time planning and training with workers focusing only on executing their tasks efficiently.

The military pursued the same practice, focusing on making the soldier more disciplined and efficient. McChrystal found, though, that while his task force in Iraq was eminently efficient it was no match for AQI. What worked for the military (and business) in the twentieth century was no longer enough. The reason it is not enough is that the world has evolved from complicated to complex.

The need to build resiliency

Another takeaway from Team of Teams was that given the complex environment it is imperative to build a resilient organization. Resiliency means you and your company accepts you will have to deal with unpredicted challenges and threats. You then build an organization and systems that can “roll with the punches” rather than erecting strong, specialized defenses. Resilient systems are those that can encounter unforeseen threats and, when necessary, put themselves back together again.

Creating resiliency requires admitting you do not know everything. McChrystal writes, “[r]esilience thinking is the inverse of predictive hubris. It is based in a humble willingness to ‘know that we don’t know and ‘expect the unexpected.’”

Resiliency puts managing complexity before managing for complication. You manage complication by creating robust systems that are strengthened on their weakest links. Resilience is the result of linking elements that allow them to reconfigure or adapt in response to change or damage, like a coral reef.

You must pivot away from seeing efficiency as the managerial holy grail. Instead, McChrystal writes “the key lies in shifting our focus from predicting to reconfiguring. By embracing humility — recognizing the inevitability of surprises and unknowns — and concentrating on systems that can survive and indeed benefit from such surprises, we can triumph over volatility…. In effect, we needed a system that, without knowing in advance what would be required, could adapt to the challenges at hand; a system that, instead of converting a known x to a known y, would be able to create an unknown output from an unpredictable input.”

Building adaptability

A key to building resiliency is building adaptability. According to McChrystal, “one can make contingency plans, but these can account for only a modest number of possibilities. A contingency plan is like a tree that branches at every variable outcome (if they fire when we arrive, choose path A, if not, choose path B). But when dozens of saplings shoot out from those branches every second, the possibilities become so overwhelmingly complex as to render complete contingency planning futile.”

To create adaptability you need to focus on teams. Fundamental structural differences separate commands from teams. Command structures are rooted in reductionist prediction, and very good at executing planned procedures efficiently. Teams are less efficient, but much more adaptable.

McChrystal the navy SEALs as a model team. SEALs are widely considered one of the most effective fighting units in the world. McChrystal writes, “SEAL teams accomplish remarkable feats not simply because of the individual qualifications of their members, but because those members coalesce into a single organism. Such oneness is not inevitable, nor is it a fortunate coincidence. The SEALs forge it methodically and deliberately…. The first step of this is constructing a strong lattice of trusting relationships. This will seem intuitive to anyone who has been on a team, but it runs against the grain of reductionist management; in a command, the leader breaks endeavors down into separate tasks and hands them out. The recipients of instructions do not need to know their counterparts, they only need to listen to their boss. In a command, the connections that matter are vertical ties; team building, on the other hand, is all about horizontal connectivity.,,, The formation of SEAL teams is less about preparing people to follow precise orders than it is about developing trust and the ability to adapt within a small group.”

To create an effective team you need to connect trust and purpose. Teams overcome challenges that could never be foreseen by a single manager, their solutions often emerge as the bottom-up result of interactions, rather than from top-down orders. According to McChrystal, “while building trust gives teams the ability to reconfigure and ‘do the right thing,’ it is also necessary to make sure that team members know what the right thing is. Team members must all work toward the same goal, and in volatile, complex environments that goal is changeable…. Purpose affirms trust, trust affirms purpose, and together they forge individuals into a working team.”

Teams do not become effective without effort. The establishment and continuation of a team requires both active management and the invisible hand of emergence, zigzagging the elements together and guiding their work. McChrystal writes, “parallel computing, joint cognition, and the oneness of a team all work toward the same goal: building a network that allows you to solve larger, more complex problems.”

The need for a Team of Teams structure

While trust and purpose is critical to creating effective teams, most mature businesses are too large to be one team. This problem often leads to a command structure telling each team what to do, which negates the resiliency and adaptability of teams. While the teams may be adaptable, a command superstructure will limit the overall organization. McChrystal writes, “[i]n a response to rising tactical complexity, many organizations in many domains have replaced small commands with teams. But the vast majority of these organizations have to be much larger than a single team; they consist of multiple teams, and these teams are wired together just like a traditional command.” In his case, “stratification and silos were hardwired throughout the Task Force.”

While teams bring some adaptability to previously rigid organizations, these performance improvements have a ceiling as long as adaptable traits are limited to the team level. As the world grows faster and more interdependent, you need to scale the fluidity of teams across entire organizations.

I love sports analogies as much as military ones in business, and McChrystal uses a sports one to exemplify this problem. He writes, “picture a MECE [mutually exclusive and collectively exhaustive] structure with clear roles and responsibilities sports team, and you’d have a ridiculous spectacle: players ignoring one another and the ball, their eyes fixed on the coach, awaiting precise orders. A coach might be able to devise a more efficient way to execute any given play than whatever it is the players would improvise in the heat of the game. But the coach has no way of predicting exactly how the game will develop, and no way of effectively communicating instructions in real time fast enough to be useful to all players simultaneously…. The team is better off with the cohesive ability to improvise as a unit, relying on both specialization (goalies mostly stay in goal; forwards mostly don’t) and overlapping responsibilities (each can do some of the others’ jobs in a pinch), as well as such familiarity with one another’s habits and responses that they can anticipate instinctively one another’s responses.”

The solution McChrystal devised was a “team of teams” (hence the name of the book), an organization within which the relationships between teams resembled those between individuals on a single team. Teams that had traditionally resided in distinct silos fused to one another via trust and purpose.

Creating a shared consciousness

McChrystal dealt with a larger organizational challenge than most of us will face in linking his teams, as he was responsible for thousands of soldiers across the world. To create his team of teams, he looked at a similarly complex situation that had great results, NASA’s ability to put the first man on the moon. NASA also had hundreds of very diverse teams operating at multiple locations. According to McChrystal, “because of the interdependence of the operating environment, both organizations would need members to understand the entire, interconnected system, not just individual MECE boxes on the org chart.”

To achieve this team of teams, both NASA and McChrystal’s task force had to create unprecedented levels of transparency and information sharing. It demanded a disciplined effort to create shared consciousness.

Each individual team took pride in its own performance, like the striker who celebrates his goals while his team consistently loses. The silos of the organization looked inward, where they could see metrics of success and failure. He needed to change the definition of success, not measuring the team individually but on the overall success.

McChrystal also had to foster information sharing. Both at NASA and the Army, information was closely guarded on a “need to know” basis. The problem is this approach is it depends on the assumption that some manager or algorithm or bureaucracy actually knows who does and does not need to know which material. The team of teams had access to virtually everything, it was not pre-ordained who would see what.

Functioning safely in an interdependent environment also requires that every team possess a holistic understanding of the interaction between all the moving parts. Everyone has to see the system in its entirety for the plan to work. McChrystal writes, “NASA’s success illustrated a number of profound organizational insights. Most important, it showed that in a domain characterized by interdependence and unknowns, contextual understanding is key; whatever efficiency is gained through silos is outweighed by the costs of ‘interface failures. It also proved that the cognitive “oneness” — the emergent intelligence — that we have studied in small teams can be achieved in larger organizations.”

Ensuring communications flow by embedding people on other teams

As the above shows, to achieve a team of teams there needs to be team like cooperation across your business. To achieve fluid, team like cooperation, you need to build inter-team trust. One way is to embed people from one team in another team. This helps build strong lateral ties between internal and external teams.

Where systemic understanding mirrors the sense of purpose that bonds small teams, this forced mating generates trust. McChrystal recounts the story of a SEAL who worked with his force in Iraq, according to the SEAL ‘When we started constantly talking at lower levels of the organization we could basically see where the fight was hot, where it wasn’t, and where people needed ISR the most. Plus, we could see that it was actually to our benefit sometimes to surrender that asset.’ With that awareness came a faith that when theirs was the priority mission, they would get what they needed when they needed it. Holistic understanding of the enterprise now permeated the ranks.”

By embedding people on other teams, you create idea flow.
Idea flow is the ease with which new thoughts can permeate a group, like a virus, which is a function of susceptibility and frequency of interaction. The key to increasing the contagion is trust and connectivity between otherwise separate elements of an establishment. The two major determinants of idea flow are engagement within a small team exploration, frequent contact with other units. As McChrystal writes, “in other words: a team of teams.”

Less command and less control

One of the things I found most appealing in McChrystal’s book is that his ideas run counter to the traditional beliefs of the military. That resonated with me as it shows how we need to adapt these dogmas everywhere to succeed in the 21st century. Nowhere is this more apparent than command and control.

While almost everyone acknowledges that the world has changed, many managers and leaders reflect a model and style that out of date. We often demand unrealistic levels of knowledge in leaders and force them into ineffective attempts to micromanage. McChrystal points out “the temptation to lead as a chess master, controlling each move of the organization, must give way to an approach as a gardener, enabling rather than directing…. A gardening approach to leadership is anything but passive. The leader acts as an ‘Eyes-On, Hands-Off’ enabler who creates and maintains an ecosystem in which the organization operates.”

To achieve this role, senior leaders are more important than ever, but the role is very different from that of the traditional heroic decision maker. In McChrystal’s case, he said “I needed to shift my focus from moving pieces on the board to shaping the ecosystem…. Creating and maintaining the teamwork conditions we needed – tending the garden – became my primary responsibility…. I found that only the senior leader could drive the operating rhythm, transparency, and cross-functional cooperation we needed. I could shape the culture and demand the ongoing conversation that shared consciousness required.

Leading as a gardener meant that I kept the Task Force focused on clearly articulated priorities by explicitly talking about them and by leading by example.”

The actions and behavior of an effective leader in a team of teams is very different than what is required in a traditional command system. McChrystal identified several attributes and actions critical to success:

Leading by example, he found his “most powerful instrument of communication was [his] own behavior”
Showing focus and commitment, which he did on daily calls by wearing his combat uniform against an austere plywood backdrop
Demanding free-flowing conversation across the teams during their daily meeting
Never cancelling the daily meeting and making attendance mandatory
Showing interest in what everyone said during meetings, never looking bored, sending emails or talking
Greeting everyone, regardless of rank, by their first name (and ensuring he knew their name going into the meeting)
Display rapt attention when being briefed and, at the conclusion, asking a question
Realizing that critical words were magnified in impact and could be crushing
Asking seemingly stupid questions or admitting openly “I don’t know” was accepted, even appreciated. Asking for opinions and advice showed respect.
Nonstop communications and visits, from town halls to larger groups. Important that everyone on the teams heard directly from him.

The end product

Creating a team of teams is not an overnight product but a transformation that requires changing both your thinking and your structure. It is also a process where shortcuts not only risk the outcome but can make the effort negative. For example, if you empower your teams without creating shared consciousness you are likely to have people driving to different goals.

Similarly, shared consciousness alone is powerful but ultimately insufficient. Building holistic awareness and forcing interaction will align purpose and create a more cohesive company, but will not unleash the full potential of the organization. You need to use shared consciousness to spread information and empower people at all levels.

As you transform your business, both your speed and precision should improve. Technology should enable your success if the culture change allows you to use it properly.

McChrystal writes, “At the core of the Task Force’s journey to adaptability lay a yin-and-yang symmetry of shared consciousness, achieved through strict, centralized forums for communication and extreme transparency, and empowered execution, which involved the decentralization of managerial authority. Together, these powered our Task Force; neither would suffice alone…. As complexity envelops more and more of our world, even the most mundane endeavors are now subject to unpredictability, and we can learn from those at the vanguard…. Our transformation is reflective of the new generation of mental models we must adopt in order to make sense of the twenty-first century. If we do manage to embrace this change, we can unlock tremendous potential for human progress.”

Key takeaways

The difference between complexity and complicated is central to building an organization that works in the 21st century. Things that are complicated may have many parts, but those parts are joined, one to the next, in straightforward and simple ways and are like an equation that needs to be solved. Things that are complex, such as insurgencies or the mobile gaming ecosystem, have a diverse range of connected parts that interact regularly.
Accept that you will have to deal with unpredicted challenges and threats. You then build an organization and systems that can adapt rather than erecting strong, specialized defenses.
Teams, while not always optimally efficient, are extremely adaptable. Teams overcome challenges that could never be foreseen by a single manager, their solutions often emerge as the bottom-up result of interactions, rather than from top-down orders. Your teams must then interconnect into a team of teams, rather than siloed organizations. Create an organization within which the relationships between teams resembled those between individuals on a single team.

The difference between great executives and everyone else

I came across a great blog post on The Difference Between Amateurs and Professionals and I found it a perfect proxy for highlighting what separates effective and great executives from everyone else.

It is impossible to overstate the value of a great executive, as each of their decisions is amplified over the entire company. I have seen leaders destroy companies, and in so doing cost many people their jobs and health, and have seen others grow companies to unexpected heights, not only creating billions in value but also jobs and other opportunities. Below are the key differences between amateur and professional athletes that also can be applied in the business world.

Amateurs focus on being right. Professionals focus on getting the best outcome.

The best business leaders are focused on delivering great outcomes (successful new products, profits that exceed expectations, gaining market share, etc.) and do not care if they hit these outcomes because of a “brilliant” strategy they came up with or a process that was built without their input. They are focused on results, not sound bites that sound impressive.

Amateurs think good outcomes are the result of their brilliance. Professionals understand when good outcomes are the result of luck.

Related to the above point, successful executives understand that luck plays a role in success (and failure). It comes down to Heisenberg’s Uncertainty Principle, the universe is uncertain and there is a range of likely outcomes. Given that there are multiple outcomes from any action, luck plays a (big) role into whether the outcome is good or bad no matter the choice (over time, though, you will have a big impact on the set of outcomes if you regularly pursue the ones most likely to yield a positive result). Conversely, sometimes you will make bad decisions and have a great outcome, a top executive will understand they got lucky and not feel they are brilliant.

Amateurs show up to practice to have fun. Professionals realize that what happens in practice happens in games.

This is one of the lessons I have learned from sports that is critical to business success. It is also called practice how you want to play. People will often take shortcuts when they feel it will not be in the final product, when they are preparing marketing collateral, when they are rehearsing conversations with VIPs, etc., but by not focusing at this stage the final product is often not polished. They may not spend much time preparing for a meeting, presentation or sales call. By practicing how you want to play, you and your team are always prepared to deliver an optimal performance.

Amateurs focus on tearing other people down. Professionals focus on making everyone better.

Only insecure business people feel they have to belittle their colleagues and employees. They tear people down in an effort to elevate themselves; they think they will look better in comparison if their colleagues look bad. Conversely, great executives are focused on making everyone around them better and realize they cannot perform well if their colleagues do not.

Amateurs blame others. Professionals accept responsibility.

The best executives follow Truman’s doctrine that “The Buck Stops Here.” Even if an executive was not directly responsible for a failure, the successful ones take on that responsibility and determine ways to avoid it in the future. Allocating blame does not get your business closer to its goal; what does is accepting blame and coming up with next steps does. Top executives also never blame their team, they own any mistake their team makes because they failed to put the processes, training or direction in place to avoid the mistake.

Amateurs give up at the first sign of trouble and assume they’re failures. Professionals see failure as part of the path to growth and mastery.

Rather than avoid failure, successful executives pursue it. Not that they want any initiative to fail, but embracing failure ensures they are pushing the limits to grow and improve their business. If you never fail, you are never trying anything new.

Amateurs focus on identifying their weaknesses and improving them. Professionals focus on their strengths and on finding people who are strong where they are weak.

This is a lesson I have stressed with both my children and teams. If you focus on eliminating your weaknesses and trying to do everything at a passable level, you are not doing anything special. Mike Trout does not spend two hours a day working on pitching, instead he takes hours and hours of batting practice even though he is the best hitter in baseball. You are not bringing anything unique that will elevate your company to greatness. Instead, looking at where you excel and double down on it, use your super-powers to elevate your company above the competition. Then bring in others to compensate for your weak points, it is a show of strength to compensate for weaknesses by adding complimentary pieces to your team.

Amateurs think the probability of them having the best idea is high. Professionals know the probability of that is low.

The most effective executives do not try to be the smartest person in the room. While to amateurs it may feel that the strategy of highlighting their intelligence is important for success, the opposite is true. Top executives focus on trying not to be the smartest person in the room and harness the power of tens or hundreds of colleagues, getting better input, more creative ideas and improved team performance.

Amateurs think in absolutes. Professionals think in probabilities.

As discussed above, the universe is uncertain and there is a range of likely outcomes. Rather than trying to get every decision right, what great executives do is maximize their expected value (the sum of all possible values each multiplied by the probability of its occurrence) by playing the odds. While great leaders will have some wins and losses, over time by focusing on the best expected outcome (not one they are sure will happen) their company will outperform competitors.

Amateurs stop when they achieve something. Professionals understand that the initial achievement is just the beginning.

I am not a big fan of celebrations because you should always strive for more. Business is not finite, you do not win and go home. Instead you want to continue growing. Even in sports, I remember reading how the Patriots coach watched film the night he won the Super Bowl to prepare for the next season.

Amateurs have a goal. Professionals have a process.

The New England Patriots are a perfect example of this principle. What it comes down to is building a good process to ensure continued growth and success, whether the goal is hitting revenue targets or winning football matches. If you have a system that can weather external shocks you will achieve this goal rather than spending your team in a bar rationalizing why you failed and your competitors succeeded.

Amateurs focus on the short term. Professionals focus on the long term.

Being a great business executive is not about having a blow out quarter or a good product launch, it is about showing sustained success. Like in chess, do not focus on the next move or the next five moves but the overall strategy that will help you consistently outperform (or avoid) the competition.

Putting it all together

A great business leader does not only follow these practices themselves, but pushes everyone around them to also embrace these concepts. Bill Belichick is not one of the greatest of all time simply because he followed these concepts but because he forced his coached and players to live by these beliefs.

Key takeaways

There are many similarities between what separates a professional athlete from an amateur and what separates a great business executive from everyone else.
The best executives focus on getting the best outcome, not being right.
The best executives accept responsibility and do not blame others.

How to manage your customer service to have the biggest impact on your profits

I have spent the last few months studying how to provide an exceptional customer experience, largely as a way to help grow gaming businesses. This learning has helped me identify best practices in creating a WOW experience but my perception of the opportunity changed after reading The Effortless Experience by Matthew Dixon. Unlike many of the other customer service (CS) books that focus on how to provide exceptional experience, Dixon’s book is data driven and challenges some basic assumptions.

In particular, his work shows that exceptional service does not drive the KPIs that matter (revenue and loyalty). Instead, these engagement KPIs are actually most impacted by how much effort a customer has to expend to achieve their goals while high effort translates to disloyalty. As the ultimate goal is to increase lifetime value (LTV) of your customers, and retention is the biggest driver of LTV, limiting disloyalty will have a much bigger impact on your profitability than creating WOW experiences. Dixon writes, “[c]ustomer service drives disloyalty, not loyalty. The average service interaction is four times more likely to make a customer disloyal than to make them loyal…. While most companies have for decades been pouring time, energy, and resources into the singular pursuit of creating and replicating the delightful experience for their customers, they’ve ironically missed the very thing customers are actually looking for—a closer-in, more attainable, replicable, and affordable goal that’s been sitting right in front of them all this time: the effortless experience.”

The beauty of The Effortless Experience is that it is data driven, not anecdote driven. Dixon analyzed more than 97,000 customers and found that there is virtually no difference between the loyalty of those customers whose expectations are exceeded and those whose expectations are simply met. He defined loyalty by measuring three behaviors

Repurchase (customers continue to buy from your company)
Share of wallet (customers buy more from you over time)
Advocacy (customers say good things about your company to family, friends, coworkers, even to strangers).

When Dixon looked at data from the 97,000 customers, “rather than a ‘hockey stick effect’ — where loyalty skyrockets upward — loyalty actually plateaus once customer expectations are met.”

Why customer delight does not work

Since the data shows that great service is not going to create better loyalty, and thus improve your customer lifetime value (LTV), the next step of analysis is how customer service can improve LTV. What Dixon found is that a customer service interaction is four times more likely to drive disloyalty than to drive loyalty, so you want CS to avoid leading to churn.

This data driven insight resonated upon reflection with my personal experiences. I could think of many companies I stopped purchasing from due to a bad or disappointing CS experience (online retailers, restaurants, department stores and even a car company) but it is very hard to think of one I am loyal to because their service went above and beyond what I expected. Dixon writes,“you could probably fill up a list a mile long with companies you’ve stopped doing business with because of the bad service you’ve experienced: the cable company that makes you take a day off from work because they can’t do better than an all-day service window, the dry cleaner that ruined your favorite suit and refuses to reimburse you.”

What customers want

While customer service cannot make customers more loyal by providing incredible experiences generally, it can help retention by solving customers’ problems. Dixon’s data shows that when something goes wrong, “the overriding sentiment is: Help me fix it. No need to dazzle me, please just solve the problem and let me get back to what I was doing before.”

Reinforcing this finding, Dixon’s data shows that 94 percent of customers who had low-effort experiences reported that they would repurchase from the company, but only four percent of customers experiencing high-effort interactions planned to make another purchase. What is more, 88 percent of customers with low-effort experiences planned to increase spend with the company, while only four percent of customers with high-effort experiences were going to increase spend. This phenomenon extends to word of mouth, where only one percent of all customers with low-effort experiences said they would spread negative word of mouth about the company but 81 percent of customers with high-effort experiences said they would spread adverse sentiment.

A strategy focused on customer delight does not work because delighting customers is rare, and even when delight does occur, it does not make customers much more loyal than simply meeting their expectations does. It does not work because customer service interactions are four times more likely to drive disloyalty than loyalty. Operationally, since delight does not work you should focus your resources, investments, performance metrics, and incentives on reducing and eliminating the sources of customer effort that make customers disloyal.

Where to focus your customer service strategy

As the data shows customers are not looking to be delighted, you should focus your customer service on fixing things when they go wrong. Rather than trying to get customers to feel you exceeded their expectations, you should be getting customers to think you made it easy to resolve their issue and avoid follow-ups.

To achieve this goal and reduce the effort (and perceived effort) your customer must exert, Dixon identified four best practices:

Low-effort companies minimize channel switching by boosting the “stickiness” of self-service channels, avoiding customers contacting CS in the first place.
When customers are forced to Live Chat or email or call, low-effort companies do not just resolve the current issue for a customer; but power their reps to head off the potential for subsequent calls. This is done through employing next issue avoidance practices. Low-effort companies understand that first contact resolution is not the goal —
it is only a step in the direction toward more holistic, event-based issue resolution.
Low-effort companies train their CS team to succeed on the soft side of the service interaction. Rather than soft skills that are about being nice and friendly, agents are trained to actively manage the customer interaction through experience engineering (I will elaborate later).
Companies that understand the Effortless Experience empower their agents to deliver a low-effort experience with incentive systems that value the quality of the experience over speed.

Even if you understand the value of a low-effort experience, many companies fail because they try to focus on everything, creating exceptional as well as delivering low-effort. As I have previously written, when there are multiple goals you fail to achieve any. Thus, you need to align your CS org on providing the effortless experience. If you are correctly optimizing for loyalty, you need to focus on finding ways to eliminate or reduce the hassles, hurdles and extra customer effort that leads to disloyalty.

KPIs, good and bad

To implement and optimize a successful strategy, you need to understand and track the KPIs that lead to the desired outcome. In the world of customer service, many of the commonly used KPIs and perceived best practices do not correlate with an effortless experience of increased loyalty. Dixon found virtually no statistical relationship between how a customer rates a company on a satisfaction survey and their future customer loyalty. He also looked at arguably the most common customer service KPI, CSAT score, and found that the data shows a strong CSAT score is not a very reliable predictor for whether customers will be loyal: whether they will repurchase, increase spend and say good things about your company to friends, family, and coworkers.

Dixon points out that it is unfair (and useless) to ask customers if their issue is fully resolved. They will not know that they need to start another ticket in a day or a week. Repeat contacts are by an order of magnitude the single biggest driver of customer effort. Having to contact a company again because an issue was not fully resolved is a customer experience killer and quite expensive.

To highlight how traditional CS measures fail, Dixon also uses an example of a customer whose problem was fully resolved by a rep who went above and beyond (that sounds great, but it tests relatively neutral for increased future loyalty). Unfortunately, this was the second time the customer called about that issue (a huge negative). If you were listening to this call, you would have to conclude, “We did a great job there.” But since it took the customer two tries to get to that moment, and knowing the huge negative impact that repeat contacts have on the customer experience, this person is still very likely to end up more disloyal. Thus it means there is less of a chance the customer will repurchase, less of a chance they will spend more, and a greater chance that they will say negative things to other people, despite the fact that the agent who eventually solved his problem went above and beyond. In this example, if you simply classify this experience as positive, that does not indicate whether it benefited or hurt your company.

The core metric to measure effectively whether your team is delivering an effortless experience is CES (customer experience score). When Dixon compared CES to CSAT, he found CES was 12 percent more predictive of customer loyalty. The most recent CES metric is based on a statement, “the company made it easy for me to handle my issue,” after which the customer is asked to answer (on a 1–7) whether they agree or disagree with that statement. You should then review CES against a normal distribution (10–20 percent of interactions would score as very high-or very low-effort, but most would be somewhere around the mean). Looking at the distribution to understand areas of opportunity can be far more helpful than just considering how your average CES compares to competitors.

While CES should be at the core of measuring your CS efforts, a robust customer effort measurement system includes three components. First, at the top of the pyramid, you want to understand the customer’s overall loyalty to you as a company, I am a big proponent of using NPS.

Following NPS you want to understand the amount of effort in the service transaction. As discussed above, CES is a good way to measure that. Dixon also recommends “that companies cross-check their CES results by looking at some of the operational data that underpin effort& #8212 for instance, number of contacts to resolve an issue.” CES provides a formidable indicator of transactional customer loyalty, clearly highlights friction points in the customer experience and helps you spot customers at risk of defection due to high-effort service interactions.

According to Dixon, “the next level down is to understand how the customer’s service journey unfolds — in other words, the number and type of touchpoints they used to resolve their issue, in what sequence those service touchpoints occured (e.g., did the customer just call the contact center or had they first visited the web site?), and the discrete customer experience within each channel (for example, assessing the clarity of information delivered by a service rep or the ease of finding information on the web site).”

Key tactics to achieving an effortless experience

While Dixon provides compelling evidence in in the value of creating an effortless experience and there are KPIs that can deliver an understanding of how well you are doing, the key to success (in everything) is execution. To create an effortless experience, you need to minimize channel switching, avoid repeat contacts, appropriately “engineer” the customer service interaction experience, build the control quotient, create the right culture and optimize the purchase experience.

The danger of channel switching

Channel switching is when a customer initially attempts to resolve an issue through self-service, only to have to then send an email or pick up the phone and call, and it has a disastrous impact on customer loyalty. Each time a customer switches channels, it has a significant negative impact on customer loyalty.

One of the core issues leading to channel switching is that many companies, mistakenly, believe they know how customers want to be served, but they are wrong. Customers do not prefer high touch interactions (Live Chat or phone), they see just as much value in self-service as they do in phone interactions. Dixon found that executives, however, expected customers to prefer phone: a 2.5-to-1 margin in favor of phone service.

Customers who attempt to self-serve but are forced to pick up the phone are ten percent more disloyal than customers who can resolve their issue in their channel of first choice. 58 percent of customers who are forced to switch from web to phone and fall into the “lose-lose” scenario costing companies more to serve and end up being less loyal as a result.

To avoid channel switching, you need to create a robust self service system. Dixon writes, “the more controllable drivers of channel switching (47 percent in B2C settings and 37 percent in B2B) can be categorized into three groups: The customer couldn’t find the information they needed. The customer found the information, but it was unclear. The customer was simply using the web site to find the phone number to call the company.”

As I have written frequently, more choices often results in lower satisfaction or performance (choice overload) and it is also a problem when designing customer service. Based on Dixon’s review of CS data, it “became clear that the variety of options to resolve an issue — all of which were presumably added in an attempt to improve the customer experience — were actually detracting from it. It’s an illustration of what’s known as “the paradox of choice”….[C]hoice is not nearly as powerful as we might have expected. Instead, guiding customers to the pathway that will require the least amount of effort is much more likely to mitigate disloyalty and create the best experience.”

Another cause of channel switching is when customers do not understand the self-service information. The goal is not simply to get customers to try self-service, it is about getting them to stay in self-service. Dixon suggests several ways to mitigate this problem:

Simplify language. Rather than being creative or trying to show how sophisticated your company is, write copy in a way that is very simple and easy to understand. A good way to check your copy is the Gunning Fog Index, your text should score an 8 or 9.
Eliminate null search results. Look for customer searches that yielded no responses as well as low-relevance searches. You will probably find that customers often use different words than what you used
Chunk related information. Chunking is condensing related information and spacing it apart from other text, allowing readers to scan content easily.
Avoid jargon. Many companies use phrases and words common to the company or industry but unknown to the customer (in the casino space, a word like “hold”). Scan your web site pages and FAQs carefully for internal jargon, industry lingo, and terms that would generally confuse the average customer.
Eliminate what is not vital. Most service sites fail not because they lack functionality and content, but because they have too much of it. Dixon argues, “the key to mitigating channel switching is simplifying the self-service experience.”

Avoid repeat contacts

The second key to creating an effortless customer experience is helping your customers avoid repeated contact with customer service. Repeat contacts are the single biggest driver of customer effort (and it is not even close). Needing to call a company back or send another email or start a new live chat because an issue was not fully resolved is a customer experience killer and hugely costly.

In Dixon’s research, he found a huge disconnect from what companies were judging as resolved and what customers experienced. Customers reported, on average, that only 40 percent of their issues are resolved in the first contact, which meant that an additional 30–40 percent of issues in which customers would disagree with the companies’ assessment that the problem was solved. This disconnect is driven by the company fixing the explicit issue but leaving implicit issues.

There are two main types of implicit issues driving these repeat contacts that companies are missing. The first are adjacent issues, which according to Dixon’s data account for 22 percent of all callbacks. These are downstream issues that might initially seem unrelated, but are ultimately connected to the first thing the customer called about. The second major source of repeat contacts is experience issues, which constitute 24 percent of all repeat contacts. These are primarily emotional triggers that cause a customer to second-guess the answer they were given, or double-check to see if another answer exists.

To mitigate this issue, you should focus on next issue avoidance. Next issue avoidance starts with a totally different mind-set than simply asking a customer if you resolved their issue. Agents are trained and coached to ask themselves, “How can I make sure this customer doesn’t have to call us back?” Simply letting a customer know that you are trying to save them from having to call back later and deal with another related issue goes a remarkably long way.

The fundamental difference when applying this tactic is you are not looking to simply solve the current issue, but also head off the next issue. According to Dixon, “the best companies think of issues as events, not one-offs, and teach their reps to forward-resolve issues that are related to the original issue but typically go unnoticed by the customer until later.”

Experience engineering

The third element of creating an effortless customer experience is “experience engineering.” Experience engineering refers to managing a conversation with carefully selected language designed to improve how the customer interprets what they’re being told. At its core, experience engineering reflects the importance of perception.

Dixon found “that the customer’s perception of the experience actually counts for fully two-thirds of the overall ‘effort equation.’ Put differently, how the customer feels about the interaction matters about twice as much as what they actually have to do during the interaction.” The exertion required from the customer makes up only 34.6 percent of how they evaluate customer effort. But the interpretation side — the softer, more subjective elements based entirely on human emotions and reactions — make up a shocking 65.4 percent of the total impact.

Many customer interactions that do not require a lot of exertion still feel like a lot of effort to customers. Also, most companies have been strategizing about how to reduce customer effort without focusing on the customer’s perception of the experience. Instead, Dixon proposes a three step process to engineer successfully the customer experience so they do not perceive high effort:

First, experience engineering is purposeful. It’s about actively guiding the customer and taking control over the interaction through a series of deliberate actions.
Second, experience engineering is designed to anticipate the emotional response of the customer. Agents who engage in experience engineering are trying preemptively to offer solutions that the customer will find agreeable.
Third, an experience engineering approach is focused on finding a mutually beneficial resolution to customer issues. This means matching the customer’s actual and often unstated needs with what the company can offer.

Following these steps a can significantly change outcomes, by preempting a high-effort interpretation and getting the customer to feel like it was very little effort. Dixon writes that, “reducing the interpretation of effort, particularly in situations where there’s nothing else that can be done to reduce exertion, is the ultimate win-win-win—best for the customer, best for the company, and best for the individual reps who are in the hot seat delivering bad news on a daily basis….Reps need to find a way to both be truthful (because the answer in many cases is, unfortunately, still no), but in a way that doesn’t trigger the negative emotional reaction and all the bad outcomes that come along with it.”

Delivering this bad news can be alleviated with the use of positive language. Dixon writes, “In their first attempt at positive language, many people struggle: ‘Uhhhhh, the park closes whenever the magic stops.’ (No, the park actually closes at 8 p.m.) ‘The park closes whenever you leave.’ (No, if you’re still here at 8: 01, you’ll probably get some Disney version of the bum’s rush.) Ultimately, the most correct answer is some version of, ‘The park remains open right up until 8 p.m. Then we reopen for even more fun tomorrow morning at 9 a.m. Hope you can join us then!’ How could a customer possibly have a negative reaction to that?”

To drive experience engineering, you should start by analyzing your highest-volume incoming customer requests and see how your agents responded if the customer was not going to get what they wanted. You want to train the agent so they act as the customer’s advocate, the person who’s on the customer’s side and doing everything to make it an easy, low-effort experience. An example Dixon uses is a restaurant server being trained to respond to a customer who wants a Coke while the restaurant does not serve it, responding, “I’m happy to get you a Pepsi.”

The salient point is for the agent not to be fast with the saying no. Suggesting an alternative is better than immediately sharing what is not available.

Another element of successful experience engineering is learning more about your customer so you can “engineer” their experience. Rather than putting a customer on hold or asking the customer to wait during a Live Chat while they look for the answers to the customers questions, agents can strategically use those moments when looking at their screen for information as an opportunity to learn something about that customer and their needs that could become useful later in the conversation.

By listening to the customer, agents can categorize them. For example, you may segment your CS requesters into four categories: The Feeler, who leads with their emotional needs. The Entertainer, who loves to talk and show off their personality. The Thinker, who needs to analyze and understand. The Controller, who just wants what they want, when they want it. When a customer feels that the agent they are interacting with understands them, a lower-effort experience is much more likely.

Giving different experiences to different people based on what the agent has learned (or the company has learned previously) moves from the target from consistent service where the customer service management team defines what “good” is and then expects all frontline reps to conform to this standard, to consistently tailored service, where each individual customer is treated individually. Dixon writes, “it cannot be accomplished simply by telling employees what to do in every situation. It is readily apparent that this notion of consistently excellent service will require a serious rethink about how to manage customer service employees.”

Build the Control Quotient

Another element of creating an effortless experience is giving your agents control, what Dixon refers to as the Control Quotient (CQ). Judgment and control differentiate today’s best agents. With increasingly complex live service (phone, Live Chat, real time email, etc) and heightened customer expectations due to simple issues being resolved in self-service, the most important competency for reps to possess is CQ. CQ is the ability to exercise judgment and maintain control in a high-pressure, complex service environment.

Dixon three distinct keys to unlocking CQ that are within the control of customer service leadership to enable:

Trust in agent judgment
Agent understanding and alignment with company goals
A strong agent peer support network

According to Dixon, “these three factors — with all other things being equal — are the difference makers that transform average organizations into world-class low-effort service providers….Frontline reps are made to feel that they are free to do whatever is right to serve that one customer they are interacting with right now.

Having high CQ is necessary to achieve consistently excellent service because you cannot have great service by treating all customers the same. Standardized service cannot be great because all customers are not the same. Customers have different personalities, different needs and different expectations. Their ability to understand and verbalize their problems and issues is also very different. Dixon explains, “when a company mandates that every customer call include all the standard, company-imposed criteria, and takes away the rep’s ability to deal with the customer at a more natural, spontaneous, human level, the interaction is reduced to a mechanical, rote exchange.”

While CQ is the greatest differentiator of rep performance, the reality is that most agents have moderate to high CQ potential. Instead of training effort reduction, you can coach it. Although training is helpful for building awareness, effort reduction involves frontline behavior change that can only be delivered and sustained through effective frontline supervisor coaching.

One issue is that many companies inhibit reps from exercising CQ due to the environment of strict adherence they have created and reinforced historically. Judgment and control are not welcomed in these environments. Dixon suggests you “give control to get control of the front line. To allow reps to activate their latent CQ potential, companies need to demonstrate trust in rep judgment. Approaches include deemphasizing or eliminating handle time and the QA checklist, clarifying reps’ alignment between what they do and what the company is trying to achieve, and allowing reps to tap into the collective experience and knowledge of their peers to make smart decisions.”

Only by coaching and empowering your agents will you reduce the effort your customer experiences when dealing with customer service.

Create the right culture

Another key to improving customer service by creating an effortless experience is creating the appropriate culture. You need to create a clear contrast between old and new behavior. Then explain to your team how and why an effort reduction approach differs from the current service philosophy. Given the power of stories, use a change story to continually reinforce why teams need to focus on effort reduction, what’s at stake, and the nature of support they’ll be provided.

In my experience, transforming your customer service approach cannot be another flavor of the day project. You should not make effort reduction another ask. If you are just adding effort reduction to a long list of requirements, it will signal a lack of commitment and competing priorities. Instead, remove requirements such as handle time or strict QA forms to allow pilot teams truly to focus on reducing customer effort, helping your team determine the right ways to change behavior.

You also must make effort reduction easy. Dixon writes, “asking reps to ‘go out and reduce effort’ without a clear sense of where and how will surely be met with failure and confusion.” Instead, start with a pilot with clear tactics and goals. This may include forward resolving a specific type of service issue, or using positive language techniques for a small number of common issues. Finally, provide heightened support and coaching, as pilot teams get comfortable with these approaches.

Purchase experience

The final element of creating an effortless experience is around the purchase journey. Dixon’s research showed that reducing customer effort in pre-and post-sales customer touchpoints has a strong impact on loyalty. The ease that customers can learn about products or services, make a purchase, and obtain post-sales service and support provides a dramatic opportunity for brand differentiation.

An effortless experience is the recipe for increasing LTV

Reorienting your customer service from creating great experiences and cool stories to reducing customers’ effort feels counter-intuitive and is not easy but the data is impossible to argue with. Your goal should not be to have a customer service experience you can feel good about but one that improves loyalty, and the effortless experience will have the biggest impact on loyalty (and thus LTV).

Key takeaways

Data shows that trying to create an exceptional customer experience has virtually no impact on loyalty and engagement, however, reducing the effort the customer must exert does improve loyalty,
The best way to measure this effort is CES score, which is based on a statement, “the company made it easy for me to handle my issue,” after which the customer is asked to answer (on a 1–7) whether they agree or disagree with that statement.
The keys to implementing successfully an effortless experience program are minimizing channel switching, avoiding repeat contacts, engineering the customer service interaction experience, building the control quotient, creating the right culture and optimizing the purchase experience.

Interview with the Queen of Research, Maria Cipollone

Several weeks ago, I wrote a blog post about the perils of market research and surveys. A former colleague and one of the smartest researchers I ever met, Maria Cipollone, highlighted that my post focused on the risks of (bad) research while there are some fantastic opportunities to apply user research to make better games. This argument resonated with me as I completely agree that more data is better than less, and integrating qualitative and quantitative information will lead to better products.

Given the success Maria and I enjoyed working together in the past, I invited her to discuss when and how to use different research tools. Below is our conversation:

Lloyd: Hi Maria, thanks for joining me today. Let’s start by discussing my recent blog posts, where I highlighted some of the challenges with market research and you accurately pointed out some issues with my rationale. I think at the core I was somewhat dismissive of surveys, and equated them with market research, but there is so much more that can be done to understand customers.

Maria: So, surveys often get misused because they’re usually one of the few research tools available to organizations. They pervade our culture; every consumer is used to getting a feedback survey. However, they’re really only good at measuring the following: (1) someone’s attitude about a brand/product (2) someone’s perception of their own experience (e.g., was it negative or positive) (3) their perception of an experience based on “word of mouth” or public opinion (e.g., WOMI or NPS).

Many people use surveys to: (1) Try and predict what a consumer/player will do in the future (2) Have consumers/players report on their behavior in the past. This is where surveys go wrong. If you are asking your consumer/player/user to self report the frequency of their behavior, or predict the likelihood that they’ll do something in the future, then you’re collecting bad data (spurious).

Lloyd: What would be a good use case then in your first scenario, using what it is good for? What would be a use case where a customer’s perception could provide actionable insights?

Maria: NPS surveys are good for finding out brand perception and the impact of experience on brand perception; if measured correctly. You would measure the score, but more importantly you would collect open-ended feedback on why that is so. For example, let’s say you have a casino game that plays ads every time the player wins. The player gives an NPS score of 2 (detractor) and says, I hate how you interrupt my winning celebration with an ad. You might move the ads to when the player loses, and re-survey the same cohort to see if it impacts NPS more positively.

I have an even better example. In product research, we use a survey called a Kano analysis; where we ask customers to identify which features are important to them in a product. In the example of the Casino game, do players want to give gifts? Do they want credits for watching ads?

This Kano analysis allows us, via survey, to look at the relative importance of features, according to player perception. That way, we can see which features are perceived as “must have” or “delighters”
Then, dev and engineering can concentrate their efforts on those features

Lloyd: How can you tell if the player actually knows. They might think they want gifts but actually want credits?

Maria: We can trust what the player knows for two reasons:

We want to acquire them; so gifts will get them in the door
We look at mean scores across the board; we consider the aggregate; not just one perception,/li>

But your question brings up another issue I’d like to address, market research v user research. If you’re trying to develop a product; or features you need user research to implicitly observe what users “want”.

In the example of the gifts/credits; if you suspect that users are saying one thing but wanting another — that’s a problem space issue. You need to observe qualitatively the use of the product as realistically as possible. This can done with rapid ethnography; user interviews; contextual interview, etc.). This is user research — where we observe the use of the product in its natural habitat to balance what users say against what they actually do. On top of this, you need metrics to measure against all of this.

Market research measures what consumers will buy; and what attitude about the purchase.

User research measures/observes what players will do, and what tools are needed to accomplish that (UX/UI design).

Lloyd: Makes sense but before delving further into user research (which to me is the biggest opportunity), one last question about surveys/market research. Some of the feedback I got from my post was that surveys largely fail due to being worded poorly. My gut reaction to that is sort of my reaction to the argument that socialism has only failed because it is has never been implemented as intended. Is it utopian to expect your surveys (or election polls) to be designed in a non-leading way (unless you have an awesome researcher, which not everyone has) to be designed well or are most surveys going to be flawed and a reason to use other research tools.

Maria: Haha about socialism; that’s less of a linguistic problem and more of a human problem. But I definitely have a response.

Surveys can fail for a lot of reasons: survey design is definitely one of them. You can pick up biases, because people tend to agree, but there are simple ways to get around that (e.g., always write scales from negative— positive and stay away from agreement).

But they can also fail if:

You are asking people to self report on behavior or predict the future
You have a sample that is not representative of the population you want to infer from

There are many reasons surveys fail, and biases are popular, but I think they fail because of #1, most
people use them in the wrong way. It’s like trying to use an X-Ray to look at muscle tissue. You need an MRI for that.

Lloyd: Thanks, I’d agree with that analysis. Now onto the really interesting thoughts related to user research. While everyone is familiar with surveys, can you discuss further what are the different user research tools and options? You touched on it above about observing customers in their natural habitat (sort of like Blue Planet) and I remember that was one of your super-powers when we worked together, the value of going to a player’s house and watching them experience the game. Can you delve deeper into how that is done and other types of user research maybe even some rough rank or what should be used when?

Maria: Of course– and thanks for the opportunity. I’ll list some popular methods and what they’re good for, then I’ll discuss the future of UX research as I see it; it’s certainly changing 🙂

Lloyd: Perfect

Maria:

1.) Usability research (in-lab): This is just a method. A lot of times, people unfamiliar with UX Research will say; “We need usability”. Here’s where you need usability:You have a prototype that is designed to solve a specific set of problems (that you’re sure are problems). Usability testing (with task analysis or think-aloud) is to EVALUATE that prototype.
2.) User interviews in Context: This is a very specific method where you “interview” the user, but really you’re guiding them very carefully through an experience to implicitly observe what problems, “pain points” and inefficiencies they have with a system.In this scenario, a researcher does very little talking. They want to reduce the “Babble ratio” here and really let the user demonstrate how they naturally use the system or game.This is when you are trying to uncover problems for your current product, or even a new product to solve. Good products solve problems– not just occupy a market space. Too often, at least in tech, we introduce a technology with no problem. (Look at Voice Assistants for a good example of no problem space).

Lloyd: I can think of a few examples of tech that doesn’t solve a problem and then fails, a few billion later 🙂 (maybe Samsung’s new foldable phone).

Maria: You could optimally do #2 as Rapid ethnography, where you go in a home/office/environment to conduct the interviews.

Lloyd: What’s the benefit of going into a home than doing it in a lab?

Maria: Home/Environment is always the optimal scenario because that’s how it gets used in the real world. For example, in slots; it’s important to have visual and haptic feedback, because players often sit back with the game.

The more feedback the UX gives them; the more likely they are to turn their attention from the TV or whatever is distracting them back to the game. But knowing the environment that the game/product gets used in is really important to UX design.

Lloyd:Pardon my ignorance, but “haptic feedback”?

Maria: My fault. Haptic feedback is a vibration or buzz that the product emits; like when a game controller rumbles or a phone vibrates.

Visual feedback would be fireworks; flashes, etc

Lloyd: And they would get a difference experience in a lab (or your office) than in their natural environment?

Maria: In the lab, it’s superficial. The player is paying sole attention to the game because I’m watching the player, and they want to do a good job (performance bias). In the home, I get the real deal; the dog is barking; the TV is on– I know that their attention gets pulled away from the game. And part of my actionable insight can be to redirect their attention to the game via visual/haptic/sound feedback (although most players play w sound off).

But, if you’re short on money, there’s ways to interview users in context–even remotely; with tools like UserTesting.com

Lloyd: And that was going to be my next question, where do those tools fit in?

Maria: From what I said, i would take it as watch the user experience the product, if possible, in their home (if it’s a game normally played at home), then using a tool like UserTesting.com and last in your office/lab.

Lloyd:Other than observing the player, are there other user research techniques or tools worth incorporating into your playbook?

Maria: Yes– I actually like to do quantitative UX research– methods include surveys, variant (AB) testing, and behavior modeling.

Lloyd: Would you mind elaborating on each?

Maria: I use surveys to benchmark experience. For example, before we do major game OS updates, I might send out a survey and ask players to rate how perceptively slow the game seems.
Then, I would follow up after the OS update to see if perception shifted, and the game seems faster; more colorful.

Here, perception is important — because a happy player is likely to give a good app store rating for a game that seems fast; bright; fun.

You can als measure the psychological impact of your OS improvements.

AB testing we use to measure small changes– copy changes (e.g., changing the word “Buy Now” to “Purchase Coins” and see if that creates an upsell/lift)

Behavior modeling we do to sharpen metrics and KPIs

What do we know from user interviews that might help us understand player engagement metrics?

How can we develop hypotheses from the interviews that we can model in the metrics. For example, we observe that users quit when they are asked to purchase every time they run out of coins. So, we look at our engagement metrics, and see if we can link a drop in session length related to the purchase flow timing. Maybe we switch it up; floating the purchase flow after a win, or at the open of the app.

I think this is the future of UX research. That we will use observed insight — from small sample to understand our users at metrical scale. The future of UX research is assisting data science; machine-learning practitioners understand what they’re seeing at scale. Eventually, this will help build algorithms

Lloyd: That’s really an interesting approach.

One last question around two buzz phrases: focus groups and personas. I have strong thoughts on both but do not want to create a leading question.

Maria:Sure!

Focus groups are misused like surveys are. I’m not a practitioner. I guess the nicest thing I can say is if you’re looking for group speak on about the conception of a brand or experience, then go ahead. I find that product developers like them because they make them feel better. It’s a bunch of enthusiasts; speaking enthusiasm.

Lloyd: I’ve seen that

Maria: But, it can be useful for some; I think they can be very misleading. Like a COVID test you can do cheap and quickly but has a high false positive rate. I’m not a good representative of the method haha.

Lloyd: It may be confirmation bias but it is what I was hoping you’d say.

Maria: I’ve been forced to do focus groups, mostly to satisfy power.

Personas can be very valuable, but you need to do a few things to get them right.

Lloyd: Tell me more

Maria:

(1) Are you talking about customers (people who use your products) or consumers (people who you want to use your product)? I find the latter to be more valuable
(2) You can develop personas by doing interviews in the method I described above. If you’re doing personas of your current customers, you should follow up with a segmentation, which means you take quantitative measures to back up the persona you’ve made.

Persona is a framework for design. A segment is a portion of an audience.

Lloyd: What’s the value of personas over just segmentation?

Maria: Personas help designers inhabit the psychology of the player/user so they can make a better user experience.

Segments help product people go after an audience or understand the share of the audience the persona occupies.

Personas are for designers.

Segments are for the rest of us.

Lloyd: My concern with personas is are we over-simplifying (i.e. stereotyping) players? It sounds like the profiles that authorities sometimes abuse?

Maria: Ah, that’s a good concern — and I’ve had many conversations about the implicit biases that get produced in persona. If you ground them in qualitative research (e.g., 10-20 interviews); where you make sure your sample is diverse, you can protect against that issue. And, you must always preach that people shift in and out of persona. For example, I’m a casual gamer with slots; but not poker!

Lloyd: Thanks so much Maria, I found our chat really enlightening. Before we end, anything you want to add?

Maria: Research is worth its weight in gold! Let’s do it again AND THANK YOU!

People Analytics for Online Gaming

Last month, I wrote about some applications online gaming companies can take from the world of operations analytics, which are primarily used by traditional and retail businesses, and a course on People Analytics from Wharton showed some ways this area of analytics could be used to improve our businesses. While people analytics is often the domain of HR professionals, there are valuable elements for managers across tech businesses (many of whom do not have robust HR teams). Below are some of the most important takeaways from the course.

Identifying the noise and improving performance evaluations

A critical role for any leader or manager is accurately evaluating performance of your employees. Accuracy is important to ensure you provide useful feedback that helps people improve, assists you in putting the right people in the correct roles and identifies the skills needed for success in specific functions.

The fundamental challenge in performance evaluation is that performance measures are very noisy. There is a range of outcomes possible outside of the employee’s control. The challenge is separating skill and effort from luck so that you understand true performance.

In the course, the instructors highlight how often people confuse skill with luck. They start with an example from sports, showing that professional American football teams ability to draft (select out of university) players is almost primarily luck. While some teams have had a string of success, success in one year has no predictive ability on success in future years. If skill were a key factor, then you would expect a team to repeat its success.

It also holds true with investment analysts. An analyst who has a great year is no more likely to have above market results the next year than one of the poorest performing analysts.

There are many reasons we confuse this luck with skill:

Interdependence. I have found a humbling amount of work depends on other people, if they are great we look great, if they are not, we look bad. You should not attribute individual performance to something that is at the group level. In these cases, performance should be evaluated as a group. Conversely, reliable individual evaluation requires seeing people on other teams (for example, Tom Brady’s play on the Buccaneers will help assess whether his performance was due to him or the environment).
Outcome bias. We tend to believe good things happen to those who work hard and judge by outcome, not by process.
Reverse causality. When we see two correlated factors, we tend to believe one caused the other. In reality, one there may be no causality or it may be in the other direction. This leads us to see things that do not exist and can prompt us to give people undeserved credit or blame. One example cited in the course was research that showed charisma did not impact whether a CEO was successful, but successful leaders were considered more charismatic.
Narrative seeking. We want to make sense of the world and tell a causal story.
Hindsight bias. Once we have seen something occur, it is hard to anticipate we did not see it coming. We rewrite the history in our minds the history of the process.
Context. We tend to neglect context when evaluating performance. We over attribute performance to personal skills and under attribute it to environmental factors such as the challenge of the problem the employee faced, quality of their team, etc. In psychology, this issue is referred to as the Fundamental Attribution Error, blaming or crediting personality traits to situational traits.
Self-fulfilling prophesies. People tend toward performing consistent with expectations. High expectations increase performance, low expectations decrease performance
Small samples. Small samples lead to greater variation, what we see in a small sample may not be representative of a large population.
Omitted variable bias. There could be an additional reason that is driving both what the performance and what we think is causing the performance. For example, we may think higher compensation is leading to better performance. The truth might be that extra effort is causing both higher compensation and superior performance, thus the key variable (effort) had been omitted.

When you are looking at evaluating performance, there are several tools to improve your accuracy. You need to focus on the process the employee (or potential employee) took rather than only the outcome; we normally omit almost 50 percent of the objectives that we later identify as relevant to success. Thus, you should look at a much broader set of objectives that impact the business. This process includes determining what increases the likelihood for superior performance, rather than traditional outcomes are there four or five things that may not be obvious but contribute to overall success. A few years ago, I wrote how one basketball player (Shane Battier) was much more valuable than many players who scored more points or otherwise had flashier statistics, the same holds true in traditional business.

You need to look carefully at the job and understand what drives success. Define success not only by outcomes but how well these factors predict other KPIs, attrition, rate of promotion, etc. In the course, they also point out what works for one role or company does not necessarily work for others. Google found that GPA was an awful predictor of performance, but for Goldman Sachs it is the gold standard of who will be successful.

Additional ways to improve performance evaluation include:

Broaden the sample. Add additional opinions, more performance metrics, different projects and assignments. The key is to use diverse, uncorrelated signals.
Find and create exogenous variation. The only truly valid way to tease out causation is to control an employee’s environment. Have the employee change teams, direct report, projects, offices as the variation will provide a better sense of the employee’s ability.
Reward in proportion to the signal. Match the duration and complexity of rewards to the duration and complexity of past accomplishments. For short, noisy signals it is better to give bonuses and praise rather than raises and promotions.
The wisdom of crowds. Average of guesses is surprisingly good (even the exercises like guessing the number of jelly beans in a bowl), so get multiple experts to help with your assessment. Ensure, though, that their predictions are independent of each other (they are not talking to each other, they do not have the same background, etc).
Ensure statistical significance. A small sample (one project, one season, etc) is less likely to give you an accurate measure.
Use multivariate regression. This analysis will allow you to separate out the influence of different characteristics.

At the end of the day, you need to separate the signal from the noise to evaluate current performance and predict future success. Someone may have had a great performance or year but they may be a less valuable future employee than someone else because of luck or other environmental factors.

Recruiting the right people

Evaluating performance is not only important for your current team but also recruiting the best new hires. Hiring the wrong person can have huge consequences, including missed growth opportunities, damaging your culture and decreased output. Yet, most companies find consistently recruiting the right people difficult. This is often caused by the Illusion of Validity, that we think we know more about people than we actually do. We interview somebody and believe we can judge his or her suitability for a job. This Illusion is popped by research that shows the correlation of several hiring tools to subsequent performance (Ranked from most effective to worst:

Work samples.
Cognitive ability tests (these are general intelligence tests).
Structured interviews.
Job knowledge tests.
Integrity tests.
Unstructured interviews.
Personality tests.
Reference checks.

Several of the low scoring tools reinforce the Illusion of Validity. Unstructured interviews, where you meet someone and get a sense of their strengths and weaknesses, is often the paramount driver for whether we hire a candidate, but we are not good judges of character. I remember reading when President Bush first met Russian President Putin in 2001, he said “I looked the man in the eye. I found him to be very straight forward and trustworthy.” We see how well that worked out. As the above research also shows, reference checks are even more ineffective in the hiring process for similar reasons.

What does work is seeing examples of their previous relevant work, intelligence tests and structured interviews. Structured interviews are one designed to assess specific attributes of a candidate.

Use analysis for internal promotions

As well as improving the hiring process, People Analytics can help move the right people internally into the right roles. Often, people are promoted based on having done a great job in their current role. The course shows, though, that this approach often leads to negative outcomes (both for the employee and the company). The skills needed to succeed in the next job may not be the same skills that led to success in the current job. Performance in one job is not automatically a predictor of performance in a new role.

Just as it is important to understand the key predictors of success when recruiting, you need to do the same with internal promotion. Understand what leads to success in the new role and hire internally (or externally) those most likely to succeed. The good news is that research has shown that people promoted performed better overall than new hires into comparable roles.

Reducing employee churn

Attrition is one of the costliest problems company’s face and People Analytics can help combat this problem. The expense of losing an employee includes hiring a replacement, training costs, loss of critical knowledge and the impact on customer relationships. People analytics offers help in mitigating this problem. You should start by analyzing the percent turnover at specific milestones (3 months, 6 months, 1 year, etc.) and evolve into using multivariate regressions to predict who will reach each milestone. As you get more sophisticated you can build a survival model to understand over time what proportion will stay with your company. And then finally look at a survival/hazard rate model to test what factors accelerate the risk of exit.

During the course, they also provided some interesting data on why people leave. The decision to quit is most commonly driven by external factors, comparing the current job to a new opportunity. This understanding is critical as internal factors do play a role, internal issues still have a relatively small relationship to how likely people are to churn.

To reduce churn over time, the instructors of the course suggest an informed hiring strategy (where predicting churn is integrating into who is hired) and target interventions (reduce factors that accelerate risk of exit, address unmet needs, focus retention efforts, etc).

Using network analysis to improve collaboration

Another great takeaway from the course was how to use network analysis to understand, improve and incentive collaboration. Without getting too granular, network analysis involves looking at the informal links between employees, who gets information from who and what direction(s) that information is flowing. Once you draw that map, you can understand who are central to communications, who are outside the map, areas for improvement and people who should be rewarded for their role in collaboration.

While there are many details to creating and analyzing a network, there are five key areas to focus on when looking at individuals (there are no right and wrong answers for each attribute, optimizing depends on the goal and environment):

Network size. How many people are they connected to.
Network strength. How important and often are the lines of communication.
Network range. How many different groups are they connected to. Range would be small if you are connected to everyone on your team even if it is a big team, large if you are connected to one person at every other corporate function (i.e. marketing, accounting, analytics, etc.)
Network density. Are the connections connected to different people or to each other.
Network centrality. Is everyone equally central or are there some in the middle and others on the fringes.

Understand how your company’s network works will allow you to understand collaboration patterns. For example, by deconstructing performance, you can understand if collaboration patterns impact performance. If there is a positive causal relationship, you can work to replicate or improve these relationships. If there is no relationship, your team might be wasting time on unnecessary collaboration.

You can use this analysis to understand if collaboration is needed and where. Then you can strategically build ties and bridges between different parts of the organization. This result can be achieved with:

Cross-functional meetings.
Conference calls or video conferences
Job rotations
Site visits
Events

You should also identify where collaboration is unnecessary or overly burdensome and reduce demands on people. Match overloaded people with well-regarded employees who are under-utilized, who can relieve some of the burden. Also identify a small number of new connections that would have the biggest positive impact on team connectivity and shift responsibilities more evenly across members.

Tying performance evaluation with collaboration

People analytics can be particularly helpful connecting the performance evaluation methods discussed above with analysis of collaboration. As I wrote earlier, the key to good performance reviews is understanding what drives the outcomes you are looking for. If collaboration is one of those success drivers, you need to evaluate it thoroughly and incorporate into performance reviews and internal promotions (you do not want to promote someone weak at collaboration into a role where it is vital to success).

You should revise your evaluation systems to include collaboration. First, this will provide incentive to employees to build and use meaningful relationships. Second, it will recognize team members who help others win new clients or serve current customers, even if those direct results accrue to someone else (the basketball player who passes the ball rather than dunks).

To achieve this goal, you need to have the right measures. If you are assessing individual collaboration, you need to look at elements the individual controls. You then need to make sure there is reliability, which are the assessments will remain consistent over time and across raters. Third, the measures must have validity (accuracy). There also needs to be comparability, you need to be able to use the measures to look at all people who you are evaluating. Finally, it must be cost effective, it should not be too expensive to collect the information.

Key takeaways

You need to align performance evaluations with the underlying factors that create success; deconstruct what leads to the outcomes you want and then assess people on those factors.
Some common problems when evaluating people include context (attributing results to a person when the environment drove success or failure), interdependence (assessing on an individual level a result that was driven by a team), self-fulfilling prophecies (people perform consistent with expectations) and reverse causality (we attribute causality to correlation, even though the factors may not be related or may be in the other direction).
You should assess how your team or company works as a network, looking at the relationships, and then encourage and grow ones that lead to desired outcomes.

Summer highlights from The Business of Social Games and Casino

Normally, I take the summer off from writing blog posts but 2020 is anything but a normal year. Unfortunately for many, the pandemic meant you had to cancel or postpone your holidays or had more time to add to your knowledge. Thus, I continued to post but at a slightly reduced schedule. If you were lucky enough to get away, below is a summary of my posts over the summer that you may have missed, enjoy.

Lifetime Value Part 29: Increasing Retention

Key Takeaways

Retention is the strongest driver of LTV and data from Google shows the most important retention KPI is the amount of players who return on day 2 after installing your game.
The strongest driver of D2 retention is how many minutes your customers stay/play within the first ten minutes of starting the app.
To improve retention between the first and second days, make the early experience faster and more fun by improving load times (while reducing secondary loading), making your lobby intuitive, and not distracting your player with a bad tutorial or promotions.

Behavioral Economics Tips for Gaming Companies

Key Takeaways

A key lesson of behavioral economics is that less choice often drives better results. When the number of choices increases, our ability to make a decision decreases.
Consumers hate uncertainty. Questions without answers cause fear and kills the experience and sales, it is a customer experience killer.
Use AB and multi-armed bandit tests help you understand how your players will react in the context of your game, market research conversely might provide bad information as people do not know what they want.

How to avoid meetings about the trivial, aka bikeshedding

Key Takeaways

Bikeshedding is the tendency we have to spend excessive time on trivial matters in meetings, often glossing over important ones.
Bikeshedding is damaging because it wastes very valuable time and, more importantly, leads to insufficient discussion of important issues.
To avoid bikeshedding, set a clear purpose for all meetings (and eliminate conversations about other issues), only invite necessary people, appoint a decision maker and have the decision maker set clear parameters for the meeting.

Lifetime Value Part 30: Why clumpiness should be one of the KPIs you focus on

Key Takeaways

We normally focus on analyzing recency, frequency and monetization of the customer but by adding a new KPI, clumpiness, we get a much better understanding of their expected value.
Clumpiness refers to the fact that people buy in bursts and that those customers could be extremely valuable.
Clumpiness can help you better segment players, predict VIPs and target your reactivation efforts and spend.

Why Evo’s $2 billion+ acquisition of NetEnt is more important (to both iGaming and social casino) than you think

Key Takeaways

Evolution Gaming, the largest Live Dealer provider, recently announced a bid to acquire NetEnt, the largest slots provider for $2+ billion.
The deal, in that Evo is the acquirer, shows that Live Dealer is eclipsing slots in the casino ecosystem.
For real money operators, they need to ensure they balance resources around both Live Dealer and slots while social casino companies need to figure out the best way to embrace this opportunity.

Customer analytics tips for gaming companies

Key Takeaways

People who wander around a retail location spend more than ones who immediately find what they are looking for and retailers optimize to create this jiggliness. Online casinos and games can also build in jiggliness so players find new games and offerings rather than simply quickly go to the one they are looking to play.
While satisfaction with customer service positively impacts profitability, the relationship is not linear. Improvements have a strong impact when players are highly dissatisfied (and that is corrected) or when customers with great service make further improvements, companies in the middle often do not see a positive ROI on CS improvements.
A relationship between two variables does not show one is causing the other, to have causation there must be a relationship plus temporal antecedence plus the absence of a third variable driving both factors.

How to get your big initiatives done

Key Takeaways

Many important initiatives, from new products to operational efficiency, bog down and die in the middle phase. They initially have momentum but stall once the initial burst dies down.
To MOVE projects through this middle phase, the Middle element needs a clear and concrete strategy and you need an Organizational structure with capacity to complete the initiative.
The final keys to getting through the middle phase are Valor, making tough decisions and prioritizing the initiative, and getting Everyone involved.

How Operations Analytics can help online gaming companies

Key Takeaways

While Operational Analytics are a focus primarily in retail and traditional businesses, there are many best practices that iGaming and social game companies can leverage.
Forecasting is central to generating and earmarking resources but is often a challenge for game companies, rather than trying to create a point forecast create a range based on moving averages and looking at standard deviation. For new products, create a simulation that will show the distribution of potential outcomes and the risks and rewards possible.
You need one, and only one, distinct goal and then optimize your strategy around that goal; it’s impossible to optimize for multiple goals. Use constraints to incorporate what used to be additional goals.

The risks of market research

Key Takeaways

A sole reliance on customer input and feedback, traditional market research, is built on a model of human decision making that assumes humans are rational, while in practice we are not.
Not only do people provide a response inconsistent with their actions, they often do not understand the underlying causes of their behavior.
Use one or multiple tools that show actual decision making, such as ABn testing or looking at reactions to similar initiatives in adjacent industries, rather than relying on what customers believe is their preference.

How Operations Analytics can help online gaming companies

I recently took an online course from Wharton on Operations Analytics and it was valuable because it was for traditional businesses, not game companies. Social game companies are great at player, marketing and customer analytics, to the point where it is hard to eke out competitive advantage using analytics because everyone is so good; it is largely the cost of doing business. To gain an edge it is sometimes useful to look outside the space and learn best practices from other industries, even ones considered less sophisticated than online gaming, and apply these insights to our business.

While Operations Analytics is a focus for retailers and bricks-and-mortar businesses, it often does not make it on the radar for social, mobile and iGaming companies. The foundational problem in the space is even referred to as the Newsvendor Problem, clearly based on a very traditional business. After taking the Operations Analytics course, I realized many useful applications for gaming companies from this field.

Forecasting demand and allocating resources

I have worked at many large gaming companies, including several that were public companies, and, despite the companies reliance on analytics, financial forecasting was at best a guess, at worst a hope. A retailer, especially a land-based one, lives or dies by its forecasts. Order too much and you are left with potentially worthless inventory. Order too little, and you miss profits that can then be used to market, grow or sustain during seasonal periods.

Accurate forecasting is critical to make valid decisions today (hiring, investing, M&A, etc.), and it is as critical for online gaming companies as it is for a bricks-and-mortar business. For example, a good forecast will help you hire the right size Customer Support team. It will also help you optimize cash flow, manage your marketing expense, allow you to seek financing preemptively, determine when to launch new content and features and more.

The first key learning from Operations Analytics related to forecasting is that point forecasting are usually wrong. Point forecasts are unreliable not only in business but in all elements of life. The chance of predicting the amount of rainfall next month accurately is close to zero.

Instead, a good forecast should be a range, showing the likely outcomes. When modeling an uncertain future, you build probability distributions based on past data (and possibly incorporating expert estimates). You can put likelihoods on different scenarios, such as low, normal and high. When looking at your potential sales for an online game (or your portfolio of games), which is almost a continuous distribution of scenarios (like amount of rainfall), you want to group the scenarios rather than create individual ones, a continuous distribution.

One of the core ways of using existing data is to base forecasts on moving averages, a practice I recently implemented and have seen quite useful. A moving average is a forecast based on the average of the n most recent observations. In my case, we have created much more accurate predictions looking at the past 28 days of data (so n=28), taking both the mean and standard deviation. It is not a perfect tool, though, as it misses trends (i.e. pandemics) and does not show causation.

While finding the mean and standard deviation of past data is a good foundation for creating your forecast, sophisticated companies integrate additional sources of information. These may include an aggregation of forecasts (for example from each of your product managers), customer surveys (how much they expect to change their purchasing behavior), a jury of executive opinion (what your leadership team thinks) or the Delphi Method (individual opinions that are compiled and reconsidered again and again until there is a consensus).

Smoothing data for seasonality

Another common practice in operations analytics that is not always used in online gaming is smoothing for seasonality. While game companies are keenly aware that sales (and other KPIs) will be impacted by day of the week or hour of the day (referred to as seasonality), some companies are better than others at adjusting for this seasonality.

Most game companies realize these differences and will not compare Saturday revenue with Sunday revenue. Instead, they will look WoW (week on week) and compare Saturday with the previous Saturday. The weakness in this approach is that a lot happens in seven days so you are losing very useful information in the comparison, rather than seeing immediate trends you are dealing with a seven day delay.

An approach to avoid this problem is to remove seasonality from the numbers. You can do this by looking historically at the numbers, creating a sample mean, then finding seasonal averages (if we are looking at day of the week you would have a mean for Mondays, a mean for Tuesdays, etc), divide the seasonal averages by the sample mean to create a seasonal factor and then divide each observation by the seasonal factor. By using a seasonal factor, you can identify immediately any big deviations and not have to wait a week to understand if your game is broken.

Projecting new game launches and other high uncertainty situations

While online game companies are not always sophisticated in modeling their forecasts, it gets much worse when projecting for new games. Some companies rely on the wishful thinking strategy, projecting what revenue will be if the game is a hit (despite the fact less than 20 percent of new games are successful). Others take the opposite tack, assuming no revenue from new projects until they already have data. Neither of these approaches provides useful information for companies’ planning. They either muddy the water so that people do not take the financials seriously or provide misleading data that leads to too much or too little additional resources (such as hiring support staff or preserving marketing budget).

While there is significant variance in potential outcomes for new games, there are methods to create a usable range for forecasting purposes. Given that there is limited (possibly a soft or beta launch) or no actual demand data, you need to augment that data with other sources:

Sales of similar games. How have comparable products done at launch and over time.
Composites. Estimates from the product team or marketing.
Customer surveys. Responses from target customers to market research.
Jury of executive opinion. Estimates from your executive or leadership team.
Delphi method. Individual opinions compiled and reconsidered, repeat until overall group consensus is (hopefully) reached.

You then should refine your estimate based on past experience. Look at previous estimates and compare them to actuals. If you are historically 5X overly optimistic or 20 percent overly pessimistic traditionally, adjust your estimate to reflect this historical variance. Also use the historical data to calculate a standard deviation, so you can create a forecasted range of potential values, rather than a specific number. Then incorporate this range in your planning.

A more advanced method to use when estimating a new product launch, or anything else where there is a high level of uncertainty, is simulation. You can build a simulation of a new product launch using a third party tool (StatPlus is one such tool) coupled with the projections above. You can then see 100 or 1,000 or more likely outcomes with a distribution similar to past product launches. This simulation will then not only give you a range of likely outcomes but also help you identify best and worst case scenarios, so you can accurately gauge risk and reward. You can then optimize your internal resource allocation (i.e. hiring new support agents or performance marketers) by optimizing for reward while using your risk measures as a constraint.

Focused goals

For me, the most valuable insight from operations analytics is that you should only have one goal. When creating an optimization model, you optimize for one decision variable (revenue, profit, cost, etc.), changing multiple variable within constraints. It ends up being a, sometimes complex, algebraic model with a solution. Mathematics such as this, however, can also have a broader application.

I will never forget an executive committee meeting I had several years ago where a new CEO said we have two goals, grow top line revenue like a growth company and grow margin. At the time, several people asked which is more important and the CEO responded they both are, we need to optimize for both. Fast forward several years later and the company wallowed in mediocrity, without significant organic revenue or margin growth.

So what does that story have to do with the math behind optimization models? In an optimization model you can only optimize for one objective. You can optimize for profitability or revenue but if you try to optimize for both, Excel would give you a big error message saying it is impossible to optimize for both. Instead you can optimize for one and set the other as a constraint (optimize for growth with margin >= 20%). This mathematical reality is also a strategic reality, when setting your objective variable do not set multiple objectives but make the hard decision as to what is most important and then use the others as constraints when building your strategy.

Key takeaways

While Operational Analytics are a focus primarily in retail and traditional businesses, there are many best practices that iGaming and social game companies can leverage.
Forecasting is central to generating and earmarking resources but is often a challenge for game companies, rather than trying to create a point forecast create a range based on moving averages and looking at standard deviation. For new products, create a simulation that will show the distribution of potential outcomes and the risks and rewards possible.
You need one, and only one, distinct goal and then optimize your strategy around that goal; it’s impossible to optimize for multiple goals. Use constraints to incorporate what used to be additional goals.

Customer analytics tips for gaming companies

While social and mobile gaming companies are generally at the cutting edge of applying analytics, I recently took an online course from Wharton on Coursera that provided some additional insights in how to best use analytics in online gaming. These takeaways range from ways to improve your UI to how to calculate LTV more accurately.

Make your players wander

One of the most interesting takeaways from the course is that efficiency is not always the desired player behavior in the online world. In traditional retail, retailers found they enjoy much higher revenue when customers wander around the store rather than quickly find what they have come for. In the studies cited, about 75 percent of movement inside a store is not required. Sixty percent of purchases are items people had no intention of buying when they went into the store. Instead retailers optimize for “jiggliness,” as people with the most jiggliness buy the most.

There are uses of this concept for online gaming and iGaming companies. Rather than optimize your lobby and UI (user interface) to ensure your players find what they are looking for, take them on a journey around your game. If it is a social casino, rather than finding the slot they know and love, expose them to some other content, they may find something they prefer.

Higher customer satisfaction may not improve profitability

While customer satisfaction is positively correlated with profitability, the relationship is not linear. Companies with a low level of customer satisfaction, referred to as the Zone of Pain, experience a strong impact on revenue when making improvements. That is, the firms with awful customer service see big benefits just moving out of the Zone of Pain.

On the high end, companies that provide great customer service and differentiate themselves with it experience positive ROI by making the experience even better. These companies are what is referred to as in the Zone of Delight. Retailers like Nordstroms, which enjoy high margins due to their customer service, see a huge impact when they find even better ways to provide a WOW experience.

When customer satisfaction is only a small part of a company’s value proposition, improvements do not necessarily have a positive return. There is a large flat region where increasing satisfaction does not increase profitability. The key takeaway is that the relationship between customer satisfaction and profitability is not linear, but starts with a Zone of Pain, then hits a sizeable flat region, and then moves to a Zone of Delight.

Correlation does not equal causation

We should all know by now that just because two variables have related movement, you cannot assume one is causing the other. I see this mistake made frequently, including by BI experts. Correlation only shows a relationship between two variables. Causation, more critically, shows that one variable produces an effect on the other variable. It is crucial to remember there are three requirements for causality:

Correlation
Temporal Antecedence. X must happen before Y.
No third factor is driving both. Need to control for other possible factors.

Use analytics for pricing

I am surprised at how often pricing strategy in mobile games (the cost of in-app purchases) or in iGaming (RTP and bet levels) is driven by competitive analysis and intuition rather than analytics. Regression, however, can be used to set optimal pricing (including for virtual goods) at the level that boosts profits. Regression can predict demand at prices that have not been tried, thus you can determine profitability for different options. As predictions can be completed for different future prices, you can then determine optimal price. Effectively, you answer the question what you can charge to make the maximum profit (and with virtually zero marginal cost for online products, can be simplified to maximum revenue).

Preparing better surveys

While market research is a less than reliable way to understand customer intent, it still provides valuable insights into your players. Surveys are a good way to learn about potential customers and are relatively low cost. Some best practices include:

- To improve reliability of surveys, test and then retest. If the results are consistent, it shows you are getting reliable results (people still may not know what they want though).
- There are multiple ways to ask questions in a survey (comparative, rank, paired comparison, Likert, continuous, etc.) and you should understand your end goal when deciding which format to use. Advantages of open-ended questions allow for a general reaction that can help interpret closed end questions and may suggest follow up questions. Closed end requires a lot of pre-testing but is easier to administer.
- Focus on drafting high quality questions. Use simple, conventional language and avoid ambiguity. Do not ask any questions more than 20 words. Most importantly, avoid leading and loaded questions (i.e. How bad a job is Lloyd doing?).
- Pay attention to sequence and layout. Start with an easy and non-threatening question. Have a smooth and logical flow. Have the questions go from general to specific. Keep the sensitive or difficult question at the end.
- The key to using surveys effectively is validity, how well it predicts variables you are interested in. If you find surveys effectively predict certain behavior, then they are an appropriate tool for predicting that variable.
- Make sure your results are generalizable to an appropriate population. You need to define clearly the population, choose a representative sample, select respondents will to be interviewed and motivate them to provide information.
- Pre-test your survey. Ensure respondents understand each question and the questions make sense.
- Collect data on non-respondents as they may be systemically different. Try to convert them to responding.

Recency is incredibly important

When looking at the future value of a customer, the three keys are how recently they made a purchase (recency), how many purchases they have made (frequency) and monetization (size of the purchase) recency is by far the best predictor of future value. Frequency is then significantly more indicative than monetization. Thus, focusing on increasing the size of a purchase (up-selling) is the least valuable strategy you can pursue to increase your customer’s lifetime value.

Include clumpiness in your LTV analysis

I wrote several weeks ago about the important of clumpiness in determining a customer’s future value so will not go into too much detail again. Clumpiness refers to the fact that people buy in bursts and that those customers could be extremely valuable. When calculating customer value and segmentation, we focus on analysing recency, frequency and monetization of the customer, as I discussed above. This analysis is based on customers making purchases in a regular pattern, i.e. coffee, diapers or milk. For certain products (and I would classify social and casino games here), customers actually monetize in bursts. Thus, you need to add C for clumpiness to your modeling.

Key takeaways

People who wander around a retail location spend more than ones who immediately find what they are looking for and retailers optimize to create this jiggliness. Online casinos and games can also build in jiggliness so players find new games and offerings rather than simply quickly go to the one they are looking to play.
While satisfaction with customer service positively impacts profitability, the relationship is not linear. Improvements have a strong impact when players are highly dissatisfied (and that is corrected) or when customers with great service make further improvements, companies in the middle often do not see a positive ROI on CS improvements.
A relationship between two variables does not show one is causing the other, to have causation there must be a relationship plus temporal antecedence plus the absence of a third variable driving both factors.

How to avoid meetings about the trivial, aka bikeshedding

I have written before on my general distaste of meetings and how they frequently destroy more value than they create, and one of the key reasons I detest meetings so much is they always seem to be about trivial things. A recent article, Why We Focus on Trivial Things: The Bikeshed Effect, explains why we gloss over important topics and focus on the trivial. Bikeshedding is a metaphor to illustrate the tendency we have to spend excessive time on trivial matters, often glossing over important ones.

What is bikeshedding and why it is a problem

Bikeshedding comes from an example by Cyril Northcote Parkinson, the father of Parkinson’s Law (tasks expand to fill the amount of time available), that explains the amount of time spent discussing an issue in an organization is inversely correlated to its actual importance. We discuss important complex issues minimally while simple, minor ones get the most attention.

In Parkinson’s example, an executive management team has three issues on its agenda

Build a $10 million nuclear power plant
Build a $350 bike shed
Allocate a $21/year coffee budget

Rather than focus the conversation on the big project, the power plant, the team runs through that proposal quickly. In Parkinson’s example, “[i]t’s too advanced for anyone to really dig into the details, and most of the members don’t know much about the topic in the first place. One member who does is unsure how to explain it to the others. Another member proposes a redesigned proposal, but it seems like such a huge task that the rest of the committee decline to consider it.”

When the conversation moves to the bike shed, it heats up. Everyone on the team is comfortable expressing thoughts on it. They all know what a bike shed looks like. They have different ideas on the material to use for the roof and potential small cost savings. The team spends more time discussing the bike shed than the $10 million nuclear power plant.

Finally, the team ends up spending the most time talking about the coffee budget. With coffee, everyone on the team is an authority. Each person knows about coffee and has a strong sense of cost and value. At the end of the meeting, they have spent more time discussing the coffee budget than the bike shed or the power plant combined. As they say in the post, “everyone walks away feeling satisfied, having contributed to the conversation.”

There are two critical issues caused by bikeshedding:

Not enough time is spent discussing the power plant and the critical issues related to it. This decision potentially alters the trajectory of the company and a mistake could cost a lot of money or be fatal to the firm. By not looking at the details, even a small misstep can have a big impact.
Even taking away the opportunity cost, wasting time talking about a cheap coffee plant or bike shed is likely to cost more in the value of time than any gain. Assuming each executive is well compensated, the value of their time spent talking about coffee is likely to run in the thousands of dollars.

Why do we have bikeshedding

Given the absurdity (and uncanny truth) of the above example, it begs the question how does this happen (and we have all seen it happen). According to the post, “the more people will have an opinion on it and thus more to say about it. When something is outside of our circle of competence, like a nuclear power plant, we don’t even try to articulate an opinion. But when something is just about comprehensible to us, even if we don’t have anything of genuine value to add, we feel compelled to say something, lest we look stupid. What idiot doesn’t have anything to say about a bike shed? Everyone wants to show that they know about the topic at hand and have something to contribute.”

During meetings, we reward people simply for expressing an opinion rather than for having put in the time and work to develop the judgment. Most importantly, people should focus on contributing when you have something valuable to add that would result in a better decision.

How to mitigate bikeshedding

Once you acknowledge the problem of bikeshedding, there are several steps you can take to avoid the issue and spend the appropriate time each issue demands:

Have a clear purpose. Successful meetings need to have a clear and well defined purpose. Specificity is central to having a purpose and conveying it.
Invite the right people. Only invite people who can contribute to the discussion or are needed for execution of the decision. If the purpose is to discuss the nuclear power plant, this purpose will make it clear who should and should not be in the meeting. As the post points out, “the most informed opinions are most relevant. This is one reason why big meetings with lots of people present, most of whom don’t need to be there, are such a waste of time in organizations. Everyone wants to participate, but not everyone has anything meaningful to contribute.”
Appoint a decision maker. To reach the best outcome, you need a designated decision maker. First, it avoids forcing a consensus when there should be a black and white winner, a compromise is not always better than an extreme option. Also, it is often impossible to reach a consensus when nobody is in charge. The discussion just drags on and on.
Have the decision maker set clear parameters. With one person in change, they can decide in advance how much importance to accord to the issue (for instance, by estimating how much its success or failure could help or harm the company’s bottom line). They can set a time limit for the discussion to create urgency. And they can end the meeting by verifying that it has indeed achieved its purpose.

By implementing these steps, you can help your company focus its efforts on finding solutions to the most intractable and important problems and let picking the right type of coffee to someone else.

Key takeaways

Bikeshedding is the tendency we have to spend excessive time on trivial matters in meetings, often glossing over important ones.
Bikeshedding is damaging because it wastes very valuable time and, more importantly, leads to insufficient discussion of important issues.
To avoid bikeshedding, set a clear purpose for all meetings (and eliminate conversations about other issues), only invite necessary people, appoint a decision maker and have the decision maker set clear parameters for the meeting.

S	M	T	W	T	F	S
				1	2	3
4	5	6	7	8	9	10
11	12	13	14	15	16	17
18	19	20	21	22	23	24
25	26	27	28	29	30	31

This isn’t our parent’s world

Complexity vs Complicated

The old approach

The need to build resiliency

Building adaptability

The need for a Team of Teams structure

Creating a shared consciousness

Ensuring communications flow by embedding people on other teams

Less command and less control

The end product

Key takeaways

Share this:

Like this:

Amateurs focus on being right. Professionals focus on getting the best outcome.

Amateurs think good outcomes are the result of their brilliance. Professionals understand when good outcomes are the result of luck.

Amateurs show up to practice to have fun. Professionals realize that what happens in practice happens in games.

Amateurs focus on tearing other people down. Professionals focus on making everyone better.

Amateurs blame others. Professionals accept responsibility.

Amateurs give up at the first sign of trouble and assume they’re failures. Professionals see failure as part of the path to growth and mastery.

Amateurs focus on identifying their weaknesses and improving them. Professionals focus on their strengths and on finding people who are strong where they are weak.

Amateurs think the probability of them having the best idea is high. Professionals know the probability of that is low.

Amateurs think in absolutes. Professionals think in probabilities.

Amateurs stop when they achieve something. Professionals understand that the initial achievement is just the beginning.

Amateurs have a goal. Professionals have a process.

Amateurs focus on the short term. Professionals focus on the long term.

Putting it all together

Key takeaways

Share this:

Like this:

Why customer delight does not work

What customers want

Where to focus your customer service strategy

KPIs, good and bad

Key tactics to achieving an effortless experience

The danger of channel switching

Avoid repeat contacts

Experience engineering

Build the Control Quotient

Create the right culture

Purchase experience

An effortless experience is the recipe for increasing LTV

Key takeaways

Share this:

Like this:

Share this:

Like this:

Identifying the noise and improving performance evaluations

Recruiting the right people

Use analysis for internal promotions

Reducing employee churn

Using network analysis to improve collaboration

Tying performance evaluation with collaboration

Key takeaways

Share this:

Like this:

Key Takeaways

Key Takeaways

Key Takeaways

Key Takeaways

Key Takeaways

Key Takeaways

Key Takeaways

Key Takeaways

Key Takeaways

Share this:

Like this:

Share this:

Like this:

Forecasting demand and allocating resources

Smoothing data for seasonality

Projecting new game launches and other high uncertainty situations

Focused goals

Key takeaways

Share this:

Like this:

Make your players wander

Higher customer satisfaction may not improve profitability

Correlation does not equal causation

Use analytics for pricing

Preparing better surveys