Skip to content

The Business of Social Games and Casino

How to succeed in the mobile game space by Lloyd Melnick

Day: March 23, 2016

How to manage your algorithms

While everyone is focused on creating the most advanced algorithms for their predictive analytics and optimizing your team’s performance, I have not seen anything on how to manage your algorithms. A great article in Harvard Business Review – Algorithms Need Managers, Too by Michael Luca, Jon Kleinberg and Sandhil Mullainathan – does a great job of combining the two issues and providing a solution.

The authors begin by pointing out most businesses rely on predictions throughout their organization. The decisions can range from predicting a candidate’s performance and whether to hire them, what initiatives will have the highest ROI and what distribution channels will yield the most sales. Companies increasingly are using computational algorithms to make these predictions more accurate.

The issue is, if the predictions are inaccurate (and although they are computer generated, they are still predictions not facts) they can lead you into bad decisions. Netflix learned this the hard way when its algorithms for recommending movies to DVD customers did not hold when its users moved to streaming. More relevant to digital marketers, algorithms that generate high click through rates may actually bring in poor users not interested in your underlying game or product. As the authors write, “to avoid missteps, managers need to understand what algorithms do well – what questions they answer and what questions they do not.”

How algorithms can lead you amiss

An underlying issue when using algorithms is that they are different than people. They behave quite differently in two key ways:

  • Algorithms are extremely literal, they do exactly what they are told and ignore any other information. While a human would understand quickly that an algorithm that gets users that generate no revenue is useless, if the algorithms was just built to maximize the number of users acquired it would continue attracting worthless users.
  • Algorithms are often black boxes, they may predict accurately but not what is causing the action or why. The problem here is that you do not know when there is incomplete information or what information may be missing.

Once you realize these two limitations of algorithms, you can then develop strategies to combat these problems. The authors then provide a plan for managing algorithms better.

Slide1

Be explicit about all of your goals

When initiating the creation of an algorithm, you need to understand and state everything you want the algorithm to achieve. Unlike people, algorithms do not understand the implied needs and trade-offs necessary often to optimize performance. People understand the end goal and then backward process how to best achieve that eventual goal. There are also soft goals to most initiatives, and these goals are often difficult to measure (and thus input into your algorithms). There could also be a goal of fairness, for example a bank using an algorithm to optimize loan behavior may not provide enough loans in areas where it feels a moral obligation to do so. Another example could be where you may want to optimize your business units sales but the behavior could negatively impact overall sales of your company.

The key is to be explicit about everything you hope to achieve. Ask everyone involved to list their soft goals as well as the primary objective. Ask people to be candid and up-front. With a core objective and a list of concerns in front of them, the algorithm’s designer can then build trade-offs into the algorithm. This process may entail extending the objective to include multiple outcomes, weighted by importance.

Minimize myopia

Algorithms tend to be myopic, they focus on the data at hand and that data often pertains to short-term outcomes. There can be a tension between short-term success and long-term profits and broader corporate goals. People understand this, computer algorithms do not.

The authors use the example of a consumer goods company that used an algorithm to decide to sell a fast-moving product from China in the US. While initial sales were great, they ended up suffering a high level of returns and negative customer satisfaction that impacted the brand and overall company sales. I often see this problem in the game industry, where product managers find a way to increase in-app purchases short term but it breaks player’s connection with the game and long-term results in losses.

The authors suggest that this problem can be solved at the objective-setting phase by identifying and specifying long-term goals. But when acting on an algorithm’s predictions, managers should also adjust for the extent to which the algorithm is consistent with long-term aims.

I recommend using NPS to balance out short term objectives with the long-term health of the product and company. I have written before about NPS, Net Promoter Score, which is probably the most powerful tool to measure customer satisfaction. It is also highly correlated with growth and success. By ensuring you keep your NPS high, you are providing a great way to look holistically at the success of specific initiatives.

Chose the right data inputs

Using the right data can make your algorithms much more effective. When looking at a game like Candy Crush, you can create levels by looking at when people abandon the game and decompose the levels before abandonment. However, by adding social media posts to the your data, you could get a more holistic view of which levels players are enjoying and thus build a more compelling product.

The authors also point to an example with the City of Boston. By adding Yelp reviews to what health inspectors use to determine what restaurants to inspect, they were able to maintain their exact same performance but with 40 percent fewer inspectors. Thus, the new data source had a huge impact on productivity.

The authors point to two areas of data that can improve your algorithms:

    • Wider is better. Rather than focusing on more data, the amount of data you know about each customer determines the width. Leveraging comprehensive data is at the heart of prediction. As the authors write, “every additional detail you learn about an outcome is like one more clue, and it can be combined with clues you’ve already collected. Text documents are a great source of wide data, for instance; each word is a clue.”
    • Diversity matters. Similar to your investment strategy, you should use data sources that are largely uncorrelated. If you use data that moves closely to your data sources, you will have the illusion of using multiple data sources but really only be looking at one angle of the data. If each data set has a unique perspective, it creates much more value and accuracy.

Understand the limitations

As with anything, it is also critical to understand the limitations of algorithms. Knowing what your algorithm will not do is equally important as understanding how it helps. Algorithms use existing data to make predictions about what might happen with a slightly different setting, population, time, or question. “In essence, you are transferring an insight from one context to another. It’s a wise practice, therefore, to list the reasons why the algorithm might not be transferable to a new problem and assess their significance,” according to the authors.

As the authors point out, “ remember that correlation still doesn’t mean causation. Suppose that an algorithm predicts that short tweets will get retweeted more often than longer ones. This does not in any way suggest that you should shorten your tweets. This is a prediction, not advice. It works as a prediction because there are many other factors that correlate with short tweets that make them effective. This is also why it fails as advice: Shortening your tweets will not necessarily change those other factors.”

Use algorithms, just use them smartly

This post is not intended for you to avoid using algorithms, it is actually the opposite goal. Algorithms are increasingly powerful and central to business success. Whether you are predicting how consumers will react with a feature, where to launch your product or who to hire, algorithms are necessary to get great results. Given the central importance of these algorithms, however, it is even more crucial to use them correctly and optimize their benefit to your company.

Key takeaways

  1. Algorithms are increasingly powerful and central to business success. Given the central importance of these algorithms it is even more crucial to use them correctly and optimize their benefit to your company.<
  2. Problems with algorithms result from them being literal (they do exactly what you ask) and are largely a black box (they do not explain why they are offering certain recommendations).
  3. When building algorithms, be explicit about all your goals, consider the long-term implications and make sure you are using as broad data as possible.

Share this:

  • Facebook
  • LinkedIn

Like this:

Like Loading...
Lloyd Melnick Analytics, General Social Games Business, General Tech Business, Machine Learning Leave a comment March 23, 2016February 28, 2016 6 Minutes

Get my book on LTV

The definitive book on customer lifetime value, Understanding the Predictable, is now available in both print and Kindle formats on Amazon.

Understanding the Predictable delves into the world of Customer Lifetime Value (LTV), a metric that shows how much each customer is worth to your business. By understanding this metric, you can predict how changes to your product will impact the value of each customer. You will also learn how to apply this simple yet powerful method of predictive analytics to optimize your marketing and user acquisition.

For more information, click here

Follow The Business of Social Games and Casino on WordPress.com

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 1,337 other followers

Most Recent Posts

  • Podcasts now available
  • Lessons for gaming and tech companies from the Peter Drucker Forum
  • Chaos Theory, the Butterfly Effect, and Gaming
  • How to give help without micromanaging

Lloyd Melnick

This is Lloyd Melnick’s personal blog.  All views and opinions expressed on this website are mine alone and do not represent those of people, institutions or organizations that I may or may not be associated with in professional or personal capacity.

I am a serial builder of businesses (senior leadership on three exits worth over $700 million), successful in big (Disney, Stars Group/PokerStars, Zynga) and small companies (Merscom, Spooky Cool Labs) with over 20 years experience in the gaming and casino space.  Currently, I am the GM of VGW’s Chumba Casino and on the Board of Directors of Murka Games and Luckbox.

Topic Areas

  • Analytics (114)
  • Bayes' Theorem (8)
  • behavioral economics (8)
  • blue ocean strategy (14)
  • Crowdfunding (4)
  • General Social Games Business (457)
  • General Tech Business (194)
  • Growth (88)
  • International Issues with Social Games (50)
  • Lloyd's favorite posts (101)
  • LTV (54)
  • Machine Learning (10)
  • Mobile Platforms (37)
  • Social Casino (51)
  • Social Games Marketing (104)
  • thinking fast and slow (5)
  • Uncategorized (32)

Social

  • View CasualGame’s profile on Facebook
  • View @lloydmelnick’s profile on Twitter
  • View lloydmelnick’s profile on LinkedIn

RSS

RSS Feed RSS - Posts

RSS Feed RSS - Comments

Categories

  • Analytics (114)
  • Bayes' Theorem (8)
  • behavioral economics (8)
  • blue ocean strategy (14)
  • Crowdfunding (4)
  • General Social Games Business (457)
  • General Tech Business (194)
  • Growth (88)
  • International Issues with Social Games (50)
  • Lloyd's favorite posts (101)
  • LTV (54)
  • Machine Learning (10)
  • Mobile Platforms (37)
  • Social Casino (51)
  • Social Games Marketing (104)
  • thinking fast and slow (5)
  • Uncategorized (32)

Archives

  • March 2021
  • February 2021
  • January 2021
  • December 2020
  • November 2020
  • October 2020
  • September 2020
  • August 2020
  • July 2020
  • June 2020
  • May 2020
  • April 2020
  • March 2020
  • February 2020
  • January 2020
  • November 2019
  • October 2019
  • September 2019
  • June 2019
  • May 2019
  • April 2019
  • March 2019
  • February 2019
  • January 2019
  • December 2018
  • June 2018
  • May 2018
  • April 2018
  • March 2018
  • February 2018
  • January 2018
  • December 2017
  • June 2017
  • May 2017
  • April 2017
  • March 2017
  • February 2017
  • January 2017
  • December 2016
  • November 2016
  • October 2016
  • September 2016
  • June 2016
  • May 2016
  • April 2016
  • March 2016
  • February 2016
  • January 2016
  • December 2015
  • November 2015
  • October 2015
  • September 2015
  • July 2015
  • June 2015
  • May 2015
  • April 2015
  • March 2015
  • February 2015
  • January 2015
  • December 2014
  • November 2014
  • October 2014
  • September 2014
  • August 2014
  • July 2014
  • June 2014
  • May 2014
  • April 2014
  • March 2014
  • February 2014
  • January 2014
  • December 2013
  • November 2013
  • October 2013
  • September 2013
  • August 2013
  • July 2013
  • June 2013
  • May 2013
  • April 2013
  • March 2013
  • February 2013
  • January 2013
  • December 2012
  • November 2012
  • October 2012
  • September 2012
  • August 2012
  • July 2012
  • June 2012
  • May 2012
  • April 2012
  • March 2012
  • February 2012
  • January 2012
  • December 2011
  • November 2011
  • October 2011
  • September 2011
  • August 2011
  • July 2011
  • June 2011
  • May 2011
  • December 2010
March 2016
S M T W T F S
 12345
6789101112
13141516171819
20212223242526
2728293031  
« Feb   Apr »

by Lloyd Melnick

All posts by Lloyd Melnick unless specified otherwise
Google+

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 1,337 other followers

Follow Lloyd Melnick on Quora

RSS HBR Blog

  • Building Influence Without Authority
    An emerging leader must learn how to increase her influence in a cross-functional role.
  • Audio Quick Take: Globe Telecom’s Ernest Cu on Adapting to New Customer Habits - SPONSOR CONTENT FROM AWS
    Sponsor content from AWS.

RSS Techcrunch

  • An error has occurred; the feed is probably down. Try again later.

RSS MIT Sloan Management Review Blog

  • How Executive Sponsors Can Help Address the COVID Gender Gap
    When I was growing up in former Yugoslavia, my father would arrive home from work with bouquets of carnations for my mother and me on International Women’s Day. My husband has continued the tradition for our children, but this year’s Women’s History Month feels bittersweet. I rejoiced in January when Kamala Harris became the first […]
  • An Emerging Landscape of Skills for All
    Developing the skills and infrastructure for lifelong learning was at the top of my agenda before the pandemic. But as the general economic fallout from the pandemic and the impact on jobs and job displacement become clearer, the issues of upskilling and reskilling are also moving up the agenda for corporations and executives across the […]
Website Powered by WordPress.com.
Cancel

 
Loading Comments...
Comment
    ×
    %d bloggers like this: