One of the popular buzzwords these days is “Big Data,” but few people, even in companies that use analytics extensively, really know what this phrase means. A recent article, co-written by one of my favorite authors, Thomas Davenport, in the MIT Sloan Management Review titled How Big Data is Different does a great job of explaining the concept and showing how it can be applied to social media.
Big data starts with all the data your company is collecting but goes well beyond it. It includes clickstream data from your games, web analytics, social media content (Tweets, blogs, Facebook wall postings, Pinterest Pins, etc.), AppData information and even YouTube views. Big data, however, also includes everything from customer service requests to game development processes and learnings. As the article points out, very little of this information is formatted in the traditional structure of conventional databases. Companies do three things to capitalize on this plethora of data:
- Paying attention to flows as opposed to stocks.There are several types of big data applications. One would be processes that detect hacking and fraud through continuous customer monitoring. A second is continuous process monitoring to detect changes in customer sentiment towards a game. Another could be exploring network relationships between players. All of these applications are based on a continuous flow of data rather than a “stock” in a data warehouse. Rather than using data to assess what happened previously, game companies should think in terms of continuous flows and processes. Also, some of these environments, such as player sentiment analysis, are not designed for automating decisions but are better suited for real-time monitoring. The increased volume and velocity of data means that your company should develop continuous processes for gathering, analyzing and interpreting data.
- Relaying on data scientists and product process developers as opposed to data analysts.People who work with big data need substantial and creative IT skills because of the way they need to interact with the data: obtaining it, extracting it, manipulating it and structuring it. They also need to be close to the game development process. For a social game company, it also often makes sense to have the data scientist as part of the development team rather than a separate analytic team.
- Moving analytics into core business and operational functions. Capturing, filtering, storing and analyzing big data flows can swamp networks and database platforms, even at sophisticated social media companies. This surging volume of data requires major improvements in database and analytics technology. Your company needs to be looking at the cutting-edge products designed to deal with big data, like Hadoop. Other ways to deal with big data is to leave it where it is and create virtual data marts that allow different data scientists to share existing data without replicating it. I am not an engineer so I won’t explore further the types of options available but want to stress that big data requires you to take a different approach to the role of IT in your business. Rather than a support function, building the systems for big data is central to IT’s role.
In the Davenport, et. al., article, a key principle is that the world and the data that describe it are constantly changing, and organizations that can recognize the changes and react quickly and intelligently will have the upper hand. I have written several times about how you can use analytics to create competitive advantage; this analysis of big data shows how you can also use your data better than your competitors.
