What Big Data Means For Your Business?

This article is for businesses across various sectors including retail, telco, pharma, digital marketing, airlines, utilities etc, who are interested to know more about Big Data and the multitude of opportunities it presents. It starts with how to identify if businesses have a big data problem, then proceeds to explain the key drivers of Big Data and finally describing 3 ways Big data can benefit businesses.

First, do you have a Big Data problem/opportunity?

Aberdeen Group recommends measure of anything more than 5 TB of data as Big Data. That’s a good benchmark to quickly qualify if businesses have a big data problem. There are two other dimensions as comprehensively defined by Gartner and they are a) if businesses generate data in real time leading to a torrent of data every second and b) if the data has variety i.e. not just machine generated or transactional data in databases but also videos, emails, chats, audio files etc that are not easily query-able. Some experts argue that the volume and velocity has been a familiar problem but ability to process variety is what is leading the big data opportunity. We will look at the drivers of big data adoption in the next section.

Consider data across all customer touch points

A common problem in organizations is that data about the user is spread across multiple data sources. For e.g. the online behavior of the user is captured in web logs whereas the purchase history is captured in financial data marts and the inventory levels are captured in ERPs. So, when counting the size of data, add up data across all sources that are collected from various customer touch points. These include customer support calls, emails, chats; user’s responses to online or mobile marketing campaigns, social media mentions etc.

Take the short test – do you have a big data problem?

• Volume – Is your data more than 5TB/month? Yes/No
• Velocity – Is data being generated at a rapid pace in real time? Yes/No
• Variety – Is data manifested in unstructured formats such as emails, chats etc? Yes/No

If you have answered yes to any one or more of them, then you have a Big Data problem! Congratulations! You are sitting on a gold mine!

Key Drivers of Big Data

According to the joint study by IDC and EMC, the size of the digital universe in 2012 was 2837 exabytes and is expected to grow to 40,000+ exabytes by the year 2020. That’s not far off – its only 7 more years now! (An exabyte is a billion gigabytes).

The key business driver is the fact that 90% of the data in the digital universe was made in the last 2 years alone and that we are still growing at 40% per year according to Aberdeen. We are all walking data generators, thanks to mobile devices and the internet of things! The promise of Big Data is not about the growing torrent of data but the ability to answer questions businesses have long been asking! These questions have always remained with the business but they had to be contented with optimal accuracy. For various technological and cost reasons, only sample sizes of large data sets were considered in statistical processing. Big data now makes it not only feasible but also cheaper to process large data sets. So, business starts to benefit from increased accuracy of predictions and recommendations.

The key drivers of Big data, in my opinion, are mainly technological and they are:

1. Hadoop and its ecosystem – Hadoop is a data processing platform. It is not a database and comes with a suite of scheduler, workflow and query tools. It is fundamentally a file system that can be configured over commodity hardware and that enables compute where the data is stored instead of having to move the data around to where the compute is. The advent of Hadoop has catalyzed processing large volumes of data. Not all applications are meant for Hadoop – only where the data processing can be contained to the location. There are also several other open source frameworks that extend Hadoop. More on the technological landscape of Hadoop in my next article.

2. Cheaper Storage – Cost of a gigabyte of storage has fallen from $1M in 1980 to $0.10 in 2012. And its still headed downwards!

3. Cloud computing – Elastic compute power and Infrastructure as a Service has taken away the complexity for businesses to set up their own IT infrastructure and data centers. Large businesses have their own private clouds.

Big data: what is now possible for businesses?

Imagine a customer walking into a Saks 5th Ave store in New York and receiving personalized recommendations on their mobile device for him or her on what might best suit them based on several factors including season of the year, size, likes and dislikes, word of mouth recommendations, past purchase history and more importantly – what is available at that moment in the store down to the color, size, brand and price (including promotions)! Its almost like the shop owner knows you since 20 years!

Here are 3 things that Big data can do right now for business:

1. Improve topline through unprecedented personalization at scale.

2. Improve bottomline through unprecedented efficiency at scale

3. Improve governance through unprecedented monitoring at scale

Let's look at each of them with examples. It is a matter of time big data processing will become mainstream with enormous implications for businesses and governments!

Unprecedented personalization at scale!

Businesses for a long time have asked the question – what will make my customer buy? In the olden age of relationship based, single chain selling, the shop owner used to know their customers personally to recommend products. It is now possible again with the use of Big Data. The case of Target recommending pregnancy related products to a family even before they knew their family member was pregnant is a well known case in personalized targeted promotions. Call centres are investing in technologies that derive insights out of text mining emails and chats between support centres and customers offering personalized advice there by cutting time to resolve issues and delighting customers. Personalization will transform the following areas:

a. Acquiring new customers

b. Retaining and upselling to existing customers

c. Customer support services

Unprecedented efficiency at scale

Harvard Business Review reported an airport reducing wait times for in flight airplanes about to land thereby saving millions of dollars for the airlines to help run their flights efficiently by looking at several parameters in the past such as weather, air traffic etc and asking the question what was common whenever there was a delay. Here is another example in the retail sector that could use Big Data. Suppliers right now don’t have visibility SKU level inventory turn around in real time. As a result, they have to wait for reports at the end of the day or week to know how the products are moving on the shelf. For e.g. they can’t tell right now how long a SKU has been out of stock. If suppliers can find out SKU level they can quickly take decisions to restock. Right now, there is no visibility at the movement by SKU level because the data is too big and not accessible. Likewise, efficiency can be looked across the supply chain operations including logistics, manufacturing, operations, merchandizing, shipping, stocking etc.

Unprecedented monitoring at scale

CCTV is recording streams and streams of data in public places. There is not enough manpower to watch all the recordings. It is sometimes used as a deterrent. Being able to process unstructured data of video streams and run pattern matching algorithms to identify interesting cases could have huge implications for security of citizens all over the world. Similarly, enforcing traffic rules where monitoring by limited staff means limited success in controlling accidents in an overcrowded planet. Unprecedented monitoring is applicable in the cases of anti-virus and fraud detection as well which need to look at historical data for non obvious patterns.

These are the 3 main questions businesses can start to ask. Of course, the era of Big data is just beginning and the possibilities are just emerging. We will see a lot more applications in the coming years.

Summary

Businesses should be really excited to be in this technological phase of big data! The business goals have not changed with the advent of Big data. Only the ability to answer those questions with depth has changed. Change is probably an understatement. Soon, businesses will be transformed with personalization, efficiency and governance at scale! What do you think? Please share your thoughts.

 About the Author

Ravi Padaki is the Founder of Pravi Solutions. Ravi is a Product Management and Marketing professional with technology background. Ravi has over 15 years of experience in the software industry with expertise in big data analytics, online display advertising and eCommerce. Ravi worked in US for Sybase (now SAP), Kodak, HP in the past. He was most recently with Yahoo R&D India, Bangalore, product managing line of analytics for Right Media ad exchange platform. He is also the Founder and President of India Product Management Association. He also blogs on delicious data strategies at http://datakulfi.wordpress.com

YourStory.in & Accel Partners to a host a Big Data session with Cloudera CTO and Co-founder, Amr Awadallah on 14th March in Bangalore. Register here to attend the event.