Big Data- Innovation in Data Analytics

What is Big Data?

It's one of those keywords that have become increasingly prevalent as our use and consumption of information has become more and more demanding  Here's a summary of what I've found to be the basics."The term Big Data applies to information that can't be processed or analyzed using traditional processes or tools.

IBM defines Big Data by three characteristics: volume, variety, and velocity.  What does this mean?

• Volume:  The sheer amount of information collected on a daily or even hourly basis by organizations is massive, reaching to petabytes.  Imagine having to analyze and sift through for nuggets of relevant data.
• Variety:  Data is being collected from a whole range of sources, from sensors to smart devices, emails to documents, social media forums, etc.  Combined, these data generators bring in both traditional and raw data that may or may not be structured.
• Velocity:  How fast is the data flowing?  It's not enough to know how quickly it is being produced, stored, analyzed, but we also need to know how quickly the data will become obsolete.

So the simplest way of describing “big data” is to say “lots and lots and lots of data”

What is all this data and where is it coming from? Well, it’s everywhere. Think about your working day and you’ll realise you interact digitally with organisations, hundreds, if not thousands of times: you check your emails on your phone, send tweets, order something online, interact with websites, buy your grocery from the local supermarket, and so on. All these interactions create data points that can be captured.

As it happens, every day we are generating 15 petabytes of data (where 1 petabytes is 10 raise to 15 bytes) and 12 terabytes of tweets worldwide. We create 350 billion meter readings per annum, and 500 million call data records. And those examples are just the tip of an enormous data iceberg. That’s where data analytics comes in. Data analytics is the process of making sense of all that data and drawing out useful patterns and insights.

What kind of insights can data analytics provide?

• Find new efficiencies and optimize performance
• Build incredibly detailed customer profiles on an individual and group basis
• Give customers an unprecedentedly customized user experience
• Help you draw meaningful insights and behavior analysis from social media activity
• Reduce complaints
• Dramatically improve operational efficiency, customer engagement and customer retention

All this adds up to serious competitive advantage. In fact, IBM has found that organisations that use analytics for competitive advantage are 2.2 times more likely to substantially outperform peers.

So why have big data and business analytics suddenly become such hot news?

But the power of big data and analytics are not just being realised by chief information officers and technologists; increasingly, senior management across all disciplines are coming to see that to stay ahead of the competition, they need to understand their customers, their business and their futures in a more detailed and more useful way than rivals’ organisations.

At the same time, customers are coming to expect significantly more personalised and more seamless experiences with businesses, across multiple channels. That means there is huge pressure on businesses to join the dots between all those different interactions, track them, and provide a seamless customer experience regardless of when, where and what device the customer has chosen to use at any given moment. One can also draw out invaluable insights into customers’ behaviour. Patterns and insights based on those browsing and buying habits might include: which channel gives you greatest ROI; how and when a customer is most likely to convert to a sale; how an individual customer prefers to communicate and interact with your business; how often they’re likely to buy again from you and at what price; who are your most valuable customers are; and so on.

Isn’t all this just for large corporates? It all sounds expensive and complicated. Absolutely not. Medium-sized businesses are realising the potential that analytics holds for their businesses, and how it will be helpful for surviving and thriving in the future. Different areas of industry have profited by this like healthcare, Automobile Industry: the biggest example being FORD, music, social networking – E.g. LinkedIn, insurance- Met Life Insurance, and television shows, and so on. I have explained few areas where it has been used innovatively as below

Presidential campaigns – Obama’s team built a volunteer army on Facebook using big data: The key was a model for determining who among its followers were the best messengers, who they might be able to persuade, and what actions they might be willing to take. So, rather than blast all of President Obama’s 30 million Facebook fans or 20 million Twitter followers with the same plea for cash or neighbourhood organizers, the campaign was able to make informed decisions about whom it asked for what, and how it asked them.

Highway traffic: Thanks to a pilot program called ExpressLanes between Xerox and the Los Angeles County Metropolitan Transportation Authority that uses big data to keep traffic moving for drivers on the I-10 and I-110 freeways who are willing to pay. If a driver is paying to drive in the HOT (high-occupancy tolling) lane, he’s guaranteed a consistent speed of 45 miles per hour. If traffic starts backing up, prices for individual cars will rise to discourage them from entering, saving the lanes (which, before this program were high-occupancy-vehicle lanes) for high-occupancy vehicles such as buses and those involved in carpools.

Sports: Basketball Fan Mark Phillip’s creation is an application called Are You Watching This? Or RUWT: Most of the sports fans might already be familiar. It’s already available on iOs and Android devices, as well as on web browsers, and it constantly analyzes streams of sports data to let fans know what games they should be tuning into and where they can find them on TV and enables them to vote on games. For Google TV and TiVo users, RUWT actually lets them change the channel to tune in to a game.

More than a decade ago, the music metadata company Gracenote received some cryptic advice from Apple to buy more servers. It did, Apple launched iTunes and the iPod, and Gracenote became a metadata empire.

–From Gracenote co-founder on ‘iPod day’ and better music through data 

There is a big bang expected in the next 5 years in use of big data and already the noticeable sparks have started.

About Author:

Padma Nambi is a consultant in Systems Plus Pvt. Ltd. Within Systems Plus, she actively contributes to the areas of Technology and Information Security. She can be contacted at padma.n@spluspl.com