What Is Big Data and Are We Using It Properly?

In Big Data, Data & Analytics, Featured on App, Technology by Vaughn Highfield

What Is Big Data and Are We Using It Properly? (Image: Camknows)

Every day we create 2.5 exabytes of data, which equates to 2,500 petabytes or around 2.684 x 10^9 gigabytes – which is an awful lot of information. If that wasn't enough of a mind-melt, 90% of all the worlds data was created in the last two years alone.

That is, obviously, beyond the realms of ‘big data', instead verging into something even bigger and more unfathomable.

The self-driving Google car currently puts out around 1 gigabyte of pure data a second, which is still probably far more data than any standard company will be dealing with each day.

For many retailers and suppliers, the only data they'll be making use of is customer data. What they're buying, where they're buying it from and how often they're buying it. They'll also be measuring how often a product is mentioned or a consumer interacts with the brand or spreads its message across social networks.

This is what big data really is and means. It's so much information on the nitty-gritty details of your customers' lives and shopping habits that you really just feel lost in how to deal with it.

But is anyone really using it properly and does it actually matter?

Well, you can easily understand why some people feel the need to say this is nothing more than another buzzword in the marketing cannon – ready to be fired all over data-hungry companies in a scene reminiscent of a Bugsy Malone drive-by.

But while big data may not be as big as everybody initially thought it would be – with many overcompensating with their server capacities for relatively low-level number-crunching (pdf) – it's data that's hugely important in personalising and improving customer interaction and satisfaction.

While Yahoo may needlessly make use of a huge cluster of computers to crunch 12.5 gigabytes of data – something a standard computer couldn't handle, yet a server could – they know the importance of handling this intimidating set of numbers and figures.

What hasn't helped is the confusion between big data and "data analysis", which are two vastly different things. Big data is not a synonym for analysing data, it's merely a descriptive term that encompasses the vast array of information gathered from various channels. Once it's been analysed it's digestible and is nothing more than data. Nothing big, nothing scary, just data.

This begs the question of why it's ever called "big data" if it's now easily devoured by servers and clusters. Surely the bar for big data has just grown and now everything we're working with is just data again?

Again, another valid argument, but – as I stated earlier – the term has grown to encompass the big impact it has upon customer interaction, so it's no longer about size.

One trouble does still remain, how do you make sure it's useful data? And how do you use that data usefully?

Well, that's exactly what will be discussed at Europe's Customer Festival during the Big Data World section of the event.

It revolves around ensuring that you've got all of your data pointing where you want it. Why collect the addresses of your customers unless you need it to push location-based promotions? Indeed, why record the granular details if you only plan to send out a blanket invite?

Nobody likes to be generalised, and that's why you have to make sure the data you're recording isn't either.