Converting Big Data to Smart Data

In Big Data, Data & Analytics, Featured on App by lizhen

Big Data_Smart Data

The 6 imperatives that you should know

“Big Data Analytics” is the new frontier to understanding complex consumer behaviour. The common practice right now is to get an extensive system to analyse every single byte and generate the collected information. After the deed is done, an infographic is produced, together with backed-up statistics. In short, that’s research for many companies.

There’s nothing wrong with that at all because really, that’s the only way to make sense out of massive amounts of data. However, don’t be fooled by the emphasis on size. It’s not really the about the size but the knowledge on how to convert big, ugly and unstructured data into smart compact data.

To get the most out of smart data, behavioural information should be combined with a consumer survey data and a social science perspective to generate information that guides the flow in understanding the pattern of consumer behavior.

Sure that sounds easy but how does it work? I’ve chanced across a comprehensive set of Six Rules That Should Govern Your Big Data Existence:

1) Don’t buy the hype of big data and throw millions of dollars away. But don’t stand still.

Take 15 per cent of your decision making budget and give it to the most intelligent person you know. Give that person the freedom to experiment in the cloud with big data possibilities for your company. Spend thousands for millions on people and not the other way round for developing ideas that have no structure to begin with.

2) Big thinking about what big data should be solving for is supremely important.

Big Data requires direction because essentially, it’s just a sea of information. Don’t just swim aimlessly and end up with nothing. Focus only on the important aspects to you business by developing a solid and grounded framework to the business without losing sight of the objective.

3) The 10/90 rule for magnificent data success still holds true.

The 10/90 rule is a safest bet in making smart decisions that can be applied everywhere. Invest 10 per cent in the tools and vendor services but the other 90% should be on the people. It might seem ridiculous at first glance, but the best investment made has always been proven to be people and not on products, in this case artificial intelligence that isn’t quite developed.

4) Shoot for right time data, not real time data.

Real time data analytics is impressive technology but analyse your options again, do you really need real time statistics? The truth is, every company has a complex decision making structure that is time consuming and therefore unable to react in real time. So if you can’t react in real time, why do you need real time data?

5) “Data quality sucks, just get over it.”

It’s easy to understand big data through classification. Like everything else that requires tedious explanation, drawing comparisons through segmentation helps a lot.

For this case, Big Data is separated into three key segments: Known Knowns, Known Unknowns and Unknown Unknowns. Why bother wasting time on data quality when the unknown unknowns simply refuse to be revealed? It’s just bad business to invest on diminishing returns.

Alternatively, focus on collecting, processing and storing the Known Knowns as well as taking the time to analyze and figure out the Known Unknowns.

Through this visualization, the only thing left is to think fast, move fast and slowly become godlike over time.

6) Eliminating noise is even more important than finding a signal.

With big data, it is so much more important to be magnificent at knowing what to ignore. The key is to separate out all the noise (Unknown Unknowns) in the massive sea of data to even have a fighting chance to start to look for the signal.

If you are not magnificent at knowing what to ignore, you’ll never get a chance to pay attention to the stuff to which you should be paying attention. Tuning your data algorithms to first ignore and then hunt for insights.

Need help with converting data and understanding why your consumers behave in a certain manner? Discover the solutions that will help you with just that at Big Data World Hong Kong and Big Data World Asia this September and October respectively!

[Image: The Telegraph]