In the first of two special guest blogs from our friends at EMC, Big Data World Europe keynote Chuck Hollis gives his thoughts about how we as humans need to proactively get better at making use of the data that surrounds us.
Evolved for survival on the Serengeti, we’re now attempting to thrive in a digital world that we weren’t programmed for.
Sure, we can adapt — over time.Â Through thoughtful reinforcement, we can learn to overcome many aspects of our evolutionary programming.
But being aware of the problem is half the challenge.
I’ve noticed that there’s more and more conclusive evidence that we — as a species — weren’t wired to work effectively with data.Â It’s something we learn to do, often despite ourselves.
And in the coming era of big data analytics, we’re going to have to collectively get better at it.
Much, much better indeed.
Have You Been Sucked Into The Big Data Analytics Vortex?
If you haven’t, you may be soon.Â It builds in your brain until you just can’t stop thinking about it.
My personal realization is that we’re living at a rather special time in history.
For the first time in our society, data about everything is relatively plentiful, with more coming every minute.Â The technology resources needed to move, amass and process said data is relatively plentiful.Â The skills, sophisticated math and machine learning algorithms are also becoming more widely accessible.
And there is no shortage of evidence that — when combined — the result is amazing predictive insight about the world around us.Â It doesn’t really matter all that much what topic you’re interested in: business, research, social good, technology, crime, politics, science, etc.
Gather enormous amounts of uncorrelated data, invest the time to correlate it and extract insights, and mind-bending things are now routinely happening.Â I’m not amazed by the stories any more.
At one level, it’s right up there with the discovery of fire.
I’m hooked.Â Many of my co-workers are hooked.Â Quentin Hardy at the New York Times appears to be hooked.Â McKinsey is hooked.Â The Davos crowd is hooked.Â To many of us, it looks like humankind is poised to take this great, historical leap forward as a society.
But there’s just one problem.Â It’s us, or — more specifically — our brain’s ability to reason and act intelligently in the face of analytical data.
It’s often true at an individual level.Â It’s largely true at a company and organizational level.Â It’s most definitely true at a political and societal level.Â We don’t handle data well, and we make poor decisions — or fail to act appropriately — as a result.
I came across this excellent and thoughtful article by James Guszcza and John Lucker in the Deloitte Review which is definitely worth reading in its entirety.
They had me at the opening quote:
"The most difficult subjects can be explained to the most slow-witted man if he has not formed any idea of them already; but the simplest thing cannot be made clear to the most intelligent man if he is firmly persuaded that he knows already, without a shadow of doubt, what is laid before him." â€” Leo Tolstoy
My spin? Our intelligence and perspectives can become intellectual prisons from which escape can be difficult, even in the face of compelling data.
The article also recounts one of the many psychology experiments where subjects make poor decisions, especially in the face of strong evidence to the contrary.Â We collectively tend to reject data and conclusions that don’t fit with our world model and personal experiences.
Why This Is A Real-World Problem
As part of my role, I get exposed to many organizations that are bringing big data analytics into their organizations, including here at EMC. For the first time, amazing predictive models are bubbling up, with mind-bending implications that fly in the face of current practices and current organizational wisdom.
The result can be spectacular conflicts between perceptions of how the world works, and how it can be proven to really work.Â And sparks can fly.
I joke that data science denial goes through three phases.
1 — Your data sucks.
2 — Your model sucks.
3 — You suck.
Funny?Â Yes.Â Until you see it unfold exactly that way a dozen different times.
New Leaders Required?
All of the sudden, the role of the strong business leader becomes oh-so-critically important.Â This individual has to steer the organization towards the new model, the new process and leveraging the new insights — and do so without blowing things up along the way.
Most insightfully, the Deloitte team offers practical advice on this key aspect of organizational change management in a data driven world.
Are We Making Things Worse For Ourselves?
Way back when, I took a lot of coursework in statistics and probability.Â I found the subject absolutely fascinating in college, but couldn’t — at the time — figure out how one might go about making a living at it.Â Besides, useful data was hard to get to, computing resources were expensive, the tools were a pain to use.
I think I was 30 years too early.Â If it was today, I’d probably end up chasing a data science career.
As part of the recent Data Science Summit at EMC World 2012, there was a most excellent rant on this subject from Michael Chui, Senior Fellow at the McKinsey Global Institute.Â Check this out, and see if you agree:
"We've got to stop teaching so much calculus," he said. "I think we should teach more stats. Who does an integral anymore in business? There are a few people in engineering. But who needs to understand conditional probability, and who needs to understand selection bias, and all those things that a data scientist just wakes up with and understands. One of the things we need to do in addition to solving all those technical problems is somehow effect the education â€” I don't just mean formal education, but the way that people are thinking broadly throughout our organizations."
Put differently, are we teaching our students the skills they needed for the last economy, or to thrive in the next one?
Creating The Analytically Enabled Organization
One of my personal heros on this topic has been Tom Davenport, who’s written several excellent books on what it means to be an analytically enabled organization — and how they tend to perform better than those that don’t.Â I’ve read his books.Â They’re spot on.Â And I think he was perhaps 10 years too early in his thinking.
That’s understandable, H.G. Wells was a century early:
"The time may not be very remote when it will be understood that for complete initiation as an efficient citizen of one of the new great complex world wide states that are now developing, it is as necessary to be able to compute, to think in averages and maxima and minima, as it is now to be able to read and write." H. G. Wells, Mankind in the Making (1904). Wells is commonly paraphrased as having written "statistical thinking."
It’s On Us As Leaders
The picture is clear — if we’re as leaders are going to help our organizations and those around us grow and thrive in this new era, we’re going to have to do our part.
Investing in creating big data analytics is going to turn out to be the easy part; getting people to engage and change their behaviours around the new insights provided is going to be where the really heavy lifting will need to be done.
And here’s the hard part: the more you think you know about something, the harder it will be to initiate meaningful and substantive change in the face of data.
Tomorrow’s business champions may end up being better described as “data champions” — strong, insightful individuals that seek out compelling insights, and are willing to buckle down to the unenviable task of proving to people that they don’t really know what they think they know — and getting them to do something about it.
And simply putting up the platform to do this work will end up the easy bit
Content originally seen on Chuck's Blog – June 7th 2012.
Come back tomorrow to hear more from Chuck. In the meantime check out part one of our interview with Big Data World Europe speaker Nilan Peiris.