As Forbes accurately predicted in February of last year, Big Data is the hottest trend in IT right now, and it will only continue to get hotter. Predicting a $50 billion market by 2017, Forbes identified the several industries, service providers and opportunities within the Big Data "space" that were not to be ignored. Unstructured, semi-structured and structured data amounts show no signs of decreasing or leveling off leaving enterprises from retail to finance to healthcare overwhelmed and at a loss on how to handle it all. Not only are there various service providers to choose from, but, additionally, the man (or woman!) power is just not there. Vast amounts of data like those we see today are only a recent development and most high tech professionals do not have formal training necessary to manage these large sets of data and the technology that is necessary to organize them.
Course offerings through online education sites like Coursera, Big Data University (part of IBM), INFORMS' Analytics Professional Certification, combined with client user groups with vendors like IBM, Hortonworks and Cloudera help to train analysts and IT professionals how to harness the power of data. In March 2012, Computerworld identified Apache Hadoop as the hottest high-tech skill and this prediction was clearly correct. The training sessions and events provided by service providers were consistently sold out and always in high demand. The reality of Big Data is the need for highly skilled engineers that have the ability to deploy and manage technologies like Hadoop. Unfortunately, that pool is extremely small at the current point in time. The preferred breed of Data Scientist comes with a combination of math, statistics and programming skills needed to analyze Big Data. As a result, there is a huge opening for vendors that can help enterprises fill the Big Data skills gap with training, technical and professional services.
On the other hand, several surveys show that enterprises are willing to move forward with their information management implementation despite the lack of appropriate human resources. Both MIT and UC Berkeley have recently added Hadoop to their curriculums, but instead of waiting around for those future grads, enterprises must act now to begin organizing their data. Ninety-one percent of respondents to NewVantage Partners' Big Data Executive Survey Part II: People – The Key to Success said they are hiring external data scientists to influence their analytics processes, while 69 percent are attempting to train currently employed analysts. New positions that are best fit for Big Data analysts require a new tech specialty called "data science" to analyze data extracted and processed using Hadoop and using statistical analysis to derive beneficial insights.
With only 2 percent of respondents to the NewVantage survey reporting having no problems finding the right big data talent and skilled resources, we know this challenge isn't going away any time soon. It's important for enterprises to recognize the need for a very specific skill set within their employees. While it may be okay for now to move along while Big Data is still relatively new, that business plan will not work for long. Enterprises will need to work with solutions providers to train existing employees and additionally, begin sourcing the newest crop of data scientists who will have learned skills like Hadoop from the start. Similar to the way many CMOs of large corporations are not as up-to-date on social media as their recently graduated Marketing Associates, CIOs and CTOs will need to learn best practices and newest developments from their newly trained employees.
How does your organization handle mass employee trainings of new technology? Is the role of the "decision maker" changing based on the new knowledge of non-executive employees? Is Data Scientist really the Sexiest Job of the 21st century? Learn from leading information management executives about developing the best Big Data analytics business strategies at Big Data World Canada in Toronto in March!