Business and IT leaders recognize the importance of big data. But putting it to work presents growing challenges.
A new report from consulting firm Capgemini, titled “Successful Big Data Implementations Elude Most Organizations,” found that global organizational spending on big data exceeded $31 billion in 2013 and is projected to hit $114 billion by 2018. Last year, big data far outranked any other technology trend. In fact, Capgemini found that 60 percent of executives believe it will serve as a major disruptor over the next three years.
But putting big data to work and delivering clear-cut results is no simple task. In all, 35 percent of the companies surveyed (226 big data executives overall) have integrated big data and predictive insights into some business operations, while 29 percent are still in proof-of-concept for selected use cases, and 19 percent have gotten to the budgeting stage but identified focus areas. Only 13 percent have integrated big data and predictive insights extensively into business operations, and 5 percent haven’t implemented the technology nor have they allocated a budget.
Yet organizations are struggling to achieve positive results, Capgemini reports. Only 27 percent of respondents described their big data initiatives as successful, and 8 percent described them as “very successful.” Meanwhile, proof of concepts had a success rate of only 38 percent. What is standing in the way? According to the report, three major obstacles exist: dealing with scattered data silos, ineffective coordination of analytics initiatives and the lack of a clear business case for funding big data initiatives.
In fact, 79 percent of respondents indicated that they have not completely integrated data sources across the organization, 67 percent lack well-defined criteria to measure the success of big data projects, and 54 percent do not have joint project teams that allow line of business and IT executives to collaborate on initiatives. However, among more successful organizations, big data initiatives receive funding and mindshare. In fact, 53 percent of companies that have achieved success have established a distinct business unit with an analytics team focused on big data, and 43 percent have a central team in place.
By contrast, only 27 percent of successful organizations take a decentralized approach with separate analytics teams for separate departments, and 20 percent have ad-hoc and isolated analytics teams in place. Overall, the group with dedicated analytics units outperformed those taking a more ad-hoc approach by a factor of 2.5. Other key factors, according to Capgemini, are establishing a roadmap with timelines and milestones, identifying well-defined criteria for use-case selection, and introducing well-defined KPIs to measure the success of initiatives.
Finally, the report noted that a top-down leadership approach is necessary to drive adoption and digital innovation. “Organizations that have successfully implemented big data initiatives usually have clearly defined leadership roles for big data and analytics,” the authors noted.
Yaniv Mor, CEO and co-founder of data and cloud services firm Xplenty, believes that the findings represent many of the same challenges that IT executives faced before the big data era emerged.
“Of course, there has been an evolution of these challenges, but the core issues of finding the single version of the truth, the need to have a corporate champion for your data project, and the need to have proper governance and well-defined KPI’s are more relevant than ever,” Mor said.