Major IT spending to cope with "big data"

According to an IDC Digital Universe study published in June this year, the amount of information created and replicated will surpass 1.9 zettabytes (1.8 trillion gigabytes) in 2011, growing by almost nine times in just five years. 

 

Other complicating aspects of "big data" have recently come to light. The variety of data sources is growing at a rapid rate, particularly as we move into the semi-structured and unstructured realm – such as social-media interactions, rich-media files and geospatial information. The other emerging factor is the increased velocity at which data is being generated, such as real-time sensor data feeds from smart meters. 

 

"These new aspects of 'big data' are creating unprecedented levels of complexity for IT executives, particularly as they realise that these massive data sets cannot be processed, managed and analysed using traditional databases and architectures," said IDC Asia Pacific associate vice president Philip Carter. "What is becoming clearer is that the real value from 'big data' will be derived from high-end analytics, predominantly using data mining, statistics, optimisation and forecasting type of capabilities to proactively turn this data into intelligence to drive business benefits and better decision-making capabilities."

 

In line with this trend, businesses in Asia are harnessing analytics-led solutions to gain better customer insights and manage risk and financial metrics more effectively. IDC market surveys in February and June found that data management and analytics were regarded as top priorities to gain significant competitive advantage. 

 

However, the approach to business analytics will have to be significantly different to the traditional approach. To run an analysis on 'big data', the variables and models are likely to be entirely new. Therefore, a different infrastructure strategy and perhaps most importantly, new skill sets, are required, Carter said.

 

Organisations must begin looking at deploying not only the applications traditionally used for business analytics, but also the supporting architecture in order to scale efficiently. IDC recommends looking at cloud bursting, the deployment of analytical appliances, and creating truly scalable enterprise architectures that take advantage of high-performance computing. 

 

Next-generation business analysts will be more akin to "data scientists", Carter said. These individuals will have strong statistical skills and will be able to extract information from large datasets and present value to non-analytical experts. They will also have the unique skill of understanding the new algorithms and analytical models that will have the most significant business impact in the short term.