The web is creating massive amounts of data every second of the day. Moore’s law states that the amount of digital information increases tenfold every five years. The social web is accelerating data. According to Cisco by 2013 the amount of traffic flowing over the internet annually will reach 667 exabytes.

An article in The Economist observed, we are at the point of an “industrial revolution of data,” with vast amounts of digital information being created, stored and analyzed. The rise of “big data” has led in turn to an increased demand for tools to both analyze and visualize the information.

The New Competition Is Data

Smart competitors are making significant investment in their ability to collect, integrate, and analyze data from anywhere and everywhere there is an engagement with the marketplace. By constantly testing, bundling, synthesizing, and making information instantly available across the organization the company had become a different and nimbler type of business.

McKinsey Quarterly article titled: Are you ready for the era of ‘big data’? states: Radical customization, constant experimentation, and novel business models will be new hallmarks of competition as companies capture and analyze huge volumes of data. Here’s what you should know.

What happens in a world of radical transparency, with data widely available? If you could test all of your decisions, how would that change the way you compete? How would your business change if you used big data for widespread, real-time customization? How can big data augment or even replace management? Could you create a new business model based on data?

Data Complications

Big data pools represent strategic opportunities, but leaders must also consider data complications. Talent is one of them. In the United States alone the demand for people with the deep analytical skills in big data (including machine learning and advanced statistical analysis) could outstrip current projections of supply by 50 to 60 percent. By 2018, as many as 140,000 to 190,000 additional specialists may be required education and training of key data personnel.

In the 1980’s American Industry awoke to competition from Japan. Japan’s competitiveness was gained by their advanced use of data to understand how to improve the quality of products and service and thus better serve the interest of the customer. The difference between then and now is the amount of data available and limited human resources who understand how to use it.

The social web is driving the explosive growth of data like never before in the history of mankind. The data is revealing consumer preferences, wishes and desires. Those who compete in the future will understand how to not only meet those expectations but exceed them. The only ones who will know how are the one’s whose people have deep analytical skills in mining and bringing meaning to data.