Confusion in the Age of Data

The key is human intelligence

“Contrary to popular opinion, we do not yet live in the Information Age. At best, we live in the Data Age – a time when bits of data constantly zoom past our eyes and buzz past our ears, yet few of them inform us meaningfully and usefully. We’re spending millions to put all of that “Big Data” into “The Cloud” without first learning how to separate the signals from the noise. A storm cloud of our own making is already raining confusion down upon us.”
These were the words of Stephen Few in his blog ‘Visual Business Intelligence’, following his keynote address at the recent Teradata Universe conference in Dublin. I think ‘the Data Age’ is a great description of where we currently are. There seems to be a growing belief in the somehow magical properties of big data, and a corresponding exaltation of those leading the way with big data and predictive analytics. Data is the new oil. Data analysts are now data scientists. And if we have all the facts and we apply comprehensive analytics, we will discover the past, understand the present and predict the future!
The point here is not that analytics and big data are useless. In fact the only problem with big data is the hype that surrounds it. This hype detracts from the real key to getting value from data, which is human intelligence. Information only becomes valuable when it is understood, not just when it is made available. Systems can spot subtle patterns in large data sets but, the question of what is a significant pattern and what it might mean, remains the preserve of human insight.
‘Business Intelligence’ also tends to ignore two other important aspects of decision making – intent and intuition. All decision making includes the intent of the decision maker. That intention drives everything from what data is gathered, through how it is evaluated, all the way to the final choice of action. Equally when faced with a significant decision, the intuition of an experienced manager becomes an important and influential factor.
The chatter about Petabytes, Zettabytes and Yottabytes, and the systems to manage them is unlikely to diminish anytime soon. When we can shift the analytics focus from the technology to the contribution of the human mind, we will truly start to move closer to ‘The Age of Information’.