With apologies to Chuck Hollis at EMC and James Governor at RedMonk I decided to take a crack at this whole “Why Applications are like fish and Data is like Wine” meme by extending it to posit that Information is like Cognac. Now, I’m not usually one to kick a dead horse but I think that all the talk of Big Data has maybe obscured something that I view as a problem with Big Data: knowledge workers don’t consume data, they consume information. I see Big Data as a problem quite frankly, and the IDC Digital Universe Study put it in context. If Big Data is the problem, Big Information is the goal, and to get there we need automation and analytics. So if you’ll bear with me I’ll share how I think Information is like cognac.
Cognac is made of wine, and not just one varietal or vintage, it is a carefully crafted blend, cared for and created layer upon layer to produce something otherworldly when done correctly. Cognac is crafted by real artists, masters who guide the spirit through it’s distillation and aging, they understand how to bring together the different grapes, wines, cognacs to deliver something that is greater than the sum of its parts. Information is like that, it takes in data, from many different systems, formats, time periods and through analytics is able to give the engineer, scientist, executive, whomever something meaningful and beautiful out of all that chaos. Knowledge workers and data scientists are like those artists who blend the cognac.
I was lucky enough to take a class on Knowledge Management ten years ago while I was pursuing my Master’s degree and it’s one of the classes that’s really stuck with me. One of the most intriguing concepts we studied was the Wisdom Hierarchy, which is sometimes referred to as the DIKW hierarchy: Data; Information; Knowledge; and Wisdom. Data is the raw material that this framework is built on, but ultimately the more data you have the more that stands in your way to true knowledge. Information in this system is Data + Analysis, something that has meaning attached to it. Data in and of itself isn’t really useful, Information however is, hence Big Data is a big problem, or in the very least a big distraction.
So, IDC tells us that Data is doubling every two years, it’ll be 50X what it is now in 2020 while IT Professional ranks will only grow 1.5X. So clearly we will need much more automation in our infrastructure and data management tools. We will definitely need a lot more automation in our analytics systems in order to be able to turn this huge pile of data into something useful. The number of files, or objects or containers that all this data is stored in will increase by 75X, so our solutions will need to be automated, adaptive and federated. And that’s just to turn all that data into information. Really what we want to create is Knowledge, and as someone who is engaged and enthusiastic about this space we eventually want to spread Wisdom throughout our enterprises. To enable that we’ve got to make it a lot easier to create new applications and ways of sharing not just the Information but the context and insight that makes it Knowledge. This is part of the point of Chuck and James’ posts, Applications, like fish, are best when fresh and up to date, long requirement definition and software development lifecycles pretty much guarantee that many applications are rather stinky upon arrival.
But wait, there’s hope! There are many, very cool efforts and initiatives underway to deliver on this. I’ve been very impressed with initiatives in this space from GreenPlum, R, NoSQL, Tableau and many others to make data more pliable and easier to turn into information. I am admittedly biased, but I think that what EMC’s Intelligent Information Group is doing with Documentum xCP and its Application Composition tools is outstanding. Also, I was lucky enough to attend the Data Scientist Summit in May and the work that companies like Via Science, Kaggle, 23andMe, Code for America, and Factual is really inspiring and mind blowing.
So, Big Data, Big Information, whatever we want to call it, color me all in, this is going to be a wild ride, and hopefully we’ll all get some wonderful cognac out of it. Cheers.