IDC Released their 2011 Digital Universe Study and the results are pretty amazing: data is doubling every two years! This is the fifth year that the IDC has released this study and each year I continue to be surprised by the results, just when I think things have started to reach terminal velocity around data growth they accelerate more. Currently data growth is outpacing Moore’s Law, suddenly the phrase Big Data just doesn’t seem to cut it any more. There are all sorts of findings in the study and the repercussions for our industry will require many changes.
I recently wrote here about the need for automation in security, and Christopher Hoff has suggested some practical ways to get started here and has started an initiative around Security Automata here. This is one of the ways that the growth of data is impacting security, the very framework for how we approach protecting assets needs to change in light of the deluge of data.
Another interesting finding had to do with the anticipated growth of data versus the growth of the number of IT Professionals, I think the image below sums it up rather nicely.
Over the course of the next 9 years data will grow 50x and the number of IT Professionals is expected to grow only 1.5X. Now that could sound like there’ll be good job security for IT folks, but this is a scary finding! If we don’t immediately change the way we manage our environments we aren’t going to be able to keep the best and brightest in our industry, much less be able to meet growing customer expectations around agility, flexibility and added value. The rise of consumer technologies that allow end users to have ready access to large volumes of data at an end point of their choosing has lead to an increase in end user expectations in the enterprise. This trend is only going to continue. So, higher expectations, larger amounts of data and only a marginal increase in the number of professionals curating this data sounds like a recipe for trouble.
Another interesting finding of the study is that during this same period the number of files\objects that this data is stored in will increase 75X. Not only will we have much more data but it will be spread all over the place and replicated and acted on by a number of collaborators, &c.. Thankfully it seems like object-based storage finally has some very good solutions that can readily scale to help meet these problems. We will however need engineers and architects who can help write and enforce file and object policies as a part of a holistic GRC approach to ensure that all of this data and its containers are manageable, secure, compliant and stored in the most cost efficient manner for their class.
There are many warning signs associated with the findings in this study, but it also highlights many opportunities for innovation, new career paths and smarter thinking about how we create, curate, share and protect our data.