It’s not just a hard drive, it’s a treasure map

Just like the secret treasure map hidden in the Declaration of Independence in the movie National Treasure, hard drives all over the world may hold the keys to a better world, and a better understanding of history and what it can teach us through data. In the movie Nicolas Cage played a historian. Today’s treasure hunters – mathematicians.

Tech Crunch featured a great guest post by Semil Shah titled Big Data Needs to Think Bigger.  Check it out.   The post talks about harnessing the data that exists on storage to arm mathematicians with the ability to sift through the data using complex algorithms that potentially could yield “new products or services,  find new deposits of minerals, oil, or gas buried deep in the ground or remote parts of the ocean bed, target geoengineering tactics high up in the clouds to combat global warming, when used in financial markets, not only to notify us of fraudulent behavior, but also to prevent market movers from profiting during bubbles while the masses get doused after the bubbles pop, or analyze seismic activity to predict earthquake likelihoods and tsunami arrival times.” After all, a computer just won Jeopardy…and that algorithmic wonder is just the beginning.

Let’s face it, there are mountains upon mountains of data stored on hard drives, both in private and public servers and storage arrays, and the mountain is only getting bigger.  The question being why isn’t it being released for public consumption? Well, some of it is, but most of it isn’t for reasons of security, intellectual property, or competitive advantage.  Granted, not all data should be opened up to the masses, but a heck of a lot more would be helpful in driving real innovation and growth. In this month’s Wired Magazine, Clive Thompson writes “Set Data Free –  Governments need to unlock their information vaults, it might just create jobs.”

It’s true. Why are we collecting the mountains of data in the first place?  Today, I would argue that it’s a  “you never know when you’ll need it” mentality.  Government regulation requires companies to hang on to data and secure it, and governments do the same.  Sure there is some analysis done, but think about how much more could be done when the power of the data is fully leveraged by people that know how to mine it.  Data has become a natural resource, a new global currency, and by making it available, we may just create a new era…scrap the Information Age…information is the offspring of Data.  What we need is a Data Age, and we need it now.

What’s your take?

Image by: Walt Disney Pictures