The world is drowning in data. Soon, it’s estimated there will be 1 trillion Internet-connected devices in the world. Every day, 15 petabytes (1015 or 1,000,000,000,000,000) of new information is generated — eight times more than the information in all the libraries in the United States. This year the amount of digital information generated is expected to reach 988 exabytes (1018 or 1,000,000,000,000,000,000). This is equivalent to the amount of information if books were stacked from the Sun to Pluto and back.
It’s not a new problem, of course. The information explosion can be traced back to the invention of the Gutenberg printing press in the mid-1400s. By 1500, printing presses had produced over 20 million volumes and in the following century that rose tenfold. Fast forward to 20 years ago when Information Anixiety was published and stated that “more information has been produced in the last 30 years than in the previous 5,000. About 1,000 books are published internationally every day, and the total of all printed knowledge doubles every eight years.” The data problem is accelerating faster and faster. Now, the amount of digital information increases tenfold every five years.
It’s no wonder then that companies across the board are looking to store less data, place it intelligently and fully leverage their existing assets. I am responsible for IBM Storage business strategy and products plans, so of course I could give you a check-list of the “must have” storage technologies, but I’ll defer to your technical teams to come up with the right mix of solutions that will solve your company’s data problem. What I can say with some degree of unbiased confidence, however, is the lack of action is not an option for any CIO. The problem is not going away and data growth will only continue to increase.
In fact, according to recent analyst reports, enterprise data growth over the next five years is estimated at 650 percent. And the nature of data is changing from “structured” forms such as numbers to “unstructured” information such as files full of videos, pictures, music, and e-mails. Industries such and media and entertainment are being especially hard hit by this rise of unstructured data. Between 2009 and 2015, that industry will see about a 10x increase in the required digital storage capacity per year, according to a recent study from Coughlin Associates.
True, storage innovators are trying to keep pace; providing companies with the ability to store data on an unprecedented scale. However, in an interconnected and instrumented world where trillions of sophisticated digital devices communicate every second and supercomputers are capable of generating more than a quadrillion (either 1015 or 1024 depending on what scale you use and where you live … but really whose counting!) calculations per second, for the foreseeable future we will not be getting ahead of this tide of data. The research firm IDC says that the world’s data already exceeds available storage space and demand for storage capacity will continue to grow at a compound annual growth rate of nearly 50 percent in the next three years.
So what’s a CIO to do? There are a lot of ways to manage data: ILM, proper governance, content management systems and the like, but it all has to reside somewhere while you figure out what to do with it so my advice is to make sure that your company is taking advantage of smarter storage solutions today and — as much possible — tomorrow.
Doug Balog is vice president for IBM System Storagewith responsibility for IBM’s storage strategy and product plans, including future systems and solutions. In his various roles driving technology development and strategy for IBM, Doug works closely with CIOs around the world.