Enormous Growth in Data is Coming — How to Prepare for It, and Prosper From It

  • It's 2025 - Are You A Leader or a Laggard?

Seagate today announced the publication of a major analysis of trends in a study by IDC titled Data Age 2025 — which predicts worldwide data creation will grow to an enormous 163 zettabytes (ZB) by 2025. That’s ten times the amount of data produced in 2017.

The analysis is fascinating, and we intend to explore its many aspects and trend stories in future blogs over the next months.

Why data is growing so fast

Why is data growing so fast? It’s a complicated question the paper addresses, and of course there are many types of devices generating data, including images and video created or consumed for entertainment, images and video from functional applications like advertising and security, productivity-driven data such as files on PCs and servers, log files, and massive amounts of metadata created by embedded devices, machine to machine, and IoT.

In the early days of the digital revolution, data was produced mainly as PCs were adopted and became ubiquitous, and later as music, movies, TV and games became digital. These days, believe it or not, there are far more consumer digital toys than there are humans, all constantly producing, accessing and sharing data. All of that still leads to increasing data as more traditional media is converted to digital, as our entertainment libraries grow, and as video and audio resolution keeps improving. Keep in mind, almost all the data we use on these “toys” either exists on, or also has iterations in, enterprise data centers and servers on the internet.

But moving ahead, the new growth will be coming from the plethora of “embedded devices” we depend on more and more. These devices produce data a little at a time when you look at each discrete transaction, but all together these transactions will comprise 20 percent of all data created by 2025. Embedded devices include things like RFID readers and chip cards, smart cars (and planes, trains and buses too), vending machines, smart meters, medical implants, security cameras, and on and on — and additional data is generated via the systems they interact with like shopping malls, traffic grids, and cellular networks. Even the appliances in your house and your kids’ toys will, more and more, produce metadata when they interact with cloud-powered applications as part of the IoT.

Thus, where consumers created of most the world’s data in the past, IDC predicts enterprises will soon create 60 percent of the world’s data via applications relying on machine learning, automation and machine-to-machine technologies, and significant growth of data-creation from embedded devices. IDC says “the average rate per capita of data-driven interactions per day is expected to increase 20-fold in the next 10 years as our homes, workplaces, appliances, vehicles, wearables, and implants become data enabled.”

Maintaining and leveraging the benefits of the Data Age

We’ll all benefit from these advances in so many ways — Data Age 2025 describes how Netflix today is using metadata not only to make your viewing better with its recommendations engine, but also to improve the quality of content itself by funding new productions based on what they see viewers asking for (for example, Netflix found viewers loved political thrillers, content by director David Fincher, and the actor Kevin Spacey, and decided to fund House of Cards). And by the way it’s also improving their own profitability.

But we’re also talking about serious societal benefits like real-time fraud detection, efficient traffic flow management between smart vehicles and prioritized traffic protocols for emergency response vehicles, and facial recognition for improved security at sporting events or transportation hubs.

The upshot of the shift is that data is increasingly a critical influencer for not only our businesses but also our lives, in all aspects. Embedded devices and the internet of things are already boosting the amount of “life critical” data.

To note a few key findings from the research:

  • The number of embedded devices, such as those found in smart buildings, manufacturing floors, and driverless cars, will grow from less than one per person to more than four in the next 10 years.
  • In just 8 years, the average person will interact with a connected device nearly 4,800 times a day.
  • By 2025, 75 percent of the population will be connected, creating and interacting with data.
  • Data must be immediately available whenever and wherever we need it. By 2025, over 25 percent of data created in the global datasphere will be real-time in nature, and IoT real-time data will make up more than 95 percent of this.
  • Data from new sources will open up new risks to private and sensitive information. By 2025, almost 90 percent of all data created in the global datasphere requires some level of security, but less than half will be secured.
  • And portentously — get this — twenty percent of the data in the global datasphere will have the potential to be critical to the continuity of our lives.

So a big goal of IDC’s analysis is to make sure businesses are aware of where data growth is happening and are prepared to manage that growth effectively and leverage it efficiently — and to ensure its benefits can continue.

Beyond the societal impact, if not managed well, this growing flood of data could result in businesses experiencing operational inefficiencies, delivering poor customer experiences, and losing revenue. Consider this additional highlight from IDC’s analysis: by 2025 over 20 percent of the data created in the global datasphere could be useful for analytics if only tagged — yet only 15 percent of what could be useful will actually be tagged and analyzed.

Businesses must be equipped to manage data growth

Seagate is urging CEOs to act now and make sure their data strategy is focused on storing small sets of data that have significant business impact — rather than trying to analyze and utilize anything and everything. The paper provides important insights into how this increase in data could adversely affect businesses if not managed well. It offers a clear definition of business-critical data and why this should be the data businesses focus on, and tips for executives to avoid drowning in their own data.

Conversely, there’s opportunity. Business leaders can prosper by embracing new and unique business opportunities powered by this wealth of data and the insight it provides. It’s important to understand how to identify and leverage the specific data that will deliver the most value for business. Deploying technology appropriately will have a huge impact on doing this effectively.

As Seagate executive chair Steve Luczo noted, “While we can see from this new research that the era of Big Data is upon us, the value of data is really not in the ‘known,’ but in the ‘unknown’ where we are vastly underestimating the potentials today. What is really exciting are the analytics, the new businesses, the new thinking and new ecosystems from industries like robotics and machine-to-machine learning, and their profound social and economic impact on our society.”

He noted that business leaders will be exploring these opportunities for decades to come. The time to begin is now — I invite you read Data Age 2025 now, explore IDC’s trend analysis, and consider the recommendations on how enterprises and entrepreneurs might best capture the value of the coming ten-fold increase in data.


About the Author:

John Paulsen
John Paulsen is a "Data for Good" advocate, with more than 20 years in the data storage industry. He's helped launch many industry-firsts including HAMR technology, 10K-rpm and 15K-rpm hard drives, drives designed specifically for video and for gaming, Serial ATA drives, fluid dynamic HDD motors, 60TB SSDs, and MACH.2 multi-actuator technology.