One of the big characteristics of much of that new data is that it is "persistent," meaning that it should or must be stored permanently. Yes, sensor data, such the use of RFID, may create voluminous amounts of data, but a lot of this is ephemeral information that may be retained for only seconds or minutes. Yes, data retention policies may be put in place for some data, such as e-mails retained only for a certain period of time and then deleted. However, much of the data businesses create or acquire each year will simply be kept and added to data from prior years that has not been deleted.
Note also that the vast bulk of this data is fixed content that will never change and that the pattern of access for the data follows a long-tailed (active in the beginning, then trailing off over time) distribution. Some portion of this information, perhaps even the vast majority, will never be accessed again, but some may be recalled for whatever reason. Predicting which data will be recalled may very well be very difficult if not impossible. Yet that information still qualifies as production data in the sense that it has to be recoverable "online." That means users can recall the data without outside assistance, although recovery is almost assuredly not in the sub-second or seconds range associated with data that is active and subject to change.
In conventional solutions, that class of data is placed in an active archive consisting of production data still available online but which resides on more cost-effective storage media. The first choice for these environments is capacity disk, such as 500GB to 1+TB SATA, in contrast to performance disk, such as FC or SAS. However, even capacity disk may provide higher performance and concomitant higher cost than is needed. Magnetic tape can also serve as an active archive repository for this data. If the data needs to be recalled, a copy can be put on disk and used there. Ffor example, StorNext from Quantum provides that capability.
Is that the end of the story? To date, magnetic tape has defined the most logical conclusion, but Opternity has come up with an alternative technology that extends tape media to an even more cost-effective tier of storage.