Analysis of large volumes of real-time network data can help optimize network forwarding mechanisms.
In talking with a customer recently about network management, an interesting question came up: "Is the network management industry headed toward instrumentation on the network devices and exporting the data to analysis systems?"
I've seen some early indications that this is starting to happen. For example, I'm aware of several products that use agents residing on network devices to gather and quickly send real-time data to an external analysis system. So the question stayed with me, and I began to wonder about analyzing such data to improve network monitoring and control. It finally sank into my head that the ideal terminology is "data driven network." While a quick Internet search turned up several papers and presentations on the topic, I found no products. I expect that to change.
The basic Idea
The idea behind data-driven networking is that the analysis of large volumes of real-time network data can help optimize network forwarding mechanisms. Stated another way, data-driven networking is the application of big data analysis to raw network data, with the results then used to optimize network performance. Is the subject that new? Not really. Most of the papers about data-driven networking published in 2016:
- "A Novel Framework of Data-Driven Networking" (IEEE Xplore)
- "Data Driven Networks" (The Stanford Platform Lab)
- "Data-Driven Networking: Harnessing the 'Unreasonable Effectiveness of Data' in Network Design" (Carnegie Mellon University, School of Computer Science)
- "Unleashing the Potential of Data-Driven Networking" (CMU, UC Berkeley, Conviva, Databricks)
The traditional approach to network monitoring and management, Simple Network Management Protocol (SNMP), uses a simple mechanism to collect data from network devices.
Read the rest of this article on No Jitter.