The culprits were Pandora, the Internet radio service, and two content delivery networks, Akamai and Limelight Networks. In other words, the network wasn't plagued by performance issues; it simply was being overwhelmed with video and audio traffic.
Adding complexity to the situation was the fact that while the Pandora traffic was obviously not critical, the CDN traffic was less straightforward, as Akamai and Limelight both not only serve up a lot of Web video, they also support Windows updates. So even though the congestion was affecting the performance of critical apps, blocking traffic from the CDNs was not an option.
"They had a lot of traffic being held up," Slattery, principal engineer at networking consultancy Chesapeake NetCraftsmen, said in a telephone interview. "They had to determine what real-time traffic was important to them."
The health care company's experience was emblematic of the extent to which video and audio data traffic have begun to clog corporate networks. In fact, Slattery believes differentiating video and audio from other network traffic has become the biggest challenge companies face in managing their networks today.
It's a problem that's not only afflicting end user companies. ABI Research recently predicted that annual spending on Web and video optimization and deep packet inspection (DPI) technologies would grow to more than $5 billion by 2019 as service providers contend with crushing demand for multimedia from smart phone and tablet users.
"Operators should be concerned with the amount of data consumed by their subscribers, and how this affects the overall quality of experience," Sabir Rafig, ABI research analyst, said in a statement.
While most enterprises won't need to make the kinds of investments service providers will, there are a few key steps they should take to reduce the drag video and audio traffic are having on their businesses.
[Read how Renown Health monitors quality of service on its network in "QoS Boosts Health Of Telemedicine Network."]
For instance, Slattery and his team added quality-of-service tools to the health care company's T3 Internet link, configuring the network to route business-critical traffic into a high-priority queue while traffic from the three video- and audio-generating sources was routed to the low-priority queue. As a result, critical applications began behaving better, but users saw their connectivity slow for low-priority activities, a tradeoff management obviously considered acceptable.
That kind of policy-driven approach is where companies facing similar challenges should start, Slattery said.
"You should make sure your policies reflect what you want from your network," he said. "A lot of times, the policies are not well written or don't exist."
In other words, Slattery said, companies need to establish a better understanding of how they want to handle network traffic. They also must make sure their network teams know the resulting policies so they can effectively engineer things to comply with them.
"If you get the planning right, the execution gets very simple," Slattery said.
What's more, he believes that software-defined networking technologies will play a key role as companies look to start prioritizing their network traffic dynamically, something they haven't been able to do previously. This is even more critical for companies whose businesses inherently rely on maximum agility.
"If you have real-time or near real-time needs in your datacenter, you're going to need some dynamic capabilities," Slattery said. "SDN is going to allow all this stuff to be dynamic."
[Don't miss Terry Slattery's session, "Optimize Your Network For Real-Time Apps," at Interop Las Vegas March 31-April 4. Register today!]