Network Downtime: The Costly Fallout
Human error when making changes to network cores chipped away at companies' revenue by disrupting everything from employee productivity to supply chain processes during 2013, according to a recent survey of IT professionals.
Avaya polled 210 IT pros at companies in the U.S., Canada and the U.K. with at least 250 employees, and 82% of the respondents said they'd experienced at least some network downtime as the result of human error made while configuring changes to the network core.
Of those who had experienced core errors, 80% reported having lost revenue as a result of such outages, to the tune of an average of $140,000 per incident. Hardest hit were respondents from the financial services industry, who reported having lost an average of $540,358 for each incident.
Those losses were triggered by a variety of disruptions, as 94% of respondents reported suffering some kind of consequence of downtime resulting from human-induced errors during core changes. Nearly half (49%) said employee productivity was hampered, 40% said IT projects were interrupted or delayed, and 30% reported supply chain disruptions.
"Any outage or disruption in the core is sure to affect the rest of the network," Brad Casemore, research director at IDC, said via email. "The ramifications from downtime in the core are felt everywhere."
Given the wide-ranging impact resulting from core errors, it's not surprising that one in five respondents to Avaya's survey said their companies had fired IT staff as a direct result of such outages.
Casemore said that whether a company fires a network professional for such an error depends on its corporate culture, the employee's record, the enormity of the error, and whether there were clearly prescribed policies and procedures to prevent such an error.
"It's certainly possible that errors of the sort covered by this survey could be considered sufficiently egregious for heads to roll, figuratively speaking," Casemore added. "I've known of networking professionals who've been fired for less, and of those who've survived errors that might have been considered firing offenses elsewhere."
[Read how best practices can go a long way toward preventing unplanned downtime caused in "How To Avoid Network Outages: Go Back To Basics."]
The survey findings underscore a growing need to eliminate human error by automating the management of network cores. In fact, Bob Laliberte, senior analyst at Enterprise Strategy Group, said via email that network management ranked second behind network security on the list of network investment priorities in a recent IT spending survey published by ESG.
"Not having complete visibility into the environment or properly understanding the impact of a change before making it can lead to unintended consequences," Laliberte said. As a result, "the ability to effectively manage large and complex network environments is a priority for most organizations."
Casemore said many companies are investigating SDN, network virtualization and other approaches as they seek to automate as many elements of their IT infrastructures as possible.
"Automation is a touchstone, from provisioning through to orchestration," he said.
Recommended For You
In honor of St. Patrick’s Day, there’s no better time to reflect on those instants when life threw us a curveball, but we were able to hit a home run.
The success of modern enterprises, especially those utilizing real-time communications solutions, is highly reliant on IT infrastructure availability.
To understand the critical role of HTTP/2 in streamlining operations, we must look back at the technologies and implementation gaps that got us where we are today.
A video overview and best practices on how to reduce broadcasts and find other things to tune.
This is a great example of the perfect storm of variables coming together to cause performance issues. Watch the video to see how the problem was found.
Providers should be making infrastructure work for everyone in 2019, improving efficiency and opening up networks for all apps on their infrastructure.