Network Downtime: The Costly Fallout
Human error when making changes to network cores chipped away at companies' revenue by disrupting everything from employee productivity to supply chain processes during 2013, according to a recent survey of IT professionals.
Avaya polled 210 IT pros at companies in the U.S., Canada and the U.K. with at least 250 employees, and 82% of the respondents said they'd experienced at least some network downtime as the result of human error made while configuring changes to the network core.
Of those who had experienced core errors, 80% reported having lost revenue as a result of such outages, to the tune of an average of $140,000 per incident. Hardest hit were respondents from the financial services industry, who reported having lost an average of $540,358 for each incident.
Those losses were triggered by a variety of disruptions, as 94% of respondents reported suffering some kind of consequence of downtime resulting from human-induced errors during core changes. Nearly half (49%) said employee productivity was hampered, 40% said IT projects were interrupted or delayed, and 30% reported supply chain disruptions.
"Any outage or disruption in the core is sure to affect the rest of the network," Brad Casemore, research director at IDC, said via email. "The ramifications from downtime in the core are felt everywhere."
Given the wide-ranging impact resulting from core errors, it's not surprising that one in five respondents to Avaya's survey said their companies had fired IT staff as a direct result of such outages.
Casemore said that whether a company fires a network professional for such an error depends on its corporate culture, the employee's record, the enormity of the error, and whether there were clearly prescribed policies and procedures to prevent such an error.
"It's certainly possible that errors of the sort covered by this survey could be considered sufficiently egregious for heads to roll, figuratively speaking," Casemore added. "I've known of networking professionals who've been fired for less, and of those who've survived errors that might have been considered firing offenses elsewhere."
[Read how best practices can go a long way toward preventing unplanned downtime caused in "How To Avoid Network Outages: Go Back To Basics."]
The survey findings underscore a growing need to eliminate human error by automating the management of network cores. In fact, Bob Laliberte, senior analyst at Enterprise Strategy Group, said via email that network management ranked second behind network security on the list of network investment priorities in a recent IT spending survey published by ESG.
"Not having complete visibility into the environment or properly understanding the impact of a change before making it can lead to unintended consequences," Laliberte said. As a result, "the ability to effectively manage large and complex network environments is a priority for most organizations."
Casemore said many companies are investigating SDN, network virtualization and other approaches as they seek to automate as many elements of their IT infrastructures as possible.
"Automation is a touchstone, from provisioning through to orchestration," he said.
Recommended For You
IBN is set to take network management to the next level. Is your organization ready to join the ride?
We have updated the Network Computing site to make it easier to find the information you need. Please take a look at our new features.
Low-Power WANs offer an alternative to 5G for connecting a fast-growing array of basic devices and sensors that transmit small amounts of data.
An effective network visibility strategy requires understanding the technical, financial, political, and legal aspects impacting your network operations.
Emerging organizational structures for IT include placement of IT pros in user areas and departments forming their own "micro IT's."
Comparing a good and bad trace helps identify performance issues. Dynamic baselining can be used when you do not have a good trace to reference.