09:59 AM
Greg Ferro
Greg Ferro
Connect Directly
Repost This

I'm Your Customer, Not Your QA Department!

Bugs are operational failures by vendors and that not only wastes company's time tracking down, but has the long term impact of stopping companies from adopting new products and features for the sake of reliability. Here are 4 misconceptions vendors tell themselves, and us, about software bugs.

Bugs happen. It's a fact of life. We accept them. We plan for them. We find them. We escalate them to vendors. Then we workaround. We hack. We patch. And we hope that we can mitigate the impact. Bugs are operational failures by vendors and that not only wastes company's time tracking down, but has the long term impact of stopping companies from adopting new products and features for the sake of reliability. Here are 4 misconceptions vendors tell themselves, and us, about software bugs.

1. Software is too complex to find all the bugs.
This is true. Software is so complex, and beyond human comprehension, that bugs will occur. However, this does NOT mean that bugs are acceptable. And bugs that could be detected using good procedure and controlled testing should never occur in public.

2. Bugs are OK as long as vendor support is responsive and has good support people.
No. It's not OK. I don't want to waste my time finding, reporting and fixing your bugs. I want to spend my time making things work, not troubleshooting your equipment and software that shouldn't be broken. I should never have to call your support about a software bug because it should work correctly and reliably.

In fact, if your support organization is "best in world" then perhaps you have so many bugs and failures that you are hiding your product problems.

3. Customers need to do their own testing and prove the product works.
True. Up to a point. A customer should take responsibility for ensuring that the product meets their needs and is used in the intended manner. This means selecting the right products, installing them correctly, ensuring the products integrates well with the rest of the network and meets performance goals.

But networking vendors include software QA-type testing when telling customers they should conduct their own tests. As a metaphor: When I need a new car, I'd choose family sedan that has good mileage and reputable name. I do not expect to take that brand new vehicle to a garage for testing to ensure that it works correctly.

Not only is it not cost effective for customers to find bugs, doing so undermines the value of the product. Vendors should make products that are reliable. Products should not need QA-type testing by customers once the products have shipped (except for extreme situations).

4. Bugs are a fact of life. Everyone has them.
Finding bugs in network software is expensive for the customer whose goals are reliable, fast networks that support other IT goals. When a network fails due to a software bug, IT's confidence in the reliability and integrity of the product erodes. The eroding confidence in turn leads to resistance to changes in the future such as using new features because IT predicts future failures based on past experiences.

The result is old firmware staying in place because it's stable and new features contained in newer versions of software remain available. In the long term, the network is not delivering new services to the business. Who wants to invest in a failed technology? Bugs are operational failures leading to negative investment in new technology.

Sound familiar? I don't think software quality in the networking industry is given enough focus. I've seen vendors claim that it's more important to have new features and new hardware to meet varying customer demands. And that maybe true, but right now I think we need a renewed focus on software quality. Lets get it right before we make it bigger and faster.

Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
User Rank: Apprentice
4/4/2012 | 6:26:56 AM
re: I'm Your Customer, Not Your QA Department!
You don't specify if what you refer to as "bugs". Are these functional problems with the software itself, or you calling anything that may present as a networking problem with a specific product a bug? Like you, I've spent many hours troubleshooting in labs and production, and based on my experience I find it hard to completely blame vendors for problems with new software development. It's not that I think that vendors aren't responsible for the quality of their work, they are, but I often wonder if the number of permutations within the world of IT actually makes perfect software statistically impossible.

To compare something like a network switch to an automobile would be incorrect, because the automobile (for all intents and purposes), is an appliance rather than a customization-specific device. Though both share similar degrees of complexity that's where the similarities end. For a customer to begin operating a new automobile, all that is needed is to turn a key and perform the same functions as in similar vehicles. The only customization needed is to make (optional) seat and mirror adjustments for comfort purposes only. This is the bane and beauty of the appliance model, and as a rule that vehicle will recieve no additional updates unless mandated by law. Even in a closed device like a new car, there are often needs for minor service, and I would assume that owners usually accept that with good grace unless the car fails completely.

In the case of something as complex and user-definable as a piece of enterprise networking hardware there are very few appliances. Even those that might define themselves as such usually require regular upgrades to fix problems, and very often to offer additional functionality or improve performance. But in my experience almost every IT product also depends on custom configuration that is very specific to the envronment under which they're to be used. This is a necessity because of the environment and nature of the task and not specifically a limitation of the device or its software. Once you open a device up to the level of user modification that IT users insist upon, (which alone completely explodes the appliance model) you enter into environmental permutations that even the best vendor may not be able to anticipate.

I would imagine they try, but I would also hazard a guess that any attempt to completely address sheer number of variations in business IT today would make the development of perfectly trouble-free software updates economically unfeasable, not to mention add years to the process. Even products that fit the appliance model regularly experience problems in the field, thus consumers expect and deserve good warranties. But I have yet to see a warranty that extends beyond repair or replacement and there is always legal boiler-plate that specifically excludes damages due to loss of business, premature aging, mental debilitation or loss of hair.

I would think that this type of protection is even more valid for heavily user-customizable products, where problems that appear to be the fault of a given device might actually be caused by incompatibilities with other devices, or yes; even human error. Most companies provide multiple levels of software (old, current and beta) and provide substantial documentation about them giving the end user fair warning. What more could vendors do? Does anyone else out there feel like a human guinea pig, or do you believe that vendors are doing their best to keep a good-faith balance between safe, reliable software and new/improved features?
User Rank: Apprentice
4/2/2012 | 5:44:23 PM
re: I'm Your Customer, Not Your QA Department!
Amen, brother. Slow down the "innovation" and new features and provide some stability, for corn's sake. And test on large networks, not is some offshore labs where developers have never actually used their own stuff for any real-world application.

More Blogs from Commentary
Infrastructure Challenge: Build Your Community
Network Computing provides the platform; help us make it your community.
Edge Devices Are The Brains Of The Network
In any type of network, the edge is where all the action takes place. Think of the edge as the brains of the network, while the core is just the dumb muscle.
Fight Software Piracy With SaaS
SaaS makes application deployment easy and effective. It could eliminate software piracy once and for all.
SDN: Waiting For The Trickle-Down Effect
Like server virtualization and 10 Gigabit Ethernet, SDN will eventually become a technology that small and midsized enterprises can use. But it's going to require some new packaging.
IT Certification Exam Success In 4 Steps
There are no shortcuts to obtaining passing scores, but focusing on key fundamentals of proper study and preparation will help you master the art of certification.
Hot Topics
Edge Devices Are The Brains Of The Network
Orhan Ergun, Network Architect,  4/23/2014
IT Certification Exam Success In 4 Steps
Amy Arnold, CCNP/DP/Voice,  4/22/2014
SDN Strategies Part 3: Juniper, Dell, Brocade, Alcatel-Lucent
Kurt Marko, Contributing Editor,  4/17/2014
White Papers
Register for Network Computing Newsletters
Current Issue
2014 Private Cloud Survey
2014 Private Cloud Survey
Respondents are on a roll: 53% brought their private clouds from concept to production in less than one year, and 60% ­extend their clouds across multiple datacenters. But expertise is scarce, with 51% saying acquiring skilled employees is a roadblock.
Twitter Feed