I'm Your Customer, Not Your QA Department!
Posted by
Greg Ferro
April 02, 2012
Bugs happen. It's a fact of life. We accept them. We plan for them. We find them. We escalate them to vendors. Then we workaround. We hack. We patch. And we hope that we can mitigate the impact. Bugs are operational failures by vendors and that not only wastes company's time tracking down, but has the long term impact of stopping companies from adopting new products and features for the sake of reliability. Here are 4 misconceptions vendors tell themselves, and us, about software bugs.
1. Software is too complex to find all the bugs.
This is true. Software is so complex, and beyond human comprehension, that bugs will occur. However, this does NOT mean that bugs are acceptable. And bugs that could be detected using good procedure and controlled testing should never occur in public.
2. Bugs are OK as long as vendor support is responsive and has good support people.
No. It's not OK. I don't want to waste my time finding, reporting and fixing your bugs. I want to spend my time making things work, not troubleshooting your equipment and software that shouldn't be broken. I should never have to call your support about a software bug because it should work correctly and reliably.
In fact, if your support organization is "best in world" then perhaps you have so many bugs and failures that you are hiding your product problems.
3. Customers need to do their own testing and prove the product works.
True. Up to a point. A customer should take responsibility for ensuring that the product meets their needs and is used in the intended manner. This means selecting the right products, installing them correctly, ensuring the products integrates well with the rest of the network and meets performance goals.
But networking vendors include software QA-type testing when telling customers they should conduct their own tests. As a metaphor: When I need a new car, I'd choose family sedan that has good mileage and reputable name. I do not expect to take that brand new vehicle to a garage for testing to ensure that it works correctly.
Not only is it not cost effective for customers to find bugs, doing so undermines the value of the product. Vendors should make products that are reliable. Products should not need QA-type testing by customers once the products have shipped (except for extreme situations).
4. Bugs are a fact of life. Everyone has them.
Finding bugs in network software is expensive for the customer whose goals are reliable, fast networks that support other IT goals. When a network fails due to a software bug, IT's confidence in the reliability and integrity of the product erodes. The eroding confidence in turn leads to resistance to changes in the future such as using new features because IT predicts future failures based on past experiences.
The result is old firmware staying in place because it's stable and new features contained in newer versions of software remain available. In the long term, the network is not delivering new services to the business. Who wants to invest in a failed technology? Bugs are operational failures leading to negative investment in new technology.
Sound familiar? I don't think software quality in the networking industry is given enough focus. I've seen vendors claim that it's more important to have new features and new hardware to meet varying customer demands. And that maybe true, but right now I think we need a renewed focus on software quality. Lets get it right before we make it bigger and faster.
Greg Ferro is a freelance Network Architect and Engineer. You can email him, follow him on Twitter as @etherealmind. He also has a technical blog at EtherealMind.com and is the co-host of the popular and well known Packet Pushers podcast on data networking. He is nearly as grumpy as Mike Fratto.
