About
Exploring Solution Spaces, Keith Ray's blog on Software development and other topics.
Send comments to:
keithray@mac.com
| For Agile Training, eLearning, or Coaching contact: |
| Industrial Logic, Inc. |
| 866-540-8336 (toll free) |
| 510-540-8336 (Berkeley, California) |
| Links |
| xpminifaq |
| Résumé |
| “Adopting XP” Article 2002 (pdf) |
| “ Refactoring” Article 2006 |
| AYE Conference |
| Lucien W. Dupont |
| Elisabeth Hendrickson |
| Johanna Rothman's Managing Product Development |
| Brian Marick's Exploration Through Example |
| Esther Derby's Insights You Can Use |
| Laurent Bossavit's Incipient(thoughts) |
| Dale Emery's Conversations with Dale |
| Martin Fowler's Bliki |
| Creating Passionate Users |
Archives
Subscribe
RSS Exploring Solution Spaces XML
2003.Nov.06 Thu
Cost of Low Quality
Johanna Rothman's What Does it Cost You to Fix a Defect? And Why Should You Care":
Nynke Fokma's Are We Solving the Real Problem?: The global organization had a list of 5 company values for all thousands of employees all over the world. One read "Exceed Customer Expectation." How many projects had we shipped in the last year that did not live up to customer expectation, let alone "exceed" it? I had no idea. None of us was directly in touch with customers and users so how will we know how we are doing? Elizabeth Hendrickson's You Can't Test the Wings Back On an Airplane:After release, Company A estimated that it spent 20 person-days per fix. At $500 per person-day, this comes to $10,000 per fix. Company A typically had 20 "emergency" fixes after each release, for a total post-release fix cost of about $200,000.
Company B tracked the time it took to fix a defect during implementation (before system test), and found the average time to be about two hours, or about $125 per fix. Company B generally didn't release fixes to its products once the product was in the field, so it had no extra post-release costs.
Like the others, Company C did not track defect fix time during the requirements, design, and implementation phases. However, Company C engineers used a build-every-night and fix-any-defects-immediately approach to product development. In effect, the engineers were finding and fixing defects throughout the entire lifecycle. They spent an average of one person-hour each day fixing defects from the previous night's build, or about $60 per defect. Using this approach — along with code walk-throughs and peer reviews — they generally shipped a very high-reliability, low-defect product, and were able to avoid most post-release defects costs. When they did have an "emergency fix," their average time to fix was about 5 person-days per fix, or $2500 per fix.
Company A had the highest cost, over $400,000 just to fix defects.... Company C had the largest, most complex project, with the most people, and was able to contain its defect costs to about $135,000.
Testing can tell you when something isn't right, but it can't fix what's wrong.
When I talk with individuals, most understand the difference. Yet the common behavior of software development groups shows that some people still believe that a test group really can ensure quality. The biggest indicator that this attitude persists lies in the name given to test groups: QA, Quality Assurance. Beyond that, there is an attitude, common in many organizations, that foists blame for bad releases on QA.
Quality problems originate long before the testers get their hands on the product. Even in an environment where testers are involved early, testers without training in quality techniques are unlikely to lend much to a quality effort. In short, naming a group QA seems to make some people in other groups think they are officially absolved of the responsibility to promote quality practices. "That's QA's job." I've heard more than one programmer, technical writer, and project manager utter those words.
Actually, quality is everyone's job. It begins with the requirements and ends with a solid product shipping out the door: a product that meets, exceeds, or redefines the customers needs and expectations.