Acceptance Tests Don't Just Eliminate
Bugs They Add a Feeling of Stability
When Extreme Programming
(XP) was first introduced to the VCAPS project at Ford there were no automated acceptance tests. It took a while
to add coverage. We added tests for all new functionality and old functionality as it required changes. After about
a year we had an estimated 40% covered by tests and the trouble ticket count dropped by 40% as well. We don't consider
this a coincidence. But the customers had noted
a different effect. By trapping bugs before they reached the production environment there were far fewer emergency
production releases. Previously, it was not uncommon to release to production a couple times a day for a couple
days due to bugs that
required an immediate fix. The acceptance tests improved the quality of the system to a point where a production
release was rarely re-released because an emergency fix was required. The customers experienced
this as a system with a far greater feeling of stability. They had more confidence in the system and us. The customers
also noticed that with fewer releases there was a large drop in spurious bugs often caused by quick and dirty fixes.
When Extreme Programming
(XP) was first introduced to the VCAPS project at Ford there were no automated acceptance tests. It took a while
to add coverage. We added tests for all new functionality and old functionality as it required changes. After about
a year we had an estimated 40% covered by tests and the trouble ticket count dropped by 40% as well. We don't consider
this a coincidence.
