Testers get credits nowadays on computer projects.
- industry average
experience suggests that there are 15-50 errors per 1000 lines of
delivered code.
(Survival Guide - p.610).
"Despite testing procedures, even a best-of-breed product usually still contains a margin of error, typically around 5%. At this point it may be deemed by both vendor and customer that trying to reduce this percentage is a case of diminishing returns and not worth the additional Investment." (Computer Weekly).
Some expensive bugs have been documented.
- You can try daily build/smoketest, each programmer using a private version of main source. (Survival Guide p.206)
- Use "defect seeding". Put 10 deliberate errors into your code or
documentation and get someone to check it. If they find 20 errors
but only 5 of them are your deliberate ones then you might hypothesise
that since they only found half of the deliberate errors they only found
half of the other errors, so there are 30 unintentional errors. Some
checkers find this a fun way to work. Don't forget to remove
seeded bugs!
- Supply the test suite with the program (Mozilla)
- Code walkthrough - get someone to listen as you describe the code
- Involve the customer in testing
- Run a bug-finding competition (see http://www.latex-project.org/guides/lgc2-err.pdf for an example of this).
- Web applications may present difficulties because of the number of
components that may be involved. If there's a problem (in particular
performance-related) the cause could be: the network; the database,
the web server
(with modules for PHP support), the web proxy-server or the browser itself
(which comes in various varieties each with different plug-ins, java
interpreters and so on). Black-box testing alone won't be enough.
- Beware of typos in error messages - they're commonly missed
Updated July 2009 with help from James Matheson
Tim Love