Good test coverage is important and sometimes not easy to achieve. A simple principle can lay a solid foundation for test coverage: Distinguishing two essential stages of testing.
The initial low-level stage tests basic development artifacts immediately or soon after they have become available. This is usually called component, module or unit testing.
The later high-level stage tests the entire system or its higher-level aggregates as early as possible and lasting until very close before product delivery. This is usually called acceptance test.
The concepts of low-level and high-level tests are not new. Important is to relate them to different phases of the development cycle (or to activities within agile iterations, likewise; levels become stages) and to systematically plan associated activities. This way, good test coverage can be achieved very efficiently.
| Low-level testing stage | High-level testing stage |
|---|---|
| Tests cover implementation of each basic development artifact from an implementation point of view (white or grey box testing) | Tests cover design of the entire system or major system part from a business or usage point of view (black box testing) |
| Test cases defined by developers; derived from requirements and design | Test cases defined by testers, domain experts from the software team, and/or customers; derived from explicit requirements or tacit domain knowledge |
| Tests conducted by developers | Tests conducted by testers and/or customers or users |
| Defects usually fixed immediately or otherwise entered into defect database | Defects usually entered into defect database, fixed, and being re-tested |
I have seen many projects in trouble, because they did not properly address these test stages. Sometimes, low-level testing was replaced by pure faith (“My programs always run well”). Sometimes, high-level testing was shallow and ineffective (“We don’t have any time for more tests”). Often, the relation between both stages was not managed well, lowering product quality and limiting the efficiency of testing.
Taking care that both stages of testing are being addressed is a first and important step towards improved testing. Both stages are complementing each other well, so that higher test coverage can be achieved without very little planning and qualification efforts. This is also a good basis for subsequent improvement activities.
Additional details on the concepts of low-level and high-level testing are described in the testing literature, although the relation between the levels and phases of the development lifecycle is often not explored very much. A very instructive book is TMap® Next by Koomen et al. (2006). Another elaboration on the two testing levels are Brian Marick’s agile testing quadrants. Lisa Crispin’s presentation on agile test planning provides a detailed explanation of those quadrants. Finally, an inspiring reflection on testing is Robert L. Glass’s text on The Many Flavors of Testing.