A positive look at the negative

One of our main references when testing is the specs. Be it called fspecs, rspecs, pspecs or devnotes, it essentially describes how the program is supposed to behave. At the very least, it tells us of the inputs that the program requires, the outputs that it generates, and of the logic that happens in between.

Clearly, what is written is rather limited — the scope of what is unwritten far exceeds that of what is written. In effect, you must test more for what is unwritten as opposed to what is written. In the book “Black-Box Testing”, Boris Beizer wrote that “In mature test suites, dirty tests [this corresponds to negative test cases] typically outnumber clean tests [positive test cases] in a ratio of 4:1 or 5:1.”

For the way that the program is intended to be used, we can immediately derive test cases from the specs. But we musn’t fall in the trap of confirmation bias i.e., our test cases must not be limited to confirming the specs. We must also create negative test cases which are geared towards testing the program in the way it is not intended to be used. Through subjecting the program to both positive and negative test cases, we can establish greater confidence that the program works and is robust.