Bug isolation

A friend had shared with me a paper on bug isolation. It’s a 2004 paper by Danny Faught entitled How to Make Your Bugs Lonely: Tips on Bug Isolation. It starts off with the quotation: “A problem well-stated is half-solved.” The paper then got me thinking on whether we cover or emphasize this strongly enough during tester training in our company. Sure, our testers indicate the expected behavior versus the actual behavior in their bug reports, and sometimes they provide the steps to replicate the problem. But giving the steps does not necessarily equate to giving the conditions that trigger the bug report.

In bug isolation, one attempts to really zero in on the bug by reducing the factors that obscure the problem.  For instance, you streamline the steps to get to the bug, or you narrow down the data needed to trigger the bug.  Basically, you refrain from posting shoddy bug descriptions like “It doesn’t work”, and instead you try to find out the specific conditions that trigger the bug.  You might think that bug isolation only benefits the developers since they’ve less effort to render in trying to replicate the bug.  But I think, this will also help the tester in thoroughly verifying bug fixes.  How else would one know if a problem is solved if he doesn’t know the problem to begin with.  Verifying the absence of the previously detected symptoms is not always a sure-fire guarantee.

For instance, consider this bug report:

Search doesn’t work!

Steps to replicate:
1. Enter “OMG” into the search keyword field.
2. Click Search.  No matching results are retrieved.

Had the tester investigated further, he’d have figured that the search function wasn’t such a failure as he had described.  He could have found out that the current search function had been case-sensitive.  If the developer fixed the bug without letting the tester in on the root cause, the tester might have just tried searching for “OMG” again without trying out similar cases, and without really verifying the fix applied by the developer.

Another example:

ORA- error occurs on save.

Again, had the tester investigated further, he’d have found out that the bug would only happen if the numeric ID value is greater than 999.  What actually happened is that when the ID value > 999, it got reformatted to a string value of format “N,NNN”.  And so when you try to save, a database error occurs because you’re trying to insert a string value into a numeric field.  Without knowing this, when it’s time to verify the developer’s fix, the tester might just try saving the record without considering the ID value.

Although bug isolation has its upside, oftentimes practicality (and hopefully common sense) dictates that we shouldn’t employ this for every single bug we find.  That would be overkill and would eat up our productivity.  For instance, 2 hours of trying to trace a problem for a low-priority bug would be quite difficult to justify.  Cliché but true, with bug isolation (along with almost everything else), we have to exercise good judgement in deciding whether to apply it.  But what I’d like to emphasize for testers is that we should at least try to isolate the bug and not be to eager to jump the gun and say “It doesn’t work!”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s