How our team “does Agile”

Super quick background: Our project started Jan 2015. To kick things off on the Agile methodology, our Scrum master conducted a brief training (less than half a day) to the team, and we’ve been playing it by ear ever since.

Over the course of many sprints, retros and releases, we’ve made adjustments on how we’re doing Agile. I’m not sure if there are Agile Purists who would frown down and shake their heads at us for the variations we’ve made. But the thing is, despite our possibly non-canon approaches, what still matters most is the team closely working together to deliver working software.

This post might be TL;DR. But in case you’re still interested in having a peek at how our little team does Agile Scrum, go on and have a read…

Continue reading

RIMGEA – 6 approaches to bug reporting

I’ve learned a very useful mnemonic from the bug advocacy class on some approaches to bug reporting. Here’s something I had written on the topic with some edits (I just linked to previous write-ups on the same or related topics).

The 6 factors or approaches to bug reporting are:

1. Replicate it – Try to see if you can replicate the bug.
If you can’t replicate it, it might be more difficult to provide information to the developer and persuade her to provide a fix for a problem she can’t see.

[Link:  How helpful are my bug reports]

2. Isolate it – Try to limit the steps or the conditions that trigger the bug.
Here you try to narrow down your repro steps or to find what exactly are the critical conditions. Here you want to get to the bug in the easiest way possible.

[Link: Bug isolation]

3. Maximize it – Try to do follow-up steps to see if you can trigger a worse failure.
The bug you find might be just the tip of the iceberg, or a symptom of an even bigger bug. Follow-up tests could help to uncover the bigger problem if there is indeed one. There are various tactics that can be tried:

  • vary your behavior – e.g., instead of doing A then B, try changing the sequence; or try a different way of doing the task like using a shortcut key instead of the button
  • vary your program settings – e.g., there could be program settings that you can toggle on or off or adjust; for browser testing, you can try adjusting the zoom sizes or enable/disable caching
  • vary your inputs – e.g., if the bug was encountered when file X was used, try a similar file Y or another file Z
  • vary your configuration – e.g., try using a different OS/browser combination

4. Generalize it – Try to broaden the extent of the bug
Here we try to uncorner corner cases. We try to find other ranges in which the bug can be reproduced. For instance, we find that a bug occurs when we have 1M records. In generalizing, we try to see if the bug occurs at lower more realistic values or if the 1M records is truly a critical condition.

5. Externalize it – Try to go see the value/impact of the bug in other stakeholders’ perspective.
Here we try to go beyond our roles as testers and try to get a sense of the bigger picture. In understanding the value that could be lost, we could paint a more compelling picture of why the bug needs to be fixed in our bug reports.

6. And, say it clearly and dispassionately – Try to have our bug reports as easy to understand and as neutral as possible.
Clarity will make it easier for the dev/triage team to find out what exactly needs fixing and hopefully why it needs fixing. Neutral bug reports will make our work easier to read as opposed to reports that are angry or disrespectful in tone. It is difficult enough to have to deal with a problem in the product, antagonistic reports don’t help as they can only increase the difficulty, cause conflict and discredit the reporter for lack of professionalism.

[Link: Article: How to report bugs effectively]

Email != defect tracker

I suppose one could expect that having a buglist or a defect tracking system should already be pretty standard in a software project. But it’s funny how it’s the basics that gets forgotten or foregone sometimes.

Well, we did have Quality Center set up, but one of the concerns was that the devs weren’t paying attention to it and testers weren’t logging into it. It was a chicken-and-egg thing. Email got inundated with issues, follow-ups and such which kinda sucked since I always got copied into issue email. Eventually, our PM (he got cc’d too) put his foot down and we’ve made the shift back into QC with me goading the testers to use our defect tracker as intended, and with someone from the dev team monitoring the defects with respect to dev assignments. Thankfully, the team has been quite cooperative.

This has also pushed me into tinkering a bit with the reporting capabilities of QC. There are built-in reports that I found to be of use, and I’ve also created my own queries for generating my own reports on defects and test case status.

So far, it’s been working out. With the shift to QC, our email is no longer as abused or misused for defect tracking. We can now misuse it for something else (jk). One major advantage is that we can now, if needed and as needed, easily extract defect data. Instead of having to dig through old email, getting the list of open issues across the many applications that we’re handling at a time is now quite easy. Having all bugs logged into QC also allows for easier detection of red flags e.g., if several testers are reporting similar issues at the same time it’s possible that there’s already a global issue; or if devs are deferring a significant number of defects as non-issues, that could be a red flag on the quality of defect reporting, or that valid issues are getting dismissed.

The lesson learned is simple: use things as intended. Email for comms and the defect tracker for defect tracking. They’re there specifically for those purposes so use them accordingly. And, this is a team thing, so even if the testers were so disciplined in logging the defects, it won’t be as efficient if the devs aren’t using the tool as well. Work to have an alignment within your team so that the available tools can be optimized.

How helpful are my bug reports

I reckon that question is something we ought to hardwire into our systems. After all, the main reason why we log bugs is to have them fixed, and helpful bug reports just make it a bit easier for us and our devs to close the bugs. How so?

Helpful bug reports allow the devs and testers to easily replicate the bug. In order to fix a problem, you’d need to be able to see it first. This also goes for testers who need to verify a bug fix (especially when reported by someone else) i.e., we ought to know what was wrong to begin with and what has been changed in order to close the item. If you happen to be a forgetful tester like me, logging enough details on how to replicate the bug and other background info on the bug helps refresh the memory.

Unavoidably, there are cases wherein the bug is very difficult to replicate or it happens intermittently. For such cases, indicate that info in the bug report so that the dev or the tester who’s trying to replicate it knows that the bug would be a challenge and so that the bug won’t immediately be dismissed. Provide as much background info as you can recall when you first encountered the problem, and provide additional details whenever you encounter it again.

Helpful bug reports help isolate the problem. The problem’s easier to address if you know which particular cases trigger it. Say, it’s a problem that occurs across all screens, then it could be a framework bug or a browser-related problem. If it’s a problem that only occurs for particular data or in particular environments, indicate it in the bug report.

Helpful bug reports are written with some consideration on who’s going to read them. It’s easier when you report a problem to your dev since both of you are working on the same function. But sometimes, you have to report a bug to a framework dev who may not have as much background on the domain as you do. Or sometimes you have to report a bug that will be handled by another tester. Or sometimes you have to report a bug that will need to be reviewed and approved by the HK SA. For such cases, you need to provide enough background information for the bug report to be better understood.

Helpful bug reports provide information on the expected results. Say, a tester found a UI bug wherein a particular text that’s supposed to be bold is italicized. The report shouldn’t be something like the ‘text “blah blah” must not be italicized.’ The dev might just remove the italics and not set it to bold. If you know the expected behavior, then provide the information. If you know of a forum topic discussing the bug, then post a link to it and post a summary of the resolution once it’s already available.

Helpful bug reports are concise, without leaving out the details that could be useful. Provide the details without being too wordy about it. Too much text tends to make the eyes glaze over. And with that note plus some links below, I end this post.

How to report bugs effectively
How to write a good bug report

Bug isolation

A friend had shared with me a paper on bug isolation. It’s a 2004 paper by Danny Faught entitled How to Make Your Bugs Lonely: Tips on Bug Isolation. It starts off with the quotation: “A problem well-stated is half-solved.” The paper then got me thinking on whether we cover or emphasize this strongly enough during tester training in our company. Sure, our testers indicate the expected behavior versus the actual behavior in their bug reports, and sometimes they provide the steps to replicate the problem. But giving the steps does not necessarily equate to giving the conditions that trigger the bug report.

In bug isolation, one attempts to really zero in on the bug by reducing the factors that obscure the problem.  For instance, you streamline the steps to get to the bug, or you narrow down the data needed to trigger the bug.  Basically, you refrain from posting shoddy bug descriptions like “It doesn’t work”, and instead you try to find out the specific conditions that trigger the bug.  You might think that bug isolation only benefits the developers since they’ve less effort to render in trying to replicate the bug.  But I think, this will also help the tester in thoroughly verifying bug fixes.  How else would one know if a problem is solved if he doesn’t know the problem to begin with.  Verifying the absence of the previously detected symptoms is not always a sure-fire guarantee.

Continue reading

Some nifty tools – bugshooting, pixie, jruler

Here are some nifty tools. Their usage is not limited to testing though.

Bug Shooting. It’s another screen capture tool which has a more catchy name than what I normally use i.e., Gadwin PrintScreen. Its edge over Gadwin is that if you intend to make some edits (e.g., add some arrows, circles, or some text), it has those functions immediately available after taking the screenshot. Whereas in Gadwin, you’d still need to open some image editing tool like Paint. With Gadwin, your screenshots are automatically saved into your capture folder. With Bug Shooting, you’d still need to save the file manually. But alternatively, Bug Shooting has an option of automatically sending the screenshot to an application. E.g., after taking a screenshot, you can send it to your email application and the image gets automatically attached.

Pixie. This tool acts like a color picker. So while it’s running, you’d just have to point your mouse over any part of the screen and Pixie returns the color you’re pointing at in hex, rgb and other formats. It also gives out the x and y positions.

JR Screen Ruler. If I remember correctly, this one was shared by Tats a long time ago. It offers a better alternative to using your “man-calipers” (as Chry puts it), or resizing some other window to use as a make-shift ruler. With this you can measure the length or width of certain screen objects in pixels, inches or centimeters.

Article: How to report bugs effectively

“… When you report a bug, you are doing so because you want the bug fixed. There is no point in swearing at the programmer or being deliberately unhelpful: it may be their fault and your problem, and you might be right to be angry with them, but the bug will get fixed faster if you help them by supplying all the information they need…”  Read more… How to Report Bugs Effectively.

You can also provide some references to supplement your report. For instance, you can take screenshots using a tool like Gadwin Printscreen. You can also capture the bugs together with the steps to replicate them on video through a tool like CamStudio.

Acknowledgements: Thanks to Roy for sharing the file.