The End of a Bug’s Life

I recently wrote an article describing five best practices that will increase the odds of your bugs getting fixed. If you were killing time at work, you might have read it. One  practice I suggested was closing your Resolved bugs as early as possible. Although I follow this practice now, I didn’t always do so.

Not long after I became a Microsoft SDET, I was assigned to a project that was approaching its release date. Two weeks before our deadline, it looked like we were in trouble. (I’ve since learned it always looks like you’re in trouble two weeks before a deadline.) There were a lot of bugs that had been Resolved by Dev, but not validated and Closed by Test.

A couple of days before our deadline, the Program Manager emailed the team a status update. We were making great progress, and there were only a few bugs left to Resolve and Close. The email had a chart that looked like this:

clip_image001

The graph showed we went from over 300 Resolved, but un-tested, bugs to almost zero in two weeks. How did we do this? A lot of long days. Working over the weekend. Meals at the office. Little sleep. Lots of stress. As good teams do, we rallied together to meet our deadline. It felt good.

It felt good until our next deadline, when we repeated this cycle all over again.

I’ve since learned there’s a simple solution to avoid the stress that comes with validating so many bug fixes just before a deadline: don’t wait until the last-minute to close your bugs!

Closing your bugs should be part of your weekly routine, not something you do in batch before a deadline. This approach has six advantages.

  1. You won’t have to work long hours to close bugs at your deadline.
  2. You’ll never be tempted to cut corners to close bugs at your deadline.
  3. Bugs are easier to close when they’re fresh in your mind; if you wait too long, you’ll have to re-learn them.
  4. If you find a problem with the bug fix, it gives the developer more time to resolve it.
  5. If you have to re-open the bug because it still doesn’t work, it won’t be rejected because the “bug bar” has been raised.
  6. When a bug isn’t closed, it’s not known whether it’s fixed or not, or whether the fix breaks something else. The more Resolved bugs you have, the less you know about the real state of your software.

Since there are so many advantages to closing your bugs in realtime, rather than in batch, why don’t more testers do it? Let’s create another list.

  1. It’s not very interesting to close your bugs. After automating a test, finding the bug, and investigating the root cause, all the interesting work is done. Verifying the fix is either a matter of running an automated test, or manually testing the feature again; neither of which are favorite pastimes of most testers.
  2. Although many organizations encourage the timely resolution of bugs, few encourage the timely closing of bugs.  I’ve worked on many teams that had guidelines on how quickly bugs must be Resolved, such as P0 bugs within 24 hours, P1 bugs within a week, and P2 within the current milestone. However, there were rarely similar guidelines for testers to verify and Close these bugs. It seems reasonable that testers should be held to  the same standards as developers, and have to close P0 bugs with 24 hours of them being Resolved. One way I’ve seen teams encourage the Closing of bugs is by having a “bug jail”. Once a tester or team exceeds a pre-defined number of Resolved bugs, they have to close these bugs before moving on to other tasks.
  3. Some testers work on teams that reward employees based on the number of bugs they find. What effect does this have on testers? It encourages the finding of bugs, not the Closing of them. If anything, testers should be rewarded based on the number of bugs fixed and closed, which would at least encourage the resolution of quality bugs.
  4. Sometimes testers don’t Close their bugs in a timely manner because they’re disorganized. Most testers have a hectic schedule, and it can be hard to keep track of which bugs you need to Close. Fortunately, Microsoft testers have access to a tool called Bugger. Bugger docks on your desktop, queries the bug database, and displays the number of bugs assigned to you. Clicking on Bugger shows the details. With Bugger docked on your desktop, it’s almost impossible to forget which bugs are assigned to you.

clip_image002

If you work at Microsoft, install Bugger and avoid some of the stress that comes at project deadlines. If you don’t work at Microsoft, consider writing your own version of this tool. Your manager and co-workers will appreciate it.

Advertisements

2 Responses

  1. Nice article! I would follow up on your first list, your points 3, 4, and 5 and move them up to top priority in the list: resolving bugs quickly is better all around for the team in terms of keeping a better handle on overall product quality and reducing churn and risk. Point 3 is from the tester’s point of view, but from the point of view of the whole team, keeping resolved bugs around represents risk. If closing a bug but discovering a new one (or re-opening it if not fixed) is done sooner rather than later, risk and cost are reduced relative to where they would be if the tester should sit on it for a week or more. Any change in product code has risk, which (everything else being equal) increases later in the SDLC towards ship time.

    So, as far as metrics used by management to reward and evaluate testers, I’d advocate for a good balance between finding new bugs and following through with bugs that are resolved back to you, whether fixed or not.

    Hey Andrew, please check out my MetaAutomation blog! I think it has value for you. Critical comments are welcome, as long as they are clear and well-reasoned (I’m not worried about that from you, it’s clear you’re a lucid writer)

  2. One motivation I finds useful, is to remind myself that when closing a bug, I have an opportunity to find either a new bug resulting from a fix or uncover an undiscovered bug.

    Thanks for the blog post. I did not find a Bugger type tool for Fogbugz on OS X, but did find a utility, Quick Bugz (http://www.quickbugzapp.com) that gives me quick access to issues assigned to me.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: