Expert Testers

Estimating the time it takes to test a feature is an important, yet often overlooked, skill. If your estimate is too low, it can affect your product’s quality and release date. If your estimate is too high, it can lead to an inefficient use of resources.

The topic of project estimation is complex enough topic to fill entire books. (See what I mean?). But here are a few simple tips that can quickly improve your estimation skills, not to mention your work/life balance and general happiness.

Consider non-functional testing

A common mistake among less-experienced testers is to only consider functional testing when providing an estimate. In this type of testing, software is evaluated against the functional requirements. On many projects, however, non-functional testing can take longer than functional testing. Non-functional testing includes areas such as security, performance, stress, accessibility, and long-haul.

I once badly underestimated my testing effort because I didn’t consider long-haul testing. (This was mostly because I didn’t know what long-haul testing was.) I worked some crazy hours to make sure my oversight didn’t negatively affect my deadline. My extra hours however, did negatively affect my wife’s mood, as she was left by herself many evenings to care for our newborn.

Consider the non-testing tasks

Non-testing tasks include writing your test plan, conducting a test plan review, participating in specification reviews, attending meetings, updating the automation framework, and logging your test cases and results. These tasks can easily take 25% or more of your time. You might want to track the time you spend on these tasks for a project or two to get a good idea of how long they take.

Assume bugs will be found.

All complex software contains bugs. One of the goals of testing is to find these bugs so a decision can be make on whether or not to fix them. If you’re not planning on finding bugs, then you’re essentially planning that your tests will be ineffective. Don’t plan on writing ineffective tests. Account for the time it’ll take to investigate test case failures, reproduce issues, log bugs, follow-up with dev, fight for bugs in triage, validate fixes, and re-run your tests.

Be cautious about giving an estimate before doing your research

One of the biggest mistakes you can make is to offer an early ballpark estimate before doing your due diligence. Your initial guess will greatly influence your final estimate due to a behavior known as anchoring. People have a tendency to “anchor” to their first guess and make small adjustments from there. These adjustments, however, typically under-estimate new information, leading to an inaccurate final estimate.

For example, the always-reliable Wikipedia cites a study in which people were asked to guess the percentage of African nations that are in the United Nations. Those who were asked “Was it more or less than 10%?” guessed lower values (25% on average) than those who were asked if it was more or less than 65% (45% on average).

I was once asked to estimate my test effort during the project kick-off meeting. I had just learned about the project, and had no business giving an estimate. But I was pressed to give a rough estimate, so I guessed two weeks.

When wrote my test plan, I realized my original estimate was too low. I had forgotten to consider some non-functional testing tasks and found the automation framework I planned on using wasn’t reliable. I spoke to my manager and we both agreed to adjust the estimate, but the damage had already been done. My two-week guess had become our anchor. We doubled the estimate to four weeks, but it turned out we should have raised it to six. As a result, my work/life balance took a turn for the worse, and once again, my wife was not happy.

Ask the developers for their estimate

Although it’s not ideal, the reality is that at some point you will be asked to estimate your test work before properly analyzing the project. In these situations, the amount of development work is a good gauge for the amount of test work. The more complex the feature, the longer it takes to both develop and test.

I’ve found that a reasonable rule of thumb is that the test effort is 1 to 2 times the development effort. If this ratio sounds high, consider how many times you gave a test estimate, but then wished you had more time as the project completed. I want to stress, however, that this is just a rule of thumb; it should only be used if you absolutely have to give an estimate before doing your due diligence.

If the developers haven’t yet made their estimate, then you have a stronger case for not yet giving yours. You can’t reasonably be expected to make an accurate test estimate if the developers don’t even know how complex their code will be. If management still insists on an estimate, have them speak to your wife.

Filed under: Test Planning | Tagged: functional testing, long-haul testing, project estimation, test estimation |