If you’re like many testers I know, you hate test plans. Most testers fall into three groups: Group A doesn’t like writing test plans, Group B thinks test plans are a waste of time, and Group C thinks groups A and B are both right.
Let’s start with Group A. Some testers don’t like writing test plans because they simply don’t enjoy writing. Their dislike has less to do with what they are writing, than the fact that they are writing. Perhaps writing does not come naturally to them. One tester recently told me, “I write like a third grader.” It’s no surprise he doesn’t like writing test plans–we like to do things we’re good at.
Some testers don’t enjoy writing test plans because they are not really testers at all–they are developers working in Test. They would much rather if someone else analyzed the functional specifications, wrote the test plan, and told them exactly what they needed to automate.
Like many testers, all of my previous jobs had been in development. I thought writing a test plan would be easy–until I saw my first test plan template. Performance testing? Stress testing? Long-haul testing? I quickly learned that I had no idea how to write a test plan, and I dreaded writing one. But I started reading plans from testers I respected, and I started reading books on testing. I slowly made the transformation from developer to tester. Now I enjoy writing test plans because I know it makes my testing better.
Perhaps the most interesting reason testers dislike test plans is because they don’t think the plans are useful. They point to the fact that most test plans are never updated. As the product evolves, test plans become out of sync with the current functionality. I’ve found this to be true. Of all the test plans I’ve written in my career, I know exactly how many I updated after the product changed: zero.
This is a problem for two reasons. First, it would be one thing if test plans were quick and easy to write; they are not. Depending on the feature, it can take me a week to write a solid detailed test plan. Some would argue this time could be better spent automating test cases or performing exploratory testing.
Even worse, test plans that are out of sync with product functionality give inaccurate information to the reader. Recently, I worked on a product that was being updated for its next release, and I was assigned to test a feature I was completely unfamiliar with. The first thing I did was review the original test plan to learn how the feature worked and how it was first tested. I assumed, incorrectly, that the document was up to date. As a result, I reported bugs even though the feature was working properly.
James Whittaker, Test Director at Google, recently debated the value of creating test plans on the Google Testing blog:
As to whether it is worth doing, well, that is another story entirely. Every time I look at any of the dozens of test plans my teams have written, I see dead test plans. Plans written, reviewed, referred to a few times and then cast aside as the project moves in directions not documented in the plan. This begs the question: if a plan isn’t worth bothering to update, is it worth creating in the first place?
Other times a plan is discarded because it went into too much detail or too little; still others because it provided value only in starting a test effort and not in the ongoing work. Again, if this is the case, was the plan worth the cost of creating it given its limited and diminishing value?
Some test plans document simple truths that likely didn’t really need documenting at all or provide detailed information that isn’t relevant to the day to day job of a software tester. In all these cases we are wasting effort.
I agree that there may be some wasted effort in writing a test plan. For instance, I’m guilty of spending too much time tweaking the wording of my documents. It’s a test plan, not a novel. Bullet points and sentence fragments are fine–you shouldn’t be spending time using a thesaurus to find synonyms for the word “feature”.
But that doesn’t mean it’s all wasted effort. In fact, I believe the benefit far outweighs the effort. This is true even if the test plan quickly becomes obsolete.
Consider feature specification documents: much like test plans, feature specifications often become outdated as the product evolves. This doesn’t this mean we shouldn’t write feature specifications. Potential document “staleness” is not a valid argument against writing spec documents—or test plans. Just don’t make the mistake of assuming old specifications are still accurate.
One of the most important reasons for creating a test plan is to get your thoughts about testing the feature onto paper and out of your head. This unclutters your mind and helps you think clearly. It also documents every idea you’ve had about testing the feature, so none are forgotten.
The writing process often leads you to think of both more and better ways to test the feature. Your first pass at writing a test plan may include mostly positive functional test cases, as well as a handful of negative functional tests. But the process of refining your document leads you to consider more negative tests, more boundary tests, more concerns around scalability and security. The more time you spend planning your testing, the more complete your testing will be.
Detailed test plans can also help you find bugs in the feature before the code has even been implemented, when they are much less costly to address. Two techniques that are excellent for finding design holes during the test planning phase are Decision Tables and State Transition Diagrams. I remember creating a large Decision Table as part of a test plan for a security feature which uncovered that nearly 10% of the possible input combinations didn’t have an expected result in the design specification document.
Test documents are also valuable for conducting test plan reviews. After creating your test plan, it’s important to have it reviewed by other testers–no single tester will ever think of all possible test cases. It’s also valuable to have your plan reviewed by both the developers and program managers. In my most recent test plan review, a program manager told me that one of the features I planned on testing had been cut. In the same review, a developer informed me about existing test hooks that saved me hours of development time.
When testers say they don’t want to write a test plan, I can sympathize. Most of us got into this business either because we like to program or because we like to break things, not because we like to write documents. But when testers say they don’t want to write a test plan because it’s not valuable, I have to disagree. Good test plans make your product better. So what if they become obsolete? As both Dwight D. Eisenhower and Wile E. Coyote once said, “plans are useless, but planning is indispensable.”