The importance of documenting test plans

There's a popular misconception that being Agile is all about a lack of documentation.  That instead of writing up huge functional specification documents, we're just going to document nothing.  Heck, there's even a Dilbert comic strip about just that.  That fairly common belief couldn't be further from the truth.

While we do practice minimal documentation in Agile methodology, we also believe in necessary documentation.  For example: documenting functionality and test scripts in a written record somewhere.  That starts with a well written story, and a test plan.

I love working with physically co-located teams.  However, in this day and age, physical presence is sometimes an impossibility and in those instances it becomes even more important to document conversations and the agreed on definition of "done."  For example, here's a great comment I'd love to see in Jira from a distributed team member: "I spoke with John, we agreed that the new delete functionality will only show for users that are company-wide admins, not regional admins."  Without this critical insight, no one will know how the story's definition of "done" evolved from the original version.

When we move into a test cycle this information becomes even more critical.  Without accurate descriptions of how the story changed along the way, a tester has no way of knowing how to test the story without going repeatedly back to the Dev and the Product Owner and asking question after question.  Additionally, describing the exact functionality that is being tested is valuable for the team's understanding (everyone on the same page now!), the clarity of purpose of the QA (what am I testing again?), and for historical purposes.  The first two make obvious sense, but the last is what motivates me.  I'll give you an example from my day to day life:

A few years ago I had worked with an engineer to design and develop a new reporting module.  We were transitioning some customers from our legacy system to our new platform and as a result had to do some data conversion.  During the testing cycle, this conversion process was detailed as part of the test plan: "Take a legacy customer, generate X activity, push them through the upgrade cycle, in report Y you should see that their balance is Z."  That sort of thing.  Four months after this feature was released I got an escalation kicked up to me because a customer was furious about their account and couldn't understand what was going on.  I looked at the report and couldn't figure it out myself until I did a quick Jira search, found the record of the story, and checked the test plan.  I was then reminded of the decision we had made a third of a year ago and could accurately help my customer.  

So please document your test plans.  Don't just write "I tested it, everything's cool."  That doesn't help anyone.  Like stamps in your passport, story comments and test plans let us know how the story traveled and where it ended up.