Methodologies along with software projects management habits and customs have gone from one extreme to another in respect to the amount of generated documentation, the present trend is to keep the documentation volume small enough so that it doesn´t over burden a software development project, but big enough to sustain a project through time. Tools and practices have been created to avoid information duplicity between tests, user stories and other documents containing acceptance criteria (functional or nonfunctional). These practices were born with TDD (test driven development), BDD (behavior driven development), ATDD (acceptance test-driven development) and test automation tools and frameworks.
Redundant and inconsistent documentation
I must stress the importance of scaling/adapting agile frameworks to support enterprise-scaled projects, more specifically requirements, stating that using user stories as disposable artifacts, with a single sprint lifespan as the main requirement placeholder type is not enough. The requirement definitions end up dispersed along several iterations.
Despite that the previously described problem can be solved creating and maintaining a requirement repository, there underlies a problem that makes teams stop documenting and it is discovering, time and time again, that the same expected behavior definitions are written everywhere in the documentation, just using a slightly different language and/or format. This is especially so between user stories acceptance criteria and test cases.
Many projects write the acceptance criteria using the scenario´s format:
Scenario 1: Title
AND [more context definitions]…
AND [another outcome]…
Scenario 2: …
Essentially the scenarios are a definition of expected behaviors under a given context and after certain events.
Test cases tend to cover, but in a different format, the same scenarios already present as user stories, duplicating the documentation. It is also true that projects are prone to define just the most likely scenarios in the user stories and then the QA team extends the definition of the missing scenarios in the test cases, though this is not recommendable being that the requirements definition becomes inconsistent between test cases and user stories, and the acceptance criteria remains partially duplicated in the documentation.
Getting the right balance
We should ask whether it is necessary to write the acceptance criteria within the user stories, and when I say use story I mean the issues of the type user story you use in your issue tracker, not just the abstract concept of user story.
I understand that it is more efficient using user stories with sprint planning and discussions on requirement purposes and define the acceptance criteria in another more permanent and consolidated placeholder, using a TDD, BDD and/or ATDD tool or framework.
If you are able to practice TDD “test first” practice you obtain the benefit of limiting the amount of code generated to only what is necessary to pass the tests. Of course test first is more easily done within low and medium abstraction levels and much more difficult to practice in UI features.
Using scenario language and structure we force even the unit test to be more descriptive and comprehensible.
The requirement definitions and tests are stored in a source code versioning repository like SVN, so it will be easier to follow and achieve configuration management good practices like product baseline acquirement (including tests and requirements acceptance criteria).
The downside of using this requirements documentation architecture is that no matter how good is the tool you use, it will be harder for a non-technical person accessing, reading and writing the documentation, but I think in general there are more benefits than the negative aspects.
Related frameworks and tools:
- Specflow (http://www.specflow.org)
- Nspec (http://nspec.org/)
- StoryQ (http://storyq.codeplex.com/)
- Cucumber (http://cukes.info/)
- Rspec (http://rspec.info/)
- Fitnesse (http://fitnesse.org)
- Codeception (http://codeception.com/)
- Concordion (http://www.concordion.org/)
- EasyB (http://easyb.org/)
- JBehave (http://jbehave.org/)
- JSpec (https://github.com/wycats/jspec/)
- Jasmine (https://github.com/pivotal/jasmine/wiki)
- JSSpec (http://code.google.com/p/jsspec/)
- Behat (http://behat.org/)