A cross-functional team will agree to deliver an IT product with a specific quality level. This quality level is defined by the acceptance criteria. As soon as the product meets these criteria it is "done". The acceptance criteria are specified per user story. The Definition of Done will define that a user story is done when all acceptance criteria have been met. The product owner is the primary stakeholder to be involved in defining the acceptance criteria, but the team will assist on this.
In high-performance IT delivery a common way to describe the individual deliverables is by creating user stories. These have three critical aspects, the Card, Conversation and Confirmation. [Jeffries 2001]. The card holds the short and basic information of the user story. The conversation is an exchange of thoughts, opinions and feelings between the stakeholders and the team. This may result in supporting documents. The confirmation is the way the stakeholders will confirm that the team has delivered what they expected. This is measured using the acceptance criteria. The acceptance criteria are defined in terms of quality characteristics and quality risks.
| Acceptance criteria are the criteria a test object must satisfy to be accepted by a user, client or other stakeholder. |
Acceptance criteria will help the team understand what is included in the scope and what is not in scope of the user story. The acceptance criteria may relate to just one user story, for example when the acceptance criterion is related to a specific piece of functionality. On the other hand, acceptance criteria may also be related to multiple user stories, to epics or even to the entire system or the whole organization. In that case, the acceptance criterion can be included in the Definition of Done so that it does not have to be repeated for each user story. The DoD typically evolves over time as teams are improving their way of working. An acceptance criterion in the DoD may for example be related to a non-functional aspect such as security.
Testing for the acceptance criteria in DevOps is done by automated test execution within the CI/CD-pipeline. Additional manual testing may also be needed.
When the team and the stakeholders discuss the acceptance criteria, the various quality characteristics are a good reference. The quality risks that are used to prioritize the user stories will also hold valuable information for defining acceptance criteria.
The acceptance criteria may be specified as examples as intended in the "Specification by Example" approach [Adzic 2011]. These examples can later serve as test cases for manual or automated testing.
When defining acceptance criteria, you may find these seven tips useful [Ravlani 2017]:
- Each product backlog item or user story should have at least one acceptance criterion.
- Acceptance criteria are written before the implementation of the user story.
- Each acceptance criterion is independently testable.
- Acceptance criteria must have a clear Pass / Fail result.
- It focuses on the end result – What. Not the solution approach – How.
- Include functional as well as non-functional criteria.
- Team members write acceptance criteria and the product owner verifies these.
We would like to emphasize that creating valuable acceptance criteria requires the team, the product owner and other stakeholders to collaborate closely so that the acceptance criteria are supported by everyone involved.
Do not neglect the acceptance criteria as they – being simple and approachable – solve multiple problems at once. They document customer expectations, provide an end-user perspective, clarify requirements and prevent ambiguity, and eventually help quality assurance verify if the development goals were met. [Altexsoft 2019]