Creating a Test Strategy is a natural thing for testing professionals. It pays a lot of attention to the detective quality measures. But the column “other quality measures” (see figure below) does not properly value the importance of these other quality measures. With this insight, we extended the traditional test strategy to a quality engineering strategy, which gives proper focus to quality measures to build quality in (preventive), quality measures to demonstrate the quality level (detective) and quality measures to improve the quality (corrective).
The benefit of applying a quality engineering strategy is that a team (or group of collaborating teams) creates easy and clear guidance for everybody involved, about which activities to apply to achieve built-in quality in fit for purpose IT systems. This guidance will have differing focus, some quality measures are linked to individual user stories, other quality measures apply to features or epics and the quality engineering strategy will also contain generic quality measures that the team applies always. Apart from the choice of quality measures, also the stage in which a quality measure is applied, matters, especially when multiple teams are involved. In that case the quality measures are also linked to the relevant fundamental DevOps activity (for preventive and corrective quality measures) and to test varieties (for static and dynamic quality measures).
DefinitionThe quality engineering strategy describes the allocation of quality measures to IT delivery items (e.g. user stories, features etc.), to balance the investment in quality engineering activities and to make an optimal distribution of effort over fundamental DevOps activities and Test Varieties. The quality risk class is used to assign the intensity of the quality measures that must be applied. |
Here you can download a template for the Quality Engineering Strategy which you can use as-is, or adapt to your specific needs. An explanation of the use of the spreadsheet-template is on a separate sheet of the document, and it contains examples to clarify the use.
This template also contains a list with over a hundred examples of possible quality measures. Although we strive to keep the list up to date, please be aware that you can always add your own quality measures which may be very specific to your situation or so new that we have not yet added them to the list.
Of course, this template is just a tool to support you in creating your quality engineering strategy. The way you register the connection between IT delivery items and quality measures is up to you. For example, teams use the template to set up their strategy and afterwards register the risk class, intensity, and quality measure with a user story in the tool they use (such as Jira or Azure DevOps) and then maintain and update it in that tool only. In practice you do not always need to decide upon quality measures for every user story, the Generic Test Agreements (which may also be called Generic Quality Engineering Agreements) or a Master Test Plan (Master Quality Plan) may give guidance for quality measures that must be applied for all user stories and therefore these may be part of acceptance criteria in the Definition of Done.
When creating your quality engineering strategy please be aware that this must be continuously maintained, as user stories are added or deleted, but also user stories change, or the risk-perception may change. High-performance IT delivery teams therefore should ask the question “do we need to change the quality engineering strategy for any IT delivery item?” very regularly, for example at every daily standup meeting and in retrospective meetings. For the team the items in the strategy are mainly a to-do-list.
Some people may struggle with the variance of the type of IT delivery items. Some items are small user stories, others may be complete epics. For large scale IT delivery items other quality measures may apply than for small IT delivery items. A quality measure for an epic for example might be to prepare and execute an end-to-end regression test, which is of a completely different scale than the unit test for a piece of code. Also keep in mind that some quality measures will relate to multiple fundamental DevOps activities and/or multiple test varieties.
ExampleIn QualityLand the product owner has put a user story on the backlog to create a new button in the app for people that are waiting in line and want to know their exact waiting time. The team has determined several quality measures that must be applied for this high-risk function (it’s high-risk because if the time is given wrong the people in the queue will be annoyed). As a preventive quality measure (to build the right level of quality from the start) the team decides to apply pair-programming which they position in the fundamental DevOps activity “code”. As a detective quality measure (to demonstrate that the quality level is as expected) they apply functional suitability testing and for performance testing (response time), in the test variety “unit testing”. As a corrective quality measure (to pro-actively improve quality) they apply the quality measure refactoring in the fundamental DevOps activity “integrate”. So with this example you see how several quality measures are selected, based on the risk level and matched with the fundamental DevOps activities and the test varieties (note: this is a partial example, in real situation other quality measures in other activities and test varieties will also be needed). |
And as a closing remark, a tip for any team: Be careful to only select quality measures that you will be able to implement. If you select a quality measure that is new to your team, make sure you also arrange coaching or training!
Related building block:
Quality risk analysis & Test strategy
Creating a Test strategy and Test plan in high-performance IT delivery