Improvement of the application being delivered
TMAP is all about improving products, processes and people. The Goal Question Metric (GQM) is about improving products.
The improvement of the application being delivered is strongly related to the metrics as defined in Metrics. You could use the GQM approach to identify Goals, Questions and Metrics [Basili 1994]. When using GQM you can distinguish between two types of goals:
- Knowledge objectives ⇒ Where do we stand right now?
- Improvement objectives ⇒ Where do we want to go?
We will explain this using an example. First, we formulate the goal. Let's assume the knowledge goal is: "Provide insight into the quality level prior to release".
Then you have to ask questions concerning this goal, questions such as: "Are the most important risks covered/found early in the DevOps QA and testing activities?", "Are the most important risks covered/found in the DevOps QA and testing activities?", and "What is the stakeholder's perception of the covered/found risks?"
The next step is deriving relevant metrics from these questions. You could use some of the already defined metrics in, or even better, use the metrics for which data is already available. Possible metrics are:
- Percentage of critical anomalies found early in test execution (effectiveness)
Ratio between the number of critical anomalies detected in the first quarter of the test execution lead time and the total number of critical anomalies found in the total test execution lead time.
- Percentage of critical tests run early in test execution (effectiveness)
Ratio between the number of critical tests run in the first quarter of the test execution lead time and the total number of critical tests run in the total test execution lead time.
- Percentage of identified risks covered by (executed) tests (effectiveness)
Ratio between identified risks and the total number of (executed) tests in relation to these risks.
- Average cost per risk item covered during testing (efficiency)
Total test costs divided by the total number of risk items.
- Stakeholder's perception of accepted risks prior to release and after it has been released into production (customer satisfaction)
- Survey of risk perception prior to release to be completed by the stakeholders, and the same survey after three months of production. Make a comparison between both survey results.
After finishing the GQM cycle, improvement actions are defined and executed, and you could start a new GQM cycle. In this way you are continuously improving.