|
Risk comes from NOT knowing what you’re doing: identify your SAP quality risks! |
In order to identify where the major quality risks are in your SAP project, execute an SAP Quality Risk Analysis (SQRA). The general approach for the SQRA is to conduct a workshop with all involved stakeholders from the business streams (or SAP Solution Trains/ Line of Businesses (LoB’s)) like Finance, Supply Chain, HR, BW and even for the Non-SAP systems to determine the processes and areas with the most risk. Participants for this workshop are from both the Business side and IT side.
From the Business side, the following stakeholders should be invited to participate: the Key User, Functional SAP Consultant, Functional Business Analyst, Product Owner. Stakeholders from the IT side are: Technical Leads, Technical SAP Consultants, Integration Consultants, System Integrator.
During the workshop, how to rate each process is explained. IMPORTANT: to do a good SQRA make sure that the detailed scope is clear (preferably on process level), defined and signed off. In case this is not 100% clear, the SQRA still will identify the risk areas, however it will not be on the lowest desired detail level. In such case, when the scope becomes more detailed, additional SQRA workshops should be organized to update the risks.
During the Quality Risks Analysis, the scope of the change or project (regardless of an implementation, a rollout or an upgrade) is important. Global and local requirements, legal requirements, language requirements etc., should be included in the Quality Risk analysis.
A quality risk is the chance that the process fails in relation to the expected impact if this failure occurs. The SQRA is a weighted risk approach where risk input from both Business and IT are merged into a risk classification:
- The Business input consists of the “Frequency of Use” and the “Business Impact” (Business: end Key User/ Functional SAP Consultant/ Functional Business Analyst/ Product Owner)
- The IT input consists of the “Process Complexity” and the “Technical impact” (IT: Technical Leads, Technical SAP Consultants, Integration Consultants, System Integrator (SAP domain specialist))
This means that choices must be made. In addition, the test efforts must be distributed as effectively and efficiently as possible over the total test scope. This principle is the basis of the risk-based test strategy.
To determine the risk, each process in scope is assessed and rated using specific rating definitions. The rates in the following tables are indicative and can be modified for each SAP Project.
Combined/Weighted risk
The values for each risk component rate from 1 to 5 and have been predefined using the definitions and guidelines above. The definitions are adaptive and best practices, it is always possible to adjust and finetune according to the project specifics and or client needs. The risk classification is the combined and weighted outcome of the “Business input” and the “IT input” risk score = (FU+BI)*(PC+TI). With a minimum score of 1 and a maximum score of 100 per process/ transaction/ test case. (score 0 is No Risk which implies No Test, and also implies no need for development & implementation).
The values are auto calculated in the template based on the weighted rates. These risk ranges are indicative and adaptive. Based on the totals per risk class (the total amount of processes, SAP transactions, E2E scenarios) the risk boundaries can be adjusted to come to a balanced risk analysis.
[NOTE: Every SAP project is different and will lead to their own balanced overview. Not all objects can be classified as critical, not all can be classified as low. And there will be objects with no change, so no risk, no test. There must be a well-balanced average. Therefore the given risk class boundaries can be adjusted to meet or come close to these references.]
Example sheet
The results can be captured in an Excel sheet calculating the risk classes.
You can download the calculation sheet template here.
The output of a SQRA is input for an SAP Test Strategy and SAP Test Plan.
For example: for all objects identified as critical, we have to define 1 or more test cases to thoroughly test this object (positive flow, negative flow, happy flow etc.). For each risk class a metric and assumption can be defined to determine your test planning. This metric and assumption will help to calculate, roughly, the total effort for designing and executing test cases. Critical risk test cases might take more time to test then low risk test cases.
Test Life Cycle
Based on the location of the test object/ process in the risk matrix (Risk Class), all test objects and processes are prioritized. The result is ordering of all processes, the most important / highest risk ones first. In addition to prioritization, a differentiated test approach for the processes can be defined based on their Risk Class.
This can be used throughout the whole development life cycle:
- Delivering test objects
- When specifying tests
- When executing tests
- When fixing anomalies
Confidence
Based on the relative level of risk (Risk Class), test design techniques can be selected that will give the proper level of confidence (as per VOICE model). Keep in mind that within SAP the focus is on business and E2E processes, meaning that process-oriented test design techniques will bring the most value. Some other test design techniques are still applicable but should be used earlier in the development life cycle (e.g., during Unit or System Test).
Example: Use SQRA in test strategy and test plan
Below an example how the outcome of an SQRA can be translated, converted and used for a test strategy, test plan and an automation approach.
The outcome of the SQRA will give insights in the Critical to Low-risk areas/ processes. Critical processes should be tested as soon as possible, at least focus on the happy flow in an early testing stage, for example SIT. In many big SAP programs, multiple SIT cycles are planned, divide the risk classes over the several SIT cycles (see picture above) and add negative flows for Critical risk classes as well. When the happy flows are successfully tested, they can be nominated for (manual) Regression Test Suite and for Automated (Regression) Testing. Per cycle, expand the Regression Test and Automation Suite.
During UAT, the focus shift to E2E scenarios and processes – Vertical and Horizontal and all risk classes are involved in these processes as the process steps can be linked to each other as a chain.
Control mechanisms (quality gates, indicators and metrics) must be in place after each cycle to reflect insight in status and quality and be input for Go/No Go decisions.