Many organizations today use ERP systems to support their customers, vendors, and employees in doing their tasks, such as ordering goods, filing their expenses or changing their personal details in the system. Because these self-service systems (such as parts of SAP-solutions) support the core business without any manual interference of back-office employees, the business process must fulfil high quality standards. The people involved in acceptance of these systems (including acceptance testing) often are no professional IT personnel, but key-users, maintenance people or operations engineers. Because of these reasons, acceptance testing requires a specific approach which we describe in this blog.
Self-service solutions require more effort for acceptance testing
Two trends in IT today support the implementation of large solutions based on ERP systems. One is the limited availability of IT people, so that home-grown solutions are no longer an option. The other is the limited availability of back-office people, so that there is a need for self-service solutions.
Self-service platforms are a growing trend, allowing users more control over their own activities to create, manage or execute specific tasks. It doesn’t require intermediaries to interact with the required service as it is performed using self-explaining processes or via dashboards.
Well-known ERP-vendors such as SAP facilitate rapid implementation of these large solutions. However, there is one new risk: if the system fails, there are no people in the loop to prevent a failure to cause trouble for the user. So, there is less tolerance for failures. Which means that more effort must be put in making sure the system meets the required quality level, involving well-organized and well-performed acceptance testing.
Confidence as the North Star
The main goal of testing is gathering information about the quality of an IT solution and the related risks, to establish the confidence the stakeholders have that they will get their pursued business value. So, confidence is the ultimate goal of acceptance testing.
Confidence is built using information about the quality level of the IT solution. Step one in acceptance testing is to perform a quality risk analysis to establish what the quality level should be. Risk classes are used while organizing the tests to mitigate the identified risks, and thus determine the quality level.
The time available for testing is limited, not everything can be tested with equal thoroughness and intensity. The number of possible tests in a self-service system is simply too large. Teams should focus their testing efforts on the high-risk areas first, which are usually the new additions, configurations, integrations, and changes.
Quality is a necessity, not a luxury, risk is an important starting point
To efficiently evaluate the quality of self-service solutions like SAP, it is important to know what to test and where the focus of testing should be. An SAP Quality Risk Analysis (SQRA) is a standardized risk-based approach to determine the focus for testing. The first step of the SQRA is to determine the scope of the platform. Do we need to test the complete scope?
For a well-founded Quality Risk Analysis, two main points of view are relevant: the view of the Business and the view of IT.
- The Business input consists of the “Frequency of Use” and the “Business Impact”.
- The IT input consists of the “Process Complexity” and the “Technical impact”.
The formula used in the SQRA is shown in the picture below:
All attributes (“Frequency of Use”, “Business Impact”, “Process Complexity” and “Technical impact”) are rated with a number between 0 and 5, resulting in an SAP quality risk score between 0 and 100.
Business & IT stakeholders will rate and classify each process in scope during a workshop. The outcome of the SQRA is input for the Test Strategy and Test Plan. Risk class Critical (A) and High (B) classified business processes need extensive and broad testing. Medium (C) and Low (D) classified business processes can be covered by a smaller number of tests.
The outcome of the SQRA can also be used throughout the whole development life cycle. Starting with deciding the order for delivering various parts of the solution, going to when to specify tests and when to execute tests, and the risk class is even used to decide which anomaly to solve first.
Test design with both coverage-based and experience-based approaches
The key-users, maintenance people and operations engineers that are involved in acceptance testing often don’t start of as trained test professionals. So, they need to acquire some relevant skills. One major and important skill set is test design.
In the TMAP body of knowledge for quality engineering & testing we distinguish two general approaches of test design: coverage-based testing using structured test design techniques and experience-based testing using the exploratory testing approach.
You may wonder: “Why would knowing multiple test design techniques be useful?”. There are two main reasons: different test design techniques are suited to different areas of interest, and a high-risk area requires another approach than a low-risk area.
We have developed a certification training course for the audience of SAP acceptance tests, that includes four test design techniques and exploratory testing (and various other relevant subjects).
Examples of risk-based selection of test design techniques and approaches
A well-known technique for test design is path testing, which determines test cases based on the flow of a business process. Path testing has several test depth levels. With a high risk class goes a high test depth level. When there is a moderate risk, a lower test depth level is selected. In a low risk situation, the exploratory testing approach can be selected. Exploratory testing means that test cases are not designed up-front, but during the test execution, based on the test ideas from the charter.
Conclusion
When the organization does not have a dedicated test team, businesspeople (e.g. key users) and operations & maintenance personnel will be involved to prepare and perform the acceptance tests. Usually, these people are not professional testers. To ensure that they have enough confidence whether the self-service solution meets the user’s needs, the people involved need relevant skills which they can acquire with focused and proper training.
For this reason, the TMAP certification scheme contains the training course "TMAP: Quality engineering for SAP". This module is specifically designed for professionals with a crucial role in testing, accepting and implementing SAP systems. It is a solid pathway to expertise in quality engineering for SAP projects, and it's backed by the practical expertise of a great number of professionals.
- More information about the TMAP training courses: Training courses.
- More information about TMAP certification: Certification.
A risk-based approach to testing by the people involved in accepting your self-service solutions is the essential way to work towards delivering business value to your users, vendors and customers!
[This article was also published in SQ Mag nr. 15]
Published: 8 July 2024
Authors: Pepijn Paap, Rik Marselis
