In well-designed systems, checks are built in that protect the system against invalid situations, such as:
- Offering invalid input.
- Executing an action without the associated preconditions being met.
In such situations, the system has to carry out correct error handling. That means that the system may not produce unpredictable or undesired results, but should discontinue its processing in a controlled manner.
The aim in the testing of right paths/fault paths is to test both the valid and invalid situations in every defined error situation. An invalid situation should lead to correct error handling, while a valid situation should be accepted by the system without error handling.
The aim is not to test all the possible variations within the valid situations. For example, in the testing of the function "Create order", it is sufficient to establish one time that the system creates a permitted order correctly. If you also wish to test how the tens of variations in permitted orders are handled by the system, other coverage types for this can be applied, such as equivalence classes, modified condition/decision coverage, pairwise testing, etc.
Inventorying test situations: right paths/fault paths
The test situations are the collection of all the right paths and fault paths with every defined error handling. To inventory which error processings are to be tested, in principle 2 sources can be drawn on:
- From the data
"Are there certain values or combinations of values defined that are not permitted for the relevant functionality?" In fact, this refers to a special application of the principle of equivalence classes: Each considered entity is divided into one valid and one or more invalid equivalence classes.
- From the process steps
"Have specific control steps been nominated in the process or algorithm that precede the processing?" In fact, this refers to decision points with the standard outcomes: Continue with the processing steps (right path); produce error message and discontinue processing (fault path).
Points of focus in the coverage of the fault paths
With an error processing, several conditions may play a role in determining whether the relevant situation is permitted or not. That means that several fault paths also exist, namely one for every condition that is not met. However, there is only one right path, namely if all the conditions are met.
With the testing of the fault paths, it should be ensured that no more than one invalid situation at a time is tested. If several invalid test situations are combined on one test case, there is the risk of "masking" faults: in the processing of the first invalid situation, the system correctly discontinues processing and consequently the second invalid situation is not tested. This is illustrated by the following example:
In more detail
For the issue of a loan, the following 2 conditions must be met:
Suppose that both error processes were to be tested with one test case, in which the age is lower than 18 and the loan sum greater than the annual salary. The system rightly produces an error message after it has established that the age does not meet the condition. The check on the loan sum is now skipped. If in the system the error processing of the loan sum is wrong or even not implemented at all, this will not be found with the one test case.
An additional disadvantage of test cases with more than one invalid situation is that if the error message does not explicitly state what is wrong, it cannot be determined which invalid situation has caused the error message.
More information about: Coverage types