State Transition Testing

Many systems show state-based behavior. State-based models are used to define this behavior. These models can also be used to design tests. With state transition testing several distinct coverage levels can be achieved.

Description

State Transition testing is a process-oriented test design technique that focuses on states, events that initiate a transition to another state and actions resulting from such events. Tests are designed to execute valid and invalid state transitions. State transition testing is used to test whether the system correctly responds to events for example by transitioning from one state to another. Multiple coverage levels can be achieved, indicated as n-switch coverage.

State transition testing is often used to test embedded software that controls machines, but also to test menu-structures in GUI-based systems or other types of systems that have distinct states and a process for getting from one state to another.

The correct behavior of the system is described in a state transition diagram that gives an overview of all states, the transitions between these states, the events that trigger transitions and the actions that result from events.

	A state is a distinguishable situation of a system. A system can only be in one state at any point in time. A system can transition from one state to another.
	A transition is a change from one state to another. A transition may also be to the same state (a “transition-to-self”).
	An event is an occurrence inside or outside a system that can result in a transition.
	An action is behavior executed at a particular point in the system. It can also be a string of actions.

Pictured here is an example of a simple state transition diagram. Our example system is a lamp.

The lamp can be off or on. If the on-switch is pushed, the electric current is enabled and the lamp is turned on. If the off-switch is pushed, the electric current is cut and the lamp is turned off.
The transitions between states can also be shown in a state table as shown below. 

The left side of the table contains the states, and the top contains the events. When a transition is valid the resulting state is shown in the corresponding cell of the table. Mentioning the action with the resulting state is optional. When the transition is invalid, it is shown as a hyphen (‘-‘). Above the resulting states, the actions may be added. (For a more extensive example of a state table, please download the template and see the example sheet)

Within state transition testing you can choose to test individual transitions or combinations of transitions.

Since a state transition diagram doesn’t show invalid transitions, only test cases with valid transitions can be derived from a state transition diagram.

A state table indicates valid transitions and invalid transitions, so a state table can be used for testing both.

The level of test coverage is related to the number of consecutive transitions that are covered. If every single transition is tested, we achieve “0-switch coverage”. 0-switch coverage means that we do not focus on testing consecutive transitions. If sequences of two transitions are tested, so all combinations of two consecutive transitions are tested, we achieve “1-switch coverage”. If a higher number of consecutive transitions are tested, we speak of “n-switch coverage” where ‘n’ is the number of consecutive transitions minus 1. For example, “2-switch coverage” tests combinations of 3 consecutive transitions.

Usually only 0-switch coverage and 1-switch coverage are applied, sometimes 2-switch.
 

1 - Identifying test situations

Test situations for state transition testing can be:

1. The individual states – this enables testing if all states can be reached
2. The individual transitions – this can be used to ensure 0-switch coverage
3. The combinations of “n” transitions – this can be used to ensure “n-1”-coverage

For our lamp example the test situations for different coverage levels are:

1. Coverage of all states: “lamp off” and “Lamp on”
2. 0-switch coverage: “push on-switch” and “push off-switch”
3. 1-switch coverage: “lamp off – push on-switch – lamp on”
   and “lamp on – push off-switch – lamp off”
4. 2-switch coverage:
   “lamp off – push on-switch – lamp on – pushoff-switch – lamp off”
   and
   “lamp on – push off-switch – lamp off – push on-switch – lamp on”

etcetera for n-switch (higher coverage levels are usually only sensible with complex diagrams)

2 - Create logical test cases

There are two possibilities for creating logical test cases:

1. Create a test case for every individual test situation.
2. Combine multiple test situations in a test case.

When would you use one test case per test situation, and when would you combine multiple test situations in a test case?
In general for progression testing you would use many short test cases so that in case of a failure it will be relatively easy to investigate the problem. Also in unit testing you want to test small parts of the code, so you would also use many short test cases.

For regression testing you would use a few long test cases since you mainly want to establish whether the system that previously passed the test was not negatively impacted by a change. As an example in end-to-end testing you want to test an entire business process and don’t bother about all possible exceptions, so you would also create few long test cases.

3 - How to calculate the number of test situations

For calculating the number of test situations the first question is whether you want to calculate valid transitions only, or both valid and invalid transitions. (This only applies to 0-switch coverage because for 1-switch coverage the test situations combine two transitions, but combining with an invalid transition doesn't result in an executable test case.)

For calculating the number of test situations the first question is whether you want to calculate valid transitions only, or both valid and invalid transitions. (This only applies to 0-switch coverage because for 1-switch coverage the test situations combine two transitions, but combining with an invalid transition doesn't result in an executable test case.)

To calculate the number of valid test situations needed for 0-switch coverage, count the number of transitions in the graph. To achieve 0-switch coverage, every test situation must be part of at least one testcase.

Note: To calculate the valid test situations (both 0-switch and 1-switch) the calculation sheet in the excel template can be used. To calculate the total of valid and invalid test situations (only applicable to 0-switch coverage) the state table sheet in the excel template can be used.

The number of test situations for 1-switch testing is calculated as follows:

• For every state, multiply the number of in-coming transitions by the number of out-going transitions
• Add the results for all states
• This gives the number of test situations needed for 1-switch coverage.

To achieve 1-switch coverage, every test situation must be part of at least one test case.

3.1 Calculate test situations for a more complex example

To illustrate the calculation of the number of test situations for 1-switch coverage, we also give a more complex system as an example. (please remember that for 0-switch coverage the number of transitions is counted to get the number of valid test situations, and the state table is used to find all valid and invalid test situations.)

For every state that has outgoing transitions, make all combinations of two consecutive transitions to create all test situations as shown below:

Test situation 1: State 1 – Event 1 – State 2 – Event 2 – State 1
Test situation 2: State 1 – Event 1 – State 2 – Event 3 – State 3
Test situation 3: State 1 – Event 1 – State 2 – Event 5 – State 4
Test situation 4: State 2 – Event 2 – State 1 – Event 1 – State 2
Test situation 5: State 2 – Event 3 – State 3 – Event 4 – State 1
Test situation 6: State 3 – Event 4 – State 1 – Event 1 – State 2

For this complex state transition diagram example, we also show how to create test cases based on the test situations.

As described above there are two approaches to creating test cases:

1. Create a test case for every individual test situation (“many short test cases”)
2. Combine as many test situations as possible in one test case (“a few long test cases”)

For this current example, both approaches can be used. First we use short test cases (one test case for each test situation):

Test case 1: State 1 – Event 1 – State 2 – Event 2 – State 1
Test case 2: State 1 – Event 1 – State 2 – Event 3 – State 3
Test case 3: State 1 – Event 1 – State 2 – Event 5 – State 4
Test case 4: State 2 – Event 2 – State 1 – Event 1 – State 2
Test case 5: State 2 – Event 3 – State 3 – Event 4 – State 1
Test case 6: State 3 – Event 4 – State 1 – Event 1 – State 2

Next we combine as many as possible test situations, this example all test situations can be combined into just one long test case:

Test case 1:
State 1 – Event 1 – State 2 – Event 2 – State 1 – Event 1 – State 2 – Event 3 – State 3 – Event 4 – State 1 – Event 1 – State 2 – Event 2 – State 1 – Event 1 – State 2 – Event 3 – State 3 – Event 4 – State 1 – Event 1 – State 2 – Event 5 – State 4

The easiest approach to creating combined test cases is to compile all possible individual test situations together. This ensures comprehensive coverage but can lead to very long test cases, which may contain duplicate test situations in the test case, which can be optimized.

To optimize the testing process, it's beneficial to reduce repetition by identifying and eliminating redundant transitions. This involves analyzing the state transitions to find common paths and combining them where possible. By doing so, you can achieve the same test coverage with fewer or shorter test cases, making the testing process more efficient without compromising on coverage. This optimization helps in better managing resources and eases the test execution process.

When you look at the 'Events' and 'States' in the example it is possible to achieve an even more simplified testcase. The series "State 1 – Event 1 – State 2 – Event 2 – State 1 – Event 1 – State 2 – Event 3 – State 3 – Event 4 – State 1 – Event 1 – State 2 – Event 2 – State 1 – Event 1 – State 2 – Event 3 – State 3 – Event 4" contains two times the same steps so one part of this series can be deleted and you will get the next testcase:

Test case 1: (optimized)  
State 1 – Event 1 – State 2 – Event 2 – State 1 – Event 1 – State 2 – Event 3 – State 3 – Event 4 – State 1 – Event 1 – State 2 – Event 5 – State 4

This optimized test case still covers all 6 test situations mentioned above, but the test case now has 15 steps instead of 25 steps.

4 - Create physical test cases

To create physical test cases, for each logical test case describe the actions needed to position the system in the starting state, then trigger each of the consecutive transitions and verify if the resulting state and the actions performed are according to the expected outcome.

5 - Create test scenarios

Usually every physical test case is a test scenario in itself so there will be little need to create specific test scenarios.