Test design techniques

Test design can be as simple as hitting some keys unprepared to see if a system works at all (called ad hoc testing). And it can be as sophisticated as identifying test situations, assuring coverage by designing test cases, automating these in a test script and executing the script to automatically create a test report. Both unstructured testing and structured testing can be used as a quality measure. To determine which approach is preferred very much depends on the situation (especially the quality risks involved). For the preparation and performance of tests, we distinguish two overall approaches:

• Coverage-based testing
• Experience-based testing
  

In both approaches we can identify higher and lower degrees of structured work, although basically the coverage-based approach will be more structured than the experience-based approaches. However, more structure doesn’t necessarily mean it better contributes to establishing confidence in the pursued value. It is always wise to combine both approaches; sometimes more experience-based, sometimes more coverage-based, but never exclusively one or the other.

For a more in-depth explanation: Read more

A test design technique is a standardised method to derive, from a specific test basis, test cases that realise a specific coverage. The implementation of test design techniques and their definition in the test specifications have several advantages:

• It provides a well-founded elaboration of the test strategy: the agreed coverage in the agreed place.
• It is a more effective way to detect defects than e.g. ad-hoc test cases.
• The tests are reproducible because the order and content of the test execution are described in detail.
• The standardised method ensures that the test process is independent of the individual who specifies and executes the test cases.
• The standardised method ensures that the test specifications are transferable and maintainable.
• It becomes easier to plan and manage the test process because the processes of test specification and execution can be split up into clearly definable blocks.

Introduction

Why test design techniques?

The aim of testing is to provide advice on quality and risks. To do this, the tester needs to collect information about the system behaviour. The most important means to achieve this is to execute test cases. The big questions in this context are: ‘Which test cases? How many? And how do we get these test cases?’ Test design techniques play an important part in answering these questions.

Definition A test design technique is a standardised method to derive, from a specific test basis, test cases that realise a specific coverage.

Designing test cases is the vital link between the test strategy and the concrete test cases that are used to implement that test strategy. This is realised in the context of test assignment to test cases, the theme of which can be outlined as follows:

• It is not possible to test everything within the confines of the preconditions of time and costs defined in the job specification. Choices will have to be made as to the lengths one wishes to go to in testing.
• The more important something is, the more intensive the tests required. Something is very important if its failure would result in severe damage to the business or client. This is mapped by means of a risk analysis.
• A test strategy is used to create an overview of what will be tested and how intensively, such that the risks defined earlier are covered as adequately as possible.
• The decisions concerning intensive and less intensive testing are translated to concrete statements about the targeted coverage.
• Depending on the available test basis, among other things, appropriate test design techniques are selected to achieve said coverage.
• The implementation of these techniques eventually results in the set of test cases that is needed to execute the test assignment satisfactorily.
  

Importance of test design techniques

You will find several arguments below that indicate the importance and benefits of using test design techniques and their definition in the test specifications.

• It provides a justified elaboration of the test strategy: the agreed coverage in the agreed place
• Because a test design technique focuses on achieving a specific coverage to detect specific types of defect (e.g. in the interfaces, the input checks or the processing), such defects are detected more effectively then by specifying ad hoc test cases
• The tests are reproducible because the order and content of the test execution are described in detail
• The standardised method ensures that the test process is independent of the individual who specifies and executes the test cases
• The standardised method ensures that the test specifications are transferable and maintainable
• It becomes easier to plan and manage the test process because the processes of test specification and execution can be split up into clearly definable blocks.
  

Essential test design concepts

Test situation, test case and test script

In the ideal case, testing would give us the certainty that the system behaved as required or desired under all circumstances. In reality, not every circumstance can be tested – only a subset that is a direct result of the decisions and choices made during the test design phase. The test design leads to a hierarchical structure of test situations – test cases – test scripts. These terms are discussed in detail further on in this section. The relationship between the concepts can be summarised as follows:

• Every test situation occurs in at least 1 test case.
• A logical test case covers 1 or more test situations.
• Every logical test case is worked out concretely into 1 physical test case .
• Every physical test case occurs in 1 test script.
  
  

The distinction between the logical and physical parts of the test design is:

• The logical test design consists of the test situations and the logical test cases. This is the part that demonstrates that the required coverage is achieved, thereby complying with the test strategy.
• The physical test design consists of the concretely created physical test cases, laid down in test scripts. This guarantees a thorough preparation of the ‘Execution’ phase. The physical creation of test cases therefore adds nothing to the thoroughness of the test.
  
  

For a more in-depth explanation: Read more

Overview of the test design techniques

The table below summarises the most important features of the test design techniques discussed here. This can be of assistance in comparing the techniques and in selecting the most suitable ones.

Test design technique	Coverage types / basic techniques	Test basis	Quality characteristic / Test type
Decision Table Test	Multiple condition coverage	Individual conditions of decision tables, without structure	Detail functionality
Data Combination Test	Equivalence classes and (Multiple condition coverage or pairwise testing)	All types of test basis	Overall functionality Detail functionality
Elementary Comparison Test	Modified condition / decision coverage	Structured functional specifications, such as pseudo-code	Detail functionality
Error Guessing	—	All types of test basis	Diverse
Exploratory Testing	Diverse, according to choice	All types of test basis	Diverse
Data Cycle Test	CRUD and Decision coverage	CRUD matrix Data-integrity rules	Overall functionality Connectivity Suitability
Process Cycle Test	Path coverage test depth level 2	Structured description of business or operating processes	Suitability
Real-Life Test	Statistically responsible simulation	Operational profilesLoad profiles	Effectivity Connectivity Performance
Semantic Test	Modified condition / decision coverage	Input and output specifications Business rules	Functionality / Validation test
Syntactic Test	Checklist	Input and output specifications Descriptions of attributes	Functionality / Validation test User-friendliness
Use cases Test	Checklist	Use cases	Suitability Effectivity User-friendliness

Another view:
Selection of Coverage types and Test design techniques

There exist many coverage types and test design techniques. For the sake of simplicity and practicality we will only highlight the most commonly used test design techniques and hence the application of the underlying coverage types.

To give you a practical overview we highlight the most commonly used coverage types and some test design techniques in which they can be applied.

Coverage type group	Test intensity: Light	Test intensity: Average	Test intensity: Thourough
Condition	Condition Decision Coverage – Elementary Comparison Test	Modified Condition Decision Coverage – Elementary Comparisson Test or Condition decision coverage – Decision Table Test	Multiple Condition Coverage Elementary Comparisson Test orMultiple Condition decision coverage – Decision Table Test
Data	One or some data pairs – Data Combination Test	Pairwise – Data Combination Test	N-wise or all combinations – Data Combination Test
Process	Statement coverage and Paths test depth level 1 – Process Cycle Test	Decision coverage andPaths test depth level 2 – Process Cycle Test	Paths test depth level 2 – Algorithms Test andPaths test depth level 3 – Process Cycle Test

We left out the coverage group “Appearance”. In the cases where that coverage group is applicable, the coverage type and thourougness depends too much on the specific interface and the result the tester wants to achieve.