To use ratios as a basis to create an estimate, it is important to collect the greatest possible amount of experience figures. This makes it possible to derive 'standard' ratios for similar projects. Similar projects are projects that are the same in terms of certain key properties. For instance the same development method, the same development platform, the same software environment, the same experience level of the developers, etc.
Naturally, the own ratios of an organisation generally are the best ones to use within that organisation. Ratios can be used at all estimation levels. At the level of test activities in test phases however, the ratios are so specific that they can only be used within one organisation and often even within the area of application (project or system).
Below please find a number of ratios between tests and other development activities from actual practice. An organisation can use these observations as a starting point. By then keeping track of its own experience figures, the organisation can match the ratios more and more adequately to its own practice.
The various observations are based on the following standardisation of terms:
- Functional design (FD) = functional detailed design.
- Realisation, consisting of the technical design (TD), programming (P), unit and unit integration test (UT and UIT).
- Functional test. This concerns the testing of the functionality quality characteristic, with the FD as the test basis. The ST and AT test levels are used for this purpose.
Observed ratios in an average risk profile are as follows:
- FD : Realisation : Functional test = 2 : 5 : 3 In an environment with a formally complete FD, waterfall development method, 3GL programming language, and a structured test approach. These figures were found to apply for the activities in the maintenance phase as well, with testing only involving a test of the change.
- (FD+TD) : (P + UT + UIT) : Functional test = 1 : 3 : 3 In an environment with an incompletely detailed FD, experienced builders who fill the FDs themselves, and a starting test approach.
- FD : Realisation : Functional test = 1 : 2 : 1.2 In a test environment with a formally complete FD, waterfall development method, experienced builders, and a functional test that does not have maximum test coverage but is driven by risk, and a maximised budget. The test approach is structured. Within a test level, ratios can be used to estimate the various phases. Here, too, observations from actual practice are available:
- For a system test with good but complex specifications, the observed ratio is as follows: Preparation 6%, Specification 54%, Execution 21%, Completion 2%, and 17% for Control and Setting up and maintaining infrastructure taken together.
- The following ratio was observed for a system with an inadequate test basis: Preparation 21%, Specification 33%, Execution 24%, Completion 5%, and 17% for Control and Setting up and maintaining infrastructure taken together.
Note: in both cases, 160 hours were spent on creating the MTP.