Dealing with incomplete information

Adequate information must be available for the product risk analysis in order to gain insight into both the damage and the chance of failure. However, test managers regularly face the impossibility of creating an overview of the damage for the client’s processes or the system’s chance of failure (yet).

Despite a full and thorough understanding of the situation, the test manager is still expected to provide a justified test strategy. One solution to this problem is to start by basing one’s choices in terms of test intensity exclusively on the level of damage for the client organisation or the test object’s chance of failure. Once the required information becomes available later on in the process, the product risk analysis can be elaborated.

The PRA decision table below helps determine which steps of the product risk analysis can be executed.

^{PRA decision table}

At the start of the test process, the test manager is usually able to gain insight into the damage because the processes and requirements are known already. Often, the chance of failure cannot be mapped yet because the technical solution still requires elaboration. In this case, the damage inventoried in the master test plan can be used to indicate which processes and product requirements must be tested more or less thoroughly (without reference to the object parts because these are not yet known). At this point, the test object is not yet split up into object parts in the master test plan. The test manager can provide insight into the chance of failure when planning the separate test levels in order to select the test intensity for the object parts.

Another possibility is that, at the start of the test process, the stakeholders only have an understanding of the chance of failure and not the extent of the damage suffered by the client organisation in the event of failure. In this case, the tests are executed under the responsibility of the vendor and there is a big physical or organisational distance between the product vendor and the client. This generally means that there is no agreement between the vendor and client in the form of a master test plan. In such a situation the test intensity of the object parts can be determined on the basis of the chance of failure of the object parts and the damage the vendor will suffer if he delivers a product that does not comply with the agreements made with the client.