Defect Management

What is defect management?

What is a defect and why is it important? During all activities in software development, defects can be found. A defect is an observed difference between the expectation or prediction and the actual outcome of a test. Many people see the finding of defects as the purpose of testing. While it should be clear that the purpose of testing is much more ( the provision of information and advice concerning risks and quality) the fact remains that finding defects is one of the most important activities of testing. That is why defects are often recorded and tracked. This way the defect can’t get lost or forgotten.

A defect is also termed a 'fault'. Confusion sometimes arises concerning the various terms, such as errors, faults and failures. In this book, the following distinction is made:

• Error
  • Human mistake; this action takes place prior to any faults and/or failures
• Fault
  • Results from an error. Fault is the view from inside the system. Fault is the state where mistake or error exists. Developers will see the fault
• Failure
  • When the system is performing differently from the required behaviour, from a viewpoint outside the system. Users will see the failure.

In TMap the following definition of defect is used:

So why is defect management important?

To be able to provision information and advice concerning risks and quality insight in the quality is required. Defect management is a means that gives insight in the quality level of the code or documentation that is being tested: the number of defects found during testing and the number of unsolved defects that are remaining in the code (or documentation).

The purpose of defect management may vary. The purpose of defect management for your project or organisation can be:

• Operational support for solving and retesting defects;
• Give input for status and progress reports;
• Give input for the release advice.
• Each defect can also be the trigger for development process improvement - what was the reason that the defect occured and can we prevent the defect from occuring in the future.

What makes a good defect administration? A good defect administration must be able to monitor the lifecycle of a defect and make clear who should make a decision or take action to get a defect in the next state. It also shows the severity of a defect. This categorisation reflects the damage to the business operations, for example ‘Production-obstructive’ or ‘Cosmetic’. Sometimes the procedure to handle defects depends on the severity of the defect.

A good defect administration provides various overviews. These overviews are used, among other things, to make well-founded quality statements.

Of course, administration is not enough. For successfull defect management we need a working defect procedure as well  A defect procedure facilitates the handling and managing of defects. In a waterfall environment, this procedure can best be defined at master test plan level. This also makes it possible to detect overall trends, over and above test varieties.

Defect management can be complex and voluminous. Several parties, in different locations, can be involved in handling defects. Ideally, the defects procedure is supported by a defect tool. Defect management tools support the registration and handling of defects found during test activities. Most tools also enable the creation of management reports and metrics. There are various freeware or commercial tools available. The latter group of tools often has the advantage that the defects administration is integrated with testware management and plan and progress monitoring.

In an Agile environment, defects can be handled differently than in a Waterfall environment. Also the purpose of defect management can be different. Agile, based on Lean, strives for zero defects. Defects need to be prevented. Theoretically, if a defect occurs, it should be found as quickly as possible and fixed immediately at the source. The fact that a defect was found is the trigger to an improvement action with the purpose to prevent such errors from happening again. The purpose of defect management is to provide information to improve the development process. This way of defect management is important in the continuous monitoring of product quality throughout the whole lifecycle of the product.

Here are some practical points were defect management is concerned. 

• Keep the defect procedure simple. That makes it easier to understand and follow. Keep explaining it to all people involved (especially when participating in a agile team).
• Part of the defect procedure could be that defects that are (structurally) solved the same day as they were found, are not recorded in the defect administration. The other defects (the ones that take longer to solve it at the source) are recorded by the team member who has detected the defect. This limits bureaucracy but ensures that defects do not get lost or forgotten. 
• Use a tool for defect management. Try to use an integrated tool for facilitating the development process (product backlog, user stories, tasks etc) and for defect management.

People Involved

The administration and monitoring of the defects is formally a project matter and not one of the testers. However, testers (and the test manager) are usually very closely involved.

Most people in the development project or scrum team have a role in the defect procedure, for instance testers, test coordinators, the test manager, testers from the user organisation, product owner, functional application management, system development, technical application management, people that use defects as trigger for continuous improvement..

From within the test process, the testers are the submitters of defects, and they check the solutions of these. The test manager communicates with the other parties concerning (the handling of) the defects. Sometimes, defect management is assigned to a dedicated role in the project: the defect administrator.

The purpose of this is to channel the defects and the associated solutions effectively. In this regard, the intermediary maintains external contacts at the level of staff doing the actual work. This person has an overview of all the defects and acts as a relay and inspection post for the defects on the one hand, and the solutions on the other. Advantages of this are that the quality of the defects and solutions is monitored better and that communication is streamlined.

Artifacts

Defects descriptions are stored in de the defect administration. Management reports and metrics concerning all characteristics of defects (and trends) can be generated from it.

Success Factors

The value and usefulness of a defect procedure (implemented in a defect tool) depends heavily on the extend that is used. Simple defect procedures are easier to use.

There are great advantages to be gained in organising one single defects administration and defects procedure for the entire project or the entire line department. All the parties involved – developers, users, testers, QA people, etc. can deposit both their defects and solutions here. Communication on the handling of the defects is thus considerably simplifi ed. A central administration also offers extra possibilities of obtaining information. The authorisations are a point to note here; it should not be possible for unauthorised persons to be able to amend or close defects by this means.