Find your template, checklist or other download to help you in you tasks.
Find your way to be trained or even get certified in TMAP.
Start typing keywords to search the site. Press enter to submit.
Deliver value with the right quality at speed. That’s what organizations ask of their IT-teams. To determine the right quality the teams use the quality characteristics. These are a very useful tool to identify various characteristics of quality that are important for the stakeholders of an IT-system.
Quality has many different aspects, and quality characteristics help identify which aspects are most relevant in a specific situation. Also quality characteristics are one of the important aspects to consider when defining test varieties.
Over the years, we have seen various standards for quality characteristics (e.g. ISO9126), as well as specific lists for specific situations, such as specified in the TMap Next book [Koomen, 2006].
A very common set of quality characteristics is the ISO25010 standard (published in 2011) to which an extension for AI and robotics was added in the TMAP book Testing in the digital age [Ven, 2018].
Functional testing usually takes up most of the effort and time spent on testing. Often however, the non-functional testing gets less attention than it should, because many IT-failures relate to non-functional problems. [more info: functional and non-functional testing]
Today, with the rapid development of GenAI, we are convinced that the ISO 25010 — even with its most recent edition of 2023 — is not sufficient to cover all relevant aspects as quality.
Also, with the introduction of the separate ISO25019 standard for quality in use (in 2023), it has become more complex for quality engineers to determine which characteristics are relevant according to ISO.
In our opinion — and we are not alone in this — the latest changes to the ISO standards for quality characteristics have not really improved the situation. Moreover, the lists of quality characteristics have become too long to be practical. Therefore we advise to make your own shortlist of relevant quality characteristics for your context.
When looking into quality characteristics, people discovered long ago that some aspects of quality can be assessed in the product itself (referred to as product quality), while other aspects depend on how the product is used (referred to as quality in use). In the sections below, we list a couple of relevant quality characteristics for product quality and for quality in use. These apply to AI-based systems, GenAI-based systems and other types of intelligent machines, including robots.
We do realize that we deviate from existing standards, this is because developments are progressing at a pace that no standardization body can keep up with. We also recognize that this will not be the ultimate list for all time, but we are convinced that it is helpful today and in the foreseeable future. If you have feedback about improvements and additions, please let us know.
Looking at all attempts to make a complete standard list of quality characteristics, we have come to the conclusion that, especially in the quickly evolving context of AI and GenAI, those involved should create their own shortlist of characteristics relevant to their specific context.
On this page, we introduce our current view on a group of particularly important quality characteristics for AI and GenAI that always should be tailored to the specific context.
Keep in mind that other quality characteristics may very well be relevant in your context, or they may be called by different names, so create your own shortlist of the characteristics that resonate with your team members and stakeholders. Also keep in mind that when physical aspects of intelligent machines are relevant, some specific quality characteristics for robotics need to be selected (such as embodiment, see [Ven 2018]).
The shortlist of important quality characteristics for product quality is:
The shortlist of important quality characteristics for quality in use is:
Both can be viewed as further elaborations of the quality in use characteristic “freedom from risk”. In our opinion, it is important to describe these two characteristics explicitly and separately in the context of GenAI.
Links to detailed information
The following links guide you to the pages where the quality characteristics are explained in more detail.
Related wiki’s
