Find your way to be trained or even get certified in TMAP.
Start typing keywords to search the site. Press enter to submit.
The way we think about quality of IT systems and software code changes with the use of GenAI. On the one hand, more quality characteristics are relevant (such as personality for a chatbot); on the other hand, existing lists of quality characteristics often bring too many possibilities to support the elaboration of the required quality level in a useful and practical manner. That is, before GenAI came people already struggled to select the relevant quality characteristics, but with GenAI extra quality characteristics (e.g. related to morality, ethics and humaneness) pop up while traditional quality characteristics also remain valid.Resulting in too many quality characteristics to practically use. So careful context-aware selection is needed instead of blindly following standard lists.More information about quality characteristics and their use can be found in here.
In today’s world many people and organizations rely on IT systems; many things would not even be possible without IT systems. So, we need to be able to trust that these IT systems are working well enough to support business processes. This means that the quality must be at the level that fits the purpose and delivers business value.
Quality is the totality of features and characteristics of a product or service that bear on its ability to satisfy stated or implied needs.
To get a clear view on the level of quality of an IT product, various quality characteristics need to be taken into account. Over the years, TMAP has used various different views on quality characteristics. Quality characteristics are a very useful tool to identify various characteristics of quality that are important for the stakeholders of an IT-system in a specific context. Which quality characteristics are relevant to determine the right level of quality, very much depends on the context of the IT-system at hand. GenAI brings new quality risks but also new possibilities for achieving the right quality. Therefore, we defined a specific subset of quality characteristics that are particularly relevant in this world of GenAI-impacted systems. For this shortlist, we used multiple sources and combined these to a useful set of ‘top-of-mind’ quality characteristics relevant for both quality engineering with GenAI and quality engineering for GenAI.This, however, by no means implies that other quality characteristics are irrelevant. On the contrary, any team should always investigate which quality characteristics are relevant in their context. But keep your selection clear and to-the-point to keep it manageable.
With so many quality characteristics available, which ones should you use? The TMAP perspective is that you shouldn’t just rely on a single standard list. Instead, you should look at the various sources of quality characteristics and select the relevant subset for your context.
People are happy to use tools to make their work easier, faster, better. With the rise of AI, and especially GenAI, information technology has become so accessible that the decision to automate is made almost by default.The main question should not be whether a task can be automated, but whether automation contributes to the overall goal of the process. Does it create value for the individual, the organization, and the world as a whole?Once the decision to automate has been made, an equally important question follows: how should automation be implemented?AI and GenAI seem to be capable of performing any task. But will it be the best way to perform such task, looking from various perspectives? Automation decisions should be taken considering multiple quality dimensions, not just functionality (will the task be fulfilled?). Quality characteristics such as security, performance, maintainability, and sustainability all matter. Regarding the last characteristic, keep in mind that retrieving information via a simple web search consumes significantly less energy than obtaining the same knowledge through an LLM chat. At least as important is humaneness as a core consideration, for example in decision support systems such as insurance claims processing. An IT system should treat people as individuals with diverse situations, levels of knowledge, and skills, rather than reducing them to anonymous data points.
A new situation, such as the adoption of GenAI, may reveal new challenges that need to be addressed. However, many of the issues that emerge are not unique to AI-infused applications. For example, working with unclear requirements is a long-standing problem. Addressing this issue is primarily about improving the requirements elaboration process, rather than introducing new technology. We advise organizations not to invest disproportionate effort in solving problems that already existed before GenAI was introduced. At the same time, GenAI can be used as part of the solution and may unexpectedly support improvements in the IT delivery process.
AmpQE overview
