Find your way to be trained or even get certified in TMAP.
Start typing keywords to search the site. Press enter to submit.
These modules are mainly about GenAI and Quality Engineering. Basically, this is an ambiguous subject, which we chose on purpose.When people talk about the combination of GenAI and Quality Engineering, immediate confusion may arise. Are we talking about Quality Engineering FOR GenAI (meaning using quality engineering to create & improve GenAI-based solutions) or are we talking about Quality Engineering WITH GenAI (meaning using GenAI to support quality engineering activities)?The answer is: We elaborate on both.
In these sections we briefly describe both aspects. Note that we used “GenAI” in the section titles, but keep in mind that we don’t want to limit ourselves. AI in general is broader than GenAI, and intelligent machines may even be broader than AI. So, keep an open mind; however, for now the primary focus is on GenAI.
GenAI basically is a tool, and this tool can serve many different purposes. One of those purposes is supporting quality engineering and testing activities. This purpose is already very broad. GenAI can support the planning, monitoring, control, design, preparation, execution, reporting, and investigation activities that are part of quality engineering, and many more.
Related:
AI-based systems today are used for all sorts of applications. Earlier we addressed that the quality engineering activities for AI-based systems differ from what we are used to for traditional systems. The TMAP book Testing in the digital age: AI makes the difference [Ven 2018] elaborates on how quality engineering and testing activities can be performed to get the right quality at theright moment for an AI-based system.
Although the term Quality Engineering has been used since the end of the last decade, many people still don’t know exactly what it is. Is it a fancy new name for testing? No, it’s not. It’s much more. The TMAP definition is this:
Quality Engineering is about team members and their stakeholders taking joint responsibility to continuously deliver IT systems with the right quality at the right moment to the businesspeople and their customers. It is a principle of software engineering concerned with applying quality measures to assure built-in quality.
So, quality engineering is about all activities related to building the right quality and ensuring it continuously stays at the right level. It is about development and maintenance, performed by cross-functional teams.This modules are about “amplified” quality engineering. This is the next step, applying modern approaches and tools to amplify (that is: support and strengthen) the quality engineering activities and the applied quality measures.
Quality engineering consists of a great number of possible activities, the so-called quality measures.
A Quality Measure is a group of activities that is aimed at achieving a certain level of quality.
Quality measures are grouped by their goal. The three goals areto build quality in, to provide information about the quality level,and to improve quality:
More examples of quality measures can be found on a separate page in the template of the quality engineering strategy that can be downloaded here.
It is of vital importance that the various quality measures are cohesive. No single quality measure is an independent activity; it is only a small cog in the quality management wheel. Testing, for example, is only one of the forms of quality engineering that can be employed.
In addition, prompting the model to explore diverse reasoning paths can be beneficial for uncovering edge cases and alternative perspectives. By converging these different lines of reasoning, the model is more likely to produce a well-rounded and comprehensive answer.
If we want to know if our IT system satisfies the needs, we must assess its quality. We need to define quality indicators and measure them. Most of this measuring is a testing task. An example of an indicator is Quality risks covered compared to quality risks identified. More examples can be found on the building block “Indicators in the VOICE model.”
Testing consists of verification, validation and exploration activities that provide information about the quality and the related risks, to establish the level of confidence that a test object will be able to deliver the pursued business value.
Testing is one of many quality measures, but was (and sometimes still is) often seen as the only quality measure. Since the start of Information Technology, the quality of information systems has been an often debated subject. When faults exist in IT products, failures may occur, and people will not get the services they need. Testing provides information about the quality and where the quality is not at the right level, it helps to identify faults and failures so that they can be fixed before they cause problems for the users.
In today’s world more and more tasks are delegated to GenAI, which brings risks in conveying the proper information about the vision and goals humans have for the IT system, causing the GenAI to generate wrong solutions. So, testing becomes evenmore important!
By the way, keep in mind: not all faults need to be fixed. With quality engineering we strive for the right quality at the right moment. Therefore, the people involved want to continuously have insight into the quality level and the remaining risks, and when necessary improve quality throughout the lifecycle. If this means some faults remain, that is fine as long as the stakeholders know it and have the necessary workarounds. This way, they can still have confidence that they will achieve their pursued business value.
Information about the quality and the related quality risks (also known as product risks) is the key input for stakeholders to establish their confidence.Since testing can never cover all possible situations and combinations of parameters and situations in relevant contexts, testing is always based on sampling methods. Analyzing the quality risks is crucial to determining where to focus the testing efforts.
A quality risk is a specific chance that the product fails in relation to the expected impact if this occurs. The chance of failure is determined by the chance of faults and the frequency of use. The impact is related to the operational use of the product.
Based on the identified quality risks and their risk level, the team(s) involved will determine their quality engineering strategy that specifies which quality measures are needed, and with what intensity, to deliver an IT product with the right quality at the right moment. For more information about quality risk analysis and creating a quality engineering strategy.
Information about quality and related risks is used to determine the kind of quality measures needed and the intensity of those quality measures that meet the risk level. Read more about quality risk analysis and creating a quality engineering strategy.
When we talk about quality engineering and testing with stakeholders, such as business owners, C-level managers, etcetera, they often ask why they should put time and effort into those activities.Basically, we need to answer three core questions for those stakeholders.
Quality engineering and testing bring peace of mind by providing information so teams can make informed decisions that help to build systems that deliver business value with the right quality at the right moment, and both quality risks and project risks are managed effectively.
Focus the quality engineering & testing efforts where the greatest business value can be gained and where the highest quality risks are present.Deciding where to spend the efforts of the team is the core reason to elaborate a strategy. We refer to the description and the downloadable excel-template of the quality engineering strategy.
Often, speed is a priority, and in these situations, the need to optimize the process is considerable. Focusing on shift left, improving the quality of requirements, applying good coding practices, and using automation (with common tools, bots, and/or agentic AI) accelerates the IT delivery process while maintaining the required quality level. (As explained in the building block test automation.
AmpQE overview
