Amplified Testing with GenAI

The goal of using machine intelligence, such as GenAI, in quality engineering and testing is not to take people out of the loop. Instead, the goal is to make quality engineering and testing
easier and faster, just as traditional tools support any kind of activity. Some tasks that couldn’t be done before are now possible thanks to GenAI, such as creating draft requirements based
on the transcript of an online meeting. So, it is about enablement, effectiveness and efficiency.

Using GenAI-based tooling is the means to a goal. To actually set the goal and work towards that goal, amplified testing (abbreviated to Amp testing) is used. GenAI, imitating intelligent human behavior and applying predictive analytics, can help us to get there. To meet the challenges
presented by accelerating delivery speed with increasing technical complexity, we need to follow a very simple imperative: Test smarter!

Test Design Supported by GenAI

The first test design techniques were documented in the 1970s, half a century ago. However, these techniques are often not applied because testers find them difficult to use, so it’s very helpful when a GenAI tool can facilitate their application. We could supply design documents and requirement specifications, preferably in a structured format. The GenAI tool will then identify which paths, and/or conditions and/or input and output values are in this test basis and creates test cases to validate the test object against it.
For some test design techniques (for example, boundary value analysis with 3 values) a tester can write a straightforward prompt for an LLM and get satisfactory test cases. For more complex test design techniques, however, a simple prompt is not enough. Because of the complexity of the prompts needed — including RAG to instruct the GenAI tool on the workflow — a better approach is often to have a “low-prompt” GenAI-based tool that uses a user interface to gather the required input from the tester and, after executing one or more pre-crafted prompts, provides the results back to the user through the same interface.
This relieves the burden of creating (and testing!) large prompts to apply more complex test design techniques. An example of such GenAI based tool is Sogeti’s “GenAI Amplifier”.
It is also important to recognize that GenAI is non-deterministic. Certain test design techniques rely on well-defined algorithms, which GenAI may struggle to follow reliably. In such cases, use a traditionally programmed algorithm and instruct GenAI to call that program and further process the results for the user. An example of a tool to be included in a prompt for GenAI is the well-known long-existing pairwise testing algorithm “PICT” (Pairwise Independent Combinatorial Testing) of Microsoft that has been relied upon by testers for about 3 decades.