Tosca and AI... What can we do with it?

This blog is the first in the series “Tosca & AI”. In the coming series I will introduce practical examples of using AI in testing (including automation), especially when using Tosca. Each blog will answer a specific question regarding Tosca and AI, and sometimes compared with other test tools.

Everybody knows that test automation is still becoming increasingly important for companies looking to increase their productivity and streamline their (test or development) processes. By automating tedious and time consuming tasks, businesses can save money, reduce human errors, and increase their overall efficiency. Furthermore AI has the potential to transform the way (software) testing is done, and we can expect to see more and more organizations adopting AI-based testing tools and techniques. AI can help in automating testing tasks, identifying defects and vulnerabilities, and even predicting future issues.

Since the worldwide introduction of ChatGPT last November (2022), the world of AI regarding testing and development made a “revolutionary” change and transformation. Why “revolutionary”? Testing with help of AI and development was already a rapidly evolving field with many advancements being made regularly. AI models such as GPT-1, 2 or 3 were already in existence and being used for a variety of applications such as natural language processing, image recognition, and decision-making systems. And with the introduction of ChatGPT to the public, the possibilities are now endless. One of the big challenges is that testing and development of these models require a significant amount of computational power and expertise in machine learning and data science. But experience shows that the global (online) community is very flexible and will come up with solutions for these challenges.

Let’s further dive into AI and test automation! Especially the comparison between Tosca and Testim. We know that Testim and Tosca are both software testing tools but they differ in several ways. Testim is a test automation platform in the cloud, that is designed to use AI in testing processes, and focuses on web applications. Furthermore it uses natural language processing and machine learning to understand test cases and generate automated test scripts. The platform is, just like Tosca, designed to be easy to use and requires no coding knowledge. It also easily integrates with various CI/CD tools. Not unimportant to mention is that Testim is partly free to use (“Community” version). On the other hand, Tosca is an enterprise-level test automation tool that supports testing of both web and desktop applications. Tosca uses the so-called “model-based approach” to test automation, which means it creates models of the application and its components to generate test cases. Because Tosca supports various types of testing, and also provides features such as test data management and reporting, it’s much more an “All-in-One” solution. In general both tools support a specific level of AI in testing, and the “Self-healing” feature is a very good example.

One of the common problems “self-healing” can address, is that testers are dealing with dynamic id’s and locators when testing web applications. Dynamic id’s are identifiers that change each time a page is loaded or refreshed, while locators are the code used to identify an element on a web page, such as an input field or a button. Dynamic id’s and locators can pose a challenge for testers because they can make it difficult to reliably locate and interact with elements on a web page. When id’s and locators change dynamically, automated tests that rely on these identifiers can fail or produce unexpected results, leading to false positives or negatives.

To work around this problem, testers may use various techniques to identify elements, such as using relative locators, XPath expressions or CSS selectors. They may also use tools that can identify elements based on their visual properties, such as their size, position, or color. In general, dealing with dynamic id’s and locators requires testers to be resourceful and adaptable, and to use a combination of tools and techniques to ensure that their tests are reliable and accurate. The Self-healing feature then provides support for these types of issues. So one of the great benefits of “Self-healing” is in terms of refactoring and maintenance. Issues and problems are detected by the AI and automatically fixed without (direct) human intervention. In the animated figure 1 below you see the “Self-healing” feature in action with Testim.

[Video opens in new screen]

 

 

Figure 1. Self-healing in action: “Smart Locator” feature (testim.io)
[Video opens in new screen]

Tosca has an additional AI feature, the so-called “VisionAI” engine, for smart or “intelligent” scanning modules when designing test cases. From Tosca version 14.3 it’s possible to create test cases, even without the code of an (web)application is written or build. Please note that Tosca is now on version 16 patch 1 (at the time of writing this article). That means that Tosca versions 14 and 15 will still have some “teething problems” regarding VisionAI. Most Tosca experts among us know that Tosca with VisionAI is not yet a resounding success. But despite that, the VisionAI engine is a powerful feature when developing tests. Many testers and developers know the so-called "Shift-Left" principle. With that principle in mind, VisionAI can theoretically even go further towards "Extreme Shift-Left". 

How do I start with VisionAI? First of all the VisionAI engine is a part of the standard Tosca Test Suite installation process (an option or feature to install whether or not). Let’s assume you’re not a Tosca admin, and you want to use VisionAI. Then you need to get access via your organization. The “organization” should have a VisionAI enabled account in the Tricentis User Administration (TUA). The TUA is a server-based user administration service (Tosca server feature) and enables an organization to manage users and groups centrally. Furthermore the Tosca license administrator will register and sign up all the users of the organization when they want to use VisionAI, see for more information, see link at the end of the article. After you have installed VisionAI and are registered successfully, then you can use Tosca as you normally did.

The VisionAI operates on different levels, from designing test cases to refactoring. In general you can divide the modus operandi of Vision AI into 3 levels:

• Test Creation
• Test Execution
• Self-healing

Regarding test creation, VisionAI uses computer vision algorithms to analyze the GUI and identify the different graphical elements, such as buttons, fields, and menus. Based on this analysis, it generates a model of the GUI and creates test cases that simulate user interactions. Furthermore during test execution, the VisionAI engine uses its model of the GUI to simulate user interactions such as clicks, inputs, and selections. It then validates that the application responds correctly to these actions and reports any errors or defects. Finally the VisionAI engine has also a self-healing capability that can automatically fix any test failures caused by changes in the GUI, such as layout or color changes. It uses image comparison algorithms to detect these changes and updates its model and test cases accordingly. So how do we use these capabilities in our daily tasks as a Tosca test engineer?

Let's start with a practical example! Suppose a multinational company wants to create a new insurance website to better manage their customer interactions. In order to ensure the success of the project, a team of experts is brought together to create a blueprint for the website. This team includes a Tosca test engineer, a functional designer, an architect and a business analyst. One of the first blueprints the team will pick up (for the coming sprints) is a login page with a new dynamic design. How does this blueprint or mock look like? In figure 2 you will see an example of this mock.

After a final decision has been made to use the mock above as a blueprint, the Tosca test engineer can already start designing a test, without the need for a developer or infra team, respectively developing the application or setting-up a test environment. Speaking of "Extreme Shift-Left"….Then the power of the VisionAI engine comes into play. In Tosca Commander the mock (picture of the blueprint) can act as a temporary system under test. So in this case the Tosca test engineer will use XScan to scan the mock (in this case “login.jpg”), and select VisionAI as the engine for designing test cases (figure 3).

When you start scanning, the following message can appear, see figure 4 below.

In that case you click on “Continue”. Furthermore, let’s keep it simple! For this example test case you can choose to scan the “email” field, the “password” field and the green “sign in” button. The special thing about the VisionAI engine is that it actually "sees" the image (picture/blueprint) as a login screen with various front-end components, so it identifies the 3 attributes, 2 text fields and 1 button (figure 5).

After finishing your scanning, the “login” VisionAI module will be saved like the “regular” Tosca modules based on the “regular” engines you’re already familiar with, see figure 6 below.

Note that in the "Properties" section the parameter "Engine" has the value "Vision AI". The next step is a “drag and drop” action from the "login" module to the TestCases section/folder. In the example below, a test scenario is created based on a successful login ("happy flow"), see figure 7.

And finally, without a test environment or test web application set-up, you can run the Login TestCase in Tosca’s Scratchbook. That means that the VisionAI engine will run the test against a mock, based on a picture/blueprint (figure 8).

Once the “login” feature of the insurance application is built by the developer, the same “VisionAI” Login TestCase is already available and stable enough for testing against the real(-time) login page. This all sounds great right? Would you like to know more about Self-healing, VisionAI or AI in general? More information can be found below or keep an eye on my blogs!

Sources used in this blog:

• Self-healing (Tosca)
• Self-healing (Testim)
• Vision AI
• ChatGPT