TMAP Glossary online

See Starting point.

See End-to-end test.

The ease or difficulty to make adaptations to the system. This is a subset of the quality subcharacteristic modifiability.

A test approach in which a tool turns off parts of an IT system to assess the robustness of the system.

Charisma is the compelling attractiveness or charm that can inspire devotion in others. This is a sub-quality characteristic of the quality characteristic personality for Artificial Intelligence and Robotics, that we added to extend the ISO25010 standard.

A structured or unstructured list of all situations that are (to be) tested. (this definition relates to testing, checklists can of course be used for many more purposes)

See Continuous integration (CI), Continuous Delivery (CD) and Continuous Deployment (CD).

The use of various services, such as software development platforms, servers, storage and software, over the internet.

A collaborative robot.

Use of cobots (see Cobot) for performing tasks by people together with robots.

A method of assessing (and possibly) improving the quality of written code by evaluating the work against the specifications and/or guidelines and subjecting it to peer review.

Knowing and perceiving.

Cognitive information technology is not just rule-based but is able to react and adapt its behavior based on perception and knowledge.

The use of cognitive IT to assist quality assurance and testing.

Collaboration is about how well a robot works alongside with humans. This is a new sub quality characteristic.

The degree to which a product, system or component can exchange information with other products, systems or components, and/or perform its required functions, while sharing the same hardware or software environment. This is an ISO25010 product quality characteristic.

The certainty that all inputs and changes are processed by the system.

The criteria a team must satisfy to have completed a (group of) activity(ies).

This is one of the coverage groups of coverage-based test design techniques that relate to testing conditions.

CC is a coverage type, from the coverage group Condition, that ensures the possible outcomes of (“true” or “false”) for each condition are tested at least once.

CDC is a coverage type, from the coverage group Condition, that ensures the possible outcomes of each condition and of the decision are tested at least once. This implies both “condition coverage” and “decision coverage”.

This is the fifth element of TMap HD, where the other elements (People, Integrate, Simplify and Industrialize) lead to, in order to gain the trust that the IT system is fit for purpose, often based on the faith that the people involved do the right things.

See re-test

The ease with which an interface can be created with another information system or within the information system, and can be changed. (This is a quality characteristic from TMap NEXT)

The degree to which a product or system can be used with effectiveness, efficiency, freedom from risk and satisfaction in both specified contexts of use and in contexts beyond those initially explicitly identified. This is a characteristic of quality in use of the ISO25010 standard.

A lightweight alternative to full machine virtualization that involves encapsulating an application in a container with its own operating environment. This provides many of the benefits of loading an application onto a virtual machine, as the application can be run on any suitable physical machine without any worries about dependencies.

The certainty that the information system will continue uninterruptedly, which means that it can be resumed within a reasonable time even after serious interruptions. This is a quality subcharacteristic of the quality characteristic Reliability of the ISO25010 standard.

An extension of the principles behind continuous integration. It entails that the code is always in a state in which it is deployable. This practice leans heavily on the automation of the testing and delivery processes, to ensure that these happen without any hiccups. The testing happens in stages, where if the code passes one stage of tests, it automatically passes on to the next stage, and so on and so forth until the application is ready for deployment.

An extension of continuous delivery, where the application is not just delivered but also automatically deployed so that it runs on the production infrastructure.

By making it a practice to integrate code into the main branch of a shared repository as early and often as possible, continuous integration minimizes the cost of integration in a build process and allows for more frequent and automated testing. By kicking off automated tests as soon as a developer merges a new code, test suites can be run to check whether any new integration errors were introduced.

Based on tools, continuously gather feedback of the indicators of the IT system during live operation and use the information to forecast the future behavior of the system.

Automated test execution that is run every time software is integrated in a build before a new build is deployed.

To take adaptive measures, based on monitoring information, to ensure proper behavior of the IT system throughout the IT delivery cycle.

The degree to which the system processes the input and changes entered correctly, in accordance with the specifications, to produce consistent data sets.

Coverage deals with aspects of the test object that you would like to assess and the thoroughness with which you do that. It is the ratio between that which can be tested and that which is tested with the test set.

A structured approach to testing that aims to demonstrate a specific type of coverage by applying one or more test design techniques.

A group of coverage types and test design techniques that aim at testing the same aspect of an IT system or Business process. The four coverage groups are: Process, Condition, Data and Appearance.

The percentage of test situations, as defined by the coverage type, that is covered by the test.

The form in which the covering of test situations deducible from the test basis, is expressed.

A cross-functional team is a group of people with different but overlapping sets of knowledge, skills and capabilities, working together toward a common goal.

Testing performed by a variety of individual testers from outside the team. They usually test on remote locations. It is often used to test on a wide variety of devices by people with a wide variety of backgrounds and experience.

This is one of the coverage groups of coverage-based test design techniques that relate to testing data.

Data analytics is the isolation, aggregation and analysis of data based on different criteria relating to specific use cases.

The ease with which the correctness and completeness of the information (in the course of time) can be checked.

Data mining serves to garner important information out of large quantities of data from an information repository. Data mining can take on multiple variations, such as anomaly detection, where the aim is to get a broad picture of general trends in large amounts of data and then be able to detect when something is out of place, or cluster detection, where the aim is to identify “clusters”, or subgroups of data that fall under the same category. Another type of data mining is classification, although this requires a clear preexisting structure, with which a data-mining algorithm can automatically classify incoming data.

DC is a coverage type, from the coverage group Condition, that ensures the possible outcomes of the decision are tested at least once.

A combination of one or more conditions that define the conditions for the various possibilities in the subsequent system behavior.

Deep learning is a form of machine learning based on learning data representations, based on the information processing in biological nervous systems, using neural networks.

The test object is not working as expected. The term Defect is very confusing in the testing profession. Therefore please refer to the related terms Anomaly, Error, Fault and Failure.

The exit criteria to determine if outputs of a sprint meet the specified criteria and are consequently ‘done’. The outputs of a sprint may have to be integrated with other products before they can be deployed, therefore see definition of shippable.

The start criteria to determine if all inputs for a sprint such as user stories are refined and a sprint can be successfully launched.

The criteria to determine if a product can be released (deployed) to the production environment and used.

The ease with which the core of the information system can continue after a part has failed.

A cross-functional systems engineering culture that aims at unifying systems development (Dev) and systems operations (Ops) with the ability to create and deliver fast, cheap, flexible and with adequate quality, whereby the team as a whole is responsible for the quality. Usually other expertises like Business Analysis and Quality Assurance (including testing) are integrated in the team. A DevOps culture has an Agile mindset that can be supported/implemented by, for example, the Scrum framework.

Testing using knowledge of the technical implementation of the system, usually by developers.

The use of data to raise human performance and implement a cyclical dynamic, where processes and capabilities are constantly evolving based on inputs from the customer, fostering ongoing product or service loyalty. (this definition refers to digital processing)

Disclosing historic data of a retired IT system.

Digital manufacturing is considered “manufacturing’s next act”. It is part of the new wave of industry 4.0, where CAD design, digital manufacturing, robotics, sensors and data & analytics are combined to redefine industrial production. Digital manufacturing defines the process of designing a product or prototype in CAD, and then creating the physical version via additive manufacturing, laser cutting or CNC.

A solution where elements of IT, electronics, mechanics, but also new technologies such as Artificial Intelligence (AI), Internet of Things (IoT) and additive manufacturing, have a place. Existing IT systems are digital solutions too. And the digital age also converts Operational Technology (OT) systems to digital solutions.

Digital transformation refers to when a business or organization shifts from their traditional mode of operation to modern, technology-enabled ways of management and operations.

A digital twin is a digital representation of a physical process, product or service. A digital twin can take on the form of a data sheet, an interactive CAD model of a product, or a dashboard with a lot of data readings. Using these models, tests can be conducted to gather information on behavior.

A simulation program that replaces a program that should take care of the control and/or the calling of the test object.

A model used to dedicate test environments to specific use. The letters stand for Development, Test, Acceptance and Production. Often (but not necessarily) the test varieties are aligned with the test environments and use similar names.

Testing by execution of the test object, that is the running of an application.

The accuracy and completeness with which users achieve specified goals. This is a characteristic of quality in use of the ISO25010 standard.

The resources expended in relation to the accuracy and completeness with which users achieve goals. It is the relationship between the performance level of the system (expressed in the transaction volume and overall speed) and the amount of resources (CPU cycles, I/O time, memory and network capacity, etc.) that are used. This is a characteristic of quality in use of the ISO25010 standard.

TMap HD describes the five elements of quality-driven testing. These elements have two goals. At the one hand they are elements of the development of the quality- and testing profession. The profession of quality is changing and these elements give direction to this change. On the other hand the elements asist when making choices to achieve better results and answer testing challenges. The five elements are: People, Integrate, Simplify, Industrialize and Confidence.

The tangible or visible form of an intelligent machine (e.g. a robot or chatbot). It refers to the physical looks of a robot, and mainly is about whether it looks right for its purpose. This is a subcharacteristic that we added related to artificial intelligence and robotics, to the ISO25010 quality characteristic Usability.

Empathy is the ability to understand and share the feelings of another. This is a subcharacteristic of the quality characteristic Personality that we added related to artificial intelligence and robotics, to the ISO25010 quality characteristics.

The person responsible for organizing end-to-end quality. (Often indicated as orchestrator)

A test variety where the end-to-end functionality of a business process, supported by one or more systems is tested with end-to-end test cases.

Engineering is the creative application of science, mathematical methods, and empirical evidence to the innovation, design, construction, operation and maintenance of structures, machines, materials, devices, systems, processes, and organizations.

A software tool (often large and complex) that supports organizations in their business processes. Instead of offering a solution for one department or one business activity, ERP systems are generally used throughout the organization and are connected to one overarching enterprise database which contains the organization’s Master Data. Also known as ‘ERP solution’.

The criteria an object (for example a test basis document or a test object) must satisfy to be ready to be used in a specific activity.

In the application of equivalence classes, the entire value range of a parameter is partitioned into classes. In a specific class the system behavior is similar (equivalent) for every value of the parameter.

See Enterprise Resource Planning system.

Human mistake that may, but not necessarily needs to, lead to faults or failures.

Ethics is about acting according to various principles. Important principles are laws, rules and regulations, but for ethics the unwritten moral principles are the most important. This is a  subcharacteristic of the quality characteristic Morality for Artificial Intelligence and Robotics, that we added to the ISO25010 standard.

This is an old term that is replaced by Static Testing.

One of the approaches to machine learning. An EA uses mechanisms inspired by biological evolution, such as reproduction, mutation, recombination, and selection. Candidate solutions to the optimization problem play the role of individuals in a population, and the fitness function determines the quality of the solutions.

Formal testing of knowledge and skills by question and answer. An example is that an intelligent machine (e.g. an autonomous car) has to pass a formal exam before it is allowed to operate in a specific environment.

The criteria an object (for example a test basis document or a test object) must satisfy to be ready at the end of a specific project activity or stage (e.g. an iteration).

The part of a test case that describes what result should be observed when the test case is executed. This will be compared with the actual result.

Testing based on the experience, skill and intuition of the tester(s). This is a test approach that can be subdivided into several test approaches.

Defining and executing specific test situations and test cases for testing a specific situation (as opposed to implicit testing).

The activity of investigating, and establishing the quality and risks of the use of an IT system through examination, inquiry and analysis.

Simultaneously designing and executing tests to learn about the system, using your insights from the last experiment to inform the next. In other words every form of testing in which the tester designs his tests during test execution and the information obtained is reused to design new and improved tests.

The ease with which (part of) the information system can continue elsewhere. This is a quality sub-characteristic from the TMap NEXT list. It is covered by the quality characteristic Reliability of the ISO25010 and its sub-characteristics.

A deviation of the system from its expected delivery or service. The result or manifestation of one or more faults. A failure may be detected by dynamic testing.

The manifestation of an error residing in the code or a document or a system. This may cause a failure. A fault may be detected by static testing.

See: Feature Toggle

A powerful technique, allowing teams to modify system behavior without changing code. A mechanism that enables deployment of features that are not finished yet, or of which the quality is uncertain. Code can be deployed to the production environment without being available to the users by turning off the feature toggle. At a later stage it can be made available by just turning the feature toggle on. And if a problem occurs it can be turned off again. 
(also called Feature Flag)

A weakness in a process or system that makes it vulnerable to security threats.

The degree to which the user may introduce extensions or modifications to the information system without changing the program itself. (this is a TMAP NEXT quality characteristic)

The degree to which a product or system mitigates the potential risk to economic status, human life, health, or the environment. This is a characteristic of quality in use of the ISO25010 standard.

Unit to measure the functionality and/or the size of application software.

A standardized method aiming to measure the size of the functionality of an automated system. The measurement is independent of the technology. This measurement may be used as a base for the measurement of productivity, the estimation of the needed resources, and project control.

A test carried out by the future user(s) in an optimally simulated production environment with the aim of demonstrating that the developed system meets the functional requirements.

A test carried out by the future user(s) in an optimally simulated production environment with the aim of demonstrating that the developed system meets the functional requirements.

Functional suitability is the degree to which a product or system provides functions that meet stated and implied needs when used under specified conditions. This is an ISO25010 product quality characteristic.

The overall approach for the setup and organization of test processes that applies to more than one project or release. General agreements on e.g. the test process, standard strategy, estimating method, procedures, organization, communication, documentation, etc.

Testing with knowledge of and insight into the internal operation of a system or program. This is the opposite of Blackbox Testing.
(The term whitebox testing is often used but glassbox better signifies the true meaning of being able to see what is happening)

A strategy derived from previous experiences with similar problems, that employs a practical method, not guaranteed to be optimal, perfect, or rational, but instead sufficient for reaching an immediate goal. It can involve using a rule of thumb, an educated guess, an intuitive judgment, a guesstimate, profiling, or common sense.

An approach that enables cross-functional teams to continuously improve the products, processes and people that are required to deliver value to the end users. Examples are Scrum and DevOps.

The roadmap of testing in the digital age consists of five hops that each describe a specific part of the roadmap.

Human friendliness refers to the level to which intelligent machines don’t cause harm to humans or humanity. This is a subcharacteristic of the quality characterisitic Morality for Artificial Intelligence and Robotics that we added to the ISO25010 standard. It refers to Isaac Asimov’s laws of robotics.

Humor is the quality of being amusing or comic, especially as expressed in literature or speech. This is a subcharacteristic of the quality characterisitic Personality for Artificial Intelligence and Robotics that we added to the ISO25010 standard.

See: Industrial Internet of Things

Improvisation is the power of the intelligent system to make right decisions in new situations. This is a subcharacteristic of the quality characteristic Intelligent Behavior for Artificial Intelligence and Robotics that we added to the ISO25010 standard.

Observing behavior of a test object during execution of test cases that were not designed for testing that specific behavior (as opposed to explicit testing).

An unplanned interruption to an IT service or reduction in the quality of an IT service or a failure of a configuration item.

A quantitative assessment for comparing or tracking the current state or level of the product or a part of the product, together with the business process it supports, and of the IT delivery process and of the people involved. Indicators are used to determine whether the business value and the IT objectives are achieved.

The use of IoT technologies in manufacturing.

Industrialize is one of the five elements of TMap HD. It aims at making testing activities repeatable and demonstrable by standardization and/or automation.

Industry 4.0 is a name for automation and data exchange in manufacturing technologies. It includes cyber-physical systems, the Internet of Things, cloud computing and cognitive computing.

A static testing technique where a product is submitted to one or more reviewers to assess it’s quality. There are no specific rules for the review so the result of the review highly depends on the reviewer(s).

The collection of activities involving the design, development, testing, maintenance, and use of computers and software for the processing and distribution of data and the generation of information.

The suitability of hardware, network, systems software and database management system for the application concerned and the degree to which the elements of this infrastructure interrelate.

The process of managing and provisioning computer environments through machine-readable definition files, rather than physical hardware configuration or interactive configuration tools.

The initial situation, needed before a test case can be executed, consists of everything that is needed to prepare the system for receiving the required input. This includes not only the data that are needed for the processing, but also the condition in which the system and its environment must be. For instance, one might think of setting a specific system date, or running specific week and month batches that bring the system to a specific status.

The most formal static testing technique with products (usually documents) being read thoroughly (reviewed) by a group of experts. In addition to determining whether the solution is adequately processed, an inspection also aims at improving the process of creating a document.

Integrate is one of the five elements of TMap HD. It aims at reducing IT complexity and quality risks by organizing a shared way of working with a shared responsibility for quality.

The ability to comprehend or understand. The Ability to learn is the ability to comprehend, to understand and to profit from experience. This is a new main quality characteristic for artificial intelligence and robotics that we added to the ISO25010 standard.

The Internet of Things (IoT) is the network of devices embedded with electronics, software, sensors, actuators, and network connectivity which enables these objects to connect and to exchange data.

See: Internet of Things

A problem that has actually occurred and has a negative effect on (the chances of) achieving a goal.

See: Information technology

The chance of failure combined with the impact when a failure occurs. This is determined with risk poker and measured in risk points.

See: Known fault

An anomaly that has been classified as an actual fault but has not been solved (yet).

A description of the various expected levels of load on an IT-system, which is the basis for specifying and executing performance tests.

Describes, in logical terms, the circumstances in which the system behavior is examined by indicating which test situations are covered by the test case.

Machine intelligence (MI) is a unifying term for what others call machine learning (ML) and artificial intelligence (AI). We found that when we called it AI, too many people were distracted by whether certain companies were “true AI”, and when we called it ML, many thought we weren’t doing justice to the more “AI-esque”-like aspects, such as the various flavors of deep learning. So, machine intelligence is a term that combines “artificial intelligence”, “machine learning” and other related terms.

Machine learning is one of the ways to achieve artificial intelligence. It contains different algorithms – each with its own strengths and weaknesses. These algorithms are often grouped into three categories: supervised learning, unsupervised learning, reinforcement learning.

The degree of effectiveness and efficiency with which a product or system can be modified by the intended maintainers. This is an ISO25010 product quality characteristic.

Testing the changes to an operational system or the impact of a changed environment to an operational system.

Data related to business entities which can be used in multiple business transactions, especially in an ERP system. Master data is stored in one unique database and can then be used by several different programs/users. Common categories are customer, supplier and employee details, products identifiers, financial structures and locations.

Test plan by which the various test levels are geared to one another.

See Model Based Review

See Model Based Testing

A software testing technique that attempts to alleviate the test oracle problem. A test oracle is the mechanism by which a tester can determine whether a system reacts correctly. A test oracle problem occurs when it is difficult to determine the expected outcomes of selected test cases or to determine whether the actual outputs accord with the expected outcomes.

A quantitative assessment for comparing or tracking the current state or level of (a part of) the IT delivery process and the people involved. Metrics are used for continuous improvement.

A version of a (software) product which has the least amount of features necessary for the product to be working and to be usable for the end user. Based on product feedback, further product development can be done. This concept is used in high-performance IT delivery models such as Scrum or DevOps.

Testing together with a group of people, for example the whole Agile team. This brings together a lot of brainpower for hard testing problems and/or it enables the team to learn from each other about the system under test and about testing.

A method or object that simulates the behavior of a real method or object in controlled ways, usually to be called by other methods or objects when the final method or object is not yet available or unpractical to test with.
Also see: stub

Model-based development is the process in which a model is at the center of the (software) development process, from establishing requirements through to design and implementation. This allows for a common design environment across multiple project teams and links all design directly to requirements.

Reviewing of the test basis by creating a model. Model-based review employs the use of models to reduce test base ambiguity, it may include creating a model. The reason for MBR is that models are unambiguous, faults like incompleteness, inconsistency, and incorrectness will be detected. This activity may create a test basis, it does not deliver test cases, these will result from model based testing.

See: MBR

Model-based testing revolves around using a model of the object or process under development to create or refine test cases. This can range from a fully automated set of tests that are designed and tested by a MBT suite, to model-based review (MBR).

MCDC is a coverage type, from the coverage group Condition, that ensures that every possible outcome of a condition is the determinant of the outcome of the decision, at least once. MCDC implies also “condition/decision coverage”.

Continuously gathering feedback, using tools, of the indicators of the IT system throughout the IT delivery cycle and use that information to forecast the behavior of the IT system.

A mood is a temporary state of mind or feeling. This is a subcharacteristic of the quality characteristic Personality that we added related to artificial intelligence and robotics, to the ISO25010 quality characteristics.

Morality is about the principles concerning the distinction between right and wrong or good and bad behavior. This is a new main quality characteristic for artificial intelligence and robotics that we added to the ISO25010 standard.

A group of people with different (and not overlapping) sets of knowledge, skills and capabilities, working together toward a common goal. A downside is that team members can’t replace each other which brings the risk that the team is not effective when just one team member is not available. (see also cross-functional team)

MCC is a coverage type, from the coverage group Condition, that ensures that all possible combinations of outcomes of conditions in a decision are tested at least once. This also implies “modified condition/decision coverage” and “condition coverage” and “decision coverage”. (This coverage level is reached for example with a complete decision table)

A type of testing where certain statements in the source code are changed (mutated) to check if test cases will identify the fault that was introduced this way. This is a manner to verify the quality of the test set (instead of the test object).

See Minimum Viable Product.

Natural interaction refers to the level of natural dynamics in communication between between an intelligent machine and a person. It is important both in verbal and non-verbal communication between humans and machines. This is a new sub quality characteristic for artificial intelligence and robotics.

Testing aimed at one or more quality characteristics other than functionality.

An object part is a logically coherent part of the test object from the perspective of the characteristic to be tested.

Function mode directly connected to an information system in which the information system immediately processes the commands and directly shows the answer (output) on the screen (or other user interface).

Operational intelligence is the use of everything from data mining to analytics to gather, correlate and use all of disparate data to reveal important patterns, gain deeper insights, reduce time to detect important events, leverage live feeds and historical data to understand what is happening, identify anomalies and make effective decisions, and quickly deploy a solution and deliver flexibility needed now and in the future.

The planning and/or coordination of the elements of a situation, to produce a desired effect.

A discipline that supports organizations and their employees in smoothly and successfully introducing an intended change into their ways of working. (Sometimes abbreviated to OCM)

An orthogonal array LN(sk, t) is a 2-dimensional array of N rows and k columns consisting of elements that can assume s values, whereby every combination of t columns contains all the combinations of the s values in equal proportion.

Operational Technology (OT) is the hardware and software dedicated to detecting or causing changes in physical processes through direct monitoring and/or control of physical devices such as valves, pumps, etc.

Test design technique that results in testing all possibilities of any combination of 2 factors.

People is one of the five elements of TMap HD. It aims at having the right people with the right skills, the right knowledge and the appropriate mindset to wisely apply the agreed methods and way of working in a team and organization. Hence the name TMap HD: Human Driven.

Performance efficiency represents the performance relative to the amount of resources used under stated condition. This characteristic is composed of the sub-characteristics Time Behavior, Resource Utilization and Capacity. This is an ISO25010 product quality characteristic.

A line organization that offers test services.

The combination of characteristics or qualities that form an individual’s distinctive character. This is a new main quality characteristic for artificial intelligence and robotics that we added to the ISO25010 standard.

The concrete elaboration of a logical test case, with choices having been made for the values of all required inputs and settings of the environmental factors. In principle, one logical test case has one physical test case.

The degree of effectiveness and efficiency with which a system, product or component can be transferred from one hardware, software or other operational or usage environment to another. This is an ISO25010 product quality characteristic.

A checklist which provides insight into what needs to be addressed in an SAP Test Strategy. It is an acronym for all SAP focus areas: “Processes, Reports, Authorizations, Configuration, Transports, Interfaces, Conversions, Enhancements, Screens, User experience and Platform”.

Testing the delivered product in such a way that it is determined whether or not the product is of sufficient quality to execute a complete test of this product.

Privacy is the state of being free from unwanted or undue intrusion or disturbance in one’s private life or affairs. This is a new sub quality characteristic of the characteristic Morality that we added to the ISO25010 standard.

A cause, or potential cause, of one or more anomalies or incidents.

This is one of the coverage groups of coverage-based test design techniques that relate to testing processes.

Product lifecycle management is the approach to managing a product from inception to disposal, and includes all aspects of the product’s life, albeit human involvement in the form of skillsets, product information, engineering and manufacturing.

A specific chance that the product fails in relation to the expected impact if this occurs. The Chance of failure is determined by the Chance of faults and the Frequency of use. The impact is related to the operational use of the product.

Analyzing the product to be tested with the aim of achieving a joint view, for all stakeholders, of the more or less risky characteristics and parts of the product to be tested so that the intensity of testing can be related to this view.

A test carried out by the future operations people in an optimally simulated production environment, with the aim of demonstrating that the developed system meets the requirements set from the operations perspective.

Testing of new or adapted parts of a system (used as the opposite of regression testing).

A method of submitting contributions to a development project by which a developer, after making a change to code in a topic branch, asks for this change to be committed to the main branch (that is to be included in the main repository). This involves static testing (i.e. reviewing) of the changed code, for example to check if the change was done properly and if it complies with maintainability and other guidelines to code quality.
(the static test may be, partly, automated)

The totality of features and characteristics of a product or service that bear on its ability to satisfy stated or implied needs.

All planned and systematic activities needed to provide adequate confidence that a product or service meets the requirements for quality.

A description of a property of an information system.

A person that performs quality assurance and testing tasks. This is a team role, not necessarily a function.

Quality Engineering is about team members and their stakeholders taking joint responsibility to continuously deliver IT systems with the right quality at the right moment to the businesspeople and their customers. It is a principle of software engineering concerned with applying quality measures to assure built-in quality.

Allocation of quality measures to IT delivery items (e.g. user stories, features etc.), to balance the investment in quality engineering activities and to make an optimal distribution of effort over fundamental DevOps activities and Test Varieties. The quality risk class is used to assign the intensity of the quality measures that must be applied.

A milestone in an IT project that requires predefined criteria to be met before the project can proceed to the next phase or stage. It is used throughout the IT delivery process to provide information on whether a delivery complies with quality expectations and if it is ready to be handed over to the next party involved.

A group of activities that is aimed at achieving a certain level of quality. There are preventive, detective and corrective quality measures.

A person that supervises the quality measures that are applied in a situation where multiple teams collaborate on delivering IT systems that support one or more business processes.

A specific chance that the product fails in relation to the expected impact if this occurs. The Chance of failure is determined by the Chance of faults and the Frequency of use. The impact is related to the operational use of the product.

Analyzing the aspects of quality of the product to be tested with the aim of achieving a joint view, for all stakeholders, of the more or less risky characteristics and parts of the product so that the intensity of quality engineering and testing activities can be related to this view.

The ease and speed with which the information system can be restored after an interruption. This is a quality sub-characteristic of the quality characteristic Reliability of the ISO25010 standard.

An approach in security testing where the security of entire the organization is assessed by attacks. The red team is the attacking team and the blue team is the defending team.

The phenomenon that the quality of a system deteriorates as a whole as a result of individual amendments.

A test aimed at verifying that all unchanged parts of a system still function correctly after the implementation of a change.

A variety of machine learning that determines various options to find the option that maximizes some notion of cumulative reward. Reinforcement learning differs from standard supervised and unsupervised learning in that correct input/output pairs are never presented, nor sub-optimal actions explicitly corrected. Instead the focus is on performance.

The degree to which a system, product or component performs specified functions under specified conditions for a specified period of time. This is an ISO25010 product quality characteristic.

Execute a previously failed test case to verify whether a problem has been properly fixed. (also called confirmation test)

The degree to which parts of the information system, or the design, can be reused for the development of different applications. This is a quality sub-characteristic of the quality characteristic Maintainability of the ISO25010 standard.

A review is a static test of a product. This means testing without “running” software or systems. There are various review types (also called static test techniques), such as: walkthrough (transfer of knowledge and exchange of visions), technical review (e.g. selecting solution direction/alternative), management review (e.g. determining project status), informal review (review by colleagues with no formal process), and inspection (the most thorough review technique that establishes the quality level of a product).

Not to be confused with the word review from the Scrum guide which is used to demonstrate the product to the product owner.

A function of the probability of occurrence of a given threat (likelyhood) and the potential adverse consequences (impact) of that threat’s occurrence.

A description of the extent to which the system meets the quality requirements and of the quality risks associated with it. This report will contribute to establishing the confidence level.

A machine that gathers information about its environment by input of sensors and, based on this input, changes its behavior. Combined with machine learning and artificial intelligence the robot’s reactions over time become more adequate. The use of Internet of Things (IoT), Big Data Analytics and Cloud technology make a robot versatile.
Robots come in many different shapes and forms. It’s not just the metallic man. Robots may equally be a smart algorithm on social media, an autonomous vacuum cleaner or a self-driving car.

The use of data-entry tooling that automatically inputs data in an administrative IT system. The data can be generated by an AI system.

Robotics is a branch of engineering that involves the conception, design, manufacturing, and operation of robots. This field overlaps with electronics, computer science, artificial intelligence, mechatronics, nanotechnology and bioengineering.

The degree to which the information system proceeds as usual even after an interruption. This is a quality sub-characteristic from the TMap NEXT list. It is covered by the quality characteristic Reliability and its sub-characteristics, of the ISO25010 standard for quality characteristics.

Describes one or more tasks and the knowledge and skills required to carry them out.

Software company founded in 1972 in Germany, focused on developing and selling Enterprise Resource Planning systems. Company name is an acronym for “System and Analysis Program development” (in original German: “SystemAnalyse und Programmentwicklung”).

The degree to which user needs are satisfied when a product or system is used in a specified context of use. This is a characteristic of quality in use of the ISO25010 standard.B116

See: Software Bill of Materials

The degree to which a product or system protects information and data so that persons or other products or systems have the degree of data access appropriate to their types and levels of authorization. This is an ISO25010 product quality characteristic.

A test with which the validity of data input is tested using the semantic rules for the relationships of the data on the input device and other data, for example in the database, in the system or on the input device.

One of the five elements of TMap HD. It aims at ending the upward spiral of growing complexity of IT systems that requires increasing testing efforts, by standardizing and decoupling. 

A part of the informationsystem that was specifically created for testing which simulates the environment of the (part of the) system that is to be tested. It can be a driver, or a stub, or both.

see: Pre-test

A formal record containing the details and supply chain relationships of various components (e.g. libraries, open source components, proprietary code, etc.) used in an IT system. (also abbreviated to SBOM)

The ease or difficulty to keep the system in a consistent state during modification. This is a subset of the quality subcharacteristic modifiability of the ISO25010 standard.

Anyone with viable interest in the business value delivered by the team, at all levels in the organization and even outside the organization.

Initial situations often contain the same data for several test cases. Such data are therefore included in a so-called starting point for the entire test and not separated for each test case. It is called a central starting point if this is intended for more tests or testers.

See: Central starting point

Testing by examining products (such as requirements specifications, manuals or source code) without programs being executed.

A simulation program that replaces the actual program, including the in- and output-flows, which is called by the test object.

The degree to which manual procedures match the automated information system and the fitness for use of these manual procedures for the organization. 

Supervised learning is the machine-learning task of learning a function that maps an input to an output based on example input-output pairs. It infers a function from labeled training data consisting of a set of training examples. 

A focus point of quality engineering that aims to minimize the unfavorable impact that business processes, the IT components that support them, and the infrastructure that hosts them, have on the planet. It promotes an approach to software and systems design, development, implementation, deployment, operation, maintenance and retirement, that emphasizes environmental perdurability and energy efficiency. 

System Under Test, see: Test object

A test with which the validity of input data or output data is tested based on the syntactic rules that describe what attributes the data should comply with, for example the value domain. 

Artificial data, for example to be used for testing without compromising privacy-regulations. Often this is generated using AI models, that are trained to reproduce the characteristics and structure of the original data.

A test carried out by the future user(s) in an optimally simulated production environment, with the aim of demonstrating that (sub)system interface agreements have been met, correctly interpreted and correctly implemented. 

System management is responsible for technical operation of the software in its intended infrastructure in production.

A test carried out by the supplier in a (manageable) laboratory environment, with the aim of demonstrating that the developed system, or parts of it, meet with the functional and non-functional specifications and the technical design. 

see: Test object

A static testing technique where a product (that’s about 60-80% complete) is submitted to a number of reviewers with the question to assess it from a certain perspective.

A way of working for designing and executing tests. There are two groups of test approaches: experience-based testing and coverage-based testing.

The information that defines the required system behavior and is used as a basis for test design.

The information that defines the required system behavior and is used as a basis for test design.

A concise document containing the starting points for an exploratory testing session.

see: test situation.

A test management task that deals with developing and applying a set of corrective actions to get a test project on track when monitoring shows a deviation from what was planned.

Data that exists (for example, in a database or in a test case) before a test is executed, and that affects or is affected by the component or system under test.

A coverage type that aims to demonstrate the certainty that all combinations of N consecutive paths in a process or program flow are covered. A path in this context consists of all steps between a decision point and the next decision point, or between the start and the first decision point, or between the last decision point and the end.

The complex of activities to create test situations, test cases, test data, test scenarios and test scripts.

In coverage-based test design we use a number of different terms for specific entities in the test design, they are: test situation, test case and test scenario. The test design entities relationship diagram describes their relationships.

A standardized method of deriving test cases from a specific test basis that will achieve a certain coverage. Applying a test design technique results in test situations, logical test cases and/or physical test cases.

The person in a cross-functional team that specifies and/or executes tests and evaluates the test results. Test engineer is a role, not necessarily a function.

A composition of parts, such as hardware and software, connections, environment data, tools and operational processes in which a test is carried out.

The execution of tests by running the system under test and thus obtaining the actual results that can be compared with the expected results to determine whether the tests have passed or failed. This is part of dynamic testing.

A goal for testing that is relevant for the client or stakeholder, formulated in terms of IT-supporting processes, achieved user requirements or use cases, critical success factors, change requests or specified risks to be covered.

A collection of software and test data configured for a development environment with the purpose of dynamically testing one unit or a series of units, whereby the behavior and output are checked.

Any useful thought, piece of data, technique, heuristic or whatever that you write down on a charter so that during your exploratory testing session you have an abundance of possibilities to vary your testing.

The facilities and resources necessary for the satisfactory execution of the test. It consists (among others) of test environments, test tools and workplaces.

Indicates whether a specific part of the test object must be tested lightly, moderately or intensively.

Guides a team in deciding how to achieve the desired test intensity by defining what test approaches and/or test techniques should be used for which level of quality risk. The test intensity table uses the test approaches and coverage groups as defined on TMAP.net.

A group of test activities that are managed and executed collectively.

The operational organization to provide test services to one or more clients. A test line has a fixed team of testers, infrastructure, test tools and standardized work procedures.

A record of the test steps, expected results and actual results, together with observations about the system behavior, which is registered during testing, for example during an exploratory testing session.

The activity that checks the status of testing activities, identifies any variances from planned or expected, and reports status to stakeholders.

The business process and/or information system (or part thereof) to be tested.

see Test goal.

The alignment of a large number of test automation tasks and other quality assurance related tasks for all teams involved in a CI/CD process, for optimized test execution. This term refers to both the process of orchestration and the technical implementation thereof in the pipeline.

The whole of the people, test functions, facilities, procedures and activities including their relationships.

A general solution for a specific recurring test problem.

Description of the general structure and the choices with respect to the tests to be executed and the way to supply information. The test plan forms the reference during organizing and performing of the tests and also serves as an instrument to communicate with the client. The test plan is a description of the test project, including a description of the activities and the planning. So a testplan is NOT a description of the tests, e.g. test cases, themselves.

Unit of measurement for the size of the test to be executed.

Test Point Analysis (TPA) is used to objectively estimate the effort for a system test or acceptance test. It uses the results of a function point analysis (FPA). Development tests are estimated together with development activities and therefore are out of scope for TPA.

Description how an organization deals with the people, resources and methods involved with the test process.

see: test scenario

Sequence of physical test cases to manually or automatically execute them in an efficient manner.

A detailed overview of testing activities to be performed and executed in a specific sequence and time.

The test automation code related to one or more test scenario’s to automatically execute tests.

A collection of test cases.

An isolated condition under which the test object displays a specific behavior that needs to be tested.

The complete documentation of the test design, the logical test cases and test scenarios for a specific test unit.

See: Test Design Technique.

A part of a test case, or when not related to a specific test case, a part of a test scenario and/or a test script.

The allocation of quality measures to balance the investment in testing and to make an optimal distribution of effort over test varieties and test approaches to give insight in test coverage and test intensity. Often this is based on the quality risk levels and the pursued business value.

A part of a test case, or when not related to a specific test case, a part of a test scenario and/or a test script.

The whole of the people, test functions, facilities, procedures and activities including their relationships.

A group of test scenarios and/or test scripts that logically belong together.

A group of people who, led by a test manager, test coordinator or testleader, undertake test activities.

An automated instrument that supports one or more test activities, such as planning, control, specification or execution.

Describes how an organization handles the acquisition, implementation and use of test tools.

A group of test activities with the intention of checking the information system in respect of a number of correlated (sub) quality characteristics.

A collection of processes, transactions and/or functions that are tested collectively.

The term “test variety” aims at making all stakeholders aware that there will always be different needs for testing, and therefore different varieties of testing will have to be organized. Whether these are organized separately or combined depends on the situation. Test varieties are defined based on the relevant quality characteristics and other relevant perspectives such as progression or regression.

The degree of effectiveness and efficiency with which test criteria can be established for a system, product or component and tests can be performed to determine whether those criteria have been met.

The detailed review (i.e. static test) of the test basis with respect to testability.

A person that performs quality assurance and testing tasks. This is a team role, not necessarily a function.

Testing consists of verification, validation and exploration activities that provide information about the quality and the related risks, to establish the level of confidence that a test object will be able to deliver the pursued business value.

All test documentation and test products, such as test plans, test specifications, test scripts, test data, description of test infrastructure, etc., that are created in the course of the test process. Testware must be suited for maintenance purposes and should therefore be transferable and maintainable.

The extent to which information is available in time to take the measures for which this information is intended.

A set of generic activities for a specific theme, that are always relevant for quality engineering, regardless of the applied IT delivery model. TMAP describes 20 topics divided in two groups: organizing and performing.

Transparency of choices indicates if a human involved can understand how a machine comes to its decisions. This is a new quality sub characteristic for AI and Robotics that we have added to the ISO25010 standard.

see: User Acceptance Test.

A test carried out by the developer in the development environment, with the aim of demonstrating that a logical group of units works together as defined in the technical specifications.

A test carried out in the development environment by the developer, with the aim of demonstrating that a unit meets the requirements defined in the technical specifications.

Any testing lacking a plan containing what to do and what to expect of a system, or lacking preparation of the test, is unstructured. This is also called ad-hoc testing. Some people see a great advantage in unstructured testing because, as they say: “You can start testing right away.” That is, without “wasting” any time on preparation.

The machine-learning task of inferring a function to describe hidden structure from “unlabeled” data (i.e. a classification or categorization is not included in the data that is input to the machine-learning algorithm).

The degree to which a product or system can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use. This is an ISO25010 product quality characteristic.

A test carried out by (or on behalf of) the future user(s) in an optimally simulated production environment, with the aim of demonstrating that the developed system supports the operational process of the users.

The ease of operation of a system by the end users. This is a quality characteristic defined in the TMap NEXT book. This is outdated. See the ISO25010 quality characteristic usability.

Confirmation by examination and through provision of objective evidence that the demands for a specific intended use have been fulfilled. (“is the right system built”)

Also see: verification.

The average amount of work— often measured in story points — that the team can perform in one sprint.

Confirmation by examination and through provision of objective evidence that specified requirements have been fulfilled. (“is the system built right”)

Also see: validation.

Design and validation activities occur collaboratively in order to prove early product designs, support decision-making and enable continuous product optimization within interdisciplinary and cross-enterprise partnerships.

The VOICE model is about establishing the level of confidence that the pursued business value can be achieved. It consists of 5 terms: Value, Objectives, Indicators, Confidence and Experience.

A flaw in an IT-system by which it is exposed to the possibility of being attacked or harmed.

A static testing technique by which the author explains the contents of a product during a meeting. Several different objectives are possible: bringing all participants to the same level of understanding, transfer of information, asking the participants for additional information or choose from the alternatives proposed.

See: glassbox testing.

Observing tests that are specified and/or executed by another party to give evidence about the level of confidence that these tests can give.

A transaction code in an SAP system, that was created for a specific customer process. Often consists of ‘Z’ plus an abbreviation of the activity it needs to execute. For example, ZTEST.