Usability testing

What is usability testing?

Usability testing is the most important test variety for creating an intuitive system, leading to productive, efficiënt and satisfied users of that system. It is generally an iterative process where different techniques should be employed throughout all phases of a project. This way the final product becomes optimally suited to the demands and the needs of the end users.

Usability testing covers a wide range of techniques applicable to different phases or iterations of a project or system lifecycle. For instance personas can be used during requirements elicitation, while observation can take place during testing or acceptance phases. These techniques may be executed by usability engineers, for example a heuristic evaluation on a functional system, together with end users, such as an observation of users interacting with a prototype of a system, or even in a group setting, such as a walkthrough of a design with users, developers and designers.

Therefore it is advisable to start with a planning phase specifically for usability testing to determine which techniques will be used in particular phases of a project to cover all product risks related to the usability of a system.

People Involved

A great usability of a product is best achieved if everyone involved in the development of this product has a sharp focus on the needs and wishes of the users . We specifically look at three important roles, however others may also be involved in usability testing, such as product owners, stakeholders, developers or analysts.

End users

For most projects the end users are the people who will actually use the system once it is realized. To add the most value to their experience with the system it is important to involve them in the project. Usability testing is an excellent way to do this. Their input provides legitimacy to any statement about the usability of a system and helps acceptance within the organization.  Therefore any end user involvement will provide added value to the system.

Usability engineer

The usability engineer engages in assessing and making recommendations to improve usability. He is the person who determines which techniques will be used. The usability engineer will also coordinate, prepare or execute these techniques. They may also perform design activities, such as creating wire-frames or other prototypes, or creating redesign proposals.


There are many different types of designers. In usability testing there may be a role for the functional, graphic or interaction designer. They may have different tasks depending on their expertise and the techniques being used. Designers may provide input for test preparation, use test results to create a redesign, participate in session based techniques or even perform techniques themselves.


A usability plan may describe what the product risks specifically concerning usability are and which techniques will be used to cover these. Additionally topics such as scoping, tooling, resourcing, planning, budget or reporting may be addressed.

Other artifacts may depend on the technique being used. Many techniques will result in findings, often compiled into a report. Also proposals for redesign of certain system parts may be included.

Some techniques by nature result in specific artifacts, such as personas, prototype testing and observation. Other techniques may yield quantitative data which may be statistically analyzed for objective conclusions on the usability of a system, such as surveys, reverse card sorting or A/B testing.

Success Factors

Start early

Applying inspection techniques to the requirements or the design of a system at an early stage of a project may prevent defects. If not, defects found after the system has been developed may severely impact the design causing excessive and unnecessary amounts of rework and delays. By then such rework often needs to be planned for another iteration or release. As described earlier it is best to use several techniques throughout a project so the usability of the product can be optimized in an iterative process, regardless of the project methodology being used.

User Focus

A user focus within the project (and even more so in the entire organization) helps to keep attention to the specific demands and needs of the end users. The entire project team should be aware that the users of its product have a unique perspective on it. They will have needs and demands that may not be described in the requirements and some may even be subconscious. Also project team members are often not representative of end users, who might not share their IT domain knowledge and don’t work with the product on a daily basis. A seemingly minor obstacle may become a major frustration when encountered multiple times a day. An awareness of these differences in perspectives will help a project team keep their focus on the end users.

Involve users

User focus may be achieved by trained interaction designers or usability engineers, however it cannot replace input by the actual end users of the system. They alone are the people who determine the added value of the system since they will be using it. Users are the main focus in certain techniques, such as card sorting, observation or thinking aloud, or provide their point of view in other techniques, such as walkthroughs, focus groups or personas.


While it is always good to ask the user for his opinion, it is even better to observe users as they interact with a system (or a prototype). Users may behave differently than they predict, since often they may not be fully aware of their own needs. When users develop a routine in their work they perform their tasks partially or wholly without even needing to think. This creates blind spots for users concerning the way they work and consequently how a system should work for them. A common pitfall during observations is to interact with a user too much. As a result users may start to rely on guidance during test execution or provide comments that they think are socially desirable rather than their own opinions.