This chapter describes an overview of security testing, which is a test variety based on the ISO25010 quality characteristic "security" within to so called non-functionals.
[More info: functional and non-functional testing]
DefinitionSecurity is the degree to which a product or system protects information and data so that persons or other products or systems have the degree of data access appropriate to their types and levels of authorization. [ISO25010] |
What is security?
Security enables us to operate in a world full of threats and vulnerabilities. Security testing is an important test variety that is aimed at guaranteeing the confidentiality, integrity and availability (often referred to as CIA) of all the forms of information and data.
DefinitionA vulnerability is a flaw in an IT system by which it is exposed to the possibility of being attacked or harmed. |
Security testing focuses on whether unauthorized users are not allowed to access information they should not access, and also on whether authorized users are allowed to access information they should be able to access. On top of that, security testing should also examine whether information can be made unavailable to authorized users (in which case there is a flaw in the system). The ISO25010 standard defines five subcharacteristics for security:
- Confidentiality – the degree to which a product or system ensures that data is accessible only to those authorized to have access.
- Integrity – the degree to which a system, product or component prevents unauthorized access to, or modification of, computer programs or data.
- Non-repudiation – the degree to which actions or events can be proven to have taken place, so that the events or actions cannot be repudiated later.
- Accountability – the degree to which the actions of an entity can be traced uniquely to the entity.
- Authenticity – the degree to which the identity of a subject or resource can be proved to be the one claimed.
In security testing, it is common practice to also include one subcharacteristic of reliability, which is:
- Availability – the degree to which a system, product or component is operational and accessible when required for use.
What is security testing?
To keep an IT system secure, a wide variety of measures is taken. Some are physical security measures such as installing locks, some are IT measures such as installing firewalls in a network and others are procedural measures such as assigning passwords to people that need to have access.
Building a secure system starts with security awareness of every member of the team. Securely using an IT system also requires security awareness of everybody involved in using the system.
When developing an IT system, the team needs to design for security and work according the security guidelines that are applicable. Static testing is an important part of the quality engineering for security.
After the system is deployed, dynamic security testing needs to demonstrate that the system indeed complies to the security guidelines, rules and regulations. Because of the specialized nature of security testing, most teams can only perform part of the security tests and will call in the help of a specialized person or team for further security testing.
Security testing has three separate areas that work together. These areas are commonly referred to as AAA: access control, authentication and auditing.
There are various reasons for testing the level of information security, for which the most important reason is to obtain insight into the performance of the implemented measures. This is essential, in order to be sure that the measures adopted meet the requirements set by the organization. Certifying bodies also make pronouncements on this, whereby organizations can obtain norm accreditation regarding particular compliance or legislative issues. The point at which it is decided to carry out testing varies. This depends on the status of an organization at the time attention is drawn to the issue of information security.
Organizations are all different and information security is always present at a certain level (network and application login, firewalls, badges etc.). A distinction should also be made between the security of the organization and the security of the individual applications that the organization administers, sells or makes available to suppliers and purchasers, whether or not via a web or mobile application. With the previously mentioned security environment, the measures focus mainly on the availability and confidentiality of information. This is also important, of course, in respect of the applications, but the integrity of the information is also an issue. The testing approach for the various quality aspects will be different.
Security testing approaches
Like any testing, security testing should start early with static testing, such as reviews of system architecture, focused on secure design of the information technology. A static testing technique which is often used for this is threat modelling. This method uses a range of attack categories to investigate whether the architecture is secure for that specific category. A common acronym for attack categories is STRIDE, which stands for:
- Spoofing
- Tampering
- Repudiation
- Information disclosure
- Denial of service
- Elevation of privilege
An important aspect of security is the behavior of the people involved. Security must be part of everyone's role in a team: it simply starts with security awareness.
In functional testing, use cases are a common design technique. In security testing the techniques are abuse cases and misuse cases. Creating these abuse and misuse cases starts with adding the word "not" to functional use cases. For example: the use case "a child of 12 years or younger has access to this amusement park ride" can be transformed to the abuse case "a child that is not 12 years or younger has access to this amusement park ride".
In security testing there's a distinction between faults and flaws. Where a fault means there is something wrong in the system, a flaw means there is something wrong in a process or procedure. In such a case, the system perfectly supports the procedure (so there is no fault) but the procedure itself is not secure. For example, the procedure to get a new access card for a building if someone has lost their card, does not require them to show identification.
DefinitionFlaw is a weakness in a process or system that makes it vulnerable to security threats. |
Generally, the flaws have a much larger impact and are also usually harder to find and fix than faults. Therefore, security testing should put more emphasis on detecting flaws. An important angle in finding flaws is the behavior of users, because users often don't do what would normally be expected. They make mistakes or deliberately try to fool the system. Still, there should be some attention to the faults as well because some faults can also have a large impact.
Focus of security testing
Security testing in general has 3 focus areas: infrastructure, organization and applications.
Security testing infrastructure and applications
A good start for testing applications and infrastructure of web applications is to use the OWASP top 10. OWASP is the "Open Web Application Security Project", which is a foundation dedicated to enabling organizations to conceive, develop, acquire, operate and maintain applications that can be trusted [OWASP 2019]. They also publish two "top 10s". The best known is the "OWASP ten most critical web application security risks"; the other is one the "OWASP proactive controls top 10" with security concepts that should be included in every software development project. When testing the security of infrastructure and applications, we distinguish security testing and security scanning. To start with the latter, security scanning is a type of static testing where tools are used to assess the security of the IT system. Security testing is the dynamic testing which is typically done manually by people (who will have supporting tools). One of the test varieties for security testing is the penetration test. This test variety (that is also referred to as a pen test) is an authorized simulated attack on the IT system performed to reveal vulnerabilities. Usually, a penetration test is performed by a specialized person or team.
Security testing the organization
For testing an organization, testers often apply "social engineering" in which they try to get information by misleading employees. An example of social engineering is when the security tester, who is looking for the computer server room, walks into a building, approaches the reception desk and asks: "The server room is on the second floor isn't it? I need to do some maintenance." The surprised receptionist usually responds with: "No, the server room is on the third floor", which reveals a first vital piece of information for the hacker to get in.
Red teaming as an integral approach
For comprehensively testing the security of an organization, including its people, infrastructure and applications, a relatively new approach is "red teaming".
DefinitionRed teaming is an approach in security testing where the security of the entire organization is assessed by attacks. The red team is the attacking team and the blue team is the defending team. |
A team of ethical hackers, often from outside the organization, is the so-called red team. This team is invited by an organization to attack it, mostly with the aim to reach a specific target such as a secured piece of computer hardware.
The red team uses all their skills and techniques to attack the target. They may hack an application, install malware on computers, physically enter the building, apply social engineering, and much more. Of course, the red team stays within the restrictions that were agreed with the client, often a high-level manager who has not informed the rest of the organization about the security test.
The blue team is the defending team, which usually consists of employees of the security operations center (SOC) of the organization. This way of serious gaming comes close to what real attackers would try to do and gives a lot of feedback about the actual security risks for an organization and whether the organization responds adequately.
In red teaming, a lot of effort is put in the "pre-testing" activities where the team gathers as much information as possible (from public sources but also for example by using social engineering) to carefully prepare their attack, which is usually over 1/3 of the effort. The actual testing takes about 50% of the effort. The remaining work is the post-testing reporting to describe the results of testing, and also give advice how to improve the security. This results in an evaluation of the security for the specific scenario, but often also gives an enterprise-wide view on the information security.