Posted on 2024-08-30 by David Chadwick in Industry
I have seen this topic frequently over the years with many clients asking: “which is the best approach?”. I have compiled as much information as possible for businesses to make the most informed decision.
TL;DR
Allowlisting allows the penetration tester to focus on testing the ‘in-scope’ system, rather than testing the ‘security controls’ sitting in front of it.
Not allowlisting can reduce the testing time as the tester needs to find a way to bypass the security control.
It is possible to do both, such as having a portion of the testing period with the tester allowlisted. Another option is to allowlist and when a vulnerability is found, it can be tested to verify if the security control can block exploitation.
Before deciding, it is best to define what is the actual goal of the penetration test? Is it to test the system, the security control, or both?
My general recommendation is that using allowlisting provides a better result in most cases, and should be the preferred option.
Definitions
First some definitions.
- ‘Threat actors’: A threat actor is any entity, ranging from hackers to state-sponsored groups, that poses a risk to computer systems and data.
- ‘Security control’: When mentioning a security control, I am referring to Web Application Firewalls (WAF) and Intrusion Prevention Systems (IPS) throughout this article.
- ‘Web Application Firewalls (WAF)’: A WAF is a security control designed to protect web applications and websites from a variety of online threats and attacks.
- ‘Intrusion Prevention System (IPS)’: An IPS is a security control designed to protect systems from a variety of online threats and attacks.
- ‘In-scope’: The agreed upon systems to assess in a penetration test.
- ‘SQL injection (SQLi)’: Is an attack where malicious SQL code into a web application’s input fields, exploiting vulnerabilities to potentially gain unauthorised access or manipulate a database
- ‘IP hopping’: Involves changing the source IP address of the penetration tester, typically utilising various Virtual Private Networks (VPNs).
So what is allowlisting?
In the context of penetration testing, allowlisting (also known as ‘whitelisting’) is the practice of ensuring that traffic from a specific source IP address is not blocked by security controls. This is crucial for enabling penetration testers to conduct thorough assessments without interference.
Security controls, such as a WAF and IPS, can automatically start blocking traffic that is considered malicious and even block/ban the source IP address. This can greatly hinder the penetration test and prevent the discovery of vulnerabilities.
Allowlisting does not involve providing more access to the penetration tester, such as allowing additional access to ports/services that are not normally accessible.
To allowlist?
Allowlisting ultimately allows the penetration tester to focus on testing the ‘in-scope’ system, rather than testing the ‘security controls’ sitting in front of it.
Penetration tests are a performed over a limited time and normally don’t last longer than a week or two. It can take a long time to understand a system in order to find its weakness to exploit, so having to try and bypass a security control can cut into the testing time. It’s important to remember that ‘threat actors’ have as much time as they want to spend looking for vulnerabilities, a persistent attacker could even spend months, even years if they wish!
This extensive time frame also allows them to use other techniques that penetration testers don’t have the luxury of, such as…‘low and slow’. The ‘low and slow’ technique is where the frequency of requests will be slowed down significantly to avoid detection, such as sending one request per minute.
A security control bypass could have been released on the dark web that the penetration tester does not have access to. Exploits are often sold and traded on the dark web, and these groups are often exclusive and difficult to join.
Not to allowlist?
So what if you don’t allowlist the penetration tester?
The tester may not find a way around the security control, and cannot test the entire system. There have been situations in the past, where the WAF was aggressively configured to block the source IP addresses after one malicious request for twenty four hours. This made it very challenging to use any form of automation, or send any request that could be deemed malicious.
Penetration testers will typically provide their source IP address to the client, so they know if it is the tester sending malicious requests, or if they are actually under attack from a ‘Threat actor’.
Using techniques to avoid IP blocking, such as ‘IP hopping’, isn’t always feasible for a tester. It can require a lot of time to setup and configure.
From the clients side: it also makes tracking the tester’s source IP a nightmare! A new source IP address may be required after only a couple requests, and testers can potentially send thousands of requests. Imagine trying to map back a couple hundred or even thousands of IP addresses to confirm it was the tester or not?
To sum up all of the above, the main concern to ‘not allowlist’ a penetration tester, means vulnerabilities that exist in the system might not be detected, simply because the tester was blocked from identifying it. It can also mean that you are relying on the security control as a ‘band-aid’, rather than fixing the root cause of the vulnerability.
It isn’t all negatives for not ‘allowlisting’ though, as it can find misconfigurations in the security controls. There has been a test I conducted, where there was a ‘WAF’ configured and I was not allowlisted. I found an ‘SQLi’ vulnerability, and the WAF did not detect exploitation at all. I was able to dump the entire database without a single alert or being blocked.
In this example, while the WAF was enabled, detection of SQL injection attacks was not configured, as it was in its default configuration. It is also worth noting that a lot of third party security controls have been tested to death over the years.
Why not both?
Is your IT team able to action the request to be allowlisted in a reasonable time frame? If you don’t think they can action the request within a couple hours, then it would be best to not go down this route, as it may cut into the testing period too much and impact the results.
Typically, splitting involves being allowlisted for a duration of the test, but not the entire period.
Splitting between allowlisting and not allowlisting is absolutely possible. Splitting the test can reduce the time spent on testing the system, as the security control now needs to be included in the test. This is a factor you may need to consider.
Most WAFs/IPSs have a “monitor” mode, so they can allowlist a source IP and also receive alerts without actively blocking. It is even possible to be allowlisted and when a vulnerability is found, it can be tested to verify if the security control can block exploitation.
Conclusion
Like most things in the security space, there is no silver bullet for which approach to take.
My general recommendation is that using allowlisting provides a better result in most cases, and should be the preferred option.
Before deciding on which option, it is a good idea to really define what is the actual goal of the penetration test? Is it to test the system, the security control, or both?
If you want to test the security controls, consider having it scoped into the test with additional time added or a separate test all together. Speaking to the consultant when making this decision could also provide lots of insight.
About the author
David Chadwick is a Senior Security Consultant at Volkis. He is a part-time meme master and caffeine enthusiast. You can catch him on LinkedIn.
If you need help with your security,
get in touch with Volkis.
Follow us on Twitter and
LinkedIn