Posted on 2020-01-29 by Alexei Doudkine in Business Security
Nothing grinds my gears more than seeing companies flog cheap, crappy scans as penetration tests. It insults penetration testers like myself, but worse than that, it exploits the unsuspecting clients that genuinely want to improve their security.
When a company realises that they need a penetration test, this task is usually delegated to one person who is almost never a penetration tester themselves. They may have had some experience in the past with selecting a company to perform penetration testing and the outcomes may or may not have been satisfactory. A lot of uncertainty in that last sentence, isn’t there?
The fact of the matter is, penetration testing can be a bit of a mystery and it can be extremely hard to know if the one you chose will be good or not. It’s like when you go to the mechanic to service your car. You drive your car in, leave it for a day, pick it up and drive it out. The car feels exactly the same. Did the mechanic do anything, or did you just pay for some very expensive parking?
My aim is to give people some tips for how to choose a good company for penetration testing and avoid disappointing results. I encourage you to use these tips on/against us as well, if you’re considering our services!
1. Scanners vs. Pentesters
Actually, during a pentest, we would use a combination of scans (predefined checks) and our own instinct (manual checks) to find vulnerabilities.
When selecting a pentester, ask them to go through their methodologies. You should expect them to answer at a conceptual level instead of naming tools. “We would try to perform x attack, and if that worked, we would do y.” rather than, “We would run tool x, then run tool y.” Feel free to push them as well. Ask, “what if x attack fails, what might you do then?”
Someone who understands infosec at a conceptual level will be able to adapt to your specific needs much better than someone who follows the same checklist every time.
2. Community activities
Companies that truly care about infosec and, as an extension, your infosec will contribute to the community in some way. It’s hard for me to give you an exhaustive list of things to look for, but here are a few:
- Participating at conferences. This could be speaking, volunteering and even running the conference itself;
- Researching and publishing new infosec findings such as 0days or attack techniques;
- Writing and sharing open source tools for the larger community, be it other hackers or defenders;
- Creating educational content such as courses, CTFs, guides and posts.
Be open to other answers as there are so many unique ways people get involved. People who get involved typically care about what they do, and that will translate into an excellent service for you.
3. Drilling deeper into your business
How attentive is the company to your needs? How many questions do they ask? A pentest firm that cares about doing a good job will most likely ask you follow up questions about your business, the expected outcome of the pentest and your larger security strategy.
If you’re a bit defensive against sales people (who isn’t), you may be thinking that this is a tactic to discover more work. In fact, these questions should show you that the company is making an effort to get you meaningful results. For example, we will always try to understand:
- Why do you want this pentest? We’ll dig deeper than just “we want to find the vulns.”
- What threats are you protecting against? Are you likely the target of an opportunistic attack? A targeted one? An insider attack? All 3?
- What data/thing are you trying to protect in this case? The scope might be too limited to give you meaningful results and we try to discover that early.
- How does this pentest fit into the bigger picture? What is your overall security goal? Exactly how secure do you need to be to make toasters?
I would expect other firms to ask similar questions. But if they don’t, then I would be cautious.
4. Sample work/report
Ask the pentest company for an example of their previous work or a sample report. Almost all firms will have this available. However, what you will want to see is if that report contains vulnerabilities that are not found using tools. This is a great prover for number 2 and shows you that you are paying for quality.
Reporting styles can be different, so make sure that their style aligns with what you want out of the pentest and try to get the contact details for a reference.
5. Industry oversight
When I talk about industry oversight, I mean 2 things:
- A governing body that holds pentesting companies to a predefined minimum standard;
- Respected pentest companies that hold each other to a high level of standard.
Unfortunately, the industry does not yet have a good governing body. Many have tried with some level of success such as CREST and Offensive Security, but they all have shortcomings. The industry is still trying to work itself out and agree upon how pentest quality should be measured. You should certainly ask if a company has CREST certification or OSCP certified testers, but it shouldn’t be your defining factor.
One thing I love about the infosec industry is that people do not shy away from calling out snake oil companies. We’ve all seen them before with taglines such as, “We stop all 0day attacks,” “100% security guarantee” and “We will find all vulnerabilities.” Be extremely cautious of these and pay attention to respected pentesters and companies when they call them out.
6. Passion
By far the best way to ensure you get a great penetration test is by seeing how passionate someone is about it. 99 times out of 100, a person who loves what they do is going to do a better job than someone who doesn’t. (The 1 other time is an off-day.) However, this is easier said than done. How do you “see” passion? Well, this is something I have gained some experience in conducting multiple job interviews.
Ask to speak to one of the pentesters (preferably the one performing the pentest) and ask some of these questions:
“How did you get started with hacking/pentesting?”
I would expect as answer that indicated that the person explored hacking as a hobby or an interest rather than “I was promoted into it”.
“Tell me about a recent hack you did that you felt proud of.”
You should feel some excitement in this answer. Passionate hackers will feel a rush when they have broken into a system and that should come out in their answer.
“How do you keep up to date with infosec and hacking techniques?”
Passionate hackers will spend a lot of their personal time reading and researching. Simply “learning on the job” is usually not enough.
“What is recent vuln or attack you read about that you thought was really interesting, and why?”
This question acts as a prover for the previous one. Make sure that the answer is real and is recent. People who struggle to answer will fall back to very old attack techniques.
Of course, there is no metric to measure “passion”, but hopefully the answers will give you some level of certainty.
Choosing the right pentest company can be daunting task. But whether you’re using us or another company for your pentests, use these 6 tips to hopefully make your decision a little easier. Happy hunting!
About the author
Alexei Doudkine is Co-Founder and Offensive Director at Volkis. Hacker, tinkerer, car modder and dog person, Alexei has been in the infosec game for over 10 years focusing on the “attack” side of security. You can catch him on Twitter and LinkedIn.
Cover photo by Javier Allegue Barros on Unsplash.
If you need help with your security,
get in touch with Volkis.
Follow us on Twitter and
LinkedIn