Posted on 2021-02-16 by Matt Strahan in Industry
When setting up Volkis, we wanted to set up a team the way we perceive that it should be set up. With quality, skill, effectiveness, ethics, and transparency. We didn’t only look at the security industry for inspiration, though. Instead of just looking in we looked around at other industries as well. Cyber security is barely a child, only having really been around for a few decades. Other industries have centuries if not millenia on us.
We looked over at finance and found that what their auditors do is in essence similar to what we do, but their processes and standards have a maturity that we don’t have. After all, cyber security isn’t known for being mature in processes, standards, personality…
Let’s take a look at one standard in the finance industry but practically unheard of in cyber security: the independence policy.
What does independence mean anyway?
When I look around, I see a lot of security organisations talk about “independent penetration testing” or being an “independent third party”. Independence must be important, right, otherwise people wouldn’t want it and companies wouldn’t be advertising how “independent” they are.
Great, so what does that mean? “Uhh…we’re not you…?”
I delved in a bit deeper to see what companies could actually mean, but when I tried to find something actually written down by a cyber security organisation that says “this is what I mean by independent” I came up empty. Maybe I didn’t look in the right direction (and please let me know if you have something) but as far as I could tell there was no security organisation that has put together an independence policy around penetration testing or cyber security auditing. Even compliance standards that mandate “independence” didn’t seem to say specifically what independence meant. Although APRA CPS 234 does have an entire section on what it means to be independent…a section that’s a whole paragraph long that mostly just says “they need to be independent”.
The grey area of independence
I ended up diving into the weeds here and things got even more murky. Late last year we had our Christmas presentation day with a mix of people both in penetration testing and outside of penetration testing. My own presentation was quite simple: I presented a bunch of scenarios and we had a poll asking “does this violate independence?”
The list of scenarios was deliberately challenging. It started off easy with “The auditor implemented the system” (of course that’s a violation of independence). Then it got harder, with other scenarios like:
- The auditor’s company has provided security products used by the system
- The auditor’s company is monitoring the system for incidents
- The auditor is friends with the main client contact
- The auditor provides consulting outside of the scope of the engagement (say they’re there for a pentest but they end up discussing how best to implement MFA or chatting about mitigations for the latest vulnerabilities)
- Vulnerabilities are fixed during the engagement with assistance from the auditor
All of these are messy scenarios and, what’s more, we couldn’t agree on whether these would still remain independent! 1 and 2 above are pretty standard for organisations that provide “full stack” security. 4 and 5 in that list in particular got a bit of discussion. If you look at our penetration testing engagement guide we encourage this and make it part of our standard method of operation, but people outside of penetration testing declared it a violation of independence!
It wasn’t quite the “should headings be sentence case or title case” dispute that went for what seemed like an eternity and a show of hands ended up in a tie, but it was quite a fun discussion trying to sort out whether all of those scenarios are alright or not.
The policy
The final policy is available in our handbook. It covers:
- Independence from implementation
- Being free from undue influence
- Being free from bias
- Personal relationships
- Company relationships
- Additional engagement
Given we couldn’t look around at other cyber security organisations for inspiration, I’ve taken feedback from that presentation and ideas from the finance industry to make the policy. As far as I can tell this is a first in the cyber security penetration testing world. When putting it together I had to make decisions and assumptions which may or may not come through, so please let us know if you have any feedback, suggestions or complaints. Arguing about independence policy requirements is fun!
Otherwise, when talking about independence go ahead and challenge organisations who just say “we’re independent” without saying what that means. If they don’t know, get them to look at our policy for tips! The more clarity we have here in the industry the better we’ll all be.
About the author
Matthew Strahan is Co-Founder and Managing Director at Volkis. He has over a decade of dedicated cyber security experience, including penetration testing, governance, compliance, incident response, technical security and risk management. You can catch him on Twitter and LinkedIn.
Photo by krakenimages on Unsplash.
If you need help with your security,
get in touch with Volkis.
Follow us on Twitter and
LinkedIn