Posted on 2022-02-07 by Matt Strahan in Business Security
Cyber security is now one of the top-of-mind topics for boards in Australia. Security assessment reports including technical reports such as from penetration testing are being placed in board papers. Although cyber security skills are becoming more commonly represented in boards, it is still the case that boards are called to interpret and act on the results of cyber security assessments without really understanding the practicalities that can underly it all.
What makes it even more difficult for board members is that the regulation and standards around cyber security reports aren’t as mature as for financial reports. While there’s a lot the cyber security assessment report might say, there’s also information that you might not be able to get until you ask the right questions.
While this is not a comprehensive list of questions you might want to ask, I’ve put together some questions that might uncover some of the gotchyas and catches you might not expect when reading a cyber security assessment report.
The questions you should ask
How could the results affect our strategy?
The linkage between security assessment reports and strategy is often opaque.
You’re not here to be secure, your company is there to do what it does, be it performing services, playing its part in the supply chain, or making toasters. Yet the security assessors that have written the report in the board papers may not have full information on what systems are key to supporting your organisation’s strategy and may not even have information about the strategy itself. This means the report is unlikely to have clear linkages between its findings and the strategy. Instead the risks are often presented from the point of view of the assessor (these are the security issues we identified), not from the point of view of the organisation (this is how it could affect our business).
With the digital world, it’s becoming more and more likely that your strategy may be dependent on your IT infrastructure. The risks that are contained could potentially determine the success or failure of your strategy. Or maybe not. Exposure of customer information, for example, could financially impact a construction organisation in the short term but may not harm the organisation long term at all. A ransomware attack on the logistics systems of the same organisation could completely stop operations. Without clear linkages you may not be getting the full ramifications of the issues that are identified.
What was not in scope and why?
Some of the reasons a piece of your organisation could be excluded from the scope of a cyber security assessment might surprise people who do not have a background in IT. Reasons I’ve personally come up against have included:
- This system is too important and so we cannot risk technical testing potentially impacting its availability
- We already know of security issues in this system and we do not want to spend money testing it again
- Although the system has privileged access over our data or forms a key component of our value chain, it is managed by a third party and we are not contractually able to include it in testing
All of these reasons are perfectly valid. Even so, if something is out of scope it is unlikely to be mentioned in the resulting report, no matter how important it is or how much risk it could potentially present to the organisation. It is extremely easy to get a misleading view of the security of the organisation by reading a report that has key systems out of scope.
Was the assessment independent? What conflicts of interest were identified in this assessment?
For financial assessments and audits it is usually mandatory to engage an independent third party and have potential conflicts of interest clearly documented. Unfortunately in cyber security the regulation around conflicts of interests has not yet caught up. It can be routine for organisations wanting to run a cyber security assessment to engage the same security organisation that manages cyber security products, writes their security policies or processes, or monitors their environment for attacks. If the results of the assessment pick up gaps in the security that assessor company themselves have implemented, can you trust that the gaps will be accurately reported?
In the finance space this would be like getting the organisation that does your bookkeeping to audit your books. It would be considered a clear conflict of interest. In cyber security, however, you may receive such a report without any identification or discussion of potential conflicts. What’s worse, consolidation in the cyber security industry and so called “full stack security organisations” have made this practice common to the point of being routine. At Volkis we have built an independence policy and, from what we can tell, we are still the only cyber security consultancy to have such a policy.
How are the results reflected in the risks in the risk register?
When talking to executives about their cyber security I will often ask “are cyber security risks in your risk register?” and “do the risks appropriately reflect what is on the ground?” You might be surprised to find that the risk register doesn’t properly cascade down into a cyber security risk register and that the report you have in your board papers isn’t accounted for at all in the risks in your risk register.
It is a difficult process to connect the realities on the ground to the risks in the cyber security risk register. I would say that this process is in fact the primary purpose of industry cyber security frameworks such as ISO27001. Without this process, though, the board will be left with a potentially misleading view of cyber security in their organisation.
The new assessment report can be a way to not only gain a bit of insight into your security but also the accuracy of your helicopter view of the organisation.
How did you verify the personnel performing the testing are appropriately skilled?
It can be extremely difficult for even cyber security professionals to tell whether a security test was done properly. One of the key ways of making sure is simply to ensure you are working with skilled personnel and a reputable organisation.
Unfortunately again there are no quick answers here due to a lack of regulation and some disreputable organisations in the industry. Even certifications and standards are not a great indicator. The CORIE Framework, for instance, refers to old and obsolete certifications which can no longer be attempted and there is currently a schism between CREST International and CREST ANZ.
What is more important from a director’s view is that the executive have considered the quality and reputation of the organisation rather than just going for a low cost test.
How are you making sure these kinds of issues won’t come up again?
“Cyber security is a journey” is an oft-quoted message, but sometimes it feels more like playing whack-a-mole. If you fix security issues in one area of the organisation, the same people end up making the same mistakes again and again. Without appropriately addressing root causes of these issues you will end up being exposed.
The cyber security plan from executives should not be limitied only to fixing the immediate issues but looking long-term at improving the underlying processes and security architecture.
How were the risks of the assessment itself managed?
When engaging a company to perform a cyber security assessment you don’t necessarily think of the assessment itself as being an avenue for being breached. When performing a penetration test or red team engagement, though, you are often handing over direct access to your network, permission to perform security attacks, and the ability to exfiltrate data. At the end of the engagement they will have a list of security issues, potentially including information on how to hack your organisation again. This is a lot of trust to put in a third party!
Some of the risks you might want to ensure are managed as part of an engagement are:
- Offshoring risks: If a company outside of Australia is performing the testing, or the team that is performing the testing is located outside of Australia, you may be opening yourself up to a level of offshore risk that is significantly higher than what you’d normally be used to. If access to your systems or information about security vulnerabilities in your organisation is leaked then you may not have criminal recourse available for you due to the change of jurisdiction. In certain jurisdictions, sharing information about security vulnerabilities with the local authorities is standard. Offshoring may occur without your knowledge if the assessor company’s contracts allow it.
- Supplier risks: Is the company you’re working with actually doing the testing? At Volkis, for instance, we do work through our partners. We try to be upfront about this but still in the industry the actual testing may be subcontracted without your knowledge.
- Data sovereignty and retention: The information collected during security assessments must be protected whether it is held within your systems or the systems of the assessors. Clear guidelines including data retention should be supplied by the assessor.
Getting outside help
As a director you shouldn’t be afraid to get an outside perspective of the report, independent of the executive or the security assessors. The results of a security assessment can be confusing, subtle, and even provide a misleading view of the risks an organisation faces. An independent voice can help you clarify the information and find these hidden catches that may not be evident on a surface reading of the report.
If the skills around cyber security are not available in your board, I would definitely recommend that your board engage a third party directly for education and assistance.
About the author
Matthew Strahan is Co-Founder and Managing Director at Volkis. He has over a decade of dedicated cyber security experience, including penetration testing, governance, compliance, incident response, technical security and risk management. You can catch him on Twitter and LinkedIn.
Photo by Towfiqu barbhuiya on Unsplash.
If you need help with your security,
get in touch with Volkis.
Follow us on Twitter and
LinkedIn