Posted on 2020-01-21 by Matt Strahan in Business Security
Every year there’s a guy who comes out and tests my smoke alarm. The smoke alarm guy visually inspects the alarm, runs the internal test, and then uses a small device that, in my head, I ignorantly name “the smoke gun” to trigger the alarm. It’s a simple process that makes sure that the alarm still works.
Watching him work, I thought it curious how so many organisations check their smoke alarms this way but have probably never actually tested whether their security systems are working or not. Probably most organisations don’t even know specifically what their security systems will detect and probably don’t have the capability or know how of testing their security system themselves. I’m going to go a bit further and show my cynicism here: Probably most organisations don’t actually know what happens when their security system alerts, don’t know what the alert looks like, and wouldn’t know what the alert would mean. It’d be like someone wandering through their home looking at the box on the roof and saying “that’s an alarm. I don’t know what it sounds like. When it goes off something is wrong – but I don’t know what it could be!”
Alternatively they might have been deluged by their tools with so many alerts that they turned them off, or at least cannot figure out which alerts are important. That’s not any better than not having the alerts at all.
I’ve seen the results of this lack of validation again and again. Security systems end up not working as they’re supposed to. I have, for instance, seen SOCs that aren’t ingesting any of the logs the organisation is supposed to be sending, and SIEMs that won’t ever raise an alarm because the SMTP settings and SMS gateway details were wrong. The systems are set up and the organisation says to themselves “job done, let’s move on” without any validation routines to ensure the systems are actually working.
What’s worse, since the actual attacks aren’t causing any meaningful alarms, you could be just sitting in comfortable obliviousness. You don’t know what you don’t know. Organisations are left with detection tools that aren’t any better than XKCD’s TornadoGuard:
With all these issues, it’s no wonder dwell time is still over three months!
Triggering the alarm
Ideally as part of a sensible security strategy each control will be performing a designated task, mitigating some risk. Like any other critical controls, you would have validation and audit routines that ensure the control is working as intended. It’s not enough to just do the visual review and the internal check. For security controls in particular, validation should always involve getting out the smoke gun and triggering the alarm.
Triggering the alarm is more than just finding out whether the detection control is working, it involves testing the entire chain of action:
- When the detection control sends out an alert, does it go to the right people?
- Do those people know what this alert means?
- Do those people know what to do when they get the alert?
- Is the information they get good enough to action?
Even just the simple action of having the recipients get the alert has huge value. They gain familiarity with the alert. They know for future cases that this is what the alarm means, what triggers it, and what they should do when it is triggered.
All this validation might sound burdensome, adding more work to a security team. Validation takes time, and you probably already have a full book. There’s something you’re probably already doing, though, that can perform all this validation for you. You can have all this done as part of your annual penetration test.
Many organisations see penetration testing as a one way process. You go to a security company, give them the scope, and then you wait until the test is done. For those organisations it’s like online shopping – click to order and pick up your report in a few weeks’ time. Those organisations are missing out – there’s so much more that your penetration testers can do for you. We have a lot of components to our testing, one of which is performing these validations.
As penetration testers we have to know what is likely to trigger your alerts. Most of the time we study your detection controls so that the alerts don’t trigger, but we also have pretty simple tests that we can perform to deliberately trigger the alerts where beneficial. These could be triggering callback requests, downloading files that will have all AVs alert, deliberately exceeding account lockout, etc. We’re happy to ensure we trigger those alerts during the engagement with specific timing, and you can get the active feedback to see on your end what alerts came through. We can also work with you to create a list of test cases to provide a more comprehensive range of triggered alerts.
This doesn’t just apply to internal penetration testing but also project and web application penetration testing as well. Do you want to be alerted if there’s been a brute force attack against an administrator account? Ask your tester to trigger the alert.
If you’d like to incorporate response into all this, then the easiest way is to respond to the alerts! See if what you’d do as part of the response is actually effective with feedback from the testers.
I believe if you’re not incorporating this as part of your penetration testing or even red team engagements, you are missing a huge part of the value a penetration test can bring.
About the author
Matthew Strahan is Co-Founder and Managing Director at Volkis. He has over a decade of dedicated cyber security experience, including penetration testing, governance, compliance, incident response, technical security and risk management. You can catch him on Twitter and LinkedIn.
If you need help with your security,
get in touch with Volkis.
Follow us on Twitter and
LinkedIn