Key Points
How’s your vulnerability management program doing? Is it effective? A success? Let’s be honest, without the right vulnerability management metrics any program is pretty pointless. If you’re not measuring, how do you know it’s working?
And even if you are measuring, faulty reporting or focusing on the wrong metrics can create blind spots and make it harder to communicate any risks to the rest of the business.
“What gets measured gets managed.” (Peter Drucker)
So how do you know what to focus on? Cyber hygiene, scan coverage, average time to fix, vulnerability severity, remediation rates, vulnerability exposure? The list is potentially endless. Every tool and solution on the market offers different metrics, so it can be hard to know what’s really important.
In this article, we will help you identify and define the key metrics that you need to track the state of your vulnerability management program, progress you’ve made, and create audit-ready reports that:
- Prove your security posture
- Meet vulnerability remediation SLAs and benchmarks
- Help pass audits and compliance
- Demonstrate ROI on security tools
- Simplify risk analysis
- Prioritize resource allocation
Why vulnerability management needs metrics
Metrics play a critical role in gauging the effectiveness of your vulnerability and attack surface management. Measuring how quickly you find, prioritize and fix flaws allows you to continuously monitor and therefore optimize your security.
With the right vulnerability management metrics, you can determine which issues are most critical, prioritize what to fix first, and measure the overall performance of your efforts. Ultimately, the right metrics allow you to make properly informed decisions, so you’re allocating the resources in the right places.
Intelligent prioritization
The number of vulnerabilities found is always a good starting point, but it doesn’t tell you much in isolation – without prioritization and advisories, where do you start? Finding, prioritizing and fixing your most critical vulnerabilities is far more important to your business operations and data security than simply finding every vulnerability.
Intelligent prioritization and filtering out the noise are so important because overlooking genuine security threats is all too easy when you’re being overwhelmed by non-essential information. Intelligent results make your job easier by prioritizing issues that have real impact on your security, without burdening you with irrelevant findings.
For example, your internet-facing systems are the easiest targets for hackers. Prioritizing issues that leave this exposed makes it easier to minimize your attack surface. Intruder makes vulnerability management simple even for non-experts, by explaining the real risks and providing remediation advice in easy-to-understand language.
Discover our top picks of vulnerability management solutions.
Measuring your time to fix
Next up in terms of importance is the average time to fix – because when you do find an issue, you want to be able to fix it as quickly as possible. Especially as the average time between an attacker discovering and exploiting a vulnerability is just 12 days - and can be as little as five hours according to a recent SANS Institute survey of ethical hackers.
Intruder interprets the output from industry-leading vulnerability scanning tools and prioritizes results according to context, saving you time to focus on what really matters.
Discovering issues is what scanners do best, but how long it takes to fix them is down to you. For many of our customers, seven days to fix a critical issue fits their internal SLAs, while for others, fixing a medium severity issue in 24 hours is the benchmark when holding particularly sensitive data. At Intruder, we call this your Cyber Hygiene Score.
What’s the Cyber Hygiene Score?
This score helps you keep track of how long it takes for your team to fix issues and how you stack up against the industry benchmark.
Your Cyber Hygiene Score’s aim is to give you a current snapshot of your ‘cyber hygiene’ - in other words, the scan coverage, the time taken to fix issues over a period of six months and the average time to fix issues overall. Let’s look at these in more detail.
3 top metrics for every vulnerability management program
1. Scan coverage
What are you tracking and scanning? Scan coverage includes all the assets you’re covering and analytics of all business-critical assets and applications, and the type of authentication offered (e.g., username- and password-based, or unauthenticated).
As your attack surface evolves, changes and grows over time, it’s important to monitor any changes to what’s covered and your IT environment, such as recently opened ports and services. A modern scanner will detect deployments you may not have been aware of and prevent your sensitive data from becoming inadvertently exposed. It should also monitor your cloud systems for changes, discover new assets, and automatically synchronize your IPs or hostnames with cloud integrations.
2. Average time to fix
The time it takes your team to fix your critical vulnerabilities reveals how responsive your team is when reacting to the results of any reported vulnerabilities. This should be consistently low since the security team is accountable for resolving issues and delivering the message and action plans for remediation to management. It should also be based on your pre-defined SLA. The severity of the vulnerability should have a corresponding relative or an absolute period of time for planning and remediation.
3. Risk score
The severity of each issue is automatically calculated by your scanner, usually Critical, High or Medium. If you decide not to patch a specific or group of vulnerabilities within a specified time period, this is an acceptance of risk. With Intruder you can snooze an issue if you’re willing to accept the risk and there are mitigating factors.
For example, when you’re preparing for a SOC2 or ISO audit and you can see a critical risk, you may be willing to accept it because the resource required to fix it isn't justified by the actual level of risk or potential impact on the business. Of course, when it comes to reporting, your CTO may want to know how many issues are being snoozed and why!
What vulnerability management metrics do you need to show management?
What metrics you want to report depends on who you’re reporting to. If it's the CTO or senior management, they will just want to know the business is protected and they’re getting ROI. For example, have there been any new critical issues, how quickly were they fixed, and how many are still open (and why).
We’ve produced useful guides on how to create a vulnerability assessment report, and how to report to the board. Of course, you need to go into the weeds a bit more with your technical or security team who are actually fixing the issues!
Make sure everything is covered
We’ve seen why measuring is so important for success, and identified the vulnerability management metrics that should be included in your audits and reporting. But are you capturing everything from every asset in your IT environment?
Modern scanners like Intruder provide automated, audit-ready reports, but it’s important to know where all your digital assets are to avoid blind spots, unpatched systems and inaccurate reporting – which is why asset discovery is integral to successful vulnerability management. By making sure all your digital estate is covered, you can validate what to prioritize in your remediation plans of your most critical systems. Learn more about asset discovery tools.
Where are vulnerability management metrics heading?
By Andy Hornegold, VP Product
The goal of Intruder is to reduce the number of vulnerabilities that exist across your attack surface. If vulnerabilities do exist, you need to find them quickly, make sure you get the right recommendations to the right people as quickly as possible, and make sure they’re fixed as quickly as possible before an attacker can compromise your network.
1. Average (or mean) time to detect
This is the point from a vulnerability going public, to us having scanned all targets and detecting the vulnerability. Essentially, how quickly are we detecting vulnerabilities across your attack surface to reduce the window of opportunity for an attacker.
If it's taking four months to detect a vulnerability because you’re not able to scan frequently enough, that's a huge window where you’re potentially exposed. If you want to understand the effectiveness of your vulnerability management processes, mean time to detect is an important metric to understand.
So, what does this mean in practice? If your attack surface is increasing, you may find that it takes you longer to scan everything comprehensively causing your mean time to detect to increase as well. Conversely, even though the number of vulnerabilities that come out every day is increasing, if your attack surface stays flat and your mean time to detect stays flat or goes down, you're using your resources you have effectively. But if you start to see the opposite, you start asking questions, why is it taking us longer to detect things? And if the answer is the attack surface has ballooned, maybe you need to invest more in your tooling and security team.
2. Attack surface visibility
Very few people are lucky enough to manage and see 100% of their attack surface. So that's where attack surface discovery comes in.
You'll have a total number of assets that you know are exposed or that you’ve found, but how many of those are covered by the vulnerability management program? What you want to see is the percentage of assets that are protected by your vulnerability management program across your attack surface, discovered or undiscovered.
But teams increasingly spin up apps too quickly for most vulnerability management programs. Our automatic vulnerability scanner and CloudBot check when a new network service is exposed to help solve that problem.
3. Mean time to inform
What happens if you’re getting results quickly, but you’re not logging into your vulnerability management tool, actioning a ticket or assigning it to somebody to fix? Prioritization – or what we call intelligent results – is increasingly important to measure and help you decide what to fix first, because of their impact on the business.
If the number of issues is exorbitantly high because it's full of noise, there is a tangible impact on your risk profile as an organization because you’re spending so much time going through fluff that you're not fixing the things that are going to prevent an attacker from getting into your environment. And the more time you spend on it, the more likely you’re going to get compromised by one of the other vulnerabilities in that list.
Intruder trims out a whole bunch of noise and helps reduce the false positive rate, which is a key metric to track. And once you reduce the amount of noise, then you can circle back and focus on the most important metric, which is the time to fix.
4. Looking to the future: Time to fix 0
We want the right people – the people who will actually be fixing issues – to get the information they need as quickly as possible to close the window of opportunity on hackers.
This means we’ll be adding features like role-based access control (RBAC) which can reduce the time to fix from hours or days down to a matter of minutes, and integrate Intruder into the full lifecycle and remediation process by automatically sending notifications and recommendations to the individuals who are actually fixing the vulnerabilities. A few steps closer to reducing time to fix down to 0 - watch this space!
Vulnerability management metrics done right with Intruder
Clearly defined and agreed metrics are a critical element of vulnerability management. Without them, it’s impossible to understand the risk to your organization.
Intruder measures what matters most. It provides audit-ready reports for stakeholders and compliance auditors with vulnerabilities prioritized and integrations with your issue tracking tools. See what’s vulnerable and get the exact priorities, remedies, insights, and automation you need to manage your cyber risk.
Want to see it Intruder in action? Request a demo or try it for free for 14 days.
Senior Product Manager
