OpenVAS vs. Nessus - A Comprehensive Analysis

I’ve spent a little time going over some of the data available for Tenable’s Nessus and OpenVAS. It can’t provide all the answers, but hopefully by the end of this post you’ll have a better understanding of similarities and differences between the two vulnerability scanners.

Intro

Over the last 10 years the rate at which vulnerabilities are being discovered has been increasing rapidly. In 2010 there were 4,992 CVEs released and over the course of 2020 there were 17,253, over three times the number. As a result, organizations of all sizes now include vulnerability scanners in their suite of tools to help identify and manage vulnerabilities within their infrastructure. With an increasing number of vulnerability scanners available within the market though, a common question for many organizations is – which one?

‍

Two that are most frequently compared are Tenable’s Nessus and Greenbone’s OpenVAS. Perhaps because they’re the most pervasive, or maybe because they have a shared history.

Nessus and OpenVAS started as the open-source Nessus Project back in 1998 by Renaud Deraison and in 2005 Tenable (co-founded by Renaud) changed the Nessus version 3 license model to closed-source, looking to improve the solution by dedicating time and resources, and create a professional commercial product. Nessus was forked in 2005 to keep an open-source version alive, and in 2006 one of these forks was rebranded to OpenVAS . Since 2008 it is Greenbone Networks who develop and drive forward OpenVAS providing the feed of checks.

While they share a common history, and some similarities remain, there are significant differences in their coverage. Before we get into the numbers though, let’s make sure we understand what we’re comparing.

All of the metrics in this analysis are based on the number of CVEs that are covered by checks within each scanning engine, rather than how many checks (also known as “plugins”) are available. Common Vulnerabilities and Exposures (CVEs) are unique identifiers for publicly known vulnerabilities, while plugins can contain checks for multiple CVEs, and multiple plugins can check for the same CVE. It made sense to base our analysis on the number of CVEs checked for.

Raw CVE Coverage

We can begin by talking raw numbers and answering the question “what is the total number of vulnerabilities covered by each scanner?” You can see from Figure 2 that in terms of raw numbers Tenable covers more CVEs (49,572) than OpenVAS (44,306) a difference of 5,266 CVEs. So, overall OpenVAS has checks for ~10.6% fewer vulnerabilities than Tenable.

But that’s not the whole story. At the time of writing, since 2010 there have been 118,523 CVEs published. So, we can see in Figure 3 that Tenable covers 41.82%, and OpenVAS 37.38%, of all publicly disclosed vulnerabilities (that have a CVE number), a difference of around 4% when compared to the total number of CVEs.

‍

The numbers in the graphs above are purely on the number of CVEs that each scanner will detect. But what is the overlap between the CVEs detected by both scanners, and what about CVEs that are unique to each? In Figure 4 we can see that there is a considerable overlap in the detection of CVEs between OpenVAS and Tenable. However, Tenable checks for 12,015 CVEs which OpenVAS does not check for and OpenVAS checks for 6,749 CVEs which Tenable does not check for.

‍

We’ll take a look at the difference between types of CVEs that are covered by each scanning engine later in this analysis.

So, in terms of raw numbers of coverage, Tenable wins. But OpenVAS is relatively close. However, this could easily lull you into a false sense of security. In vulnerability management we need to consider how severe each vulnerability is, rather than simply worry about the basic number of vulnerabilities you’re looking for.

Risk Rating Coverage

Using just the raw numbers above means that one scanner may have more checks for lower risk vulnerabilities, but fewer for high-risk vulnerabilities, giving us a false sense of its effectiveness. Let’s dig a little deeper and review the number of vulnerabilities checked by each scanner but broken down into the risk of each vulnerability. This gives us a better understanding of the types of risks that each scanner can identify for us. I have broken down the ratings into the following approach:

‍

We can see from Figure 5 that Tenable covers more CVEs at every risk category, but both OpenVAS and Tenable track closely with each other. The greatest difference in raw numbers (2786 additional CVEs checked) is in the medium risk category but may be expected since the vast majority of CVEs that have been released since 2010 also fit within this category. We can see from Figure 6 that the percentage coverage of vulnerabilities increases as the risk rating increases. This is positive, we want to make sure that we’re detecting as many of the critical rated vulnerabilities as possible.

‍

One of the striking differences is that Tenable will check for 258 additional critical vulnerabilities, that means OpenVAS can identify 4.6% fewer critical vulnerabilities than Tenable. Critical-rated vulnerabilities are the vulnerabilities that will often cause your IT/security staff to lose sleep and turn to drink.

‍

On a more serious note, though, these are the kind of vulnerabilities that you hear about in the news that allow ransomware attackers to (almost) effortlessly gain initial access to a victim’s network, leading to tens of millions in ransoms, regulator fines, closed national health services, and in some cases job losses.

So, in terms of coverage of checks for vulnerabilities when broken down by risk level Tenable wins at every risk level. But OpenVAS is still relatively close, and importantly we can see where it matters most - at the critical risk level.

It’s important to note here though that not all types of scanning are created equal, OpenVAS and Nessus have different features available on this front – and the type of scanning you implement affects the CVEs you get checked for, so let’s take a look at the difference that makes.

Remote Check Types

It’s important to understand the difference between remote and local checks, and the coverage they can provide. Remote checks are run over the internet against your systems with no authentication or access beyond that of a normal attacker. Local checks are run either on the system via an agent, or via an authenticated session (such as through SSH etc). There are pros and cons to each type of scanning, but these are discussed in detail in another blog by Chris, and are beyond the scope of this blog post.

What really matters to us here is that many of the CVEs included in the stats above require you to either enable authenticated scanning or install an agent on your systems to get that level of coverage. But what if we’re not in a position to do authenticated scanning or install an agent on our systems, or we’re specifically interested in what an attacker can see. In that case we need to focus on remote checks only. Let me be clear, when I mention “remote” checks I mean unauthenticated checks (uncredentialled) against your systems, just as if they’re scanned by an attacker over the internet. Tenable handily has a field plugin_type which we can use to determine if it’s a remote check, and with OpenVAS I have retrieved the QoD type and if it begins with “remote”* then I consider it a remote check.

In terms of raw numbers, OpenVAS has remote checks for 11,014 CVEs beating Tenable’s 9,497. So, on the surface it seems like OpenVAS is the choice for scanning your systems using unauthenticated remote scans. But as before we need to break this down to understand what risks we are identifying with these scanners.

We can see from Figure 8 that OpenVAS leads in remote checks for low (480 more CVEs covered) and medium (1,444 more CVEs covered) risk ratings. OpenVAS then loses out to Tenable for remote checks of high (214 fewer CVEs covered) and critical (193 fewer CVEs covered) risk vulnerabilities. If we’re concerned about remote coverage of high or critical vulnerabilities, which most organizations should be, then Tenable appears to have the best coverage.

‍

It is close within the high and critical categories between Tenable and OpenVAS. With the Security Feed from Greenbone, it is possible that OpenVAS would have more remote checks in these risk categories and potentially propel the scanner forward. But, as we’re comparing the community feed, and as we’re most concerned about high and critical risk vulnerabilities, as a result I have to give this round to Tenable again. Tenable wins but was bloodied up something fierce by OpenVAS in the early rounds.

*Unfortunately, this is not an ideal solution, as also possible that the “exploit” QoD type may be a remote exploit, rather than an authenticated local exploit but I haven’t been able to find a distinction between local and remote exploits.

Check Publication Lead Time

One of the concerns I have as someone who uses vulnerability scanners is the delay between a vulnerability’s details being released to the public and the availability of a check for that vulnerability. As I mentioned previously, I’m using CVE publish date from the NVD dataset as a point in time to measure against. However, vulnerabilities can be found and exploited by a threat actor before the vulnerability is published publicly (and before a fix for the vulnerability is available); if you’re interested in the difference between a patch being made available and exploitation being seen in the wild take a look at the Mandiant Threat Intelligence team’s article titled Time to Exploit.

In an ideal world vulnerabilities and exploits wouldn’t exist. But we’re not there yet. This means we need checks to be published so that we can find vulnerabilities in our own systems. So, let’s take a look at the different levels of ideal for when these checks are published compared to CVE publish date:

Local/Authenticated vs Remote Check Prioritization

It is possible for any given check to detect multiple CVEs, for example the “Potential exposure to Hafnium Microsoft Exchange targeting” plugin checks for 3 CVEs (CVE-2021-26857, CVE-2021-26858, and CVE-2021-27065), and conversely a single CVE can be covered by multiple plugins (CVE-2021-26857 is associated with 3 Tenable plugins).

In the numbers above I’ve specifically only included the time at which a remote check is available. Not when another type of check is released first. I’m interested in working out whether when a vulnerability is published I will be able to execute a scan immediately (assuming I’m only running remote checks), or do I have to wait for the scanners to publish a local check (which is potentially of no use to me) first before moving on to publishing a remote check? This will depend on the priorities of each of the organizations. In Figure 13 we can see that Tenable releases a remote check first 14.34% of the time, and OpenVAS releases a remote check first 26.46% of the time. These are numbers for vulnerabilities which specifically have a CVSSv2 Access Vector of “Network”, meaning the vulnerability can be exploited remotely, and where User Interaction Required is “False”.

‍

Tenable have the ability to carry out checks using their agent, and OpenVAS do not. Both are able to carry out authenticated/credentialed scans which are not included in these numbers. What this means is that both scanners are more frequently releasing either local or authenticated checks before they release remote checks. This means that if you’re looking to keep up with the latest vulnerabilities perhaps agent-based/authenticated scanning is the correct approach for you. Additionally, if you’re only focused on what an attacker can see from the internet then perhaps OpenVAS is a good alternative.

My primary focus for this article is specifically for remote checks, so when it comes to prioritization for remote checks over local checks, OpenVAS wins.

Software Vendor & Package Coverage

Another area where it is useful to understand the major differences between the two scanning engines is CVE coverage in different types of software. It’s all well and good checking for thousands of critical CVEs, but are they appropriate for the type of software my organization uses?

We can use the Common Platform Enumeration (CPE) information within each CVE to determine which software and software versions are affected by that CVE.

CPE is a structured approach to naming information technology systems, software, and packages. The official CPE dictionary is maintained by the NVD (those same helpful people who oversee the population of CVE details – thank you NVD 😍).

In Figure 14 there are 40 different vendors listed. In the top half, we can see the 20 software vendors where there is the greatest difference in critical CVE coverage with remote checks in favor of Tenable. In the bottom half, we have 20 vendors where the difference is greater but in favor of OpenVAS. We can see that if we were to use Tenable, we’d get greater coverage with the types of commercial software provided by the likes of Sun, Oracle, HP, Cisco, Microsoft, F5, Citrix and Nagios. Some of the software vendors have no coverage at all from remote OpenVAS checks (CODESYS, F5, Citrix), this is something we’d expect given how Greenbone have marketed their community feed. OpenVAS, however, have greater coverage of CVEs for software from the likes of Lexmark, D-Link, Netgear, Adobe, Western Digital, Mozilla, SonicWall, and QNAP. OpenVAS also provides a pretty large number of remote checks for critical CVEs in Schneider Electric products.

‍

We can even take this one step further and take a look at the difference between specific software packages. Figure 15 shows the greatest differences in critical CVEs covered by remote checks for software packages. At the top of Tenable’s leader board are the Java Runtime Environment (both Sun’s JRE and Oracle’s JRE), the numbers for these two packages are exacerbated in this chart because Tenable will check for vulnerabilities in commercial software like Apache Tomcat and JBoss where there are CVEs that affect multiple CPEs. Tenable also seems to clean up on remote checks for MacOS critical CVE’s. At the bottom of Tenable’s leader board (in the middle of the chart) we can see the first entry for Cisco VPN firmware, where Tenable has a large number of checks spread across multiple firmware versions, which OpenVAS lacks. OpenVAS on the other hand has more checks for specific printer firmware, QNAP NAS software, the Mozilla suite and some IBM domino products.

But we can also see there are more remote checks for critical CVEs within Windows 10 from OpenVAS, this could be because Tenable have more checks specifically Windows 10 within their agent-based scanning.

‍

In summary if you are a small company/start-up with few internet-facing systems then you may find that OpenVAS meets your requirements. The types of software that the feed covers are more focused towards open-source software vendors and packages. If you’re a large organization with a large number of commercial systems exposed to the internet you may want to look more towards Tenable’s scanning solution. There is no winner in this section both have their places.

Recent Headline Vulnerabilities

Since the beginning of 2021 there has been a steady stream of vulnerabilities that have made sysadmins and security professionals alike cry themselves to sleep at night. They have CVEs, and sometimes they also get the VIP (VIV?) treatment and are given catchy names, so in 2021 you may have heard of ProxyLogon or PrintNightmare. While these aren’t always the most important vulnerabilities (some big ones can sometimes slip through the headlines), you can be sure that my boss or my board members are going to ask me if we are vulnerable to these, so it’s important to me to understand whether or not these killer vulnerabilities are going to be detected by either scanner. Some of these are listed in the 2021 section of CISA’s Top Routinely Exploited Vulnerabilities alert.

Microsoft Exchange Server

‍

Microsoft Exchange is the email and calendaring server software used by thousands of organizations across the globe. It is often exposed by organizations to the internet to allow for email to be received, and for remote users to access Outlook Web Access (OWA) and view their inboxes or calendar in a browser.

The Microsoft Exchange vulnerabilities that have been actively exploited in the wild from the beginning of 2021 consist of 7 CVEs:

• ProxyOracle (CVE-2021-31195, a reflected XSS vulnerability, combined with CVE-2021-31196, a padding oracle attack which can allow attackers to decrypt user cookies)
• ProxyLogon (CVE-2021-26855 a flaw which allows an attacker to bypass authentication and impersonate an admin, chained with CVE-2021-27065 a flaw which allows an authenticated user to write arbitrary files resulting in code execution)
• ProxyShell (CVE-2021-34473 a flaw in the autodiscover service which fails to appropriately validate a URI allowing attackers to bypass access controls, with CVE-2021-34523 a flaw in exchange PowerShell backend where user access tokens are not validated correctly allowing an attacker to bypass authentication, and CVE-2021-31207 a flaw in mailbox export functionality allowing an authenticated attacker to execute code as SYSTEM by uploading arbitrary files).
• ProxyToken (CVE-2021-33766 a flaw in the Exchange Control Panel allowing an attacker to bypass authentication)

Tenable has local or remote checks for all 8 of the vulnerabilities associated with the Microsoft Exchange CVEs. Greenbone’s community feed has checks for none of these CVEs.

Microsoft do not provide bug bounty rewards to security researchers if they find a vulnerability in Exchange, unless it affects Microsoft O365 (the cloud alternative to running your own Exchange server). We may see less interest from security researchers to hunt for bugs in Exchange if they’re not going to be rewarded for their labors, which means we may also see more of these bugs being found by attackers and exploited before the vulnerability information is publicly available.

Pulse Secure

‍

The Pulse Secure VPN (or Pulse Connect Secure) is a VPN solution which establishes a secure connection between an organization's networks, or to allow remote workers to connect to their organization's internal network. It’s often exposed to the internet to allow a legitimate user to connect from any location.

There are 5 CVEs which are associated with active exploitation of Pulse Secure VPNs in 2021:

• CVE-2021-22893 (A flaw in Pulse Connect Secure gateway which allows a remote unauthenticated attacker to execute arbitrary code)
• CVE-2021-22894 (A flaw in Pulse Connect Secure allows remote authenticated users to execute arbitrary code as the root using maliciously crafted meeting rooms)
• CVE-2021-22899 (Allows remote authenticated users to execute arbitrary code using the Windows Resource Profiles feature)
• CVE-2021-22900 (A flaw in how Pulse Secure VPNs handle archive files allows remote authenticated admins to write arbitrary files)
• CVE-2021-22937 (A flaw which allows an attacker to bypass fixes for CVE-2020-8260 which allows attackers to overwrite arbitrary files and execute code as the root user)

Tenable has local or remote checks for all 5 of these Pulse Secure CVEs. The community feed from Greenbone has checks for none of these CVEs.

Accellion File Transfer Appliance

Accellion File Transfer Appliance (FTA) allows users to transfer large and sensitive files and there are 3 hosting models (private cloud, on-premises, and hosted). As of 30th April 2021, FTA is considered End of Life.

There are 4 CVEs which are associated with active exploitation of Accellion FTA in 2021:

• CVE-2021-27101 (A flaw in how FTA handles host headers in HTTP requests which allows an attacker to carry out SQL injection)
• CVE-2021-27102 (A flaw in a local web service which allows attackers to execute arbitrary OS commands)
• CVE-2021-27103 (A Server-side Request Forgery flaw which allows an attacker to force the system to make web requests to a domain under the attacker’s control)
• CVE-2021-27104 (OS command injection)

Both Tenable and Greenbone’s community feed have zero checks, local or remote, for the 4 CVEs associated with the Accellion FTA CVEs.

PrintNightmare

‍

PrintNightmare is a vulnerability that affects the Print Spooler service on Windows, the vulnerabilities allow an attacker to execute code on affected systems. Usually, Windows systems do not have their SMB port exposed to the internet. There are 2 CVEs which are associated with PrintNightmare:

• CVE-2021-1675 (Originally a local privilege escalation vulnerability in the Windows Print Spooler service, the impact was later updated when CVE-2021-34527 was released)
• PrintNightmare (CVE-2021-34527, similar to CVE-2021-1675, but allows authenticated remote attackers to execute code with SYSTEM privileges)

Both Tenable and Greenbone’s community feed have checks for both CVEs associated with the PrintNightmare vulnerability.

Given the lack of checks in the Greenbone community feed for the Exchange vulnerabilities which have grabbed headlines for a significant portion of the year, as well as the Pulse Secure vulnerabilities Tenable wins this category. Tenable wins.

It may seem that a lack of checks for these vulnerabilities is bad for Greenbone’s community feed and OpenVAS, but this is expected since Greenbone state that their community feed focuses more on open-source and commercially available software rather than enterprise-grade self-hosted software. If you are still hosting your own Exchange servers, your own Pulse Secure VPN endpoints and exposing a solution like FTA to the internet then you should consider using a professional vulnerability detection feed.

Conclusion

We’ve taken a look at both OpenVAS (with Greenbone’s community feed) and Tenable’s vulnerability scanning solutions, using the following categories:

• Raw number of CVEs checked (Tenable wins)
• Number of CVEs check by risk rating (Tenable wins)
• Number of CVEs checked using remote checks (Tenable wins)
• Local/Authenticated vs Remote Check Prioritization (OpenVAS wins)
• Lead time for publication of checks (Tied result)
• Software coverage (No contest)
• Headline vulnerability coverage (Tenable wins)

Some of you may well have expected this kind of a result, seeing as Tenable is the incumbent commercial offering. But we can see that both scanners have their places and their own specific use cases.

Intruder utilizes both scanning engines – our Essential plan is powered by OpenVAS, and our Pro, Premium, and Vanguard plans are built upon Tenable Nessus.

If you’re interested in the greatest coverage then consider moving to an agent-based scanner, like the Pro or Premium offering from Intruder. These will be useful if:

• You’re an organization which hosts your own servers (Exchange, VPN endpoints etc),
• You’re worried about coverage of vulnerabilities in enterprise solutions, or
• You need agent-based scanning of internal systems which aren’t exposed to the internet.

While being the runner-up in most of the important areas we’ve analyzed, OpenVAS still provides a good level of coverage, if you’re running fewer on-prem corporate products. Which means Intruder’s Essential plan is a potential cheaper alternative if:

• Your budget is limited,
• You’re an early-stage start-up or SME with a limited internet footprint because all of your software uses SaaS/cloud solutions, or
• You don’t have the time to invest in what can be a fairly complicated installation, setup and usage journey. Because even the experts at Cloudflare have run in to difficulties trying to implement open-source scanning.

If any of the above fits, or you’re only interested in remote-only scanning, the Essential plan from Intruder is a viable and cost-effective alternative.

Analysis Decisions

The data for Tenable plugins is retrieved from the Tenable API and focuses specifically on the Vulnerability Management plugins which are associated with the Nessus network scanner.

The data associated with checks for the OpenVAS open-source vulnerability scanner are extracted directly from the NASL files published by Greenbone (the OpenVAS maintainers) within their community feed only. Greenbone also provide a commercial feed of NASL files but for this article I have only reviewed the checks available in the community feed with the checks available from Tenable.

Both Nessus and OpenVAS execute checks for vulnerabilities using NASL files. NASL is an abbreviation of the Nessus Attack Scripting Language and describes how the vulnerability scanner should conduct a specific check/attack to validate whether a vulnerability exists. These NASL files are delivered via feeds where you can retrieve the latest NASL files from the feed to keep your scanner up to date (you don’t need to do this if you’re using a cloud solution as it’s handled for you). Tenable’s NASL files are referred to as plugins and are proprietary; you need to purchase a license to be able to access them (even then they’re often encrypted/compiled). OpenVAS NASL files from Greenbone’s community feed are free and available to download and view. I refer to an individual check as a plugin for this article. A plugin can check for the existence of multiple vulnerabilities.

CVE information is based on data from the NVD API. MITRE CVE information feeds into the NVD where the CVE is enriched with additional information (ratings are applied etc), this means that a CVE may have been reserved by the MITRE organization but will not be reflected in this data UNTIL such a time as the NVD have assigned a rating and provided a description of the vulnerability. The time to triage a vulnerability will vary based on the organization and the vulnerability itself, but the impact of using the NVD data is that 38 CVEs from 2020 and 296 CVEs from 2021 are not fully reflected in the analysis because they have not yet had their CVSS scores or supporting information applied.

All vulnerability data used within this analysis is from the NVD including CVE IDs, risk ratings, CPEs, and when I refer to publish dates, I am referring to the date that the CVE information is published in the NVD. While this may not be a perfect representation of the existence of a vulnerability/exploit at a point in time, it does provide a common variable from which to analyze both scanners lead-time publishing checks.

We’re relying on data from 3 different sources over which we have no control. Therefore, any inaccuracies in the data provided by any of these sources will result in inaccuracies in the analysis. I cannot guarantee that the statistics here are 100% accurate.

Additionally, the data in this analysis only includes vulnerabilities that have had a CVE number assigned since the beginning of 2010. Any checks for CVEs before this date are excluded. I had to limit the volume of data, and 11 years seemed like a reasonable timeframe.

TL;DR

Raw CVE Coverage

This table summarizes some of the high-level findings from the Raw CVE Coverage section.

‍

Risk Rating Coverage

This table summarizes some of the high-level findings from the Risk Rating Coverage section. Percentages within the table are the percentage of all CVEs from 2010 which are covered by each scanner at each risk rating.

‍

Remote Check Types

This table summarizes some of the high-level findings from the Remote Check Types section. Percentages within the table are percentage of all CVEs from 2010 that are covered by a remote check (rather than local or authenticated check) by each scanner at a given risk rating.

‍

Check Publication Lead Time

These tables summarize some of the findings from the Check Publication Lead Time section.  The first section shows when CVEs have a check published (remote, local or authenticated) by each scanner, in relation to the date that the CVE details are published. Percentages within this table show what percentage of each scanners total CVE coverage was released at each time delay. We are most interested in checks released before or on the same day as a CVE’s details are published, anything greater than 31 days after is not ideal.

‍

The following table shows how many critical CVEs had remote checks released by each scanning engine. Percentages within this table show what percentage of each scanners critical CVE coverage using remote checks, were released at each time delay.

‍

Local/Authenticated vs. Remote Check Prioritization

For all critical CVEs where Tenable has a check available, they have released a remote check first, before releasing a local or authenticated check, 14.34% of the time. However, OpenVAS release a remote check first 26.46% of the time. For more information and what this means please refer to the Local/Authenticated vs Remote Check Prioritization section.

Software Vendor & Package Coverage

The following table gives an overview of the types of software vendor that each scanning engine has the greatest difference in checks (the greatest contributors to the distinct areas of each scanning engine in Figure 4). For more information, including specific package differences, please refer to the Software Vendor & Package Coverage section.

‍

Headline Vulnerabilities of 2021 Coverage

This table shows the number of CVEs associated with some of the headline vulnerabilities that have been reported since the beginning of 2021, and the associated number of CVEs that are covered by each scanning engine.

‍

Effortlessly utilize industry-leading scanning engines to protect your organization, with Intruder. Start your 14 day free trial.

‍