The recent, high profile software vulnerabilities have raised a number of questions about the security of our software.
Three questions which have been on my mind:
- Is software less secure now... than it used to be?
- Which has more vulnerabilities... Open Source Software or Closed Source Software?
- Which Operating System has the most vulnerabilities... and which has the least?
These seem like fair questions to ask. And, considering the massive amount of data available, we should be able to arrive at some definitive answers. Yet, when we see discussions around exactly these topics, most of the statements seem to be based on feelings and preferences... rather than hard facts.
Let's fix that.
First we need to grab details on all publicly known CVEs (aka "Common Vulnerabilities and Exposures") -- the standard way of publishing details around exploits and vulnerabilities -- and drill down into that data. Luckily CVEdetails.com makes obtaining this data incredibly simple (the data below is obtained from there).
Is software less secure now... than it used to be?
The easiest way to begin answering this question is to track the number of CVEs reported per year... and put that data into a pretty graph.
And the results are... not exactly difficult to read. It goes up pretty much every year -- accelerating, significantly, over the last few years.
- 2022: 25,083
- 2023: 29,065
That's a roughly 16% increase in the total reported CVEs... in just one year.
And, at the current rate (January through March of this year), 2024 is on track to hit 35,484 by the end of the year. Which would be a 22% increase, year on year.
There are two likely possibilities which could explain this:
- We are getting much better at finding the vulnerabilities in software.
- We are adding more vulnerabilities to software.
And, in fact, both could absolutely be true.
Considering the ever-increasing complexity of our software systems (both in terms of total Lines of Code and number of interdependent systems), it seems reasonable that at least some portion of this dramatic increase in CVEs is thanks to us simply having more vulnerabilities in software.
This is about as close to a definitive answer as we are going to get: Based on the available data, yes. Software is less secure now than it used to be.
Which Operating System has the most vulnerabilities... and which has the least?
Now let's pull data on all known CVEs... and sort them by Operating System (again, using data gathered from CVEdetails.com).
Behold.
Your eyes do not deceive you. Debian Linux has had the highest number of reported vulnerabilities, clocking in at a whopping 8,751.
- Android is in second place, with 7,008 CVEs.
- And Ubuntu Linux was trailing in third place, with 4,058.
Windows, iOS, and macOS all had significantly lower total numbers of reported vulnerabilities.
Note: I left a variety of BSD and UNIX systems off this list as their number of total CVEs was lower than the lowest entry on the chart. FreeBSD: 488, OpenBSD: 188, NetBSD: 167, Solaris: 532.
But... that chart above only provides part of the picture, as it includes all CVEs ever reported.
Therefore, while it is a fascinating glimpse into past (and overall) vulnerability, it does not give us a good indicator of the current security of each given OS.
To solve that, let's look at a singular recent major version of each OS. While the versions below are not all of the same age, each was chosen as: 1) a recent release, 2) publicly available for enough time to be somewhat well tested, and 3) with sufficient data available to be worth evaluating.
The results clearly show Android as the Operating System with the largest number of known vulnerabilities (currently).
- iOS (for iPhone) has roughly 10% of the reported CVEs as Android. Or, to put another way, "1,000% more secure".
- Both Windows and macOS clock in as measurably more secure than Ubuntu (in terms of total number of vulnerabilities found).
The old narrative that "Linux is more secure" appears to be... mostly untrue.
That said, it's entirely possible that the Open Source nature of Linux (and the software ecosystem around it) has enabled a higher percentage of vulnerabilities to be found, compared to Closed Source systems. But that is purely speculative, and we need to go on what data we have available.
No matter which way you slice it -- modern versions of major Linux Distributions have significantly more known vulnerabilities than modern versions of Windows or macOS.
The Findings
We can safely declare, based on available data, the following:
Q: Is software less secure now... than it used to be?
A: Yes. Demonstrably so. And it's getting worse, year on year.
Q: Which Operating System has the most vulnerabilities... and which has the least?
A: Linux based systems contain the most reported vulnerabilities, with Android (Linux-based) leading the pack by a large margin. Windows, macOS, iOS (and most BSD / UNIX systems) all have significantly fewer known vulnerabilities.
Q: Which has more vulnerabilities... Open Source Software or Closed Source Software?
A: This is a mixed bag. Open Source BSD systems have significantly less known vulnerabilities (both in total, and per version) than the Closed Source Microsoft Windows. At the same time, Open Source Linux (and Android) led the pack in vulnerabilities. One thing we can say for sure: The most vulnerable systems are Open Source (to one degree or another).
I don't like these numbers any more than you do. Don't shoot the messenger.
