I’ve figured out why software users aren’t requesting SBOMs

I realized about a year ago that the reason why SBOMs aren’t being used by non-developers for vulnerability management purposes – even though the developers themselves are using them very heavily for those purposes, as shown previously – isn’t that most developers don’t want to supply them. Rather, it’s because the users aren’t asking for them.

I have until recently blamed that reticence on the fact that there are few low-cost or open source tools to “process” SBOMS (i.e. ingest them and look up component vulnerabilities in the NVD or another vulnerability database) available for end users; moreover, there’s only one tool (at any price point), Dependency-Track, that ingests SBOMs and VEX documents, and these tools have to do both. Of course, the whole point of creating a machine-readable SBOM (as opposed to just a listing of components in a PDF) is that…strange but true!…it needs to be read by a machine. I can attest that SBOMs in JSON or XML can be read easily by someone with a little knowledge of English, but there’s still no reason to create such an SBOM if a machine isn’t going to read it.

And when you get to SBOMs with hundreds or thousands of components (the average software product now has about 150 third-party components, but some have thousands or even tens of thousands), there’s no question that you need to process it using a tool. That is, you probably have better things to do with your time than print out an SBOM and look up 150 components in the National Vulnerability Database (NVD) using “manual” searches.

But just having a tool that can “process” SBOMs isn’t enough. The end user should look not just for component vulnerabilities, but for the small subset of those that are exploitable in the product itself; those are the ones that pose a problem. VEX documents are required to separate the exploitable wheat from the unexploitable chaff. Since at least 90% of component vulnerabilities aren’t exploitable, an SBOM consumption tool that doesn’t also read VEXes is probably less than useful.

When Dependency-Track recently started supporting VEXes as well as SBOMs (it’s supported SBOMs since 2012, long before almost anybody else got interested in them), I happily announced that fact. And since D-T is open source and therefore free, I wondered if this development might at least get a few more end users interested in getting SBOMs from their suppliers. The result was (drumroll, please)…

Thunderous silence. Of course, when Dependency-Track (which was developed in 2012 by Steve Springett, co-leader of the CycloneDX SBOM format project) is downloaded, the person who did that doesn’t fill out a questionnaire saying what they’re going to use it for. However, Steve developed D-T for use by developers, and Steve says that what he hears from users is that they’re overwhelmingly developers.

To be honest, just changing some wording and some screens would make D-T a perfectly usable consumer product, but Steve has never had requests for that. He knows of a small number of non-developers who are using D-T to track vulnerabilities in products they use, but he doesn’t believe that number is growing.

This isn’t to say that, if someone wants to make the investment to create a consumer-oriented product that “processes” both SBOMs and VEXes, and make it available at low cost, there won’t be takers for it. Until recently, I thought that was all it would take to make people start using SBOMs.

However, just asking myself a question recently – and answering it with some simple math – made it clear to me that the lack of a low-cost, end-user-friendly tool isn’t at all what’s holding back the use of SBOMs.

The (rather long) question I asked myself was, “Let’s assume there’s a low-cost, end-user-friendly SBOM/VEX processing tool, and that a user feeds an SBOM into it. The tool looks up every component listed in the SBOM in the NVD. For every successful lookup, the user receives a list of any open vulnerabilities in the component. They then use any available VEX from the supplier, for the product and version that’s the subject of the SBOM, to determine which of these vulnerabilities are not exploitable. They remove them from the list of open vulnerabilities, ultimately yielding a list of only exploitable vulnerabilities.

“Given this, what is the probability that the user will a) learn about every exploitable component vulnerability in the product, but also b) know exactly which of those vulnerabilities are exploitable or not exploitable?” We will call these the two target probabilities.

There are three probabilities that need to be considered, to answer this question:

1. The probability that the user will be able to find each component in the NVD when they look it up, and learn of any vulnerabilities (CVEs) that apply to the component;
2. The probability that a vulnerability identified for a component will be exploitable; and
3. The probability that there will be VEX documents available from the supplier, which will identify all non-exploitable component vulnerabilities. This will leave just the exploitable ones.

The first probability is based on the “naming problem”. This is the problem that the informal group I lead, the SBOM Forum, addressed in a recent paper; I discussed it in this post. Briefly, the aspect of the naming problem that we addressed can be stated as, “When a user looks up the components found in a newly-created SBOM in the NVD, they will find only a small fraction of them, unless they try other manual processes.” These processes include AI, fuzzy logic, references to other databases, etc. The point is that a purely automated process will only find a small fraction of the vulnerabilities in the NVD (six reasons for this are described in the paper); the person interpreting the SBOM will need to use these other processes, and mix in a good measure of dumb luck, in order to learn about any more than a small fraction of the vulnerabilities that apply to the product.

And what is that small fraction? Our paper quotes the Director of Product Security of a very large US-based software developer, who said that they can find fewer than 20% of the components that appear in one of their SBOMs (they’re creating SBOMs regularly for all their products, but they’re not distributing them now). However, in a recent discussion in one of our meetings, this person admitted that the real number, in their experience, is less than 10%, and maybe even less than 5%. Another participant in the meeting – who works for a very large US-based producer of both software and intelligent devices – said their experience shows that the number is under 5%.

What this means is that, even if an end-user has a tool that ingests SBOMs and looks up all the components in the NVD, they will probably learn about vulnerabilities in fewer than 5% of the components. Of course, the SBOM Forum’s proposal aims to raise that percentage far higher – and it will ultimately do that. But I estimate that it will be one to two years before our proposal is fully implemented (I’m optimistic that it will be implemented, based on the reception it’s received so far).

Even after it’s implemented, it will mostly affect new vulnerability reports, not existing ones. So, it could be a number of years before we’ll be able to state that the percentage of successful NVD component searches is over say 70% (it will never be 100%, of course). The bottom line is that, for at least the next couple of years, the first probability above is no more than 10% and more likely below 5%.

I’ve already discussed the second probability, the percentage of component vulnerabilities that are exploitable, in previous blog posts. I normally use the figure of 90%, but most people I talk to believe it’s over 95%. In other words, once a user has looked up whatever components they can find in the NVD and listed all the vulnerabilities identified for those components, at least nine of every ten will be non-exploitable. This might seem like good news until you consider that you don’t know which ones aren’t exploitable – so you have to treat them all as exploitable for the time being.

The third percentage, the probability that the supplier will produce enough VEX documents to identify every non-exploitable component vulnerability, is zero currently since no supplier is producing VEXes, period (and BTW, don’t point to a vulnerability report and tell me that’s a VEX, even if it’s in the CSAF format). Will it become non-zero later?

I have recently come to believe there’s a good likelihood that VEX documents will never be produced, for various reasons I’ve discussed already and will in the future. This is why I’m now advocating for the idea of real-time VEX, although I’m also still working with the VEX committee as it makes slow progress toward producing a VEX playbook – which is needed no matter whether the VEX document or real-time VEX emerges as the winner. VEX information is required for SBOMs to be widely used, no matter how it ultimately gets distributed. And SBOMs will be widely used.

So, what’s my estimate for the third probability? It’s zero now, but in a couple of years, it will be nonzero. However, I want to remind you that the only way a user will be sure that the list of component vulnerabilities they have for a product only contains exploitable vulnerabilities is if they can be sure that the supplier has notified them of every non-exploitable component vulnerability in the product and version in question – that is, they have informed them which nine of every ten component vulnerabilities aren’t exploitable.

Now that we know these three probabilities, what about the two target probabilities we want to derive from them? These are the probabilities that the user will a) learn about every component vulnerability in the product, and b) learn exactly which of those vulnerabilities are exploitable or not exploitable.

Finding a) is easy, since it’s just the first probability listed above: the probability that the user will find every component in the NVD. We’ve already said that’s (conservatively) 10%. But finding b) is harder, since this requires receiving a VEX for each of the non-exploitable vulnerabilities in the product. So, if there are a total of ten vulnerabilities but only one is exploitable, the user will need to receive a VEX for each of the other nine vulnerabilities, saying it’s not exploitable (remember, the user initially needs to assume that every component vulnerability they identify is exploitable until they receive a VEX stating otherwise).

Let’s say the user just receives seven VEX documents, each stating that one of the ten component vulnerabilities isn’t exploitable. Does the fact that only seven were received mean the remaining three vulnerabilities are exploitable? The supplier could always later end the uncertainty by sending another VEX, which states that the three are exploitable. However, would that be a good idea, especially if the supplier doesn’t have a patch for the vulnerabilities yet? It definitely wouldn’t be a good idea. Only if the supplier sends out a VEX stating that the vulnerabilities have all been patched, which includes the URL(s) of the patch(es), can the user be sure the remaining three vulnerabilities are exploitable in that product and version.[i]

Unfortunately, there’s no assurance that the supplier will always make sure to close the loop and inform their customers of the status of every vulnerability that the NVD lists for a component of a particular version of their product. Thus, it’s very possible that a customer will waste time (theirs and the supplier’s) by contacting the supplier regarding non-exploitable vulnerabilities. This is one of the reasons why I now think real-time VEX is a better option than VEX documents.

So what is the value of target probability b)? As I said earlier, currently it’s zero, since no supplier is currently producing VEXes. However, I do think that the idea of a VEX API will ultimately be adopted. This will allow the customer to be certain whether or not each component’s vulnerability is exploitable. But, since we’re concerned with the next year or so at the moment, I can confidently say that b) is a very low percentage, and currently, it’s zero percent.

Finally, we’re able to answer the big question: Why are so few users interested in receiving SBOMs for use in software vulnerability management – even though there’s one open-source tool available that could help them in this job? There are at least two reasons:

1. They have heard that it’s difficult to look up components from an SBOM in the NVD. An estimated maximum ten percent success rate doesn’t inspire them to try doing that, either.
2. They know that, at least in the near term, they won’t be able to learn which component vulnerabilities are exploitable in the full product, without calling up the supplier’s help desk and running down their entire list of component vulnerabilities with the unfortunate person who answers their call, forcing them to find out whether each of those vulnerabilities is exploitable or not. Of course, this is one of the main reasons why suppliers aren’t keen on distributing SBOMs to their customers now; they suspect their help desks will be overwhelmed with calls about false positives.

You may notice a seeming contradiction in this post: I first talked about the fact that SOBMs have been hugely successful with developers, but now I’m implying they’re lying dead at the starting gate, as far as end users are concerned. Clearly, the developers know something that the users don’t. Can they share this?

As far as the first reason, the naming problem goes, the suppliers definitely have knowledge. The naming problem has been known since the beginning of the NVD, but SBOMs have turned it from an annoyance into a crisis, because of the huge number of components that now need to be looked up. Developers and consultants have developed a whole array of tricks – including AI and fuzzy logic routines, databases that shed light on different aspects of software names, etc. – to get around the problem.

Since these tricks are very ad hoc, they can’t be shared as part of a tool. However, developers could apply them on behalf of their customers, so that the SBOMs they send out have valid identifiers (“CPE names”) that allow customers to look up components in the NVD (meaning the customers would probably still end up looking up the vulnerabilities in the NVD manually). But, more likely, the suppliers will just outsource the whole “SBOM processing” activity to third-party service providers

The service providers will provide an “SBOM service” for end users. This service will take in SBOMs, as well as either VEX documents or real-time VEX information (whichever is available), and produce for their customers a list of exploitable component vulnerabilities in a product/version that is used by the customer. The customers won’t need even to look at an SBOM or VEX document, nor should they need to look up anything in the NVD on their own. They’ll receive what’s really the holy grail of SBOMs: continuously-updated lists of exploitable vulnerabilities in the products/versions in use by the user organization.

This is why I now think that, at least for the next few years, almost all “SBOM processing” for end users will be performed by third parties, not by standalone tools operated by the users themselves. My guess is the users will be quite happy with that arrangement.

Any opinions expressed in this blog post are strictly mine and are not necessarily shared by any of the clients of Tom Alrich LLC. If you would like to comment on what you have read here, I would love to hear from you. Please email me at [email protected].

[i] The fact that a patch has been released for a vulnerable version of a product doesn’t mean the status of the vulnerability changes in that version. Best practice dictates that application of the patch increments the version number of the instance of the product that’s patched. So the vulnerability will be fixed in the patched version but not in the unpatched version.

Tom Alrich

Tom Alrich is an independent consultant specializing in supply chain security of critical infrastructure. He is the founder and leader of the SBOM Forum, a group of SBOM professionals that is trying to remove obstacles currently preventing widespread use of SBOMs. He writes a popular blog on these topics.