Anyone should of course feel free to publish the results of their own benchmarking efforts! And indeed, we might do that ourselves at some point in the future. But before we do so, we'd like to work on a few other things, and we'd love your help. For example, we'd love to expand the number of security tools that are integrated into the CVE Benchmark through a tool driver. We'd also like to make doubly-sure that the existing drivers invoke their security tools in the right way. Last, but not least, we'd like to make sure that we don't upset anyone by publishing these results. Let's work together to build awesome security tools!
Please take a look at our Contributing guide. If you'd like to join the OpenSSF Security Tooling working group, please take a look here. If that's not what you're looking for, feel free to open an issue with your question.
Why is CVE-2020-99999 not in CVEs dataset?
Probably because no one has added it yet. If you'd like to add a CVE to the dataset, please open a pull request with data that satisfies the requirements of benchmark-CVEs.
Probably because no one has identified that yet. Please add it in a pull request, including data that satisfies the requirements of benchmark-CVEs.
Probably because it only is a partial fix. Please open a pull request or issue to discuss it.
One of the main advantages of using a very simple data format is to make it easy for anyone to contribute!
The purpose of this project is not to describe the weaknesses and alerts in detail — that is a job for the individual security analysis tools in their favorite UI environments. Our JSON data format does allow for external references (URLs) that can point to such explanations.
The purpose of this project is to facilitate easy comparison of the alerts from different benchmark tools, and a simple data format for weaknesses and alerts ensures everyone can contribute.
There ought to be! For information about adding support for a new analysis tool, see Analysis tools. We'd love to see a pull request!
It probably should. Please open a pull request that improves the driver for X. Thanks!
This section is a cookbook for selecting CVEs of interest. The
answers can be piped to command such as run and report like this:
bin/cli list ... | bin/cli run ... -.
$ bin/cli list '*'
$ bin/cli list CVE-123-456
$ bin/cli list ... > my-cves.txt
$ cat my-cves.txt | bin/cli run ... -
$ bin/cli list '*' | bin/cli filter --invert-match CVE-123-456
$ bin/cli list CWE-79 | bin/cli filter year:2019 year:2020
$ bin/cli list mitre-cwe-top:25:2020 | bin/cli filter year:2019 year:2020
That could be a very good idea! Please open an issue to discuss this further, and hopefully we can collaborate on a pull request!
Reports generally rely on the simple heuristic that an alert on the same line as a known weakness is a true positive. If a driver uses a security rule that happens to result in an alert on a line with a weakness, even though the rule technically is irrelevant for the CVE in question, then that is erroneously regarded as a true positive. The driver should only make use of relevant alerts.