We welcome contributions to the OpenSSF CVE Benchmark project. If you want to contribute benchmark data, analysis tool drivers, or report generators, please go ahead and open a pull request!
Contributions to this project are released to the public under the project's open source MIT license.
Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms.
The metadata for CVEs used for benchmarking purposes is located in the CVEs directory. The benchmark CVEs are maintained by the community and additions and changes are welcome as pull requests.
To add a CVE to the benchmark, please provide details of:
- One vulnerable and one patched commit in a real codebase
- At least one weakness for the vulnerable commit (a line of code on which you'd expect a security tool to flag up an alert)
For more information, see Benchmark CVEs.
The ossf-cve-benchmark repository contains two source code directories:
-
src/: contains the core tooling, and is primarily maintained by a core team of developers
- src/types.ts: contains the types that defines the JSON schemas used for validation of serialized data. Changes to this file should be accompanied by a run of bin/update-json-schema.sh.
-
contrib/: contains utility scripts, such as the analysis tool drivers and report generators.
If you're not sure whether a particular contribution is useful, feel free to open an issue with your proposal.
Analysis tool drivers are small scripts that run an actual static application security analysis (or SAST)
tool when invoked by bin/cli run .... The driver helps decide whether an analysis tool has (1) detected the CVE and has (2) recognized the patch.
To add support for a new driver, add a new directory to the contrib/tools, and read the requirements for the drivers in the docs.
Report generators process the CVE data and analysis tool results into
different kinds of reports when invoked through bin/cli report ....
To add support for a new report generator, add a new directory to the contrib/reports with the implementation. Adjust the implementation of src/cli.ts to support the new report generator.
These types of contributions are generally best discussed in advance by opening an issue.
This project uses GitHub Actions for CI, see
.github/workflows. For a slightly faster
turnaround time, it's recommended that you run npm run-script all-checks
locally before pushing.
The following analysis tools are run as CI, new alerts from these must be considered:
codeql:+security-and-qualityfor JavaScript/TypeScript- (CI only)
OSSAR: default configuration- (CI only)
prettier: see .prettierrc.json, .prettierignore.- Can also be run locally as
npm run-script lintornpx prettier -c . - Run
npx prettier -w .locally to fix problems
- Can also be run locally as
typescript-eslint: see .eslintrc.js, .eslintignore.- Can also be run locally as
npm run-script lintornpx eslint .
- Can also be run locally as
This project uses jest for testing.
Failures from these tests must be addressed. Use npm run-script test
or npx jest to run the tests locally.
This project uses SemVer for the JSON schemas and the bin/cli interface. In time, this is expected to extend to the implementation in src. The contrib/ directory is not expected to contribute to this repository's versioning. it's up to the individual directories in the contrib/ directory to maintain a versioning strategy.
Breaking changes (for example, major version bumps) for the JSON schemas and the associated data files, such as those in CVEs, are expected to be rare. Whenever a breaking change does take place, we'll provide an upgrade script to convert old data to the correct format.
This versioning strategy for the data and CLI enables external scripts and drivers to work reliably in the presence of version changes.