[Technical Initiative Funding Request]: OpenSSF Scorecard Security Audit

[justaugustus]

Technical Initiative

OpenSSF Scorecard

Lifecycle Phase

Incubation

Funding amount

[requires Staff/TAC input on historical audit costs]

Problem Statement

Outside of the general importance of security audits, they are also required for project graduation.

Who does this affect?

Consumers, contributors, and maintainers of OpenSSF Scorecard and its associated subprojects

Have there been previous attempts to resolve the problem?

N/A

Why should it be tackled now and by this TI?

OpenSSF Scorecard is interested in moving towards the graduation phase in 2025 and this request is being submitted in time for the first TI funding review round in order to better contextualize roadmap and maintainer priorities for 2025.

Give an idea of what is required to make the funding initiative happen

• Meeting with OpenSSF Maintainers
• Engage with audit providers to estimate cost
• Security Audit
• Security Report

What is going to be needed to deliver this funding initiative?

• Resources to deliver a security audit
• Money to pay them

Are there tools or tech that still need to be produced to facilitate the funding initiative?

No response

Give a summary of the requirements that contextualize the costs of the funding initiative

Refer to previous security audits for cost contextualization.

Who is responsible for doing the work of this funding initiative?

Stephen Augustus (@justaugustus)

Who is accountable for doing the work of this funding initiative?

Stephen Augustus (@justaugustus)

If the responsible or accountable parties are no longer available, what is the backup contact or plan?

Spencer Schrock (@spencerschrock)

What license is this funding initiative being used under?

Apache 2.0

Code of Conduct

• I agree to follow the OpenSSF's Code of Conduct

List the major milestones by date and identify the overall timeline within which the technical initiative plans to accomplish their goals. Any payments for services, sponsorships, etc., will require LF Legal and Financial review.

There are no major milestones to reference, but as a general timeline:

• Before end of Q1 2025:
  • identify and contract a security auditor
  • begin engagement
• Q2:
  • Complete audit
  • Digest audit findings
  • Build roadmap to address audit findings
  • Start addressing audit findings
• Q3/Q4:
  • Finish addressing audit findings
  • Target graduation stage for OpenSSF Scorecard

If this is a request for funding to issue a contract, then OpenSSF will issue that contract. Please provide a Statement of Work (SOW) that we may review. Any contracting action will take 4-6 weeks to issue.

It would be helpful to review prior art and work with @GeauxJD and @ossf/scorecard-maintainers to build an SOW based on that.

[riaankleinhans]

@Amir-Montazery for your information

[Amir-Montazery]

Thank you for the heads up. I will begin engaging teams to get a cost and time estimate. In the meantime, some basic information would be really helpful! Would it be possible to fill out this brief questionnaire form? Link: https://docs.google.com/document/d/1y1sKwjMO5CegdR69bGxCjXcP1dngzLxtd5s0eJoAeUU/

[justaugustus]

Can do, @Amir-Montazery!

[steiza]

Do we have a sense for what's in scope vs out of scope for the audit? My understanding of Scorecard is that there are many ways people interact with Scorecard: the web viewer, CLI binaries, a Docker container, a GitHub Action, and a REST API (did I miss any?)

We've heard from the community that the larger number of ways to interact with Scorecard is confusing. Would it makes sense for the project to be more opinionated about how people use it, deprecating some of these existing methods, and updated the website, README, and other documentation before doing the security audit? This would help scope the security audit, make the project less confusing externally, and be a good step of maturity as Scorecard works towards the graduation phase.

[justaugustus]

Thanks @steiza! I've captured that feedback as part of the Best Practices WG 2024 Q4 TAC update.

I believe that feedback is orthogonal to approving and scoping a security audit.

Using sigstore, an OpenSSF Graduated project, as a comparison:

• binaries are included as part of cosign releases
• container images are provided
• as well as a GitHub Action

cosign verify also provides a means of verifying signatures, which queries against a public API.

tl;dr — I believe the project components you mentioned are well within the scope of a project looking to attain OpenSSF Graduated status.

I would agree that there is some work we (@ossf/scorecard-maintainers) should do ahead of kicking off a security audit.
It is my opinion that that would affect the start date of an audit, but is non-blocking for TAC approval.

[lehors]

Strong yes from me!

[gkunz]

+1 to this request.

[steiza]

Thanks @justaugustus, it sounds like at this time we don't have a sense of the scope of the security audit, or maybe by default it would cover all the ways that people currently produce or consume Scorecard content?

The comparison to Sigstore is an interesting one - I gave similar feedback to the Sigstore Technical Steering Committee, and they are working on delineating what the paved path of interacting with Sigstore is ("core") vs the experiments or works in progress ("community"). There are also plans to deprecate cosign verify querying Rekor directly during verification.

[steiza]

Oh! And to be clear, I'm not saying we need to finish the work of having a primary way people produce or consume Scorecard content. But I do think we should decide what those methods are sooner than later - to help streamline onboarding people to the Scorecard project as well as help scope the security audit.

[justaugustus]

Thanks for the clarification (and additional references from sigstore), @steiza !

I'll keep these in mind as I work through the audit questionnaire and come back with any thoughts on scope.

[haydentherapper]

@justaugustus, speaking for the Sigstore audit done by OSTIF back in 2022, we focused on a code review of the (at the time) core components, namely Cosign, Fulcio and Rekor. There were other repositories we had (like policy-controller) that weren't audited. Since then, we've had a lot of new repositories added to the org, some of which we'd welcome an audit of and some where an audit would not add as much value. My two cents would be to create a rough threat model for all of Scorecard's clients and infrastructure (CI/CD pipeline, API service, viewer) and prioritize based on impact to Scorecard and its consumers.

[scovetta]

I'll wait on the +1 until we get the actual cost, but I'm supportive of this.

[Amir-Montazery]

+1 to @haydentherapper's suggestion of a rough threat model. Our audit will include threat modeling and analysis, so starting with an internal threat model and then building on it with the audit team (we have a good candidate in mind that has extensive familiarity with the scorecard project). This will help with prioritization as mentioned and hone in on focus areas to maximize the value of the audit.

[riaankleinhans]

/vote

[git-vote]

Vote created

@riaankleinhans has called for a vote on [Technical Initiative Funding Request]: OpenSSF Scorecard Security Audit (#449).

The members of the following teams have binding votes:

Team
@ossf/tac

Non-binding votes are also appreciated as a sign of support!

How to vote

You can cast your vote by reacting to this comment. The following reactions are supported:

In favor	Against	Abstain
👍	👎	👀

Please note that voting for multiple options is not allowed and those votes won't be counted.

The vote will be open for 1month 11days 13h 26m 24s. It will pass if at least 70% of the users with binding votes vote In favor 👍. Once it's closed, results will be published here as a new comment.