scorecard Documentation

OpenSSF Scorecard - Security health metrics for Open Source

5,769 stars Go Go 22 files Β· ~67,414 tokens #openssf-scorecard#scorecard
RAW Doc

Docker & Self-Hosted Deployment Verified OCI

Deploy scorecard on any server, homelab, or cloud container platform.

1-Click Terminal Command
bash (docker run)
docker run -d --name scorecard --restart always ossf/scorecard:latest

Technical Documentation: ossf/scorecard

ℹ️ Provenance: Multi-Tier Ingestion Β· CodeWiki Reference

Overview (`README.md`)

OpenSSF Scorecard









Overview

Using Scorecard

Checks

Other Important Recommendations

Scoring

Contribute

FAQ

Overview

What is Scorecard?

We created Scorecard to help open source maintainers improve their security
best practices and to help open source consumers judge whether their dependencies
are safe.

Scorecard is an automated tool that assesses a number of important heuristics
("checks") associated with software security and assigns
each check a score of 0-10. You can use these scores to understand specific
areas to improve in order to strengthen the security posture of your project.
You can also assess the risks that dependencies introduce, and make informed
decisions about accepting these risks, evaluating alternative solutions, or
working with the maintainers to make improvements.

The inspiration for Scorecard’s logo:
"You passed! All D's ... and an A!"

Project Goals

  1. Automate analysis and trust decisions on the security posture of open source
    projects.

  2. Use this data to proactively improve the security posture of the critical
    projects the world depends on.

  3. Act as a measurement tool for existing policies

    If OSS consumers require certain behaviors from their dependencies,
    Scorecard can be used to measure those. With the V5 release, we see
    Structured Results as a way of doing this if there is a supported analysis.
    Instead of relying on an aggregate score of X/10, or a Maintained score of
    Y/10, an OSS consumer may want to ensure the repo they're depending on
    isn't archived (which is covered by the archived probe). The OpenSSF
    takes this approach with its own Security Baseline for projects.

Project Non-Goals

  1. To be a definitive report or requirement that all projects should follow.

    Scorecard is not intended to be a one-size-fits-all solution. Every step of
    making our results is opinionated: what checks get included or excluded,
    the importance of each check, and how scores are calculated. The checks
    themselves are heuristics; there are false positives and false negatives.

    Whether it’s due to applicability, or feasibility, or a matter of opinion,
    what's included or excluded from Scorecard results leads to a lot of
    discussion. It’s impossible to create a Scorecard that satisfies everyone
    because different audiences will care about different subsets of behavior.

    Aggregate scores in particular tells you nothing about what individual
    behaviors a repository is or is not doing. Many check scores are aggregated
    into a single score, and there’s multiple ways of arriving at the same
    score. These scores change as we add new heuristics or refine the existing
    ones.

Prominent Scorecard Users

Scorecard has been run on thousands of projects to monitor and track security
metrics. Prominent projects that use Scorecard include:

View a Project's Score

To see scores for projects regularly scanned by Scorecard, navigate to the webviewer. You can also replace the placeholder text (platform, user/org, and repository name) in the following template link to generate a custom Scorecard link for a repo:
https://scorecard.dev/viewer/?uri=<github_or_gitlab>.com//<repository_name>

For example:

To view scores for projects not included in the webviewer, use the Scorecard CLI.

Public Data

Scorecard scans more than 1.3 million repositories every week, from
ossf/scorecard-infra. Results are
available through the REST API. The BigQuery public
dataset is no longer available; see
Scorecard Infrastructure Changes
for details.

To add a project to the weekly scan, follow the instructions in
ossf/scorecard-infra's contributing guide.

Using Scorecard

Scorecard GitHub Action

The easiest way to use Scorecard on GitHub projects you own is with the
Scorecard GitHub Action. The Action
runs on any repository change and issues alerts that maintainers can view in the
repository’s Security tab. For more information, see the Scorecard GitHub
Action
installation instructions.

Scorecard REST API

To query pre-calculated scores of OSS projects, use the REST API.
Scores calculated from our weekly scan omit the CI-Tests,
Contributors, and Dependency-Update-Tool checks due to the API costs
associated with running them at scale.

API results are cached with a CDN (thanks to Fastly and their Fast Forward program). Results are purged from the CDN when new results are available, but if you notice issues with stale data, please open an issue.

To enable your project to be available on the REST API, set
publish_results: true
in the Scorecard GitHub Action setting.

Data provided by the REST API is licensed under the CDLA Permissive 2.0.

Scorecard Badges

Enabling publish_results: true
in Scorecard GitHub Actions also allows maintainers to display a Scorecard badge on their repository to show off their
hard work. This badge also auto-updates for every change made to the repository. See more details on this OSSF blogpost.

To include a badge on your project's repository, simply add the following markdown to your README:

text
[](https://scorecard.dev/viewer/?uri=github.com/{owner}/{repo})

Scorecard Command Line Interface

To run a Scorecard scan on projects you do not own, use the command line
interface installation option.

Prerequisites

Platforms: Currently, Scorecard supports OSX and Linux platforms. If you are
using a Windows OS you may experience issues. Contributions towards supporting
Windows are welcome.

Language: You must have GoLang installed to run Scorecard
(https://golang.org/doc/install)

Installation

Docker

scorecard is available as a Docker container:

shell
docker pull ghcr.io/ossf/scorecard:latest

To use a specific scorecard version (e.g., v3.2.1), run:

shell
docker pull ghcr.io/ossf/scorecard:v3.2.1

Standalone

To install Scorecard as a standalone:

Visit our latest release page and
download the correct zip file for your operating system.

Add the binary to your GOPATH/bin directory (use go env GOPATH to identify your directory if necessary).

Verifying SLSA provenance for downloaded releases

We generate SLSA3 signatures using the OpenSSF's slsa-framework/slsa-github-generator during the release process. To verify a release binary:

  1. Install the verification tool from slsa-framework/slsa-verifier#installation.
  2. Download the provenance file multiple.intoto.jsonl from the same GitHub release as your archive.
  3. Run the verifier:
shell
slsa-verifier verify-artifact --provenance-path multiple.intoto.jsonl --source-uri github.com/ossf/scorecard --source-tag <the-tag> <the-archive>

Using package managers

Package Manager Supported Distribution Command
Nix NixOS nix-shell -p nixpkgs.scorecard
AUR helper Arch Linux Use your AUR helper to install scorecard
Homebrew macOS or Linux brew install scorecard

Authentication

GitHub imposes api rate limits
on unauthenticated requests. To avoid these limits, you must authenticate your
requests before running Scorecard. There are two ways to authenticate your
requests: either create a GitHub personal access token, or create a GitHub App
Installation.

  • Create a classic GitHub personal access token.
    When creating the personal access token, we suggest you choose the
    public_repo scope. Set the token in an environment variable called
    GITHUB_AUTH_TOKEN, GITHUB_TOKEN, GH_AUTH_TOKEN or GH_TOKEN using the
    commands below according to your platform.
shell
# For posix platforms, e.g. linux, mac:
export GITHUB_AUTH_TOKEN=<your access token>
# Multiple tokens can be provided separated by comma to be utilized
# in a round robin fashion.
export GITHUB_AUTH_TOKEN=<your access token1>,<your access token2>

# For windows:
set GITHUB_AUTH_TOKEN=<your access token>
set GITHUB_AUTH_TOKEN=<your access token1>,<your access token2>

OR

  • Create a GitHub App Installation
    for higher rate-limit quotas. If you have an installed GitHub App and key
    file, you can use the three environment variables below, following the
    commands (set or export) shown above for your platform.
text
GITHUB_APP_KEY_PATH=<path to the key file on disk>
GITHUB_APP_INSTALLATION_ID=<installation id>
GITHUB_APP_ID=<app id>

These variables can be obtained from the GitHub
developer settings page.

Basic Usage

Using repository URL

Scorecard can run using just one argument, the URL of the target repo:

shell
scorecard --repo=github.com/ossf-tests/scorecard-check-branch-protection-e2e
text
/* Detailed source-code truncated for AI context efficiency. */

Docker

The GITHUB_AUTH_TOKEN has to be set to a valid token

shell
docker run -e GITHUB_AUTH_TOKEN=token ghcr.io/ossf/scorecard:latest --show-details --repo=https://github.com/ossf/scorecard

To use a specific scorecard version (e.g., v3.2.1), run:

shell
docker run -e GITHUB_AUTH_TOKEN=token ghcr.io/ossf/scorecard:v3.2.1 --show-details --repo=https://github.com/ossf/scorecard

Showing Detailed Results

For more details about why a check fails, use the --show-details option:

text
./scorecard --repo=github.com/ossf-tests/scorecard-check-branch-protection-e2e --checks Branch-Protection --show-details
text
/* Detailed source-code truncated for AI context efficiency. */

Showing Maintainers Annotations

Maintainer Annotations let maintainers add context to display alongside Scorecard check results. Annotations can provide users additional information when Scorecard has an incomplete assessment of a project's security practices. To see the maintainers annotations for each check, use the --show-annotations option.

For more information on available annotations or how to make annotations, see the configuration doc.

Using a GitLab Repository

To run Scorecard on a GitLab repository, you must create a GitLab Access Token with the following permissions:

  • read_api
  • read_user
  • read_repository

You can run Scorecard on a GitLab repository by setting the GITLAB_AUTH_TOKEN environment variable:

bash
export GITLAB_AUTH_TOKEN=glpat-xxxx

scorecard --repo gitlab.com/<org>/<project>/<subproject>

For an example of using Scorecard in GitLab CI/CD, see here.

Self Hosted Editions

While we focus on GitLab.com support, Scorecard also works with self-hosted GitLab installations.
If your platform is hosted at a subdomain (e.g. gitlab.foo.com), Scorecard should work out of the box.
If your platform is hosted at some slug (e.g. foo.com/bar/), you will need to set the GL_HOST environment variable.

bash
export GITLAB_AUTH_TOKEN=glpat-xxxx
export GL_HOST=foo.com/bar
scorecard --repo foo.com/bar/<org>/<project>

Using GitHub Enterprise Server (GHES) based Repository

To use a GitHub Enterprise host github.corp.com, use the GH_HOST environment variable.

shell
# Set the GitHub Enterprise host without https prefix or slash with relevant authentication token
export GH_HOST=github.corp.com
export GITHUB_AUTH_TOKEN=token

scorecard --repo=github.corp.com/org/repo
# OR without github host url
scorecard --repo=org/repo

Using a Package manager

For projects in the --npm, --pypi, --rubygems, or --nuget ecosystems, you have the
option to run Scorecard using a package manager. Provide the package name to
run the checks on the corresponding GitHub source code.

For example, --npm=angular.

Note: The package ecosystem flags are to find a GitHub repo only.
These flags do not change the final evaluation for the checks.

Additionally, the flags cannot be used with --repo.

Running specific checks

To run only specific check(s), add the --checks argument with a list of check
names.

For example, --checks=CI-Tests,Code-Review.

Formatting Results

The currently supported formats are default (text) and json.

These may be specified with the --format flag. For example, --format=json.

Checks

Scorecard Checks

The following checks are all run against the target project by default:

Name Description Risk Level Token Required GitLab Support Note
Binary-Artifacts Is the project free of checked-in binaries? High PAT, GITHUB_TOKEN Supported
Branch-Protection Does the project use Branch Protection ? High PAT (repo or repo> public_repo), GITHUB_TOKEN Supported (see notes) certain settings are only supported with a maintainer PAT
CI-Tests Does the project run tests in CI, e.g. GitHub Actions, Prow? Low PAT, GITHUB_TOKEN Supported
CII-Best-Practices Has the project earned an OpenSSF (formerly CII) Best Practices Badge at the passing, silver, or gold level? Low PAT, GITHUB_TOKEN Validating
Code-Review Does the project practice code review before code is merged? High PAT, GITHUB_TOKEN Supported
Contributors Does the project have contributors from at least two different organizations? Low PAT, GITHUB_TOKEN Validating
Dangerous-Workflow Does the project avoid dangerous coding patterns in GitHub Action workflows? Critical PAT, GITHUB_TOKEN Unsupported
Dependency-Update-Tool Does the project use tools to help update its dependencies? High PAT, GITHUB_TOKEN Supported
Fuzzing Does the project use fuzzing tools, e.g. OSS-Fuzz, QuickCheck or fast-check? Medium PAT, GITHUB_TOKEN Validating
License Does the project declare a license? Low PAT, GITHUB_TOKEN Validating
Maintained Is the project at least 90 days old, and maintained? High PAT, GITHUB_TOKEN Validating
Pinned-Dependencies Does the project declare and pin dependencies? Medium PAT, GITHUB_TOKEN Validating
Packaging Does the project build and publish official packages from CI/CD, e.g. GitHub Publishing ? Medium PAT, GITHUB_TOKEN Validating
SAST Does the project use static code analysis tools, e.g. CodeQL, LGTM (deprecated), SonarCloud? Medium PAT, GITHUB_TOKEN Unsupported
Security-Policy Does the project contain a security policy? Medium PAT, GITHUB_TOKEN Validating
Signed-Releases Does the project cryptographically sign releases? High PAT, GITHUB_TOKEN Validating
Token-Permissions Does the project declare GitHub workflow tokens as read only? High PAT, GITHUB_TOKEN Unsupported
Vulnerabilities Does the project have unfixed vulnerabilities? Uses the OSV service. High PAT, GITHUB_TOKEN Validating
Webhooks Does the webhook defined in the repository have a token configured to authenticate the origins of requests? Critical maintainer PAT (admin: repo_hook or admin> read:repo_hook doc EXPERIMENTAL

Detailed Checks Documentation

To see detailed information about each check, its scoring criteria, and
remediation steps, check out the checks documentation page.

Beginner's Guide to Scorecard Checks

For a guide to the checks you should use when getting started, see the beginner's guide to scorecard checks.

Other Important Recommendations

Two-factor Authentication (2FA)

Two-factor Authentication (2FA) adds an extra layer of security when logging into websites or apps. 2FA protects your account if your password is compromised by requiring a second form of authentication, such as codes sent via SMS or authentication app, or touching a physical security key.

We strongly recommend that you enable 2FA on any important accounts where it is available. 2FA is not a Scorecard check because GitHub and GitLab do not make that data about user accounts public. Arguably, this data should always remain private, since accounts without 2FA are so vulnerable to attack.

Though it is not an official check, we urge all project maintainers to enable 2FA to protect their projects from compromise.

Enabling 2FA

For users

Follow the steps described at Configuring two-factor authentication

If possible, use either:

  • physical security key (preferred), such as Titan or Yubikey
  • recovery codes, stored in an access protected and encrypted vault

As a last option, use SMS. Beware: 2FA using SMS is vulnerable to SIM swap attack.

For an organization

  1. Prepare to require 2FA in your organization
  2. Require 2FA in your organization

Scoring

Aggregate Score

Each individual check returns a score of 0 to 10, with 10 representing the best
possible score. Scorecard also produces an aggregate score, which is a
weight-based average of the individual checks weighted by risk.

  • β€œCritical” risk checks are weighted at 10
  • β€œHigh” risk checks are weighted at 7.5
  • β€œMedium” risk checks are weighted at 5
  • β€œLow” risk checks are weighted at 2.5

See the list of current Scorecard checks for each check's
risk level.

Contribute

Report Problems

If you have what looks like a bug, please use the
GitHub issue tracking system. Before
you file an issue, please search existing issues to see if your issue is already
covered.

Contribute to Scorecard

Before contributing, please follow our Code of Conduct.

See the Contributing documentation for guidance on how to
contribute to the project.

Adding a Scorecard Check

If you'd like to add a check, please see guidance here.

Connect with the Scorecard Community

If you want to get involved in the Scorecard community or have ideas you'd like
to chat about, we discuss this project in the
OSSF Best Practices Working Group
meetings.

Artifact Link
Scorecard Dev Forum ossf-scorecard-dev@
Scorecard Announcements Forum ossf-scorecard-announce@
Community Meeting VC Link to z o o m meeting
Community Meeting Calendar APAC-friendly Biweekly on Thursdays at 1:00-2:00 PM Pacific (OSSF Public Calendar) Video Call: LFX Zoom EMEA-friendly Every 4 Mondays at 7:00-8:00 AM Pacific (OSSF Public Calendar) Video Call: LFX Zoom
Meeting Notes Notes
Slack Channel #scorecard

Maintainers are listed in the CODEOWNERS file.

Report a Security Issue

To report a security issue, please follow instructions here.

Join the Scorecard Project Meeting

Zoom

APAC-friendly Biweekly on Thursdays at 1:00-2:00 PM Pacific (OSSF Public Calendar)

Video Call: LFX z o o m

EMEA-friendly Every 4 Mondays at 7:00-8:00 AM Pacific (OSSF Public Calendar)

Video Call: LFX z o o m

Agenda

You can see the agenda and meeting notes here.

Stargazers over time

FAQ

FAQ

See the FAQ for answers to Frequently Asked Questions about Scorecard.


Overview (`attestor/README.md`)

Scorecard Attestor

What is scorecard-attestor?

scorecard-attestor is a tool that runs scorecard on a software source repo, and based on certain policies about those results, produces a Google Cloud binary authorization attestation.

scorecard-attestor helps users secure their software deployment systems by ensuring the code that they deploy passes certain criteria.

Building and using scorecard-attestor

scorecard-attestor can be built as a standalone binary from source using make build-attestor, or with Docker, using make build-attestor-docker. scorecard-attestor is intended to be used as part of a Google Cloud Build pipeline, and inherits environment variables based on build substitutions.

Unless there's an internal error, scorecard-attestor will always return a successful status code, but will only produce a binary authorization attestation if the policy check passes.

Configuring policies for scorecard-attestor

Policies for scorecard attestor can be passed through the CLI using the --policy flag. Examples of policies can be seen in attestor/policy/testdata.

Policies

  • PreventBinaryArtifacts: Ensure that a repository is free from binary artifacts, which can link against the final repo artifact but isn't reviewable.
    • AllowedBinaryArtifacts: A list of binary artifacts, by repo path, to ignore. If not specified, no binary artifacts will be allowed
  • PreventKnownVulnerabilities: Ensure that the project is free from security vulnerabilities/advisories, as registered in osv.dev.
  • PreventUnpinnedDependencies: Ensure that a project's dependencies are pinned by hash. Dependency pinning makes builds more predictable, and prevents the consumption of malicious package versions from a compromised upstream.
    • AllowedUnpinnedDependencies: Ignore some dependencies, either by the filepath of the dependency management file (filepath, e.g. requirements.txt or package.json) or the dependency name (packagename, the specific package being ignored). If multiple filepaths/names, or a combination of filepaths and names are specified, all of them will be used. If not specified, no unpinned dependencies will be allowed.
  • RequireCodeReviewed: Require that If CodeReviewRequirements is not specified, at least one reviewer will be required on all changesets. Scorecard-attestor inherits scorecard's default commit window (i.e. will only look at the last 30 commits to determine if they are reviewed or not).
    • CodeReviewRequirements.MinReviewers: The minimum number of distinct approvals required.
    • CodeReviewRequirements.RequiredApprovers: A set of approvers, any of whom must be found to have approved all changes. If a change is found without any approvals from this list, the check fails.

Policy schema

Policies follow the following schema:

yaml
---
type: "//rec"
optional:
    preventBinaryArtifacts: "//bool"
    allowedBinaryArtifacts:
        type: "//arr"
        contents: "//str" # Accepts glob-based filepaths as strings here
    ensureNoVulnerabilities: "//bool"
    ensureDependenciesPinned: "//bool"
    allowedUnpinnedDependencies:
        type: "//arr"
        contents:
            type: "//rec"
            optional:
                packagename: "//str"
                filepath: "//str"
                version: "//str"
    ensureCodeReviewed: "//bool"
    codeReviewRequirements:
        type: "//rec"
        optional:
            requiredApprovers:
                type: "//arr"
                contents: "//str"
            minReviewers: "//int"

Sample

Examples of how to use scorecard-attestor with binary authorization in your project can be found in these two repos:

Sample code comes with:

  • cloudbuild.yaml to build the application and run scorecard-attestor
  • Terraform files to set up the binary authorization environment, including KMS and IAM.

Overview (`config/README.md`)

Maintainer Annotations

Maintainer Annotations let maintainers add context to display alongside Scorecard check results. Annotations can provide users additional information when Scorecard has an incomplete assessment of a project's security practices.

Annotating Your Project

To annotate your repository, create a scorecard.yml file in the root of your repository.

You can also place your annotations in .scorecard.yml or .github/scorecard.yml.

The file structure is as follows:

yml
annotations:
  - checks:
      - binary-artifacts
    reasons:
      - reason: test-data # the binary files are only used for testing
  - checks:
      - dangerous-workflow
    reasons:
      - reason: remediated # the workflow is dangerous but only run under maintainers verification and approval

You can annotate multiple checks at a time:

yml
annotations:
  - checks:
      - binary-artifacts
      - pinned-dependencies
    reasons:
      - reason: test-data # the binary files and files with unpinned dependencies are only used for testing

And also provide multiple annotations for checks:

yml
annotations:
  - checks:
      - binary-artifacts
    reasons:
      - reason: test-data # test.exe is only used for testing 
      - reason: remediated # dependency.exe is needed and it's used but the binary signature is verified

The available checks are the Scorecard checks in lower case e.g. Binary-Artifacts is binary-artifacts.

Types of Annotations

The annotations are predefined as shown in the table below:

Annotation Description Example
test-data A check or probe has found a security issue in files or code snippets only used for test or example purposes. The binary files are only used for testing.
remediated To annotate when a check or probe has found a security issue to which a remediation was already applied. A workflow is dangerous but only run under maintainers verification and approval, or a binary is needed but it is signed or has provenance.
not-applicable To annotate when a check or probe is not applicable for the case. The dependencies should not be pinned because the project is a library.
not-supported To annotate when the maintainer fulfills a check or probe in a way that is not supported by Scorecard. Clang-Tidy is used as SAST tool but not identified because its not supported.
not-detected To annotate when the maintainer fulfills a check or probe in a way that is supported by Scorecard but not identified. Dependabot is configured in the repository settings and not in a file.

Viewing Maintainer Annotations

To see the maintainers annotations for each check on Scorecard results, use the --show-annotations option.


Overview (`cron/README.md`)

Batch scanning pipeline

This directory has moved to
ossf/scorecard-infra.
To add a repository to the weekly scan, follow the instructions in its
contributing guide.


Overview (`openspec/README.md`)

OpenSpec

This directory holds OpenSpec specifications for
Scorecard.

  • specs/ describes Scorecard v5 behavior as built, including known
    defects. A requirement that records a defect is not a compatibility
    promise; fixing it is an ordinary spec delta.
  • changes/ holds proposed changes. changes/osps-baseline-conformance/
    describes a v6 target in which one run produces both check scores and
    conformance results. In v5, running probes directly returns probe
    findings and skips checks.

Capability ownership

Each behavior is stated in one capability. Others refer to it by path
rather than restating it.

Capability Owns
aggregate-scoring Aggregate formula, weights, inconclusive exclusion
attestor Attestor commands, attestor policy, signing
check-evaluation Per-check score rules and shared evaluation behavior
check-registry Check inventory, metadata, selection, experimental gating
cli Command tree, flags, progress output, result ordering, exit status
http-server scorecard serve requests, statuses, CORS, lifecycle
maintainer-annotations Annotation discovery, parsing, applicability
package-manager-resolution npm, PyPI, RubyGems, and NuGet lookup
policy Policy file parsing, policy-driven selection, policy errors
probes Probe registration, direct probe runs, findings and outcomes
public-go-api Exported Run, its options, defaults, and errors
raw-data-collection Evidence gathered for checks and probes
remediation Remediation definitions and generated suggestions
repo-clients Remote and local repository access and host support
result-formats Output formats, schemas, destinations, score presentation

A flag is defined in cli; what it turns on is defined by the capability
that owns that behavior. A computed value is defined where it is computed;
how it is displayed is defined in result-formats.

Validation

sh
openspec validate --specs --strict

Overview (`probes/README.md`)

OpenSSF Scorecard Probes

This directory contains all the Scorecard probes. Each probe has its own subdirectory.

A probe is an individual heuristic, which provides information about a distinct behavior a project under analysis may or may not be doing.
The probes follow a camelcase naming convention that describe the exact heuristic a particular probe assesses.
The name should be phrased in a way that can be answered by "true" or "false".

Probes can return one or more findings, where a finding is a piece of data with an outcome, message, and optionally a location where the behavior was observed.
The primary outcomes are finding.OutcomeTrue and finding.OutcomeFalse, but other outcomes are available as well.
For example, the finding.OutcomeNotAvailable is often used for scenarios, where Scorecard cannot assess a behavior because there is no data to analyze.

A probe consists of three files:

  • def.yml: The documentation of the probe.
  • impl.go: The actual implementation of the probe.
  • impl_test.go: The probe's test.

Lifecycle

Probes can exist in several different lifecycle states:

  • Experimental: The semantics of the probe may change, and there are no stability guarantees.
  • Stable: The probe behavior and semantics will not change. There may be bug fixes as needed.
  • Deprecated: The probe is no longer supported and callers should not expect it to be maintained.

Reusing code in probes

When multiple probes use the same code, the reused code can be placed in a package under probes/internal/

How do I know which probes to add?

In general, browsing through the Scorecard GitHub issues is the best way to find new probes to add. Requests for support for new tools, fuzzing engines or other heuristics can often be converted into specific probes.

Probe definition formatting

Probe definitions can display links following standard markdown format.

Probe definitions can display dynamic content. This requires modifications in def.yml and impl.go and in the evaluation steps.

The following snippet in def.yml will display dynamic data provided by impl.go:

md
${{ metadata.dataToDisplay }}

And then in impl.go add the following metadata:

golang
f, err := finding.NewWith(fs, Probe,
	"Message", nil,
	finding.OutcomeTrue)
f = f.WithRemediationMetadata(map[string]string{
	"dataToDisplay": "this is the text we will display",
})

Example

Consider a probe with following line in its def.yml:

text
The project ${{ metadata.oss-fuzz-integration-status }} integrated into OSS-Fuzz.

and the probe sets the following metadata:

golang
f, err := finding.NewWith(fs, Probe,
	"Message", nil,
	finding.OutcomeTrue)
f = f.WithRemediationMetadata(map[string]string{
	"oss-fuzz-integration-status": "is",
})

The probe will then output the following text:

text
The project is integrated into OSS-Fuzz.

Should the changes be in the probe or the evaluation?

The remediation data must be set in the probe.


Overview (`docs/security/README.md`)

Scorecard threat model

This directory holds the threat model for the Scorecard scan engine, as two
Gemara v1.6.0 catalogs:

Both import matching entries from
FINOS Common Cloud Controls
Core instead of redefining them.

Scope

The catalogs cover the software this repository releases: the scorecard
CLI and container image, the pkg/scorecard Go API, scorecard serve, and
scorecard-attestor. The release pipeline and the hosted services in other
repositories are out of scope. ARCHITECTURE.md
describes the actors and the scan pipeline.

Keeping it current

Each capability ends with a Specs: line naming the
OpenSpec specs that define its behavior. When you
add or change a spec, check the capability that names it, and the threats
linked to that capability. Every spec should be named by at least one
capability.

Validate both catalogs against the pinned Gemara release:

sh
cue vet -c -d '#CapabilityCatalog' github.com/gemaraproj/[email protected] capability-catalog.yaml
cue vet -c -d '#ThreatCatalog' github.com/gemaraproj/[email protected] threat-catalog.yaml

cue checks each catalog's shape. It doesn't check that a threat's
capability references exist in the capability catalog, so check those by
hand.


Overview (`docs/checks/dependencyupdatetool/README.md`)

Supported Tools

  • Dependabot
    • Detection is based on a .github/dependabot.yml or .github/dependabot.yaml file.
  • Renovate
    • Detection is based on the configuration files listed here
  • scala-steward
    • Detection is based on the configuration files listed here

Add Support

Don't see your update tool listed?
Search for an existing issue, or create one, to discuss adding support.


Overview (`docs/checks/fuzzing/README.md`)

Supported Fuzzers

  • LibFuzzer
    • Detection is based on usages of a function named LLVMFuzzerTestOneInput in C, C++, or Swift files.
  • ClusterFuzzLite
    • Detection is based on a file called .clusterfuzzlite/Dockerfile.
  • Native Go Fuzzing
    • Looks for functions of the form func FuzzXxx(*testing.F) in Go files.
  • Jazzer
    • Detection based on the import of com.code_intelligence.jazzer.api.FuzzedDataProvider in Java files.
  • OSS-Fuzz
  • Property-based Haskell Fuzzers
  • fast-check
    • Detection based on import statements in JavaScript and TypeScript files.
  • Atheris
    • Detection based on the presence of import atheris in Python files.
  • cargo-fuzz
    • Detection based on presence of libfuzzer_sys in Rust files.
  • FsCheck
    • Detection based on import statements in C# and F# files.

Add Support

Don't see your fuzzing tool listed?
Search for an existing issue, or create one, to discuss adding support.


Overview (`docs/checks/sast/README.md`)

Supported Tools

  • CodeQL
    • Detection is based on GitHub workflows using github/codeql-action/analyze, or GitHub Action checks run against PRs.
  • Qodana
    • Detection based on GitHub workflows using JetBrains/qodana-action.
  • Snyk
  • Sonar
    • Detection based on the presence of a pom.xml file specifying a sonar.host.url, or GitHub Action checks run against PRs.

Add Support

Don't see your SAST tool listed?
Search for an existing issue, or create one, to discuss adding support.


ROADMAP (`docs/ROADMAP.md`)

OpenSSF Scorecard Roadmap

2026

Theme: Scorecard v6 β€” Open Source Security Evidence Engine

Mission: Scorecard produces trusted, structured security evidence for the
open source ecosystem.

Scorecard v6 evolves Scorecard from a scoring tool to an evidence engine. The
primary initiative for 2026 is adding
OSPS Baseline conformance evaluation as the
first use case that proves this architecture. Scorecard accepts diverse inputs
about a project's security practices, normalizes them through probe-based
analysis, and packages the resulting evidence in interoperable formats for
downstream tools to act on.

Check scores (0-10) and conformance labels (PASS/FAIL/UNKNOWN) are parallel
evaluation layers over the same probe evidence, produced in a single run.
Existing checks, probes, and scores are unchanged β€” v6 is additive. The
conformance layer is a new product surface aligned with the
ORBIT WG ecosystem. (While Scorecard is not
part of ORBIT WG, ecosystem interoperability with ORBIT tools is an overarching
OpenSSF goal, and Scorecard interoperates through published output formats.)

Target Baseline version: v2026.02.19

Current coverage: See docs/osps-baseline-coverage.md
for a control-by-control analysis.

Phased delivery

Phases are ordered by outcome. Maintainer bandwidth dictates delivery timing.

Phase 1: Conformance foundation and Level 1 coverage

Outcome: Scorecard produces a useful OSPS Baseline Level 1 conformance
report for any public GitHub repository, available across CLI, Action, and
API surfaces.

Deliverables:

  • Evidence model and output formats:
    • Enriched JSON (Scorecard-native)
    • In-toto evidence predicate (scorecard.dev/evidence/v0.1)
    • Gemara output (via security-baseline
      dependency)
    • OSCAL Assessment Results (via
      go-oscal)
    • OpenSSF Best Practices Badge Automation Proposals
  • Unified framework abstraction for OSPS Baseline v2026.02.19:
    • Checks and OSPS Baseline both use the same internal probe composition interface
    • Probe-to-control mappings maintained in Scorecard
  • Applicability engine detecting preconditions (e.g., "has made a release")
  • Map existing probes to OSPS controls where coverage exists today
  • New probes for Level 1 gaps:
    • Governance and documentation presence
    • Dependency manifest presence
    • Security policy deepening
    • Secrets detection β€” consuming platform signals where available
  • Metadata ingestion layer β€” Security Insights as first supported source;
    architecture supports additional metadata sources
  • Scorecard control catalog extraction plan

Phase 2: Release integrity and Level 2 core

Outcome: Scorecard evaluates release-related OSPS controls, covering the
core of Level 2 and becoming useful for downstream due diligence workflows.

Deliverables:

  • Release asset inspection layer
  • Signed manifest support
  • Release notes and changelog detection
  • Evidence bundle output (conformance results + in-toto statement)
  • Additional metadata sources for the ingestion layer

Note: Phase 1 focuses on automatically verifiable controls. Design of
attestation mechanisms (for non-automatable controls) is deferred to Phase 3 or
beyond.

Phase 3: Enforcement detection, Level 3, and multi-repo

Outcome: Scorecard covers Level 3 controls including enforcement detection
and project-level aggregation.

Deliverables:

  • SCA policy and enforcement detection (Scorecard detects enforcement mechanisms without being an enforcement tool itself)
  • SAST policy and enforcement detection (Scorecard detects enforcement mechanisms without being an enforcement tool itself)
  • Multi-repo project-level conformance aggregation
  • Attestation mechanism for non-automatable controls (deferred from Phase 2)
  • Attestation integration GA

Ecosystem alignment

Scorecard operates within the ORBIT WG ecosystem as an evidence engine. All
downstream tools consume Scorecard evidence on equal terms through published
output formats.

Allstar, a Scorecard sub-project,
continuously monitors GitHub organizations and enforces Scorecard check
results as policies. OSPS conformance output could enable Allstar to enforce
Baseline conformance at the organization level.

Scorecard SHOULD NOT (per RFC 2119)
duplicate evaluation that downstream tools handle:

  • Privateer β€” Baseline evaluation powered by Gemara and Security Insights
  • Minder β€” Policy enforcement and remediation platform (OpenSSF Sandbox, ORBIT WG)
  • AMPEL β€” Attestation-based policy enforcement; already consumes Scorecard probe results via policy library
  • Darnit β€” Compliance audit and remediation

Scorecard's role is to produce deep, probe-based security evidence that these
tools and downstream consumers can use through interoperable output formats
(JSON, in-toto, Gemara, SARIF, OSCAL).

Design principles

  1. Evidence is the product. Scorecard's core output is structured,
    normalized probe findings. Check scores and conformance labels are parallel
    evaluation layers over the same evidence.
  2. Probes normalize diversity. Each probe understands multiple ways a
    control outcome can be satisfied.
  3. UNKNOWN-first honesty. If Scorecard cannot observe a control, the
    status is UNKNOWN with an explanation β€” never a false PASS or FAIL.
  4. All consumers are equal. Downstream tools consume Scorecard evidence
    through published output formats.
  5. No metadata monopolies. Probes may evaluate multiple sources for the
    same data. No single metadata file is required for meaningful results,
    though they may enrich results.
  6. Formats are presentation. Output formats (JSON, in-toto, Gemara,
    SARIF, OSCAL) are views over the evidence model. No single format is
    privileged.

Open questions

The following design questions are under active discussion among maintainers:

  • Attestation identity model β€” How non-automatable controls are attested
    (repo-local metadata vs. signed attestations via Sigstore/OIDC). Decomposed
    into identity (who signs) and tooling (what generates, when) sub-questions.
  • Enforcement detection scope β€” How Scorecard detects enforcement
    mechanisms without being an enforcement tool itself

How to contribute

See the proposal
for detailed requirements and open questions. Discussion and feedback are
welcome via GitHub issues and the Scorecard community meetings.


Beginner Checks (`docs/beginner-checks.md`)

Getting Started with Scorecard Checks for Supply Chain Security

Choosing which Scorecard checks to get started with as a project maintainer can be overwhelming. This page walks through some of the most important checks to start with for project improvement, focusing on the ones that give you the biggest payoff versus effort. They're broken down into three categories based on stages of the development process: setting up your project, accepting contributions from others, and packaging the project to release to the world.

Note: not every Scorecard check or topic mentioned on this page might be relevant to your project. See below for more on customizing your checks to your needs.

1. Setting up your project

Start your project off strong by focusing on Scorecard checks that help you secure your project dependencies and workflows.

  • Secure your dependencies with the Vulnerabilities check and the Dependency-Update-Tool check
  • Secure your workflows with the Token-Permissions check

The Vulnerabilities and Dependency-Update-Tool checks secure your dependencies

Vulnerabilities are probably the most familiar security risk. By running Scorecard’s Vulnerabilities check, you’ll get information about known vulnerabilities in your project, either through your codebase or through your direct and (for most languages) indirect dependencies. Tracking vulnerabilities in an indirect (sometimes called transitive) dependency can be tricky, but it’s important: 95% of vulnerable dependencies are transitive.

If vulnerabilities are found in your dependencies, there are a few options:

  • Update the dependency to a non-vulnerable version (if available)
  • Submit a patch to the vulnerable project
  • Replace the dependency with a non-vulnerable dependency
  • Remove the dependency and write code to take its place
  • If you are sure a vulnerability does not impact your project, you may ignore the dependency by creating an osv-scanner.toml file in your project's root directory.

If you have handled the vulnerabilities in your dependencies and are still not satisfied with your score for this check, make sure there are no open, unfixed vulnerabilities in your project’s own codebase. Once you have dealt with those, your score should improve.

Next, Scorecard’s Dependency-Update-Tool check encourages developers to keep their dependencies up to date, which is a great way to stay on top of security updates. This check awards a high score to a project if it uses a dependency update tool such as Dependabot or Renovate bot. Using one of these tools helps streamline security processes by notifying you when vulnerabilities have surfaced in your dependencies or when new versions of your dependencies become available.

Automated processes like these save you time and are highly configurable; for example, you can set your bot to update dependencies every day or every week at the same time.

If you want to increase your score in this category, sign up for automatic updates with a dependency update tool. Keep in mind, though, that this check can only show that the dependency update tool is enabled, not that it is running. To benefit as much as possible from this check, be sure that you consistently run and act on the information from your dependency update tool.

Token-Permissions check helps you secure your workflows

We suggest addressing the Token-Permissions check next because it takes just a few minutes to β€œset it and forget it” and secure your workflows. The check warns you when your project’s top-level tokens have write access instead of the more restrictive read access. Not all write access permissions need to be eliminated; some workflows may genuinely require them. But ensuring your top-level permissions have read access helps your project follow the principle of least privilege, which means that permissions are granted based on the minimal necessary access to perform a function. Projects that have top-level tokens set to write are granting more access than necessary to their automated workflow dependencies. If those dependencies were ever compromised, they could become an attack vector, and could be vulnerable to malicious code execution.

To change the default setting for token permissions, add the following to the top of your workflow:

text
permissions:
  contents: read

When you add a GitHub Action, be sure to read the Action’s docs to see if it needs any additional permissions; this information is usually prominent in the Action’s README.

2. Manage contributions to your project

As projects grow, they generally start including contributions from others. Contributors can expand your project’s scope and maturity, but they can also introduce security risk. To protect your project at this stage, we recommend improving the Branch Protection check, which allows you, the maintainer, to define rules that require certain workflows for certain branches.

Branch Protection reduces the risks of errors and hacks

The Branch Protection check can help protect your code from unvetted changes. You can choose either or both of the following options:

  • Require code review. Select this if your project has more than one maintainer. Requiring review before changes are merged is one of the strongest protections you can give your code. This will also improve your Code Review check score.
  • Require status checks. All projects would benefit from selecting this option. It ensures that all Continuous Integration (CI) tests chosen by the maintainer have passed before a change is accepted, helping you catch mistakes early on in the development process. This will also improve your CI Test check score.

3. Package and release your project

Deciding how to securely share your code can be difficult. Building locally on your laptop may seem simpler at first, but using an automated build process to create your package on your CI/CD system provides you with security benefits that pay off in the long run. Scorecard’s Packaging check helps guide you through this process.

Packaging check verifies if a project is published as a package

The Packaging check assesses whether a project has been published as a package. The check is currently limited to repositories hosted on GitHub. It looks for GitHub packaging workflows and language-specific GitHub Actions that upload the package to a corresponding hub, like npm or PyPI.

Another benefit to releasing projects as packages is reproducibilityβ€”the version that new users can download and execute is identical to the one that you and other contributors have already reviewed. Packages also have clear versioning documentation that makes it easier to track whether any newly discovered security issues are applicable to your project.

Packaging your projects makes it easier for users to receive security patches as updates. It also provides information about the release details to your users, which opens the door to more collaboration from your open-source peers.

Customize your checks to your project’s needs

Based on the specifics of your project, not all the checks offered by Scorecard or discussed on this page may apply to you. For example, if you are the sole maintainer of an open-source project, the β€œCode Review” check would not be usable for your project.

The languages you use also influence which checks will be useful to you. For example, if your project is built in C or C++, the Packaging and Dependency-Update-Tool checks will not be applicable because the C/C++ ecosystem does not have a centralized package manager.

To learn more about all the checks Scorecard offers, see the checks documentation.


Checks (`docs/checks.md`)

Check Documentation

This page describes each Scorecard check in detail, including scoring criteria,
remediation steps to improve the score, and an explanation of the risks
associated with a low score. The checks are continually changing and we welcome
community feedback. If you have ideas for additions or new detection techniques,
please contribute!

Binary-Artifacts

Risk: High (non-reviewable code)

This check determines whether the project has generated executable (binary)
artifacts in the source repository.

Including generated executables in the source repository increases user risk.
Many programming language systems can generate executables from source code
(e.g., C/C++ generated machine code, Java .class files, Python .pyc files,
and minified JavaScript). Users will often directly use executables if they are
included in the source repository, leading to many dangerous behaviors.

Problems with generated executable (binary) artifacts:

  • Binary artifacts cannot be reviewed, allowing possible obsolete or
    maliciously subverted executables. Reviews generally review source code, not
    executables, since it's difficult to audit executables to ensure that they
    correspond to the source code. Over time the included executables might not
    correspond to the source code.
  • Generated executables allow the executable generation process to atrophy,
    which can lead to an inability to create working executables. These problems
    can be countered with verified reproducible builds, but it's easier to
    implement verified reproducible builds when executables are not included in
    the source repository (since the executable generation process is less
    likely to have atrophied).

Allowed by Scorecard:

  • Files in the source repository that are simultaneously reviewable source
    code and executables, since these are reviewable. (Some interpretive
    systems, such as many operating system shells, don't have a mechanism for
    storing generated executables that are different from the source file.)
  • Source code in the source repository generated by other tools (e.g., by
    bison, yacc, flex, and lex). There are potential downsides to generated
    source code, but generated source code tends to be much easier to review and
    thus presents a lower risk. Generated source code is also often difficult
    for external tools to detect.
  • Generated documentation in source repositories. Generated documentation is
    intended for use by humans (not computers) who can evaluate the context.
    Thus, generated documentation doesn't pose the same level of risk.

Remediation steps

  • Remove the generated executable artifacts from the repository.
  • Build from source.

Branch-Protection

Risk: High (vulnerable to intentional malicious code injection)

This check determines whether a project's default and release branches are
protected with GitHub's branch protection
or repository rules settings.
Branch protection allows maintainers to define rules that enforce
certain workflows for branches, such as requiring review or passing certain
status checks before acceptance into a main branch, or preventing rewriting of
public history.

Note: The following settings queried by the Branch-Protection check require an admin token: DismissStaleReviews, EnforceAdmins, RequireLastPushApproval, RequiresStatusChecks and UpToDateBeforeMerge. If
the provided token does not have admin access, the check will query the branch
settings accessible to non-admins and provide results based only on these settings.
However, all of these settings are accessible via Repo Rules. EnforceAdmins is calculated slightly differently.
This setting is calculated as false if any Bypass Actors
are defined on any rule, regardless of if they are admins.

Different types of branch protection protect against different risks:

  • Require code review:

    • requires at least one reviewer, which greatly
      reduces the risk that a compromised contributor can inject malicious code.
      Review also increases the likelihood that an unintentional vulnerability in
      a contribution will be detected and fixed before the change is accepted.

    • requiring two or more reviewers protects even more from the insider risk
      whereby a compromised contributor can be used by an attacker to LGTM
      the attacker PR and inject a malicious code as if it was legit.

  • Prevent force push: prevents use of the --force command on public
    branches, which overwrites code irrevocably. This protection prevents the
    rewriting of public history without external notice.

  • Require status checks:
    ensures that all required CI tests are met before a change is accepted.

Although requiring code review can greatly reduce the chance that
unintentional or malicious code enters the "main" branch, it is not feasible for
all projects, such as those that don't have many active participants. For more
discussion, see Code Reviews.

Additionally, in some cases these rules will need to be suspended. For example,
if a past commit includes illegal content such as child pornography, it may be
necessary to use a force push to rewrite the history rather than simply hide the
commit.

This test has tiered scoring. You receive partial credit for requirements met within your current tier. However, a tier must be fully satisfied before you can earn points from the next tier. For example, if you fulfill the Tier 3 checks but do not fulfill all the Tier 2 checks, you will not receive any points for Tier 3.

Note: If Scorecard is run without an administrative access token, the requirements that specify β€œFor administrators” can be safely ignored, and scores will be determined as if all such requirements have been met.

Tier 1 Requirements (3/10 points):

  • Prevent force push
  • Prevent branch deletion

Tier 2 Requirements (6/10 points):

  • Require at least 1 reviewer for approval before merging (for administrators, this requirement weights twice than the others in this tier)
  • For administrators: Require PRs prior to make any code changes
  • For administrators: Require branch to be up to date before merging (or require a merge queue via repository rulesets)
  • For administrators: Require approval of the most recent reviewable push

Tier 3 Requirements (8/10 points):

  • Require branch to pass at least 1 status check before merging

Tier 4 Requirements (9/10 points):

  • Require at least 2 reviewers for approval before merging
  • Require review from code owners

Tier 5 Requirements (10/10 points):

  • For administrators: Dismiss stale reviews and approvals when new commits are pushed
  • For administrators: Include administrator for review

GitLab Integration Status:

  • GitLab associates releases with commits and not with the branch. Releases are ignored in this portion of the scoring.

Azure DevOps Integration Status:

  • Default and named branches use the same applicable PR policies, including inherited policies and later result pages.
  • Only enabled, required, non-deleted PR policies establish protection. Repository push restrictions, such as secrets scanning, do not establish PR protection.
  • Reviewer counts are conservative independent-approval floors. Self-votes and path filters do not receive unconditional review credit.
  • Required build contexts use azure-pipelines/<definition-id>. These identify policy requirements, not successful test runs.
  • Permission evaluations describe the caller. A proven caller exception can show that forced updates are allowed, but a caller denial does not prove protection for everyone.
  • Unreadable or unsupported permission controls remain unknown. Administrator enforcement and CODEOWNERS enforcement are not inferred.
  • Unknown deletion or force-push protection receives no Tier 1 credit, so the score can remain zero despite mapped PR policies.

Remediation steps

  • Enable branch protection settings in your source hosting provider to avoid force pushes or deletion of your important branches.
  • For GitHub, check out the steps for rulesets here and rules here.

CI-Tests

Risk: Low (possible unknown vulnerabilities)

This check tries to determine if the project runs tests before pull requests are
merged. It is currently limited to repositories hosted on GitHub, and does not
support other source hosting repositories (i.e., Forges). This check only
considers tests which run successfully.

Running tests helps developers catch mistakes early on, which can reduce the
number of vulnerabilities that find their way into a project.

The check works by looking for a set of CI-system names in GitHub CheckRuns
and Statuses among the recent commits (~30). A CI-system is considered
well-known if its name contains any of the following: appveyor, buildkite,
circleci, e2e, github-actions, jenkins, mergeable, test, travis-ci, woodpecker.

Note: A project that fulfills this criterion with other tools may still receive
a low score on this test. There are many ways to implement CI testing, and it is
challenging for an automated tool like Scorecard to detect them all. A low score
is therefore not a definitive indication that the project is at risk.

If a project's system was not detected and you think it should be, please
open an issue in the scorecard project.

Remediation steps

  • Check-in scripts that run all the tests in your repository.
  • Integrate those scripts with a CI/CD platform that runs it on every pull request (e.g. if hosted on GitHub, GitHub Actions, Prow, etc).

CII-Best-Practices

Risk: Low (possibly not following security best practices)

This check determines whether the project has earned an OpenSSF (formerly CII) Best Practices Badge at the passing, silver, or gold level.
The OpenSSF Best Practices badge indicates whether or not that the project uses a set of security-focused best development practices for open
source software. The check uses the URL for the Git repo and the OpenSSF Best Practices badge API.

The OpenSSF Best Practices badge has 3 tiers: passing, silver, and gold. We give
full credit to projects that meet the gold criteria, which is a significant achievement for projects and requires multiple developers in the project.
Lower scores represent a project that has met the silver criteria, met the passing criteria, or is working to achieve the passing badge, with increasingly more points awarded as more criteria are met. Note that even meeting the passing criteria is a significant achievement.

Some of these criteria overlap with other Scorecard checks.
However, note that in those overlapping cases, Scorecard can only report what it can automatically detect, while the OpenSSF Best Practices badge can report on claims and claim justifications from people (this counters false negatives and positives but has the challenge of requiring additional work from people).

Remediation steps

Code-Review

Risk: High (unintentional vulnerabilities or possible injection of malicious
code)

This check determines whether the project requires human code review
before pull requests (merge requests) are merged.

Reviews detect various unintentional problems, including vulnerabilities that
can be fixed immediately before they are merged, which improves the quality of
the code. Reviews may also detect or deter an attacker trying to insert
malicious code (either as a malicious contributor or as an attacker who has
subverted a contributor's account), because a reviewer might detect the
subversion.

The check determines whether the most recent changes (over the last ~30 commits) have
an approval on GitHub/GitLab
or if the merger is different from the committer (implicit review). It also
performs a similar check for reviews using
Prow (labels
"lgtm" or "approved") and Gerrit ("Reviewed-on" and "Reviewed-by").
If recent changes are solely bot activity (e.g. Dependabot, Renovate bot, or custom bots),
the check returns inconclusively.

Scoring is leveled instead of proportional to make the check more predictable.
If any bot-originated changes are unreviewed, 3 points are deducted. If any human
changes are unreviewed, 7 points are deducted if a single change is unreviewed, and
another 3 are deducted if multiple changes are unreviewed.

Review by bots, including bots powered by
artificial intelligence / machine learning (AI/ML),
do not count as code review.
Such reviews do not provide confidence that there will
be a second person who understands the
code change (e.g., if the originator suddenly becomes unavailable).
However, analysis by bots
may be able to meet (at least in part) the SAST criterion.

Note: Requiring reviews for all changes is infeasible for some projects, such as
those with only one active participant. Even a project with multiple active
contributors may not have enough active participation to be able to require
review of all proposed changes. Projects with a small number of active
participants instead sometimes aim for a review of a
percentage of proposals (e.g., "at least half of all proposed changes are
reviewed").

Requiring review does not eliminate all risks. The other reviewers might fail to
notice unintentional vulnerabilities or malicious code, be colluding with a
malicious developer, or even be the same person (using a "sock
puppet
" account).

Remediation steps

  • If the project has only one contributor, or does not have enough reviewers to practically require that all contributions be reviewed, try to recruit more maintainers to the project who will be willing to review others' work. Ideally at least some of these people will be from different organizations (see Contributors). If the project has very limited utility, consider expanding its intended utility so more people will be interested in improving it, and make that larger scope clear to potential contributors.
  • Follow security best practices by performing strict code reviews for every new pull request / merge request.
  • Make "code reviews" mandatory in your repository configuration. (Instructions for GitHub.)
  • Enforce the rule for administrators / code owners as well. (Instructions for GitHub.)

Contributors

Risk: Low (lower number of trusted code reviewers)

This check tries to determine if the project has recent contributors from
multiple organizations (e.g., companies). It is currently limited to
repositories hosted on GitHub, and does not support other source hosting
repositories (i.e., Forges).

The check looks at the Company field on the GitHub user profile for authors of
recent commits. To receive the highest score, the project must have had
contributors from at least 3 different companies in the last 30 commits; each of
those contributors must have had at least 5 commits in the last 30 commits.

Note: Some projects cannot meet this requirement, such as small projects with
only one active participant, or projects with a narrow scope that cannot attract
the interest of multiple organizations. See
Code Reviews
for more information about evaluating projects with a small number of
participants.

Remediation steps

  • Ask contributors to join their respective organizations, if they have not already. Otherwise, there is no remediation for this check; it simply provides insight into which organizations have contributed so that you can make a trust-based decision based on that information.

Dangerous-Workflow

Risk: Critical (vulnerable to repository compromise)

This check determines whether the project's GitHub Action workflows has dangerous
code patterns. Some examples of these patterns are untrusted code checkouts,
logging github context and secrets, or use of potentially untrusted inputs in scripts.
The following patterns are checked:

Untrusted Code Checkout: This is the misuse of potentially dangerous triggers.
This checks if a pull_request_target or workflow_run workflow trigger was used in conjunction
with an explicit pull request checkout. Workflows triggered with pull_request_target / workflow_run
have write permission to the target repository and access to target repository
secrets. With the PR checkout, PR authors may compromise the repository, for
example, by using build scripts controlled by the author of the PR or reading
token in memory. This check does not detect whether untrusted code checkouts are
used safely, for example, only on pull request that have been assigned a label.

Script Injection with Untrusted Context Variables: This pattern detects whether a
workflow's inline script may execute untrusted input from attackers. This occurs when
an attacker adds malicious commands and scripts to a context. When a workflow runs,
these strings may be interpreted as code that is executed on the runner. Attackers
can add their own content to certain github context variables that are considered
untrusted, for example, github.event.issue.title. These values should not flow
directly into executable code.

The highest score is awarded when all workflows avoid the dangerous code patterns.

Remediation steps

  • Avoid the dangerous workflow patterns. See this post for information on avoiding untrusted code checkouts. See this document for information on avoiding and mitigating the risk of script injections.

Dependency-Update-Tool

Risk: High (possibly vulnerable to attacks on known flaws)

This check tries to determine if the project uses a dependency update tool,
specifically one of:

  • Dependabot
  • Renovate bot
    Out-of-date dependencies make a project vulnerable to known flaws and prone to attacks.
    These tools automate the process of updating dependencies by scanning for
    outdated or insecure requirements, and opening a pull request to update them if
    found.

This check can determine only whether the dependency update tool is enabled; it
does not ensure that the tool is run or that the tool's pull requests are
merged.

Note: A project that fulfills this criterion with other tools may still receive
a low score on this test. There are many ways to implement dependency updates,
and it is challenging for an automated tool like Scorecard to detect them all. A
low score is therefore not a definitive indication that the project is at risk.

Remediation steps

  • Sign up for automatic dependency updates with one of the previously listed dependency update tools and place the config file in the locations that are recommended by these tools. Due to https://github.com/dependabot/dependabot-core/issues/2804 Dependabot can be enabled for forks where security updates have ever been turned on so projects maintaining stable forks should evaluate whether this behavior is satisfactory before turning it on.
  • Unlike Dependabot, Renovate bot has support to migrate dockerfiles' dependencies from version pinning to hash pinning via the pinDigests setting without additional manual effort.

Fuzzing

Risk: Medium (possible vulnerabilities in code)

This check tries to determine if the project uses
fuzzing by checking:

  1. if the repository name is included in the OSS-Fuzz project list;
  2. if ClusterFuzzLite is deployed in the repository;
  3. if there are user-defined language-specified fuzzing functions in the repository.
    • currently only supports Go fuzzing,
    • a limited set of property-based testing libraries for Haskell including QuickCheck, Hedgehog, validity or SmallCheck,
    • a limited set of property-based testing libraries for JavaScript and TypeScript including fast-check.
    • a limited set of property-based testing libraries for Erlang, including proper and quickcheck.
    • a limited set of property-based testing libraries for C# and F# including FsCheck.

Fuzzing, or fuzz testing, is the practice of feeding unexpected or random data
into a program to expose bugs. Regular fuzzing is important to detect
vulnerabilities that may be exploited by others, especially since attackers can
also use fuzzing to find the same flaws.

Note: A project that fulfills this criterion with other tools may still receive
a low score on this test. There are many ways to implement fuzzing, and it is
challenging for an automated tool like Scorecard to detect them all. A low score
is therefore not a definitive indication that the project is at risk.

Remediation steps

  • Integrate the project with OSS-Fuzz by following the instructions here.

License

Risk: Low (possible impediment to security review)

This check tries to determine if the project has published a license. It
works by using either hosting APIs or by checking standard locations
for a file named according to common conventions for licenses.

A license can give users information about how the source code may or may
not be used. The lack of a license will impede any kind of security review
or audit and creates a legal risk for potential users.

Scorecard uses the
GitHub License API
for GitHub hosted projects. Otherwise, Scorecard uses its own heuristics to
detect a published license file.

On its own, this check will detect files in the top-level directory with
any combination of the following names and extensions:LICENSE, LICENCE,
COPYING, COPYRIGHT and having common extensions such as .html, .txt,
or .md. It will also detect these files in a directory named LICENSES.
(Files in a LICENSES directory are typically named as their
SPDX license identifier followed by an
appropriate file extension, as described in the REUSE Specification.)

License Requirements:

  • A detected LICENSE, COPYRIGHT, or COPYING filename, or license
    files in a LICENSES directory (6/10 points)
  • The detected file is at the top-level directory (3/10 points)
  • A FSF or OSI license is specified (1/10 points)

Remediation steps

  • Determine which license to apply to your project. For GitHub hosted projects, follow those instructions to establish a license for your project.
  • For other hosting environments, create the license in a .adoc, .asc, .docx, .doc, .ext, .html, .markdown, .md, .rst, .txt, or .xml, named LICENSE, COPYRIGHT, or COPYING, and place it in the top-level directory. To identify a specific license, use an SPDX license identifier in the filename. Examples include LICENSE.md, Apache-2.0-LICENSE.md or LICENSE-Apache-2.0.
  • Alternately, create a LICENSES directory and add a license file(s) with a name that matches your SPDX license identifier. such as LICENSES/Apache-2.0.txt.

Maintained

Risk: High (possibly unpatched vulnerabilities)

This check determines whether the project is actively maintained. If the project
is archived, it receives the lowest score. If there is at least one commit per
week during the previous 90 days, the project receives the highest score. If there
is activity on issues from users who are collaborators, members, or owners of the
project, the project receives a partial score.

A project which is not active might not be patched, have its
dependencies patched, or be actively tested and used. However, a lack
of active maintenance is not necessarily always a problem. Some software,
especially smaller utility functions, does not normally need to be maintained.
For example, a library that determines if an integer is even would not normally
need maintenance unless an underlying implementation language definition
changed. A lack of active maintenance should signal that potential users should
investigate further to judge the situation.

This check will only succeed if a GitHub project is >90 days old. Projects
that are younger than this are too new to assess whether they are maintained
or not, and users should inspect the contents of those projects to ensure they
are as expected.

Remediation steps

  • There is no remediation work needed from projects with a low score; this check simply provides insight into the project activity and maintenance commitment. External users should determine whether the software is the type that would not normally need active maintenance.

Packaging

Risk: Medium (users possibly missing security updates)

This check tries to determine if the project is published as a package. It is
currently limited to repositories hosted on GitHub, and does not support other
source hosting repositories (i.e., Forges).

Packages give users of a project an easy way to download, install, update, and
uninstall the software by a package manager. In particular, they make it easy
for users to receive security patches as updates.

The check currently looks for
GitHub packaging workflows
and language-specific GitHub Actions that upload the package to a corresponding
hub, e.g., Npm. We plan to add better support to query
package manager hubs directly in the future, e.g., for
Npm, PyPi.

You can create a package in several ways:

  • Many program language ecosystems have a generally-used packaging format
    supported by a language-level package manager tool and public package
    repository.
  • Many operating system platforms also have at least one package format,
    tool, and public repository (in some cases the source repository generates
    system-independent source packages, which are then used by others to
    generate system executable packages).
  • Using container images.

Note: A project that fulfills this criterion with other tools may still receive
a low score on this test. There are many ways to package software, and it is
challenging for an automated tool like Scorecard to detect them all. A low
score is therefore not a definitive indication that the project is at risk. If
Scorecard fails to detect the way you publish a package and you think we should
support your use case, please let us know by opening an
issue
.

Remediation steps

Pinned-Dependencies

Risk: Medium (possible compromised dependencies)

This check tries to determine if the project pins dependencies used during its build and release process.
A "pinned dependency" is a dependency that is explicitly set to a specific hash instead of
allowing a mutable version or range of versions. It
is currently limited to repositories hosted on GitHub, and does not support
other source hosting repositories (i.e., Forges).

The check works by looking for unpinned dependencies in Dockerfiles, shell scripts, and GitHub workflows
which are used during the build and release process of a project.
Special considerations for Go modules treat full semantic versions as pinned
due to how the Go tool verifies downloaded content against the hashes when anyone first downloaded the module.

Pinned dependencies reduce several security risks:

  • They ensure that checking and deployment are all done with the same
    software, reducing deployment risks, simplifying debugging, and enabling
    reproducibility.
  • They can help mitigate compromised dependencies from undermining the
    security of the project (in the case where you've evaluated the pinned
    dependency, you are confident it's not compromised, and a later version is
    released that is compromised).
  • They are one way to counter dependency confusion (aka substitution) attacks,
    in which an application uses multiple feeds to acquire software packages (a
    "hybrid configuration"), and attackers fool the user into using a malicious
    package via a feed that was not expected for that package.

However, pinning dependencies can inhibit software updates, either because of a
security vulnerability or because the pinned version is compromised. Mitigate
this risk by:

  • using automated tools to notify applications when their dependencies are
    outdated;
  • quickly updating applications that do pin dependencies.

For projects hosted on GitHub, you can learn more about
dependencies using the GitHub dependency graph.

Remediation steps

  • If your project is producing an application, declare all your dependencies with specific versions in your package format file (e.g. package.json for npm, requirements.txt for python, packages.config for nuget). For C/C++, check in the code from a trusted source and add a README on the specific version used (and the archive SHA hashes).
  • If your project is producing an application and the package manager supports lock files (e.g. package-lock.json for npm), make sure to check these in the source code as well. These files maintain signatures for the entire dependency tree and saves from future exploitation in case the package is compromised.
  • For Dockerfiles used in building and releasing your project, pin dependencies by hash. See Dockerfile for example. If you are using a manifest list to support builds across multiple architectures, you can pin to the manifest list hash instead of a single image hash. You can use a tool like crane to obtain the hash of the manifest list like in this example.
  • For GitHub workflows used in building and releasing your project, pin dependencies by hash. See main.yaml for example. To determine the permissions needed for your workflows, you may use StepSecurity's online tool by ticking the "Pin actions to a full length commit SHA". You may also tick the "Restrict permissions for GITHUB_TOKEN" to fix issues found by the Token-Permissions check.
  • To help update your dependencies after pinning them, use tools such as those listed for the dependency update tool check.

SAST

Risk: Medium (possible unknown bugs)

This check tries to determine if the project uses Static Application Security
Testing (SAST), also known as static code analysis.
It is currently limited to repositories hosted on GitHub, and does not support
other source hosting repositories (i.e., Forges).

SAST is testing run on source code before the application is run. Using SAST
tools can prevent known classes of bugs from being inadvertently introduced in the
codebase.

The checks currently looks for known GitHub apps such as
CodeQL (github-code-scanning) or
SonarCloud in the recent (~30) merged PRs, or the use
of "github/codeql-action" in a GitHub workflow. It also checks for the deprecated
LGTM service until its forthcoming shutdown.

Note: A project that fulfills this criterion with other tools may still receive
a low score on this test. There are many ways to implement SAST, and it is
challenging for an automated tool like Scorecard to detect them all. A low score
is therefore not a definitive indication that the project is at risk.

Remediation steps

  • Run CodeQL checks in your CI/CD by following the instructions here.

SBOM

Risk: Medium (possible inaccurate reporting of dependencies/vulnerabilities)

This check tries to determine if the project maintains a Software Bill of Materials (SBOM)
either as a file in the source or a release artifact.

An SBOM can give users information about what dependencies your project has which
allows them to identify vulnerabilities in the software supply chain.

Standards to be used during checks;

This check currently looks for the existence of an SBOM in the
source of a project and as a pipeline or release artifact.

An SBOM Exists (one or more) (5/10 points):

  • Any SBOM found counts for this test either in source. pipeline or release.
  • A SBOM stored with your source code is not ideal, but is a good first step.
    * It is recommended to publish with your release artifacts.

An SBOM is published as a release artifact (5/10 points):

  • This is the preferred way to store an SBOM, and will be awarded full points.
  • Checks release artifacts for an SBOM file matching established standards

Remediation steps

  • For Gitlab, see more information here.
  • For GitHub, see more information here.
  • Alternatively, there are other tools available to generate CycloneDX and SPDX SBOMs.

Security-Policy

Risk: Medium (possible insecure reporting of vulnerabilities)

This check tries to determine if the project has published a security policy. It
works by looking for a file named SECURITY.md (case-insensitive) in a few
well-known directories.

A security policy (typically a SECURITY.md file) can give users information
about what constitutes a vulnerability and how to report one securely so that
information about a bug is not publicly visible.

This check examines the contents of the security policy file awarding points
for those policies that express vulnerability process(es), disclosure timelines,
and have links (e.g., URL(s) and email(s)) to support the users.

Linking Requirements (one or more) (6/10 points):

  • A valid form of an email address to contact for vulnerabilities
  • A valid form of a http/https address to support vulnerability reporting

Free Form Text (3/10 points):

  • Free form text is present in the security policy file which is beyond
    simply having a http/https address and/or email in the file
  • The string length of any such links in the policy file do not count
    towards detecting free form text

Security Policy Specific Text (1/10 points):

  • Specific text providing basic or general information about vulnerability
    and disclosure practices, expectations, and/or timelines
  • Text should include a total of 2 or more hits which match (case-insensitive)
    vuln and as in "Vulnerability" or "vulnerabilities";
    disclos as "Disclosure" or "disclose";
    and numbers which convey expectations of times, e.g., 30 days or 90 days

Remediation steps

  • Place a security policy file SECURITY.md in the root directory of your repository. This makes it easily discoverable by a vulnerability reporter.
  • The file should contain information on what constitutes a vulnerability and a way to report it securely (e.g. issue tracker with private issue support, encrypted email with a published public key). Follow the coordinated vulnerability disclosure guidelines to respond to vulnerability disclosures.
  • For GitHub, see more information here.

Signed-Releases

Risk: High (possibility of installing malicious releases)

This check tries to determine if the project cryptographically signs release
artifacts. It is currently limited to repositories hosted on GitHub, and does
not support other source hosting repositories (i.e., Forges).

Signed releases attest to the provenance of the artifact.

This check looks for the following filenames in the project's last five
release assets:
*.minisig, *.asc (pgp),
*.sig, *.sign, *.sigstore, *.sigstore.json, *.intoto.jsonl.

If a signature is found in the assets for each release, a score of 8 is given.
If a SLSA provenance file is found in the assets for each release (*.intoto.jsonl), the maximum score of 10 is given.

This check looks for the 30 most recent releases associated with an artifact. It ignores the source code-only releases that are created automatically by GitHub.

Note: The check does not verify the signatures.

Remediation steps

  • Publish the release.
  • Generate a signing key.
  • Download the release as an archive locally.
  • Sign the release archive with this key (should output a signature file).
  • Attach the signature file next to the release archive.
  • If the source is hosted on GitHub, check out the steps here.

Token-Permissions

Risk: High (vulnerable to malicious code additions)

This check determines whether the project's automated workflows tokens follow the
principle of least privilege. This is important because attackers may use a
compromised token with write access to, for example, push malicious code into the
project.

It is currently limited to repositories hosted on GitHub, and does not support
other source hosting repositories (i.e., Forges).

The highest score is awarded when the permissions definitions in each workflow's
yaml file are set as read-only at the
top level
and the required write permissions are declared at the
run-level.
One point is reduced from the score if all jobs have their permissions defined but the top level permissions are not defined.
This configuration is secure, but there is a chance that when a new job is added to the workflow, its job permissions could be
left undefined because of human error.

Though a project's score won't be penalized, the check's details will include
warnings for more sensitive run-level permissions, listed below:

  • actions - May allow an attacker to steal GitHub secrets by approving to run an action that needs approval.
  • checks - May allow an attacker to remove pre-submit checks and introduce a bug.
  • contents - Allows an attacker to commit unreviewed code. However, points are not reduced if the job utilizes a recognized packaging action or command.
  • deployments - May allow an attacker to charge repo owner by triggering VM runs, and tiny chance an attacker can trigger a remote service with code they own if server accepts code/location variables unsanitized.
  • packages - Allows an attacker to publish packages. However, points are not reduced if the job utilizes a recognized packaging action or command.
  • security-events - May allow an attacker to read vulnerability reports before a patch is available. However, points are not reduced if the job utilizes a recognized action for uploading SARIF results.
  • statuses - May allow an attacker to change the result of pre-submit checks and get a PR merged.

This compromise makes it clear the maintainer has done what's possible to use those permissions safety,
but allows users to identify that the permissions are used.

The check cannot detect if the "read-only" GitHub permission setting is
enabled, as there is no API available.

Remediation steps

  • Set top-level permissions as read-all or contents: read as described in GitHub's documentation.
  • Set any required write permissions at the job-level. Only set the permissions required for that job; do not set permissions: write-all at the job level.
  • To help determine the permissions needed for your workflows, you may use StepSecurity's online tool by ticking the "Restrict permissions for GITHUB_TOKEN". You may also tick the "Pin actions to a full length commit SHA" to fix issues found by the Pinned-dependencies check.

Vulnerabilities

Risk: High (known vulnerabilities)

This check determines whether the project has open, unfixed vulnerabilities
in its own codebase or its dependencies using the OSV (Open Source Vulnerabilities) service.
An open vulnerability is readily exploited by attackers and should be fixed as soon as
possible.

Remediation steps

  • Fix the vulnerabilities in your own code base. The details of each vulnerability can be found on https://osv.dev.
  • If the vulnerability is in a dependency, update the dependency to a non-vulnerable version. If no update is available, consider whether to remove the dependency.
  • If you believe the vulnerability does not affect your project, the vulnerability can be ignored. To ignore, create an osv-scanner.toml file next to the dependency manifest (e.g. package-lock.json) and specify the ID to ignore and reason. Details on the structure of osv-scanner.toml can be found on OSV-Scanner repository.

Webhooks

Risk: Critical (service possibly accessible to third parties)

This check determines whether the webhook defined in the repository has a token configured to authenticate the origins of requests.

Remediation steps

  • Check if the service your webhooks is configured with supports secrets.
  • If there is support for token authentication, set the secret in the webhook configuration. See Setting up a webhook.
  • If there is no support for token authentication, request the webhook service implement token authentication functionality by following these directions.

Faq (`docs/faq.md`)

Frequently Asked Questions

This page answers frequently asked questions about Scorecard, including its purpose, usage, and checks. This page is continually updated. If you would like to add a question, please contribute!

Installation / Usage

Check-Specific Questions


Installation / Usage

Can I preview my project's score?

Yes.

Over a million projects are automatically tracked by the Scorecard project. Use the webviewer to see these scores, replacing the placeholder text with the platform, user/org, and repository name: https://scorecard.dev/viewer/?uri=.com//.

For example:

You can also use the CLI to generate scores for any public repository by following these steps:

  1. Installation
  2. Authentication
  3. Basic Usage

What is the difference between Scorecard and other Code Scanning tools?

Most code scanning tools are focused on detecting specific vulnerabilities already existing in your codebase. Scorecard, however, is focused on improving the project's overall security posture by helping it adopt best practices. The best solution for your project may well be to adopt Scorecard along with other tools!

Wasn't this project called "Scorecards" (plural)?

Yes, kind of. The project was initially called "Security Scorecards" but that form wasn't used consistently. In particular, the repo was named "scorecard" and so was the program. Over time people started referring to either form (singular and plural) and the inconsistency became prevalent. To end this situation the decision was made to consolidate over the use of the singular form in keeping with the repo and program name, drop the "Security" part and use "OpenSSF" instead to ensure uniqueness. One should therefore refer to this project as "OpenSSF Scorecard" or "Scorecard" for short.

Check-specific Questions

Binary-Artifacts: Can I allowlist testing artifacts?

Scorecard lowers projects' scores whenever it detects binary artifacts. However, many projects use binary artifacts strictly for testing purposes.

While it isn't currently possible to allowlist such binaries, the Scorecard team is working on this feature (#1270).

Code-Review: Can it ignore bot commits?

This is quite a complex question. Right now, there is no way to do that. Here are some pros and cons of allowing users to set up an ignore-list for bots.

  • Pros: Some bots run very frequently; for some projects, reviewing every change is therefore not feasible or reasonable.
  • Cons: Bots can be compromised (their credentials can be compromised, for example). Or if commits are not signed, an attacker could easily send a commit spoofing the bot. This means that a bot having unsupervised write access to the repository could be a security risk.

However, this is being discussed by the Scorecard Team (#2302).

Both Dependabot and Renovate bot won't update your dependencies immediately. They have some precautions to make sure a release is reasonable / won't break your build (see Dependabot compatibility documentation).

You can either configure the tools to only update your dependencies once a week or once a month. This way, if a malicious version is released, it's very likely that it'll be reported and removed before it even gets suggested to you. Besides, there's also the benefit that it gives you the chance to validate the new release before merging if you want to.

Another configuration possibility that would limit even more the release updates only to trusted releases is enabling to only perform Security Updates, which means you only be notified about releases that fixes a previous vulnerability you might be exposed to.

Fuzzing: Does Scorecard accept custom fuzzers?

Currently only for projects written in Go.

For more information, see the Fuzzing check description.

Pinned-Dependencies: Will Scorecard detect unpinned dependencies in tests with Dockerfiles?

Scorecard can show the dependencies that are referred to in tests like Dockerfiles, so it could be a great way for you to fix those dependencies and avoid the vulnerabilities related to version pinning dependencies. To see more about the benefits of hash pinning instead of version pinning, please see the Pinned-Dependencies check description

Pinned-Dependencies: Can I use version pinning instead of hash pinning?

Version pinning is a significant improvement over not pinning your dependencies. However, it still leaves your project vulnerable to tag-renaming attacks (where a dependency's tags are deleted and recreated to point to a malicious commit).

The OpenSSF therefore recommends hash pinning instead of version pinning, along with the use of dependency update tools such as Dependabot to keep your dependencies up-to-date.

Please see the Pinned-Dependencies check description for a better understanding of the benefits of the Hash Pinning.

Signed-Releases: Why sign releases?

Currently, the main benefit of signed releases is the guarantee that a specific artifact was released by a source that you approve or attest is reliable.

However, there are already moves to make it even more relevant. For example, the OpenSSF is working on implementing signature verification for NPM packages which would allow a consumer to automatically verify if the package they are downloading was generated through a reliable builder and if it is correctly signed.

Signing releases already has some relevance and it will soon offer even more security benefits for both consumers and maintainers.

Branch-Protection: How to setup a 10/10 branch protection on GitHub?

To get a 10/10 score for Branch-Protection check using a non-admin token, you should have the following settings for your branches:

When using an admin token, Scorecard can verify if a few other important settings are ensured:

It's important to reiterate that Branch-Protection score is Tier-based. If a setting from Tier 1 is not satisfied, it does not matter that all other settings are met, the score will be truncated up the Tier's maximum. In this case, 3/10. The following table shows the relation between branch protection settings on GitHub and the score Tier:

Name Status Required only for admin token Tier
Allow force pushes Disabled - 1
Allow deletions Disabled - 1
Do not allow bypassing the above settings Enabled Yes 1
Require a pull request before merging > Require Approvals Enabled with at least 1 - 2
Require status checks to pass before merging > Require branches to be up to date before merging Enabled Yes 2
Require a pull request before merging > Require approval of the most recent reviewable push Enabled Yes 2
Require status checks to pass before merging > Status Checks At least 1 - 3
Require a pull request before merging > Require Approvals Enabled with at least 2 - 4
Require a pull request before merging > Require review from Code Owners Enabled and has CODEOWNERS file - 4
Require a pull request before merging > Dismiss stale pull request approvals when new commits are pushed Enabled Yes 5

Osps Baseline Coverage (`docs/osps-baseline-coverage.md`)

OSPS Baseline Coverage Analysis

Analysis of Scorecard's current probe and check coverage against the
OSPS Baseline v2026.02.19.

This is a living document. As probes are added or enhanced, update the
coverage status and evidence columns accordingly.

Coverage legend

Symbol Meaning
COVERED Scorecard has probes that fully satisfy this control
PARTIAL Scorecard has probes that provide evidence but do not fully satisfy the control
GAP No existing probe provides meaningful evidence for this control
NOT_OBSERVABLE Control requires data Scorecard cannot access (e.g., org-level admin permissions)

Summary

Level Total controls COVERED PARTIAL GAP NOT_OBSERVABLE
1 25 6 8 9 2
2 17 2 5 9 1
3 17 0 4 13 0
Total 59 8 17 31 3

Automated coverage rate (COVERED + PARTIAL): 42% (25 of 59)

Full coverage rate (COVERED only): 14% (8 of 59)

Level 1 controls (25)

OSPS ID Control (short) Status Scorecard probes/checks providing evidence Gap / notes
OSPS-AC-01.01 MFA for sensitive resources NOT_OBSERVABLE None Requires org admin API access; Scorecard tokens typically lack this. Must be UNKNOWN unless org-admin token is provided.
OSPS-AC-02.01 Least-privilege defaults for new collaborators NOT_OBSERVABLE None Requires org-level permission visibility. Must be UNKNOWN.
OSPS-AC-03.01 Prevent direct commits to primary branch COVERED requiresPRsToChangeCode, branchesAreProtected Branch-Protection check. Maps directly when PR-only merges are enforced.
OSPS-AC-03.02 Prevent primary branch deletion COVERED blocksDeleteOnBranches Branch-Protection check. Direct mapping.
OSPS-BR-01.01 Sanitize untrusted CI/CD input PARTIAL hasDangerousWorkflowScriptInjection Dangerous-Workflow check detects script injection patterns. Does not cover all sanitization cases (e.g., non-shell contexts).
OSPS-BR-01.03 Untrusted code snapshots cannot access privileged credentials PARTIAL hasDangerousWorkflowUntrustedCheckout Dangerous-Workflow check detects untrusted checkouts with access to secrets. Does not cover all credential isolation scenarios.
OSPS-BR-03.01 Official channel URIs use encrypted transport GAP None Requires a source-of-truth for official URIs (Security Insights). No probe exists.
OSPS-BR-03.02 Distribution URIs use authenticated channels GAP None Same as BR-03.01. Requires declared distribution channels.
OSPS-BR-07.01 Prevent unintentional storage of secrets in VCS GAP None No secrets detection probe exists today. Could consume platform signals (e.g., GitHub secret scanning API).
OSPS-DO-01.01 User guides for released software GAP None No documentation presence probe. Would need file/path heuristics.
OSPS-DO-02.01 Defect reporting guide GAP None No issue template / bug report documentation probe.
OSPS-GV-02.01 Public discussion mechanism GAP None Could check whether issues/discussions are enabled, but no probe exists.
OSPS-GV-03.01 Documented contribution process GAP None No CONTRIBUTING file presence/content probe.
OSPS-LE-02.01 OSI/FSF license for source code COVERED hasFSFOrOSIApprovedLicense License check. Direct mapping.
OSPS-LE-02.02 OSI/FSF license for released assets PARTIAL hasFSFOrOSIApprovedLicense License check verifies repo license, but does not verify license is shipped with release artifacts.
OSPS-LE-03.01 License file in repository COVERED hasLicenseFile License check. Direct mapping.
OSPS-LE-03.02 License included with released assets PARTIAL hasLicenseFile Detects license in repo, not in release artifacts. Needs release asset inspection.
OSPS-QA-01.01 Repo publicly readable at static URL COVERED (implicit) Scorecard can only scan public repos. If Scorecard runs, this is satisfied.
OSPS-QA-01.02 Public commit history with authorship and timestamps COVERED (implicit) VCS provides this by nature. Scorecard relies on commit history for multiple probes.
OSPS-QA-02.01 Direct dependency list present PARTIAL pinsDependencies Pinned-Dependencies check detects dependency manifests but focuses on pinning, not mere presence.
OSPS-QA-04.01 Docs list subprojects GAP None Requires Security Insights or similar metadata. No probe exists.
OSPS-QA-05.01 No generated executable artifacts in VCS PARTIAL hasBinaryArtifacts, hasUnverifiedBinaryArtifacts Binary-Artifacts check. Detects binary files but may not distinguish "generated executables" from other binaries.
OSPS-QA-05.02 No unreviewable binary artifacts in VCS PARTIAL hasUnverifiedBinaryArtifacts Detects unverified binaries. "Unreviewable" vs "reviewable" classification is not yet granular enough.
OSPS-VM-02.01 Security contacts documented PARTIAL securityPolicyPresent, securityPolicyContainsLinks Security-Policy check detects SECURITY.md presence and links, but does not verify actual contact methods (email, form, etc.).
OSPS-BR-01.04 (Note: This is Level 3, not Level 1. Listed under Level 3 below.)

Level 2 controls (17)

OSPS ID Control (short) Status Scorecard probes/checks providing evidence Gap / notes
OSPS-AC-04.01 Default lowest CI/CD permissions PARTIAL topLevelPermissions, jobLevelPermissions, hasNoGitHubWorkflowPermissionUnknown Token-Permissions check evaluates workflow permissions. "Defaults to lowest" semantics need verification.
OSPS-BR-02.01 Releases have unique version identifier GAP None No release versioning probe. Needs release API inspection.
OSPS-BR-04.01 Releases have descriptive changelog GAP None No changelog/release notes detection probe.
OSPS-BR-05.01 Standardized tooling for dependency ingestion GAP None No probe detects whether standard package managers are used in CI.
OSPS-BR-06.01 Releases signed or accounted for in signed manifest PARTIAL releasesAreSigned, releasesHaveProvenance, releasesHaveVerifiedProvenance Signed-Releases check covers signatures and provenance. Does not yet check for "signed manifest including hashes" as an alternative.
OSPS-DO-06.01 Docs describe dependency selection/tracking GAP None Documentation control. No probe exists.
OSPS-DO-07.01 Build instructions in documentation GAP None Documentation control. No probe exists.
OSPS-GV-01.01 Docs list members with sensitive access NOT_OBSERVABLE None Requires org-level data or attestation. Not automatable via Scorecard.
OSPS-GV-01.02 Docs list roles and responsibilities GAP None Documentation control. May require attestation.
OSPS-GV-03.02 Contributor guide with acceptability requirements PARTIAL (related: CONTRIBUTING presence could be inferred) No probe today; could extend a contributing-file probe to check for content structure.
OSPS-LE-01.01 Legal authorization per commit (DCO/CLA) GAP None No DCO/CLA detection probe. Would check for Signed-off-by trailers or CLA bot enforcement.
OSPS-QA-03.01 Status checks pass or bypassed before merge COVERED runsStatusChecksBeforeMerging Branch-Protection check. Direct mapping.
OSPS-QA-06.01 Automated tests run prior to acceptance COVERED testsRunInCI CI-Tests check. Maps directly.
OSPS-SA-01.01 Design docs with actions/actors GAP None Documentation/assessment control. Requires attestation.
OSPS-SA-02.01 Docs describe external interfaces GAP None Documentation control. Requires attestation.
OSPS-SA-03.01 Security assessment performed GAP None Process control. Requires attestation with evidence link.
OSPS-VM-01.01 CVD policy with response timeframe PARTIAL securityPolicyContainsVulnerabilityDisclosure, securityPolicyContainsText Security-Policy check detects disclosure language. Does not verify explicit timeframe commitment.
OSPS-VM-03.01 Private vulnerability reporting method PARTIAL securityPolicyContainsLinks Detects links in SECURITY.md. Does not verify private reporting is actually enabled (e.g., GitHub PSIRT feature).
OSPS-VM-04.01 Publicly publish vulnerability data GAP None No probe checks for GitHub Security Advisories, OSV entries, or CVE publication.

Level 3 controls (17)

OSPS ID Control (short) Status Scorecard probes/checks providing evidence Gap / notes
OSPS-AC-04.02 Job-level least privilege in CI/CD PARTIAL jobLevelPermissions Token-Permissions check evaluates job-level permissions. "Minimum necessary" is hard to assess without understanding job intent.
OSPS-BR-01.04 Sanitize trusted collaborator CI/CD input PARTIAL hasDangerousWorkflowScriptInjection Dangerous-Workflow check partially covers this, but focuses on untrusted input, not trusted collaborator input.
OSPS-BR-02.02 Release assets tied to release identifier GAP None No release asset naming/association probe.
OSPS-BR-07.02 Secrets management policy GAP None Documentation/policy control. Requires attestation.
OSPS-DO-03.01 Instructions to verify release integrity/authenticity GAP None Documentation control. Could partially automate by checking for verification docs alongside signed releases.
OSPS-DO-03.02 Instructions to verify release author identity GAP None Documentation control.
OSPS-DO-04.01 Support scope/duration per release GAP None Documentation control.
OSPS-DO-05.01 EOL security update statement GAP None Documentation control.
OSPS-GV-04.01 Policy to review collaborators before escalated perms GAP None Governance policy. Requires attestation.
OSPS-QA-02.02 SBOM shipped with compiled release assets PARTIAL hasReleaseSBOM, hasSBOM SBOM probes exist but may not specifically verify compiled-asset association.
OSPS-QA-04.02 Subprojects enforce >= primary requirements GAP None Requires multi-repo scanning and cross-repo comparison.
OSPS-QA-06.02 Docs describe when/how tests run GAP None Documentation control.
OSPS-QA-06.03 Policy requiring tests for major changes GAP None Documentation/policy control.
OSPS-QA-07.01 Non-author approval before merging PARTIAL codeApproved, codeReviewOneReviewers, requiresApproversForPullRequests Code-Review and Branch-Protection probes cover this. "Non-author" semantics need verification.
OSPS-VM-04.02 VEX for non-affecting vulnerabilities GAP None No VEX detection probe.
OSPS-VM-05.01 SCA remediation threshold policy GAP None Policy control.
OSPS-VM-05.02 SCA violations addressed pre-release GAP None Policy + enforcement control.
OSPS-VM-05.03 Automated SCA eval + block violations PARTIAL hasOSVVulnerabilities Vulnerabilities check detects known vulns. Does not verify gating/blocking enforcement.
OSPS-VM-06.01 SAST remediation threshold policy GAP None Policy control.
OSPS-VM-06.02 Automated SAST eval + block violations PARTIAL sastToolConfigured, sastToolRunsOnAllCommits SAST check detects tool presence and execution. Does not verify gating/blocking enforcement.

Phase 1 priorities (Level 1 gap closure)

The following Level 1 gaps should be addressed first, ordered by implementation feasibility:

  1. OSPS-GV-03.01 (contribution process): Add probe for CONTRIBUTING file presence
  2. OSPS-GV-02.01 (public discussion): Add probe for issues/discussions enabled
  3. OSPS-DO-02.01 (defect reporting): Add probe for issue templates or bug report docs
  4. OSPS-DO-01.01 (user guides): Add probe for documentation presence heuristics
  5. OSPS-BR-07.01 (secrets in VCS): Consume GitHub secret scanning API or add detection heuristics
  6. OSPS-BR-03.01 / BR-03.02 (encrypted transport): Requires Security Insights ingestion for declared URIs
  7. OSPS-QA-04.01 (subproject listing): Requires Security Insights or equivalent metadata

Probes not mapped to any OSPS control

The following probes exist in Scorecard but do not directly map to any OSPS Baseline control:

Probe Check Notes
archived Maintained Project archival status β€” relates to "while active" preconditions
hasRecentCommits Maintained Activity signal β€” relates to "while active" preconditions
issueActivityByProjectMember Maintained Activity signal β€” relates to "while active" preconditions
createdRecently Maintained Age signal
contributorsFromOrgOrCompany Contributors Diversity signal
dependencyUpdateToolConfigured Dependency-Update-Tool Best practice, not a Baseline control
fuzzed Fuzzing Testing best practice, not a Baseline control
hasOpenSSFBadge CII-Best-Practices Meta-badge, not a Baseline control
packagedWithAutomatedWorkflow Packaging Distribution best practice
webhooksUseSecrets Webhook Security practice, not a Baseline control
hasPermissiveLicense (uncategorized) License type classification
unsafeblock (independent) Language-specific safety
dismissesStaleReviews Branch-Protection Review hygiene beyond Baseline scope
requiresCodeOwnersReview Branch-Protection CODEOWNERS enforcement beyond Baseline scope
requiresLastPushApproval Branch-Protection Push approval beyond Baseline scope
requiresUpToDateBranches Branch-Protection Branch freshness beyond Baseline scope
branchProtectionAppliesToAdmins Branch-Protection Admin override prevention beyond Baseline scope
blocksForcePushOnBranches Branch-Protection Force-push protection; related to AC-03 but not explicitly required

These probes remain valuable for Scorecard's existing scoring model and may become relevant for future Baseline versions.

Existing issues and PRs relevant to gap closure

The following open issues and PRs in the Scorecard repository are directly
relevant to closing OSPS Baseline coverage gaps. These should be prioritized
and linked to the conformance work.

Security Insights ingestion

  • #2305 β€” Support for SECURITY INSIGHTS
  • #2479 β€” SECURITY-INSIGHTS.yml implementation

These are critical for OSPS-BR-03.01, BR-03.02, QA-04.01, and other
controls that depend on declared project metadata.

Secrets detection (OSPS-BR-07.01)

  • #30 β€” New check: code is scanning for secrets

Open since the project's earliest days. Phase 1 priority.

SBOM (OSPS-QA-02.02)

  • #1476 β€” Feature: Detect if SBOMs generated
  • #2605 β€” Add support for SBOM analyzing at Binary-Artifacts stage

The SBOM check and probes (hasSBOM, hasReleaseSBOM) already exist but
may need enhancement for compiled release asset association.

Changelog / release notes (OSPS-BR-04.01)

  • #4824 β€” Feature: New Check: Check if the project has and maintains a CHANGELOG

Direct match for Phase 2 deliverable.

Private vulnerability reporting (OSPS-VM-03.01)

  • #2465 β€” Factor whether or not private vulnerability reporting is enabled into the scorecard

Direct match. GitHub's private vulnerability reporting API could provide
platform-level evidence.

Vulnerability disclosure improvements (OSPS-VM-01.01, VM-04.01)

  • #4192 β€” Test for security policy in other places than SECURITY.md
  • #4789 β€” Rethinking vulnerability check scoring logic
  • #1371 β€” Feature: add check for vulnerability alerts

Signed releases and provenance (OSPS-BR-06.01)

  • #4823 β€” Feature: pass Signed-Releases with GitHub immutable release process
  • #4080 β€” Use GitHub attestations to check for signed releases
  • #2684 β€” Rework: Signed-Releases: Separate score calculation of provenance and signatures
  • #1417 β€” Feature: add support for keyless signed release

Threat model / security assessment (OSPS-SA-01.01, SA-03.01)

  • #2142 β€” Feature: Assess presence and maintenance of a threat model

Release scoring (OSPS-BR-02.01, BR-02.02)

  • #1985 β€” Feature: Scoring for individual releases

Minder integration

  • #4723 β€” Initial draft of using Minder rules in Scorecard (CLOSED)

Draft PR that attempted to run Minder Rego rules within Scorecard,
including OSPS-QA-05.01 and QA-03.01. Closed due to inactivity but
demonstrates interest in deeper Minder/Scorecard integration.

Notes

  • The OSPS Baseline v2026.02.19 contains 59 controls. Previous coverage
    estimates against older Baseline versions should be treated as out-of-date.
    This analysis supersedes any prior mapping.
  • Controls marked NOT_OBSERVABLE cannot produce PASS or FAIL without elevated
    permissions. The conformance engine must return UNKNOWN with an explanation.
  • Many Level 2 and Level 3 controls are documentation or policy controls that
    require attestation rather than automated detection. The attestation mechanism
    (OQ-1 in the proposal) is critical for these.
  • The "while active" precondition on many controls maps to Scorecard's Maintained
    check probes (archived, hasRecentCommits, issueActivityByProjectMember).
    These could serve as applicability detectors.

Probes (`docs/probes.md`)

Probe Documentation

This page describes each Scorecard probe in detail, including description, motivation,
and outcomes. If you have ideas for additions or new detection techniques,
please contribute!

archived

Lifecycle: stable

Description: Check that the project is archived

Motivation: An archived project will not received security patches, and is not actively tested or used.

Implementation: The probe checks the Archived Status of a project.

Outcomes: If the project is archived, the outcome is OutcomeTrue.
If the project is not archived, the outcome is OutcomeFalse.

blocksDeleteOnBranches

Lifecycle: stable

Description: Check that the project blocks non-admins from deleting branches.

Motivation: Allowing non-admins to delete project branches has a similar effect to performing force pushes.

Implementation: Checks the protection rules of default and release branches.

Outcomes: The probe returns one OutcomeTrue for each branch that is disallowed from users deleting it, and one OutcomeFalse for branches where users are able to delete it. Scorecard only considers default and releases branches.

blocksForcePushOnBranches

Lifecycle: stable

Description: Check that the project blocks force push on its branches.

Motivation: Allowing force pushes to branches could allow those with write access to make insecure changes to the behavior of the project.

Implementation: Checks the protection rules of default and release branches.

Outcomes: The probe returns one OutcomeTrue for each branch that is blocked from force pushes, and one OutcomeFalse for branches that allows force push.
Returns OutcomeNotAvailable if Scorecard cannot fetch the data from the repository.

branchProtectionAppliesToAdmins

Lifecycle: stable

Description: Check that the project's branch protection rules apply to project admins.

Motivation: Admins may be able to bypass branch protection settings which could defeat the purpose of having them.

Implementation: Checks the protection rules of default and release branches.

Outcomes: The probe returns one OutcomeTrue for each branch that enforces branch protection rules on admins, and one OutcomeFalse for branches that don't.

branchesAreProtected

Lifecycle: stable

Description: Check that the project uses protected branches.

Motivation: Unprotected branches may allow actions that could compromise the project's security.

Implementation: Checks the protection rules of default and release branches.

Outcomes: The probe returns one OutcomeTrue for each branch that is protected, and one OutcomeFalse for branches that are not protected. Scorecard only considers default and releases branches.

codeApproved

Lifecycle: experimental

Description: Check that all recent changesets have been approved by someone who is not the author of the changeset.

Motivation: To ensure that the review process works, the proposed changes should have a minimum number of approvals.

Implementation: This probe looks for whether all changes over the last --commit-depth commits have been approved before merge. Commits are grouped by the changeset they were introduced in, and each changesets must have at least one approval. Reviewed, bot authored changesets (e.g. dependabot) are not counted.

Outcomes: If all commits were approved, the probe returns OutcomeTrue
If any commits were not approved, the probe returns OutcomeFalse
If there are no changes, the probe returns OutcomeNotApplicable

codeReviewOneReviewers

Lifecycle: experimental

Description: Check that at least one reviewers review a change before merging.

Motivation: To ensure that the review process works, the proposed changes should have a minimum number of approvals.

Implementation: This probe looks for whether all changes over the last --commit-depth commits have been approved by a minimum number of reviewers. Commits are grouped by the Pull Request they were introduced in. Only unique reviewer logins that aren't the same as the changeset author are counted.

Outcomes: If all the changes had at least one reviewers, the probe returns OutcomeTrue (1)
If the changes had fewer than one reviewers, the prove returns OutcomeFalse (0)

contributorsFromOrgOrCompany

Lifecycle: experimental

Description: Checks whether a project has a contributions from users associated with a company or organization.

Motivation: This probe tries to determine if the project has recent contributors from multiple organizations. For some projects, having a diverse group of contributors is an indicator of project health.

Implementation: The probe looks at the Company field on the user profile for authors of recent commits. To receive the highest score, the project must have had contributors from at least 3 different companies in the last 30 commits.

Outcomes: If the project has no contributing organizations, the probe returns 1 OutcomeFalse
If the project has contributing organizations, the probe returns 1 OutcomeTrue per organization.

createdRecently

Lifecycle: stable

Description: Checks if the project was created in the last 90 days.

Motivation: Recently created repositories have been used for malicious forks / typosquatting attacks in the past. A newly created repo is not a strong signal on its own, but can be a useful piece of information.

Implementation: The implementation checks the creation date is within the last 90 days.

Outcomes: If the project was created within the last 90 days, the outcome is OutcomeTrue.
If the project is older than 90 days, the outcome is OutcomeFalse. The finding will include a "lookBackDays" value which is the time period that the probe looks back in.

dependencyUpdateToolConfigured

Lifecycle: stable

Description: Check that a dependency update tool config is present.

Motivation: Out-of-date dependencies make a project vulnerable to known flaws and prone to attacks. Tools can help the process of updating dependencies by scanning for outdated or insecure requirements, and opening a pull request to update them if found.

Implementation: The implementation looks for the presence of various config files for different dependency update tools.

Outcomes: If a dependency update tool is configured, the probe returns OutcomeTrue for each configuration.
If no tool is detected, the probe returns OutcomeFalse.

dismissesStaleReviews

Lifecycle: stable

Description: Check that the project dismisses stale reviews when new commits are pushed.

Motivation: When a project does not dismiss stale reviews, contributors can bring their pull requests to an approved state and then make unreviewed commits.

Implementation: Checks the protection rules of default and release branches.

Outcomes: The probe returns one OutcomeTrue for each branch that dismisses the stale status of PRs, and one OutcomeFalse for branches that don't.

fuzzed

Lifecycle: stable

Description: Check that the project is fuzzed

Motivation: Fuzzing, or fuzz testing, is the practice of feeding unexpected or random data into a program to expose bugs. Regular fuzzing is important to detect vulnerabilities that may be exploited by others, especially since attackers can also use fuzzing to find the same flaws.

Implementation: The implementation looks for various fuzzing function signatures, imports, configuration files, and external integration data.

Outcomes: If a fuzzing tool is found, one finding per tool with OutcomeTrue is returned.
If no fuzzing tool is found, or the project uses a tool we don't detect, one finding with OutcomeFalse is returned.

hasBinaryArtifacts

Lifecycle: stable

Description: Checks if the project has any binary files in its source tree.

Motivation: Binary files are not human readable so users and reviewers can't easily see what they do.

Implementation: The implementation looks for the presence of binary files. This is a more restrictive probe than "hasUnverifiedBinaryArtifacts" which excludes verified binary files.

Outcomes: If the probe finds binary files, it returns one OutcomeTrue for each binary file found.
If the probe finds no binary files, it returns a single OutcomeFalse.

hasDangerousWorkflowScriptInjection

Lifecycle: stable

Description: Check whether the project has GitHub Actions workflows that enable script injection.

Motivation: Script injections allow attackers to use untrusted input to access privileged resources (code execution, secret exfiltration, etc.)

Implementation: The probe analyzes the repository's workflows for known dangerous patterns.

Outcomes: The probe returns one finding with OutcomeTrue for each dangerous script injection pattern detected. Each finding may include a suggested patch to fix the respective script injection.
If no dangerous patterns are found, the probe returns one finding with OutcomeFalse.

hasDangerousWorkflowUntrustedCheckout

Lifecycle: stable

Description: Check whether the project has GitHub Actions workflows that does untrusted checkouts.

Motivation: GitHub workflows triggered with pull_request_target or workflow_run have write permission to the target repository and access to target repository secrets. Combined with a dangerous checkout of PR contents, attackers may be able to compromise the repository, for example, by using build scripts controlled by the PR author.

Implementation: The probe iterates through the workflows looking for pull_request_target and workflow_run triggers which checkout references from a PR. This check does not detect whether untrusted code checkouts are used safely, for example, only on pull request that have been assigned a label.

Outcomes: The probe returns one finding with OutcomeTrue per untrusted checkout.
The probe returns one finding with OutcomeFalse if no untrusted checkouts are detected.

hasFSFOrOSIApprovedLicense

Lifecycle: stable

Description: Check that the project has an FSF or OSI approved license.

Motivation: A license can give users information about how the source code may or may not be used. Using a recognized license facilitates security or legal reviews for potential users.

Implementation: The implementation checks whether a license file is present and is of an approved format. The list of FSF or OSI approved license is taken from https://spdx.org/licenses/ .

Outcomes: If a license file is found and is of an approved format, the probe returns a single OutcomeTrue.
If a license file is missing the probe returns a single OutcomeNotApplicable.
If the license is not of an approved format, the probe returns a single OutcomeFalse.

hasLicenseFile

Lifecycle: stable

Description: Check that the project has a license file

Motivation: A license can give users information about how the source code may or may not be used. The lack of a license will impede any kind of security review or audit and creates a legal risk for potential users.

Implementation: The implementation checks whether a license file is present.

Outcomes: If license files are found, the probe returns OutcomeTrue for each license file.
If a license file is not found, the probe returns a single OutcomeFalse.

hasNoGitHubWorkflowPermissionUnknown

Lifecycle: experimental

Description: Checks that GitHub workflows have workflows with unknown permissions

Motivation: Unknown permissions may be a result of a bug or another error from fetching the permission levels.

Implementation: The probe checks the permission levels of a projects workflows and collects the workflows that have unknown permissions.

Outcomes: The probe returns 1 false outcome per workflow without unknown permission level(s).
The probe returns 1 true outcome if the project has no workflows with unknown permission levels.

hasOSVVulnerabilities

Lifecycle: stable

Description: Check whether the project has known vulnerabilities

Motivation: This check determines whether the project has open, unfixed vulnerabilities in its own codebase or its dependencies using the OSV (Open Source Vulnerabilities) service. An open vulnerability may be exploited by attackers and should be fixed as soon as possible.

Implementation: The implementation fetches data from OSV.dev about the project which shows whether a given project has known, unfixed vulnerabilities. The implementation uses the number of known, unfixed vulnerabilities to score.

Outcomes: The probe returns one true outcome for each vulnerability found in OSV.
If there are no known vulnerabilities detected, the probe returns one false outcome.

hasOpenSSFBadge

Lifecycle: stable

Description: This check determines whether the project has an OpenSSF (formerly CII) Best Practices Badge.

Motivation: The OpenSSF Best Practices badge indicates whether or not the project uses a set of security-focused best development practices for open source software.

Implementation: The probe checks the badge level using the OpenSSF Best Practices Badge API.

Outcomes: If the project has a badge, the probe returns one OutcomeTrue finding. The finding includes the badge level as an entry in the Values map with the "badgeLevel" key.
If the project does not have a badge, the probe returns one OutcomeFalse.

hasPermissiveLicense

Lifecycle: stable

Description: Check that the project has a permissive license.

Motivation: A permissive license allows users to use the analyzed component to be used in derivative works. Non-permissive licenses (as copyleft licenses) might be a legal risk for potential users.

Implementation: The implementation checks whether a permissive license is present

Outcomes: If a license file is found and is permissive, the probe returns a single OutcomeTrue.
If a license file is missing the probe returns a single OutcomeFalse.
If the license is not permissive, the probe returns a single OutcomeFalse.

hasRecentCommits

Lifecycle: stable

Description: Check whether the project has at least one commit per week over the last 90 days.

Motivation: A project which is not active might not be patched, have its dependencies patched, or be actively tested and used. A lack of active maintenance should signal that potential users should investigate further to judge the situation. A project may not need further features or maintenance; In this case, the probe results can be disregarded.

Implementation: The implementation checks the number of commits made in the last 90 days by any user type.

Outcomes: If the project has commits from the last 90 days, the probe returns one OutcomeTrue with a "commitsWithinThreshold" value which contains the number of commits that the probe found within the threshold. The probe will also return a "lookBackDays" value which is the number of days that the probe includes in its threshold - which is 90.
If the project does not have commits in the last 90 days, the probe returns a single OutcomeFalse.

hasReleaseSBOM

Lifecycle: experimental

Description: Check that the project publishes an SBOM as part of its release artifacts.

Motivation: An SBOM can give users information about how the source code components and dependencies. They help facilitate software supplychain security and aid in identifying upstream vulnerabilities in a codebase.

Implementation: The implementation checks whether a SBOM artifact is included in release artifacts.

Outcomes: If SBOM artifacts are found, the probe returns OutcomeTrue for each SBOM artifact up to 5.
If an SBOM artifact is not found, the probe returns a single OutcomeFalse.

hasSBOM

Lifecycle: experimental

Description: Check that the project has an SBOM file

Motivation: An SBOM can give users information about how the source code components and dependencies. They help facilitate software supplychain security and aid in identifying upstream vulnerabilities in a codebase.

Implementation: The implementation checks whether an SBOM file is present in the source code.

Outcomes: If an SBOM file(s) is found, the probe returns OutcomeTrue for each SBOM artifact up to 5.
If an SBOM file is not found, the probe returns a single OutcomeFalse.

hasUnverifiedBinaryArtifacts

Lifecycle: stable

Description: Checks if the project has binary files in its source tree. The probe skips verified binary files which currently are gradle-wrappers.

Motivation: Binary files are not human readable so users and reviewers can't easily see what they do.

Implementation: The implementation looks for the presence of binary files that are not "verified". A verified binary is one that Scorecard considers valid for building and/or releasing the project. This is a more permissive probe than "hasBinaryArtifacts" which does not skip verified binary files.

Outcomes: If the probe finds unverified binary files, it returns OutcomeTrue for each unverified binary file found.
If the probe finds no unverified binary files, it returns OutcomeFalse.

issueActivityByProjectMember

Lifecycle: stable

Description: Checks that a collaborator, member or owner has participated in issues in the last 90 days.

Motivation: A project which does not respond to issues may not be actively maintained. A lack of active maintenance should signal that potential users should investigate further to judge the situation. However a project may simply not have any recent issues; In this case, the probe results can be disregarded.

Implementation: The probe checks whether collaborators, members or owners of a project have participated in issues in the last 90 days.

Outcomes: If collaborators, members or owners have participated in issues in the last 90 days, the probe returns one OutcomeTrue. The probe also returns a "numberOfIssuesUpdatedWithinThreshold" value with represents the number of issues on the repository which project collaborators, members or owners have shown activity in.
If collaborators, members or owners have NOT participated in issues in the last 90 days, the probe returns a single OutcomeFalse.

jobLevelPermissions

Lifecycle: experimental

Description: Checks that GitHub workflows do not have "write" permissions at the "job" level.

Motivation: In some circumstances, having "write" permissions at the "job" level may enable attackers to escalate privileges.

Implementation: The probe checks the permission level, the workflow type and the permission type of each workflow in the project.

Outcomes: The probe returns 1 false outcome per workflow with "write" permissions at the "job" level.
The probe returns 1 true outcome if the project has no workflows "write" permissions a the "job" level.

packagedWithAutomatedWorkflow

Lifecycle: stable

Description: Checks whether the project uses automated packaging.

Motivation: Packages give users of a project an easy way to download, install, update, and uninstall the software by a package manager. In particular, they make it easy for users to receive security patches as updates.

Implementation: The implementation checks whether a project uses common patterns for packaging across multiple ecosystems. Scorecard gets this by checking the projects workflows for specific uses of actions and build commands such as docker push or mvn deploy.

Outcomes: If the project uses a packaging mechanism we detect, the outcome is positive.
If the project doesn't use automated packaging we can detect, the outcome is negative.

pinsDependencies

Lifecycle: stable

Description: Check that the project pins dependencies to a specific digest.

Motivation: Pinned dependencies ensure that checking and deployment are all done with the same software, reducing deployment risks, simplifying debugging, and enabling reproducibility. They can help mitigate compromised dependencies from undermining the security of the project (in the case where you've evaluated the pinned dependency, you are confident it's not compromised, and a later version is released that is compromised).

Implementation: The probe works by looking for unpinned dependencies in Dockerfiles, shell scripts, and GitHub workflows which are used during the build and release process of a project. Special considerations for Go modules treat full semantic versions as pinned due to how the Go tool verifies downloaded content against the hashes when anyone first downloaded the module.

Outcomes: For supported ecosystem, the probe returns OutcomeTrue per pinned dependency.
For supported ecosystem, the probe returns OutcomeFalse per unpinned dependency.
If the project has no supported dependencies, the probe returns OutcomeNotApplicable.

releasesAreSigned

Lifecycle: stable

Description: Check that the projects GitHub and GitLab releases are signed.

Motivation: Signed releases allow consumers to verify their artifacts before consuming them.

Implementation: The implementation checks whether a signature file is present in release assets. The probe checks the last 5 releases on GitHub and GitLab.

Outcomes: For each of the last 5 releases, the probe returns OutcomeTrue, if the release has a signature file in the release assets.
For each of the last 5 releases, the probe returns OutcomeFalse, if the release does not have a signature file in the release assets.
If the project has no releases, the probe returns OutcomeNotApplicable.

releasesHaveProvenance

Lifecycle: stable

Description: Check that the projects releases on GitHub and GitLab have provenance.

Motivation: Provenance give users security-critical, verifiable information so that consumers can verify their artifacts before consuming them.

Implementation: The probe checks whether any of the assets in any of the last five releases on GitHub or GitLab have a provenance file.

Outcomes: For each of the last 5 releases, the probe returns OutcomeTrue, if the release has a provenance file in the release assets.
For each of the last 5 releases, the probe returns OutcomeFalse, if the release does not have a provenance file in the release assets.
If the project has no releases, the probe returns OutcomeNotApplicable.

releasesHaveVerifiedProvenance

Lifecycle: experimental

Description: Checks if the project releases with provenance attestations that have been verified

Motivation: Package provenance attestations provide a greater guarantee of authenticity and integrity than package signatures alone, since the attestation can be performed over a hash of both the package contents and metadata. Developers can attest to particular qualities of the build, such as the build environment, build steps or builder identity.

Implementation: This probe checks how many packages published by the repository are associated with verified SLSA provenance attestations. It uses data from a ProjectPackageClient, which associates a GitHub/GitLab project with a package in a package manager. Using the data from the package manager (whom we rely on to verify the provenance attestation), this probe returns a finding for each release. For now, only NPM is supported.

Outcomes: For each release, the probe returns OutcomeTrue or OutcomeFalse, depending on if the package has a verified provenance attestation.
If we didn't find a package or didn't find releases, return OutcomeNotAvailable.

requiresApproversForPullRequests

Lifecycle: stable

Description: Check that the project requires approvers for pull requests.

Motivation: Requiring approvers for pull requests makes it harder to introduce vulnerable code to the project.

Implementation: The probe checks the number of required approvers in default and release branches of the project.

Outcomes: The probe returns one OutcomeTrue for each branch that requires approval for PRs, and one OutcomeFalse for branches that don't.

requiresCodeOwnersReview

Lifecycle: stable

Description: Check that the project requires dedicated code owners to review PRs.

Motivation: Code owners are expected to have deep knowledge about a code; Having experienced reviewers for PRs is expected to prevent security issues.

Implementation: The probe checks which branches require code owner reviews. The probe only considers default and release branches.

Outcomes: The probe returns one OutcomeTrue for each branch that requires code owner review for PRs, and one OutcomeFalse for branches that don't.

requiresLastPushApproval

Lifecycle: stable

Description: Check that the project requires approval of the most recent push.

Motivation: Requiring approval of the most recent push prevents contributors from sneaking malicious commits into a PR after it has been approved.

Implementation: The probe checks the protection rules of default and release branches branches.

Outcomes: The probe returns one OutcomeTrue for each branch that requires approval of the most recent push, and one OutcomeFalse for branches that don't.

requiresPRsToChangeCode

Lifecycle: stable

Description: Check that the project requires pull requests to change code.

Motivation: Changing code through pull requests promotes testing and reviews of the suggested change.

Implementation: The probe checks which branches require pull requests to change the branches' code. The probe only considers default and release branches.

Outcomes: The probe returns one OutcomeTrue for each branch that requires pull requests to change code, and one OutcomeFalse for branches that don't.

requiresUpToDateBranches

Lifecycle: stable

Description: Check that the project requires PRs to be in sync with the base branch.

Motivation: Requiring PRs to be in sync with the base branch is good practice.

Implementation: The probe checks the branch protection rules of default and release branches in the repository. For GitHub-hosted projects, an active repository ruleset that requires a merge queue also satisfies this probe, since a merge queue tests each change against the latest version of the base branch before merging.

Outcomes: The probe returns one OutcomeTrue for each branch that requires PRs to be in sync with the base branch, and one OutcomeFalse for branches that don't.

runsStatusChecksBeforeMerging

Lifecycle: stable

Description: Check that the project runs required status checks

Motivation: Required status checks can check for common errors and resolve issues in PRs.

Implementation: The probe checks the rules for default and release branches in the projects repository.

Outcomes: The probe returns one OutcomeTrue for each branch that runs required status checks, and one OutcomeFalse for branches that don't.

sastToolConfigured

Lifecycle: stable

Description: Check that the project uses a SAST tool

Motivation: SAST is testing run on source code before the application is run. Using SAST tools can prevent known classes of bugs from being inadvertently introduced in the codebase.

Implementation: The implementation checks for evidence of various SAST tools. This includes configuration files, GitHub Action workflows, and GitHub PR check annotations.

Outcomes: If the project uses a SAST tool we can detect, the probe returns one finding per tool with OutcomeTrue.
If the project does not use a SAST tool, or uses a tool we dont currently detect, the probe returns one finding with OutcomeFalse.

sastToolRunsOnAllCommits

Lifecycle: experimental

Description: Checks that a SAST tool runs on all commits in the projects CI.

Motivation: SAST is testing run on source code before the application is run. Using SAST tools can prevent known classes of bugs from being inadvertently introduced in the codebase.

Implementation: The implementation iterates through the projects commits and checks whether any of the check runs for the commits associated merge request was any of the SAST tools that Scorecard supports.

Outcomes: If the project had no commits merged, the probe returns a finding with OutcomeNotApplicable.
If the project runs SAST tools successfully on every pull request before merging, the probe returns one finding with OutcomeTrue (1). In addition, the finding will include two values. 1) How many commits were tested by a SAST tool, and 2) How many commits in total were merged.
If the project does not run any SAST tools successfully on every pull request before merging, the probe returns one finding with OutcomeFalse (0). In addition, the finding will include two values. 1) How many commits were tested by a SAST tool, and 2) How many commits in total were merged.

Lifecycle: stable

Description: Check that the security policy contains web or email links.

Motivation: URLs point users to additional information as well as online disclosure forms. Emails provide a point of contact for vulnerability disclosure.

Implementation: The implementation looks for strings "http(s)://" to find URLs; and for strings "...@..." for email addresses.

Outcomes: If links are found, one finding with OutcomeTrue is returned for each security policy file.
If no links are found, one finding with OutcomeFalse is returned for each security policy file.
If no security policy files are found, one finding with OutcomeFalse is returned.

securityPolicyContainsText

Lifecycle: stable

Description: Check that the security policy contains enough text and not just links.

Motivation: Telling security researchers how to privately disclose problems with your project is important. The more details available, the better.

Implementation: The implementation checks that the content of the SECURITY.md contains more than just a link or an email address. It does this by comparing the length of the content to the lengths of the links and email addresses.

Outcomes: If explanatory text is found in a security policy, one finding with OutcomeTrue is returned for each file.
If no additional text is found in a security policy, one finding with OutcomeFalse is returned for each file.
If no security policy is found, one finding with OutcomeFalse is returned.

securityPolicyContainsVulnerabilityDisclosure

Lifecycle: stable

Description: Check that the security policy indicates a vulnerability disclosure process.

Motivation: If someone finds a vulnerability in the project, it is important for them to be able to communicate it to the maintainers.

Implementation: The implementation looks for strings "Disclos" and "Vuln".

Outcomes: If information about the disclosure process is found in a security policy file, the probe returns one finding with OutcomeTrue for each file.
If no information about the disclosure process is found, the probe returns one finding with OutcomeFalse for each file.
If no security policy is found, the probe returns one finding with OutcomeFalse.

securityPolicyPresent

Lifecycle: stable

Description: Check if a security policy is defined in the repository or in the org's .github repository.

Motivation: A security policy (typically a SECURITY.md file) can give users information about what constitutes a vulnerability and how to report one securely so that information about a bug is not publicly visible. If you have a large organization, having a unified security policy across all your repositories may simplify the vulnerability disclosure response.

Implementation: The implementation looks for the presence of security policy files in the repository or in '/.github' repository. See https://github.com/ossf/scorecard/blob/main/checks/raw/security_policy.go#L139 for a detailed list of filenames.

Outcomes: If a security policy file is found, one finding with OutcomeTrue is returned.
If no security file is found, one finding with OutcomeFalse is returned.

testsRunInCI

Lifecycle: stable

Description: Checks that the project runs tests in the CI for example with GitHub Actions or Prow.

Motivation: Running tests helps developers catch mistakes early on, which can reduce the number of vulnerabilities that find their way into a project.

Implementation: The probe checks for tests in the projects CI jobs in the recent commits (~30).

Outcomes: The probe returns one OutcomeTrue for each PR that ran CI tests and one OutcomeFalse for each PR that did not run CI tests.
The probe returns a single OutcomeNotApplicable if the projects has had no pull requests.

topLevelPermissions

Lifecycle: experimental

Description: Checks that the project does not have any top-level write permissions in its workflows.

Motivation: In some circumstances, having "write" permissions at the "top" level may enable attackers to escalate privileges.

Implementation: The probe checks the permission level, the workflow type and the permission type of each workflow in the project.

Outcomes: The probe returns 1 false outcome per workflow with "write" permissions at the "top" level.
The probe returns 1 true outcome if the project has no workflows "write" permissions a the "top" level.

unsafeblock

Lifecycle: experimental

Description: Flags unsafe blocks of code in this project.

Motivation: Memory safety in software should be considered a continuum, rather than being binary. While some languages and tools are memory safe by default, it may still be possible, and sometimes unavoidable, to write unsafe code in them. Unsafe code allow developers to bypass normal safety checks and directly manipulate memory.

Implementation: The probe is ecosystem-specific and will surface non memory safe practices in the project by identifying unsafe code blocks. Unsafe code blocks are supported in rust, go, c#, Java, and swift, but only go, c# and Java are supported by this probe at this time: - for go the probe will look for the use of the unsafe include directive. - for c# the probe will look at the csproj and identify the use of the AllowUnsafeBlocks property. - for Java the probe will look at references to either the sun.misc.Unsafe class or the jdk.internal.misc.Unsafe class.

Outcomes: For supported ecosystem, the probe returns OutcomeTrue per unsafe block.
If the project has no unsafe blocks, the probe returns OutcomeFalse.

webhooksUseSecrets

Lifecycle: experimental

Description: This check determines whether the webhooks defined in the repository have secrets configured to authenticate the origins of requests.

Motivation: Webhooks without secret authorization have the potential to make projects accessible to third-parties.

Implementation: The probe checks all webhooks of a project and checks whether each uses secret authentication.

Outcomes: The probe returns one OutcomeTrue per webhook with secret authorization.
The probe returns one OutcomeFalse per webhook without secret authorization.
Projects without webhooks receive an OutcomeNotApplicable.


Repositories (`docs/repositories.md`)

Repository Guidelines

This document attempts to outline a structure for creating and associating
GitHub repositories with the OpenSSF Scorecard project.

Approval

New repositories require approval from the OpenSSF Scorecard Steering Committee.

Requirements

The following requirements apply to all OpenSSF Scorecard repositories:

  • Must be identified in the OpenSSF Scorecard project documentation

  • Must reside in the OpenSSF GitHub organization

  • Must utilize the topic openssf-scorecard (ref: managing topics)

  • Must adopt the OpenSSF Scorecard Code of Conduct

  • Must adopt an appropriate license, in compliance with the Intellectual Property Policy of OpenSSF Scorecard charter

  • Must include headers across all files that attribute copyright as follows:

    text
    Copyright [YYYY] OpenSSF Scorecard Authors
  • Must enforce usage of the Developer Certificate of Origin (DCO) via the DCO GitHub Application

  • All privileges to the repository must be defined via GitHub teams, instead of individuals

  • All code review permissions must be defined via CODEOWNERS

  • All contributors with privileges to the repository must also be active members of the OpenSSF Scorecard project

Donated repositories

The OpenSSF Scorecard project may at times accept repository donations.

Donated repositories must:

The addition of required copyright headers to code created by the contributors
can occur post-transfer, but should ideally occur shortly thereafter.

Note that copyright notices should only be modified or removed by the people or organizations named in the notice.

Attribution

These guidelines were drafted with inspiration from the Kubernetes project's repository guidelines.


Design Cdn (`docs/design/cdn.md`)

Scorecard CDN

  • Status: Complete

Background

Serving the weekly scan results at api.scorecard.dev represents a sizable
chunk of our monthly cloud bill, both in terms of number of requests (30 million
per week) and bandwidth egress (680 GiB). These results are largely static,
updating once per week for most repositories, representing a great opportunity
to use a CDN. The OpenSSF Scorecard project successfully applied to Fastly’s
Fast Forward program, so we have CDN credits available for use.

The Scorecard API has three main endpoints:

  • GET /projects/{platform}/{org}/{repo}: Get a repository's Scorecard
  • POST /projects/{platform}/{org}/{repo}: Publish results from Scorecard Action
  • GET /projects/{platform}/{org}/{repo}/badge: Get a repository's Scorecard badge

This document focuses on the two GET endpoints, as we don’t want to prevent
repositories from uploading their latest scores with Scorecard Action.

Request Traffic

In the last week, the Scorecard API received 30.1M requests (50 QPS), the
traffic can be classified simply by its HTTP response status code:

Response Code Description Requests Update Frequency
200 Result request for a project in our corpus 17.7M (59%) 7 days if published by weekly scan, or variable if by Scorecard Action.
404 Result request for a project not in our corpus 11.6M (38%) Never, until a project is added to the weekly scans or installs Scorecard Action
302 Badge request 0.83M (3%) Never, assuming we don’t change providers

Caching Strategy

DNS Entries

We modified our DNS to connect our app to Fastly. We did this in two phases:

  1. Added a new CNAME DNS entry (cdn.scorecard.dev), so we could test the CDN
    without affecting our normal traffic
cdn.scorecard.dev. 3600 in cname
dualstack.<letter>.sni.global.fastly.net.
  1. Changed the api.scorecard.dev
    CNAME entry away from the Cloud Run entry, towards Fastly instead.
text
api.scorecard.dev. 3600 IN CNAME dualstack.<letter>.sni.global.fastly.net.

Time-to-Live (TTL)

Our caching strategy is to cache all responses for the maximum TTL supported by
Fastly (1 year). We rely on cache purging to invalidate results when new results
are available. This provides slightly more flexibility if results ever update at
a different timeframe (either due to issues with our weekly scan, or if we
intentionally reduce frequency).

We implemented caching with the Surrogate-Control header to support split
policies, namely so we can set longer cache times for Fastly while maintaining
shorter cache times for browsers.

Note: While uncommon, approximately 80,000 requests per week (0.3%) set
the commit query parameter to get results from another commit. Requests
containing the commit query parameter should be treated as unique cache keys,
allowing them to be cached separately from the commit-less requests.

Purging

Fastly supports purging individual URLs, as well as groups of URLs, and we make
use of both in our strategy. We purge our result endpoint URLs individually when
new results become available. Our weekly scans process approximately 10,000
repos per hour, which is below Fastly’s purge API limit of 100,000/hr. The badge
endpoint would only need to be changed if we make infrastructure changes, so
they can use a surrogate key to enable purging as a group if manually needed by
maintainers.

One important thing to note is authentication. URL purges are unauthenticated by
default in Fastly, so anyone could send a PURGE request with curl -X PURGE so we have enabled authenticated purges.

Shielding

Fastly operates multiple points of presence (POP) across the globe, which act
independently, including when contacting the origin server. We have designated
their Chicago POP as an Origin Shield, which will route all origin requests
through a single POP, reducing origin load further.


Design Scalable Scorecard (`docs/design/scalable_scorecard.md`)

Scalable Scorecard

Scale OSSF Scorecard to 100k+ repositories.

  • Author: Azeem
  • Last Updated: 2021-09-23
  • Status: Complete

Background

Scorecard is an open source project by Google in
collaboration with OpenSSF to automate the analysis and
trust decisions on the security posture of an open source project. As part of
this project we (Google) maintain a cron job, which calculates Scorecard results
on a predefined list of open source projects and makes it available to the
public for consumption.

The current cron/batch job runs on ~2k repositories. It utilizes a single worker
machine to process all repositories and does so by running the Scorecard
command-line tool and processing the stdout output of this tool. This has caused
us problems in the past (TODO: add links to issues) with reliability and
maintainability of this service.

The performance of this service is also questionable. It takes ~18+hrs to
process the current list of ~2k repositories. The main bottleneck here has been
the usage of GitHub tokens which are rate-limited to 5k API requests per-hour
per token. To overcome this, the current service uses 6 different tokens of
unique GitHub accounts, making our rate limit to be 30k API requests per-hour.

We propose a re-design of this service such that the underlying machine
infrastructure can handle a larger number of repositories and a more efficient
usage of GitHub tokens for better throughput. Following the Google convention,
our aim in this document is to scale the current solution by 10x and we’ll be
proposing a solution capable of handling 100k+ repositories.

Proposal

We propose the following work areas to help improve Scorecard scalability:

  • Sharded batch job
  • GitHub token optimization
  • Reliability and Testing
  • Developer journey

These areas are listed in approximately decreasing order of priority.

Sharded Batch Job

The current cron/batch job works on ~2k repositories and utilizes a single
worker machine to process all the repositories. Below we propose an option to
provide horizontal scalability to this architecture which could help scale this
solution up further.

PubSub architecture

All repositories to be processed will be pushed to a single PubSub Topic. Each
published request will represent a single shard to be output by the PubSub
subscribers/workers. Multiple workers can read and process requests from this
Topic independently, providing required parallelization. Every processed request
will result in a new shard being output to Google Cloud Storage.

Pros:

  • Horizontal scalability - can add more workers if needed to improve
    performance.
  • Reliability - if workers go down during processing a request, requests can be
    retried easily.
  • Can use Golang’s
    generic PubSub package to make this
    work across different cloud platforms.

Token Sharing

A single GitHub token cannot be used concurrently. Along with the primary rate
limit (api_requests per hour), GitHub servers apply a secondary rate limit
(abuse detection). The abuse detection mechanism gets triggered if a token is
used concurrently. Repeatedly triggering the abuse rate limits causes the token
to be bricked. To avoid this, we need to consider how the GitHub token will be
shared across workers (nodes) in an efficient way.

We propose maintaining a TokenServer which exposes GetToken() and
ReleaseToken() RPC methods. Tokens will be provided to the service using a
Kubernetes secret which it consumes as an environment variable. The PubSub
workers use the exposed RPC method to obtain tokens for their usage. TokenServer
takes care of efficiently distributing the tokens (e.g using RoundRobin method).

Storage and Database Schema


Things to consider:

  • We expect that the most common query from our users would be to get the latest
    (or at a particular date) Scorecard results for a given dependency. So we’ll
    design our schema to optimize this query.
  • BigQuery performs best when schema is denormalized and encourages nested
    repeated columns in schema -
    images/scorecard_denormalization.png.
  • Limitation on nested repeated fields is that nesting cannot be more than 15
    levels -
    images/scorecard_limitation_nested_fields.png.
  • Does it make sense to tag our results with the package version along with the
    date? It might be more useful for users to lookup Scorecard for a package by
    version rather than date?

BigQuery Schema

Note that below specifies the schema in Protobuf format for brevity only.

protobuf
// BigQuerySchema

message Check {
  optional string name = 1;
  optional bool pass = 2;
  optional int confidence = 3;
}

message BigQuerySchemaV1 {
  optional string repo = 1;
  optional datetime date = 2;
  repeated Check checks = 3;
}

Partitioned Tables

BQ allows users to partition tables based on the column value of TIMESTAMP.
With Scorecard scaling to 100K repos and eventually to 1M, the number of rows
that will be appended periodically to the BigQuery table will be significant.
We propose partitioning the BigQuery table using the date field in the schema.
This will create a new partition for the day the results were calculated. This
will improve query performance and will allow users to access latest results
easily.

Appending Data

BQ streaming API seems limited due to quotas, constraints and language
(the efficient gRPC API is only available in Java).
So using
BQ Data Transfer Service
is currently the only viable solution.

BQ Data Transfer Service can read
Cloud Storage URIs with wildcards.
So each worker in our sharded batch job can write to a separate file inside a
Cloud Storage bucket. We can then setup a recurring data transfer job to load
data into BQ.

Interaction With Batch Job

To make sure that BQ data transfer job and the sharded batch job interact
smoothly with each other, we propose:

  • The PubSub controller owns the logic for creating shards and decides the shard
    size. This ensures that the number of shards to be expected per run are
    deterministic and available at the time of PubSub request creation.
  • The PubSub controller writes the num_expected_shards from a run to the
    GCS bucket.
  • The PubSub worker creates a single shard for every request it receives from
    the topic.
  • BQ data transfer (BQDT) periodically reads from the GCS folder of a single run
    and initiates a transfer to BQ only if:
    num_expected_shards == num_created_shards.
  • All GCS folders created by the daily batch job will have a TTL of 15 days.
    This ensures both data backup and garbage collection.

View of Latest Data

We will also create a View on our BQ dataset which points to the latest
partition in the dataset. This should provide a smooth interface for users to
query the latest results.

Alternatives Considered

MapReduce architecture (with intermediate storage)

The repositories to be processed could be sharded and Mapper jobs will then send
these smaller chunks of repositories to Workers. In addition, we could choose to
add a layer of intermediate storage between Mapper and Reducer (if one isn’t
already provided by the framework) to help improve reliability in case the
Mapper or Reducer worker goes down.

Pros:

  • More robust than the pub sub architecture in terms of reliability.
  • We can use existing MapReduce frameworks for our purpose (e.g
    BigQuery connector).
  • Horizontal scalability - easy to add more mapper/reducers.

Cons:

  • No out-of-the-box solution provided by Cloud providers and will require lot of
    custom logic and service maintenance.

Diagram:

BigQuery Schema

protobuf
// BigQuerySchema

message Check {
  optional string name = 1;
  optional bool pass = 2;
  optional int confidence = 3;
}

message BigQuerySchemaV2 {
  optional string repo = 1;
  optional datetime date = 2;
  optional Check active = 3;
  optional Check reviews = 4;
  …
}

Pros:

  • Better queriability - clients can choose to select which checks they want to
    query.
  • If in future, we choose to have different structs representing different
    checks, it would be advantageous to have this structure.

Cons:

  • Since each check is explicitly mentioned, adding a new check would require
    updating the BQ schema.
  • There is no performance or scalability advantage for this added complexity.

Token Sharing

  • Per node tokens - a set of tokens will be assigned to every node which it will
    use in a round-robin way.
    • Inefficient - tokens might be under utilized.
    • DevOps will be difficult to automate.
  • Oliver Chang: every node will have access to all tokens. This reduces DevOps
    complexity at the cost of efficiency.
    • Does not allow for a high degree of parallelism.
    • This option was used in the initial development stages before moving to
      the β€œToken Server” option.

GitHub Tokens

There are 2 problems with the current usage pattern of GitHub APIs in our code:

  • Most checks need O(n) API calls.
  • The same API call is repeated across checks.

To improve the efficiency of GitHub token usage we propose:

  1. Using the GraphQL API instead of REST. This will help bring down the number
    of API calls from O(n) to O(1).
  2. Expose high-level APIs (e.g GetMergedPRs(), ListCommits() etc.) which
    under the hood cache the results for a repository and ensure there are no
    repeated calls to the same API.
  3. At the architecture level, we don’t expect all repos to be changing as
    frequently as we run the Scorecard job. We should consider using
    GitHub’s conditional requests.
    Although not all API endpoints support this today, it's a good starting
    point.

Both (1) and (2) can be achieved by implementing a RepoClient interface as
described here:
https://github.com/ossf/scorecard/issues/462

Reliability and Testing

Discussion:
https://github.com/ossf/scorecard/issues/651

As part of improving the reliability of the cron, we propose setting up an
end-to-end integration testing framework. This will also serve as our release
process. We propose:

  • Scorecard images will be built with COMMIT_SHA and latest tags using
    CloudBuild triggers.
  • The PubSub controller will generate a shard_metadata file which will
    contain the COMMIT_SHA used to generate the shards.
  • The release tests will trigger every 24 hours and use the latest tag
    binaries.
  • The tests will run on a sample of the repos used during the weekly runs.
    • There will be a separate BQ table and a successful data transfer to this
      table will be considered as the test successful.
    • There will be sanity tests like ensuring no runtime errors occur during
      the running of the test.
    • This will not test the result quality.
  • If the release tests are successful:
    • The release tests will trigger a webhook which tags the docker image
      corresponding to COMMIT_SHA tag with stable.
  • The weekly cron job uses the stable tag binaries and goes through the same
    process as the release test. The only difference being that a successful
    run will trigger a different webhook which generates a new Git tag (using
    SemVer) for the associated COMMIT_SHA.

Developer Journeys

Scalability of a project like Scorecard greatly benefits from wider developer
usage and contributions from Open Source developers. Currently, the project is
highly developer unfriendly for anyone trying it out for the first time. Towards
this end, we need efforts to:

  • Add non-hermetic tests which are not flaky and do not fail based on
    environment variables and access to GCS.
  • Better unit test coverage to add confidence for any incoming PRs.
  • Better documentation.

Security Capability Catalog.Yaml (`docs/security/capability-catalog.yaml`)

Licensed under the Apache License, Version 2.0 (the "License");

you may not use this file except in compliance with the License.

You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software

distributed under the License is distributed on an "AS IS" BASIS,

WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

See the License for the specific language governing permissions and

limitations under the License.

metadata:
id: OSSF.SCORECARD.ENGINE.CAPS
type: CapabilityCatalog
gemara-version: "1.6.0"
version: "1.0.0"
Capabilities of the Scorecard scan engine and CLI: the scorecard binary,
its container image, the pkg/scorecard Go API, scorecard serve, and
scorecard-attestor. The release pipeline and the hosted services in
other repositories are out of scope. See ARCHITECTURE.md for the actors
and the scan pipeline. Each capability's "Specs" line names the
OpenSpec specs in openspec/specs/ that define its behavior.
author:
id: ossf-scorecard
name: OpenSSF Scorecard maintainers
type: Human
mapping-references:
- id: CCC
title: Common Cloud Controls Core
version: v2025.10
url: https://github.com/finos/common-cloud-controls/releases
title: Scorecard Scan Engine Capabilities

groups:

  • id: ingestion
    title: Ingestion
  • id: analysis
    title: Analysis
  • id: access
    title: Access
    imports:
  • reference-id: CCC
    entries:
    • reference-id: CCC.Core.CP14
      remarks: API Access. Scorecard calls repository host and auxiliary service APIs.
    • reference-id: CCC.Core.CP18
      remarks: Resource Versioning. A scan targets HEAD or a specific commit.
    • reference-id: CCC.Core.CP27
      remarks: Configurable Network Ports. scorecard serve listens on PORT.
    • reference-id: CCC.Core.CP28
      remarks: Command-line Interface. The scorecard CLI.
    • reference-id: CCC.Core.CP29
      remarks: Active Ingestion. Scorecard downloads repositories and fetches metadata.

capabilities:

  • id: OSSF.SCORECARD.ENGINE.CAP01
    title: Repository Content Retrieval
    Downloads the scanned repository at a commit as a tarball (GitHub,
    GitLab) or zip (Azure DevOps) and extracts it to a temporary directory,
    or reads a local directory. The content is controlled by whoever can
    write to the scanned repository.
    Specs: repo-clients.
    group: ingestion
  • id: OSSF.SCORECARD.ENGINE.CAP02
    title: Repository Host Metadata Collection
    Queries the repository host's REST and GraphQL APIs for branch
    protection, releases, commits, pull requests, check runs, contributors,
    and similar metadata. Hosts include GitHub Enterprise and self-managed
    GitLab instances chosen by the user.
    Specs: repo-clients, raw-data-collection.
    group: ingestion
  • id: OSSF.SCORECARD.ENGINE.CAP03
    title: Package Name Resolution
    Looks up a package on npm, PyPI, RubyGems, or NuGet and scans the source
    repository named in the package's metadata. The metadata is controlled
    by whoever publishes the package.
    Specs: package-manager-resolution.
    group: ingestion
  • id: OSSF.SCORECARD.ENGINE.CAP04
    title: Auxiliary Signal Lookup
    Fetches signals that live outside the repository: known vulnerabilities
    through OSV-Scanner and OSV, OSS-Fuzz enrollment, and the OpenSSF Best
    Practices badge level.
    Specs: repo-clients.
    group: ingestion
  • id: OSSF.SCORECARD.ENGINE.CAP05
    title: Repository File Analysis
    Parses files from the scanned repository, including GitHub and GitLab
    workflows, Dockerfiles, shell scripts, dependency manifests, and
    lockfiles, to collect raw data for checks. Scorecard parses these files;
    it doesn't build or run the scanned repository's code.
    Specs: raw-data-collection.
    group: analysis
  • id: OSSF.SCORECARD.ENGINE.CAP06
    title: Maintainer Annotation Loading
    Reads a Scorecard configuration file from the scanned repository and
    attaches the maintainers' explanations to check results. Annotations
    don't change scores.
    Specs: maintainer-annotations.
    group: analysis
  • id: OSSF.SCORECARD.ENGINE.CAP07
    title: Scoring and Result Rendering
    Turns raw data into probe findings, scores each check from 0 to 10 (or
    -1 when inconclusive), and renders the result as text, JSON, SARIF,
    probe findings, raw data, or an in-toto statement. An optional policy
    file selects checks and sets SARIF thresholds.
    Specs: probes, check-evaluation, aggregate-scoring, result-formats,
    policy, remediation.
    group: analysis
  • id: OSSF.SCORECARD.ENGINE.CAP08
    title: Host Authentication
    Authenticates to repository hosts with tokens from the environment
    (GITHUB_AUTH_TOKEN and alternatives, GITLAB_AUTH_TOKEN,
    AZURE_DEVOPS_AUTH_TOKEN) or a GitHub App key. The tokens' scope is
    whatever the user granted.
    Specs: repo-clients.
    group: access
  • id: OSSF.SCORECARD.ENGINE.CAP09
    title: HTTP Scan Service
    scorecard serve accepts scan requests over HTTP and returns results as
    JSON, running each scan with the operator's credentials. It has no
    authentication or rate limiting and is intended for trusted networks.
    Specs: http-server.
    group: access
  • id: OSSF.SCORECARD.ENGINE.CAP10
    title: Scan Invocation
    Accepts a scan request from CLI flags and environment variables or from
    Go API options, and selects which checks or probes to run.
    Specs: cli, public-go-api, check-registry.
    group: access
  • id: OSSF.SCORECARD.ENGINE.CAP11
    title: Image Attestation
    scorecard-attestor runs checks on the source repository and commit a
    container image's pipeline names, evaluates the raw results against an
    attestation policy, and when the policy passes, signs a Google Cloud
    Binary Authorization attestation with a Cloud KMS, PGP, or PKIX key.
    Specs: attestor.
    group: access

Security Threat Catalog.Yaml (`docs/security/threat-catalog.yaml`)

Licensed under the Apache License, Version 2.0 (the "License");

you may not use this file except in compliance with the License.

You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software

distributed under the License is distributed on an "AS IS" BASIS,

WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

See the License for the specific language governing permissions and

limitations under the License.

metadata:
id: OSSF.SCORECARD.ENGINE.THREATS
type: ThreatCatalog
gemara-version: "1.6.0"
version: "1.0.0"
Threats to the Scorecard scan engine and CLI, and to the people who run
it or rely on its results. Grouped by STRIDE. Capabilities are defined in
capability-catalog.yaml.
author:
id: ossf-scorecard
name: OpenSSF Scorecard maintainers
type: Human
mapping-references:
- id: CCC
title: Common Cloud Controls Core
version: v2025.10
url: https://github.com/finos/common-cloud-controls/releases
- id: OSSF.SCORECARD.ENGINE.CAPS
title: Scorecard Scan Engine Capabilities
version: "1.0.0"

title: Scorecard Scan Engine Threats

groups:

  • id: spoofing
    title: Spoofing
  • id: tampering
    title: Tampering
  • id: information-disclosure
    title: Information Disclosure
  • id: denial-of-service
    title: Denial of Service
  • id: elevation-of-privilege
    title: Elevation of Privilege
    imports:
  • reference-id: CCC
    entries:
    • reference-id: CCC.Core.TH02
      remarks: Data is Intercepted in Transit. Applies to host and service API calls.
    • reference-id: CCC.Core.TH14
      remarks: Older Resource Versions are Used. Applies to results read after the repository changed.
    • reference-id: CCC.Core.TH17
      remarks: Responses are Generated for Unauthorized Requests. Applies to scorecard serve.

threats:

  • id: OSSF.SCORECARD.ENGINE.THR01
    title: Scanned Repository Shapes Its Own Score
    Checks infer practices from what is visible in the repository and host
    metadata. A repository owner can add files, workflows, or settings that
    satisfy a check's heuristics without following the practice the check
    measures, so consumers overestimate the project's security posture.
    group: tampering
    capabilities:

    • reference-id: OSSF.SCORECARD.ENGINE.CAPS
      entries:
      • reference-id: OSSF.SCORECARD.ENGINE.CAP01
      • reference-id: OSSF.SCORECARD.ENGINE.CAP05
      • reference-id: OSSF.SCORECARD.ENGINE.CAP07
        actors:
    • id: repository-owner
      name: Owner of the scanned repository
      type: Human
  • id: OSSF.SCORECARD.ENGINE.THR02
    title: Self-Asserted Annotations Explain Away Findings
    Maintainer annotations are statements by the scanned repository's
    owners that Scorecard displays without verifying. They don't change
    scores, but they can persuade a reader to discount a low score, and
    SARIF output omits annotated checks entirely.
    group: tampering
    capabilities:

    • reference-id: OSSF.SCORECARD.ENGINE.CAPS
      entries:
      • reference-id: OSSF.SCORECARD.ENGINE.CAP06
      • reference-id: OSSF.SCORECARD.ENGINE.CAP07
        actors:
    • id: repository-owner
      name: Owner of the scanned repository
      type: Human
  • id: OSSF.SCORECARD.ENGINE.THR03
    title: Package Metadata Points at Another Repository
    When a user scans a package by name, Scorecard scans whatever source
    repository the package's registry metadata names. A package publisher
    can name an unrelated, well-maintained repository, so the package
    borrows that repository's score.
    group: spoofing
    capabilities:

    • reference-id: CCC
      entries:
      • reference-id: CCC.Core.CP29
    • reference-id: OSSF.SCORECARD.ENGINE.CAPS
      entries:
      • reference-id: OSSF.SCORECARD.ENGINE.CAP03
        actors:
    • id: package-publisher
      name: Package publisher
      type: Human
  • id: OSSF.SCORECARD.ENGINE.THR04
    title: Repository Host Returns Misleading Data
    Results depend on the host's API responses and archives. A compromised
    or attacker-run host, such as a self-managed GitLab instance, can return
    data that misrepresents the repository and produces a misleading result.
    group: spoofing
    capabilities:

    • reference-id: CCC
      entries:
      • reference-id: CCC.Core.CP14
    • reference-id: OSSF.SCORECARD.ENGINE.CAPS
      entries:
      • reference-id: OSSF.SCORECARD.ENGINE.CAP01
      • reference-id: OSSF.SCORECARD.ENGINE.CAP02
        actors:
    • id: host-operator
      name: Operator of a repository host
      type: Human
  • id: OSSF.SCORECARD.ENGINE.THR05
    title: Archive Entries Write Outside the Scan Directory
    A crafted archive could use relative paths or links to write files
    outside the temporary directory during extraction. Extraction rejects
    entries whose cleaned path leaves the destination and skips symbolic
    links.
    group: elevation-of-privilege
    capabilities:

    • reference-id: OSSF.SCORECARD.ENGINE.CAPS
      entries:
      • reference-id: OSSF.SCORECARD.ENGINE.CAP01
        actors:
    • id: repository-owner
      name: Owner of the scanned repository
      type: Human
    • id: host-operator
      name: Operator of a repository host
      type: Human
  • id: OSSF.SCORECARD.ENGINE.THR06
    title: Crafted Files Disrupt Parsing
    Malformed or adversarial workflows, Dockerfiles, shell scripts,
    manifests, or lockfiles can make a parser fail, hang, or crash, which
    makes checks inconclusive or stops the scan. The 2025 third-party audit
    fuzzed several of these parsers.
    group: denial-of-service
    capabilities:

    • reference-id: OSSF.SCORECARD.ENGINE.CAPS
      entries:
      • reference-id: OSSF.SCORECARD.ENGINE.CAP04
      • reference-id: OSSF.SCORECARD.ENGINE.CAP05
      • reference-id: OSSF.SCORECARD.ENGINE.CAP06
        actors:
    • id: repository-owner
      name: Owner of the scanned repository
      type: Human
  • id: OSSF.SCORECARD.ENGINE.THR07
    title: Large Repositories Exhaust Scanner Resources
    Scanning downloads and extracts the whole repository and may analyze
    every file. A very large repository, or an archive served by a hostile
    host, can exhaust the scanner's disk, memory, or time, or the host's
    API rate limit for the token in use.
    group: denial-of-service
    capabilities:

    • reference-id: OSSF.SCORECARD.ENGINE.CAPS
      entries:
      • reference-id: OSSF.SCORECARD.ENGINE.CAP01
      • reference-id: OSSF.SCORECARD.ENGINE.CAP02
      • reference-id: OSSF.SCORECARD.ENGINE.CAP04
        actors:
    • id: repository-owner
      name: Owner of the scanned repository
      type: Human
    • id: host-operator
      name: Operator of a repository host
      type: Human
  • id: OSSF.SCORECARD.ENGINE.THR08
    title: Host Tokens Are Exposed
    Scorecard holds the user's host tokens for the whole run. A defect that
    writes request headers or environment values to logs or output would
    expose them to anyone who can read the logs, such as viewers of a
    public CI run.
    group: information-disclosure
    capabilities:

    • reference-id: OSSF.SCORECARD.ENGINE.CAPS
      entries:
      • reference-id: OSSF.SCORECARD.ENGINE.CAP07
      • reference-id: OSSF.SCORECARD.ENGINE.CAP08
  • id: OSSF.SCORECARD.ENGINE.THR09
    title: Over-Scoped Tokens Widen the Blast Radius
    Scorecard only reads, but users often supply tokens with write or
    private-repository scopes, and some checks need broader scopes to see
    settings such as branch protection. If the token leaks, or the process
    running Scorecard is compromised, the token's extra scopes are exposed
    with it.
    group: elevation-of-privilege
    capabilities:

    • reference-id: OSSF.SCORECARD.ENGINE.CAPS
      entries:
      • reference-id: OSSF.SCORECARD.ENGINE.CAP08
  • id: OSSF.SCORECARD.ENGINE.THR10
    title: Exposed Scan Service Acts on the Operator's Behalf
    scorecard serve runs every request with the operator's credentials and
    has no authentication or rate limiting, because it is intended for
    trusted networks. If it is reachable by untrusted clients, they can
    spend the operator's API rate limit, and can read results for private
    repositories the operator's token can access. scorecard serve --help
    states this trust model.
    group: information-disclosure
    capabilities:

    • reference-id: CCC
      entries:
      • reference-id: CCC.Core.CP27
    • reference-id: OSSF.SCORECARD.ENGINE.CAPS
      entries:
      • reference-id: OSSF.SCORECARD.ENGINE.CAP08
      • reference-id: OSSF.SCORECARD.ENGINE.CAP09
        actors:
    • id: network-client
      name: Client that can reach the service
      type: Human
  • id: OSSF.SCORECARD.ENGINE.THR11
    title: Results Are Read as Describing a Different Commit
    A result describes the repository at the commit that was scanned,
    usually HEAD at scan time. Consumers who read cached or weekly results
    after the repository changed act on a posture that no longer holds.
    group: tampering
    capabilities:

    • reference-id: CCC
      entries:
      • reference-id: CCC.Core.CP18
    • reference-id: OSSF.SCORECARD.ENGINE.CAPS
      entries:
      • reference-id: OSSF.SCORECARD.ENGINE.CAP07
      • reference-id: OSSF.SCORECARD.ENGINE.CAP10
  • id: OSSF.SCORECARD.ENGINE.THR12
    title: Attestation Vouches for Source the Image Wasn't Built From
    scorecard-attestor evaluates the repository and commit its caller names,
    falling back to HEAD when no commit is given. It doesn't verify that the
    image was built from that commit, so a misconfigured or compromised
    pipeline can obtain a signed attestation for an image whose actual
    source was never evaluated.
    group: spoofing
    capabilities:

    • reference-id: CCC
      entries:
      • reference-id: CCC.Core.CP18
    • reference-id: OSSF.SCORECARD.ENGINE.CAPS
      entries:
      • reference-id: OSSF.SCORECARD.ENGINE.CAP11
        actors:
    • id: pipeline-author
      name: Author of the build pipeline
      type: Human
  • id: OSSF.SCORECARD.ENGINE.THR13
    title: Attestation Signing Key Is Exposed
    PGP and PKIX signing keys are read from files, and a PGP passphrase can
    be passed as a command-line flag, where process listings or build logs
    can capture it. Anyone holding the key can sign attestations for any
    image. Cloud KMS keys keep the key material out of the process.
    group: information-disclosure
    capabilities:

    • reference-id: OSSF.SCORECARD.ENGINE.CAPS
      entries:
      • reference-id: OSSF.SCORECARD.ENGINE.CAP11

Checks Internal Checks.Yaml (`docs/checks/internal/checks.yaml`)

Licensed under the Apache License, Version 2.0 (the "License");

you may not use this file except in compliance with the License.

You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software

distributed under the License is distributed on an "AS IS" BASIS,

WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

See the License for the specific language governing permissions and

limitations under the License.

This is the source of truth for all check descriptions and remediation steps.

Run `cd checks/main && go run /main` to generate `checks.json` and `checks.md`.

checks:
Maintained:
risk: High
tags: supply-chain, security
repos: GitHub, GitLab
short: Determines if the project is "actively maintained".
Risk: High (possibly unpatched vulnerabilities)

text
This check determines whether the project is actively maintained. If the project
  is archived, it receives the lowest score. If there is at least one commit per
  week during the previous 90 days, the project receives the highest score.  If there
  is activity on issues from users who are collaborators, members, or owners of the
  project, the project receives a partial score.

  A project which is not active might not be patched, have its
  dependencies patched, or be actively tested and used. However, a lack
  of active maintenance is not necessarily always a problem. Some software,
  especially smaller utility functions, does not normally need to be maintained.
  For example, a library that determines if an integer is even would not normally
  need maintenance unless an underlying implementation language definition
  changed. A lack of active maintenance should signal that potential users should
  investigate further to judge the situation.

  This check will only succeed if a GitHub project is >90 days old. Projects
  that are younger than this are too new to assess whether they are maintained
  or not, and users should inspect the contents of those projects to ensure they
  are as expected.
remediation:
  - >-
    There is no remediation work needed from projects with a low score; this
    check simply provides insight into the project activity and maintenance
    commitment. External users should determine whether the software is the type
    that would not normally need active maintenance.

Dependency-Update-Tool:
risk: High
tags: supply-chain, security, dependencies
repos: GitHub, GitLab, Azure DevOps, local
short: Determines if the project uses a dependency update tool.
Risk: High (possibly vulnerable to attacks on known flaws)

text
This check tries to determine if the project uses a dependency update tool,
  specifically one of:
  - [Dependabot](https://docs.github.com/en/code-security/supply-chain-security/keeping-your-dependencies-updated-automatically/configuration-options-for-dependency-updates)
  - [Renovate bot](https://docs.renovatebot.com/configuration-options/)
  Out-of-date dependencies make a project vulnerable to known flaws and prone to attacks.
  These tools automate the process of updating dependencies by scanning for
  outdated or insecure requirements, and opening a pull request to update them if
  found.

  This check can determine only whether the dependency update tool is enabled; it
  does not ensure that the tool is run or that the tool's pull requests are
  merged.

  Note: A project that fulfills this criterion with other tools may still receive
  a low score on this test. There are many ways to implement dependency updates,
  and it is challenging for an automated tool like Scorecard to detect them all. A
  low score is therefore not a definitive indication that the project is at risk.

remediation:
  - >-
    Sign up for automatic dependency updates with one of the previously listed dependency update tools and place
    the config file in the locations that are recommended by these tools. Due to
    https://github.com/dependabot/dependabot-core/issues/2804 Dependabot can
    be enabled for forks where security updates have ever been turned on so projects
    maintaining stable forks should evaluate whether this behavior is satisfactory
    before turning it on.
  - >-
    Unlike Dependabot, Renovate bot has support to migrate dockerfiles' dependencies from version pinning to hash pinning
    via the [pinDigests setting](https://docs.renovatebot.com/configuration-options/#pindigests) without
    additional manual effort.

Binary-Artifacts:
risk: High
tags: supply-chain, security, dependencies
repos: GitHub, GitLab, Azure DevOps, local
short: Determines if the project has generated executable (binary) artifacts in the source repository.
Risk: High (non-reviewable code)

text
This check determines whether the project has generated executable (binary)
  artifacts in the source repository.

  Including generated executables in the source repository increases user risk.
  Many programming language systems can generate executables from source code
  (e.g., C/C++ generated machine code, Java `.class` files, Python `.pyc` files,
  and minified JavaScript). Users will often directly use executables if they are
  included in the source repository, leading to many dangerous behaviors.

  Problems with generated executable (binary) artifacts:

    - Binary artifacts cannot be reviewed, allowing possible obsolete or
      maliciously subverted executables. Reviews generally review source code, not
      executables, since it's difficult to audit executables to ensure that they
      correspond to the source code. Over time the included executables might not
      correspond to the source code.
    - Generated executables allow the executable generation process to atrophy,
      which can lead to an inability to create working executables. These problems
      can be countered with verified reproducible builds, but it's easier to
      implement verified reproducible builds when executables are not included in
      the source repository (since the executable generation process is less
      likely to have atrophied).

  Allowed by Scorecard:

    - Files in the source repository that are simultaneously reviewable source
      code and executables, since these are reviewable. (Some interpretive
      systems, such as many operating system shells, don't have a mechanism for
      storing generated executables that are different from the source file.)
    - Source code in the source repository generated by other tools (e.g., by
      bison, yacc, flex, and lex). There are potential downsides to generated
      source code, but generated source code tends to be much easier to review and
      thus presents a lower risk. Generated source code is also often difficult
      for external tools to detect.
    - Generated documentation in source repositories. Generated documentation is
      intended for use by humans (not computers) who can evaluate the context.
      Thus, generated documentation doesn't pose the same level of risk.

remediation:
  - >-
    Remove the generated executable artifacts from the repository.
  - >-
    Build from source.

Branch-Protection:
risk: High
tags: supply-chain, security, source-code, code-reviews
repos: GitHub, GitLab, Azure DevOps
short: Determines if the default and release branches are protected with GitHub's branch protection settings.
Risk: High (vulnerable to intentional malicious code injection)

text
This check determines whether a project's default and release branches are
  protected with GitHub's [branch protection](https://docs.github.com/github/administering-a-repository/defining-the-mergeability-of-pull-requests/about-protected-branches) 
  or [repository rules](https://docs.github.com/repositories/configuring-branches-and-merges-in-your-repository/managing-rulesets/about-rulesets) settings.
  Branch protection allows maintainers to define rules that enforce
  certain workflows for branches, such as requiring review or passing certain
  status checks before acceptance into a main branch, or preventing rewriting of
  public history.

  Note: The following settings queried by the Branch-Protection check require an admin token: `DismissStaleReviews`, `EnforceAdmins`, `RequireLastPushApproval`, `RequiresStatusChecks` and `UpToDateBeforeMerge`. If
  the provided token does not have admin access, the check will query the branch
  settings accessible to non-admins and provide results based only on these settings.
  However, all of these settings are accessible via Repo Rules. `EnforceAdmins` is calculated slightly differently.
  This setting is calculated as `false` if any [Bypass Actors](https://docs.github.com/repositories/configuring-branches-and-merges-in-your-repository/managing-rulesets/creating-rulesets-for-a-repository#granting-bypass-permissions-for-your-ruleset)
   are defined on any rule, regardless of if they are admins.

  Different types of branch protection protect against different risks:

    - Require code review: 
      - requires at least one reviewer, which greatly
      reduces the risk that a compromised contributor can inject malicious code.
      Review also increases the likelihood that an unintentional vulnerability in
      a contribution will be detected and fixed before the change is accepted.

      - requiring two or more reviewers protects even more from the insider risk 
      whereby a compromised contributor can be used by an attacker to LGTM 
      the attacker PR and inject a malicious code as if it was legit.

    - Prevent force push: prevents use of the `--force` command on public
      branches, which overwrites code irrevocably. This protection prevents the
      rewriting of public history without external notice.

    - Require [status checks](https://docs.github.com/en/github/collaborating-with-pull-requests/collaborating-on-repositories-with-code-quality-features/about-status-checks):
      ensures that all required CI tests are met before a change is accepted.

  Although requiring code review can greatly reduce the chance that
  unintentional or malicious code enters the "main" branch, it is not feasible for
  all projects, such as those that don't have many active participants. For more
  discussion, see [Code Reviews](https://github.com/ossf/scorecard/blob/main/docs/checks.md#code-review).

  Additionally, in some cases these rules will need to be suspended. For example,
  if a past commit includes illegal content such as child pornography, it may be
  necessary to use a force push to rewrite the history rather than simply hide the
  commit.

  This test has tiered scoring. You receive partial credit for requirements met within your current tier. However, a tier must be fully satisfied before you can earn points from the next tier. For example, if you fulfill the Tier 3 checks but do not fulfill all the Tier 2 checks, you will not receive any points for Tier 3.

  Note: If Scorecard is run without an administrative access token, the requirements that specify β€œFor administrators” can be safely ignored, and scores will be determined as if all such requirements have been met.

  Tier 1 Requirements (3/10 points):
    - Prevent force push
    - Prevent branch deletion

  Tier 2 Requirements (6/10 points):
    - Require at least 1 reviewer for approval before merging (for administrators, this requirement weights twice than the others in this tier)
    - For administrators: Require PRs prior to make any code changes
    - For administrators: Require branch to be up to date before merging (or require a [merge queue](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/configuring-pull-request-merges/managing-a-merge-queue) via repository rulesets)
    - For administrators: Require approval of the most recent reviewable push

  Tier 3 Requirements (8/10 points):
    - Require branch to pass at least 1 status check before merging

  Tier 4 Requirements (9/10 points):
    - Require at least 2 reviewers for approval before merging
    - Require review from code owners

  Tier 5 Requirements (10/10 points):
    - For administrators: Dismiss stale reviews and approvals when new commits are pushed
    - For administrators: Include administrator for review

  GitLab Integration Status:
    - GitLab associates releases with commits and not with the branch. Releases are ignored in this portion of the scoring.

  Azure DevOps Integration Status:
    - Default and named branches use the same applicable PR policies, including inherited policies and later result pages.
    - Only enabled, required, non-deleted PR policies establish protection. Repository push restrictions, such as secrets scanning, do not establish PR protection.
    - Reviewer counts are conservative independent-approval floors. Self-votes and path filters do not receive unconditional review credit.
    - Required build contexts use `azure-pipelines/<definition-id>`. These identify policy requirements, not successful test runs.
    - Permission evaluations describe the caller. A proven caller exception can show that forced updates are allowed, but a caller denial does not prove protection for everyone.
    - Unreadable or unsupported permission controls remain unknown. Administrator enforcement and CODEOWNERS enforcement are not inferred.
    - Unknown deletion or force-push protection receives no Tier 1 credit, so the score can remain zero despite mapped PR policies.

remediation:
  - >-
    Enable branch protection settings in your source hosting provider to
    avoid force pushes or deletion of your important branches.
  - >-
    For GitHub, check out the steps for rulesets [here](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-rulesets/creating-rulesets-for-a-repository) and rules [here](https://docs.github.com/en/github/administering-a-repository/managing-a-branch-protection-rule).

CI-Tests:
risk: Low
tags: supply-chain, testing
repos: GitHub, GitLab, Azure DevOps
short: Determines if the project runs tests before pull requests are merged.
Risk: Low (possible unknown vulnerabilities)

text
This check tries to determine if the project runs tests before pull requests are
  merged. It is currently limited to repositories hosted on GitHub, and does not
  support other source hosting repositories (i.e., Forges). This check only 
  considers tests which run successfully.

  Running tests helps developers catch mistakes early on, which can reduce the
  number of vulnerabilities that find their way into a project.

  The check works by looking for a set of CI-system names in GitHub `CheckRuns`
  and `Statuses` among the recent commits (~30). A CI-system is considered
  well-known if its name contains any of the following: appveyor, buildkite,
  circleci, e2e, github-actions, jenkins, mergeable, test, travis-ci, woodpecker.

  Note: A project that fulfills this criterion with other tools may still receive
  a low score on this test. There are many ways to implement CI testing, and it is
  challenging for an automated tool like Scorecard to detect them all. A low score
  is therefore not a definitive indication that the project is at risk.

  If a project's system was not detected and you think it should be, please
  [open an issue in the scorecard project](https://github.com/ossf/scorecard/issues/new/choose).
remediation:
  - Check-in scripts that run all the tests in your repository.
  - >-
    Integrate those scripts with a CI/CD platform that runs it on every pull
    request (e.g. if hosted on GitHub, [GitHub Actions](https://docs.github.com/en/actions/learn-github-actions/introduction-to-github-actions),
    [Prow](https://github.com/kubernetes/test-infra/tree/master/prow), etc).

CII-Best-Practices:
risk: Low
tags: security-awareness, security-training, security
repos: GitHub, GitLab, Azure DevOps
short: Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.
Risk: Low (possibly not following security best practices)

text
This check determines whether the project has earned an [OpenSSF (formerly CII) Best Practices Badge](https://www.bestpractices.dev/) at the passing, silver, or gold level.
  The OpenSSF Best Practices badge indicates whether or not that the project uses a set of security-focused best development practices for open
  source software. The check uses the URL for the Git repo and the OpenSSF Best Practices badge API.

  The OpenSSF Best Practices badge has 3 tiers: passing, silver, and gold. We give
  full credit to projects that meet the [gold criteria](https://www.bestpractices.dev/criteria/2), which is a significant achievement for projects and requires multiple developers in the project.
  Lower scores represent a project that has met the silver criteria, met the passing criteria, or is working to achieve the passing badge, with increasingly more points awarded as more criteria are met. Note that even meeting the passing criteria is a significant achievement.

  - [gold badge](https://www.bestpractices.dev/criteria/2): 10
  - [silver badge](https://www.bestpractices.dev/criteria/1): 7
  - [passing badge](https://www.bestpractices.dev/criteria/0): 5
  - in progress badge: 2

  Some of these criteria overlap with other Scorecard checks.
  However, note that in those overlapping cases, Scorecard can only report what it can automatically detect, while the OpenSSF Best Practices badge can report on claims and claim justifications from people (this counters false negatives and positives but has the challenge of requiring additional work from people).
remediation:
  - >-
    Sign up for the [OpenSSF Best Practices program](https://www.bestpractices.dev/).

Code-Review:
risk: High
tags: supply-chain, security, source-code, code-reviews
repos: GitHub, GitLab, Azure DevOps
short: Determines if the project requires human code review before pull requests (aka merge requests) are merged.
Risk: High (unintentional vulnerabilities or possible injection of malicious
code)

text
This check determines whether the project requires human code review
  before pull requests (merge requests) are merged.

  Reviews detect various unintentional problems, including vulnerabilities that
  can be fixed immediately before they are merged, which improves the quality of
  the code. Reviews may also detect or deter an attacker trying to insert
  malicious code (either as a malicious contributor or as an attacker who has
  subverted a contributor's account), because a reviewer might detect the
  subversion.

  The check determines whether the most recent changes (over the last ~30 commits) have 
  an approval on GitHub/GitLab
  or if the merger is different from the committer (implicit review). It also
  performs a similar check for reviews using
  [Prow](https://github.com/kubernetes/test-infra/tree/master/prow#readme) (labels
  "lgtm" or "approved") and [Gerrit](https://www.gerritcodereview.com/) ("Reviewed-on" and "Reviewed-by").
  If recent changes are solely bot activity (e.g. Dependabot, Renovate bot, or custom bots),
  the check returns inconclusively.

  Scoring is leveled instead of proportional to make the check more predictable.
  If any bot-originated changes are unreviewed, 3 points are deducted. If any human
  changes are unreviewed, 7 points are deducted if a single change is unreviewed, and
  another 3 are deducted if multiple changes are unreviewed.

  Review by bots, including bots powered by
  artificial intelligence / machine learning (AI/ML),
  do not count as code review.
  Such reviews do not provide confidence that there will
  be a second person who understands the
  code change (e.g., if the originator suddenly becomes unavailable).
  However, analysis by bots
  may be able to meet (at least in part) the [SAST](#sast) criterion.

  Note: Requiring reviews for all changes is infeasible for some projects, such as
  those with only one active participant. Even a project with multiple active
  contributors may not have enough active participation to be able to require
  review of all proposed changes. Projects with a small number of active
  participants instead sometimes aim for a review of a
  percentage of proposals (e.g., "at least half of all proposed changes are
  reviewed").

  Requiring review does not eliminate all risks. The other reviewers might fail to
  notice unintentional vulnerabilities or malicious code, be colluding with a
  malicious developer, or even be the same person (using a "[sock
  puppet](https://en.wikipedia.org/wiki/Sock_puppet_account)" account).

remediation:
  - >-
    If the project has only one contributor, or does not have enough
    reviewers to practically require that all contributions be reviewed, try to
    recruit more maintainers to the project who will be willing to review
    others' work. Ideally at least some of these people will be from different
    organizations (see [Contributors](checks.md#contributors)). If the project has very limited utility, consider expanding
    its intended utility so more people will be interested in improving it, and
    make that larger scope clear to potential contributors.
  - >-
    Follow security best practices by performing strict code reviews for every
    new pull request / merge request.
  - >-
    Make "code reviews" mandatory in your repository configuration.
    ([Instructions for GitHub.](https://docs.github.com/en/github/administering-a-repository/about-protected-branches#require-pull-request-reviews-before-merging))
  - >-
    Enforce the rule for administrators / code owners as well.
    ([Instructions for GitHub.](https://docs.github.com/en/github/administering-a-repository/about-protected-branches#include-administrators))

Contributors:
risk: Low
tags: source-code
repos: GitHub, Azure DevOps
short: Determines if the project has a set of contributors from multiple organizations (e.g., companies).
Risk: Low (lower number of trusted code reviewers)

text
This check tries to determine if the project has recent contributors from
  multiple organizations (e.g., companies). It is currently limited to
  repositories hosted on GitHub, and does not support other source hosting
  repositories (i.e., Forges).

  The check looks at the `Company` field on the GitHub user profile for authors of
  recent commits. To receive the highest score, the project must have had
  contributors from at least 3 different companies in the last 30 commits; each of
  those contributors must have had at least 5 commits in the last 30 commits.

  Note: Some projects cannot meet this requirement, such as small projects with
  only one active participant, or projects with a narrow scope that cannot attract
  the interest of multiple organizations. See
  [Code Reviews](https://github.com/ossf/scorecard/blob/main/docs/checks.md#code-review)
  for more information about evaluating projects with a small number of
  participants.
remediation:
  - >-
    Ask contributors to
    [join their respective organizations](https://docs.github.com/en/organizations/managing-membership-in-your-organization/inviting-users-to-join-your-organization),
    if they have not already. Otherwise, there is no remediation for this check;
    it simply provides insight into which organizations have contributed so that
    you can make a trust-based decision based on that information.

Fuzzing:
risk: Medium
tags: supply-chain, security, testing
repos: GitHub, GitLab, Azure DevOps, local
short: Determines if the project uses fuzzing.
Risk: Medium (possible vulnerabilities in code)

text
This check tries to determine if the project uses
  [fuzzing](https://owasp.org/www-community/Fuzzing) by checking:
  1. if the repository name is included in the [OSS-Fuzz](https://github.com/google/oss-fuzz) project list;
  2. if [ClusterFuzzLite](https://google.github.io/clusterfuzzlite/) is deployed in the repository;
  3. if there are user-defined language-specified fuzzing functions in the repository.
     - currently only supports [Go fuzzing](https://go.dev/doc/fuzz/),
     - a limited set of property-based testing libraries for Haskell including [QuickCheck](https://hackage.haskell.org/package/QuickCheck), [Hedgehog](https://hedgehog.qa/), [validity](https://hackage.haskell.org/package/validity) or [SmallCheck](https://hackage.haskell.org/package/smallcheck),
     - a limited set of property-based testing libraries for JavaScript and TypeScript including [fast-check](https://fast-check.dev/).
     - a limited set of property-based testing libraries for Erlang, including proper and quickcheck.
     - a limited set of property-based testing libraries for C# and F# including [FsCheck](https://github.com/fscheck/FsCheck).

  Fuzzing, or fuzz testing, is the practice of feeding unexpected or random data
  into a program to expose bugs. Regular fuzzing is important to detect
  vulnerabilities that may be exploited by others, especially since attackers can
  also use fuzzing to find the same flaws.

  Note: A project that fulfills this criterion with other tools may still receive
  a low score on this test. There are many ways to implement fuzzing, and it is
  challenging for an automated tool like Scorecard to detect them all. A low score
  is therefore not a definitive indication that the project is at risk.
remediation:
  - >-
    Integrate the project with OSS-Fuzz by following the instructions
    [here](https://google.github.io/oss-fuzz/).

Packaging:
risk: Medium
tags: supply-chain, security, releases
repos: GitHub, local
short: Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.
Risk: Medium (users possibly missing security updates)

text
This check tries to determine if the project is published as a package. It is
  currently limited to repositories hosted on GitHub, and does not support other
  source hosting repositories (i.e., Forges).

  Packages give users of a project an easy way to download, install, update, and
  uninstall the software by a package manager. In particular, they make it easy
  for users to receive security patches as updates.

  The check currently looks for
  [GitHub packaging workflows](https://docs.github.com/en/packages/learn-github-packages/publishing-a-package)
  and language-specific GitHub Actions that upload the package to a corresponding
  hub, e.g., [Npm](https://www.npmjs.com/). We plan to add better support to query
  package manager hubs directly in the future, e.g., for
  [Npm](https://www.npmjs.com/), [PyPi](https://pypi.org/).

  You can create a package in several ways:

    - Many program language ecosystems have a generally-used packaging format
      supported by a language-level package manager tool and public package
      repository.
    - Many operating system platforms also have at least one package format,
      tool, and public repository (in some cases the source repository generates
      system-independent source packages, which are then used by others to
      generate system executable packages).
    - Using container images.

  Note: A project that fulfills this criterion with other tools may still receive
  a low score on this test. There are many ways to package software, and it is
  challenging for an automated tool like Scorecard to detect them all. A low
  score is therefore not a definitive indication that the project is at risk. If
  Scorecard fails to detect the way you publish a package and you think we should
  support your use case, please let us know by [opening an
  issue](https://github.com/ossf/scorecard/issues/new/choose).
remediation:
  - Publish your project as a downloadable package, e.g., if hosted on GitHub, use [GitHub's mechanisms for publishing a package](https://docs.github.com/en/packages/learn-github-packages/publishing-a-package).
  - If hosted on GitHub, use a GitHub action to release your package to language-specific hubs.

Pinned-Dependencies:
risk: Medium
tags: supply-chain, security, dependencies
repos: GitHub, GitLab, Azure DevOps, local
short: Determines if the project has declared and pinned the dependencies of its build process.
Risk: Medium (possible compromised dependencies)

text
This check tries to determine if the project pins dependencies used during its build and release process.
  A "pinned dependency" is a dependency that is explicitly set to a specific hash instead of
  allowing a mutable version or range of versions. It
  is currently limited to repositories hosted on GitHub, and does not support
  other source hosting repositories (i.e., Forges).

  The check works by looking for unpinned dependencies in Dockerfiles, shell scripts, and GitHub workflows
  which are used during the build and release process of a project.
  Special considerations for Go modules treat full semantic versions as pinned
  due to how the Go tool verifies downloaded content against the hashes when anyone first downloaded the module.

  Pinned dependencies reduce several security risks:

    - They ensure that checking and deployment are all done with the same
      software, reducing deployment risks, simplifying debugging, and enabling
      reproducibility.
    - They can help mitigate compromised dependencies from undermining the
      security of the project (in the case where you've evaluated the pinned
      dependency, you are confident it's not compromised, and a later version is
      released that is compromised).
    - They are one way to [counter dependency confusion (aka substitution) attacks](https://azure.microsoft.com/en-us/resources/3-ways-to-mitigate-risk-using-private-package-feeds/),
      in which an application uses multiple feeds to acquire software packages (a
      "hybrid configuration"), and attackers fool the user into using a malicious
      package via a feed that was not expected for that package.

  However, pinning dependencies can inhibit software updates, either because of a
  security vulnerability or because the pinned version is compromised. Mitigate
  this risk by:

    - using automated tools to notify applications when their dependencies are
      outdated;
    - quickly updating applications that do pin dependencies.

  For projects hosted on GitHub, you can learn more about
  dependencies using the [GitHub dependency graph](https://docs.github.com/en/code-security/supply-chain-security/understanding-your-software-supply-chain/about-the-dependency-graph).
remediation:
  - >-
    If your project is producing an application, declare all your dependencies with specific versions in your package
    format file (e.g. `package.json` for npm, `requirements.txt` for python, `packages.config` for nuget).
    For C/C++, check in the code from a trusted source and add a `README` on the
    specific version used (and the archive SHA hashes).
  - >-
    If your project is producing an application and the package manager supports lock files (e.g. `package-lock.json` for
    npm), make sure to check these in the source code as well. These files
    maintain signatures for the entire dependency tree and saves from future
    exploitation in case the package is compromised.
  - >-
    For Dockerfiles used in building and releasing your project, pin dependencies by hash. See [Dockerfile](https://github.com/ossf/scorecard/blob/main/Dockerfile) for example. If you are using a manifest list to support builds
    across multiple architectures, you can pin to the manifest list hash instead
    of a single image hash. You can use a tool like [crane](https://github.com/google/go-containerregistry/blob/main/cmd/crane/README.md)
    to obtain the hash of the manifest list like in this [example](https://github.com/ossf/scorecard/issues/1773#issuecomment-1076699039).
  - >-
    For GitHub workflows used in building and releasing your project, pin dependencies by hash. See [main.yaml](https://github.com/ossf/scorecard/blob/f55b86d6627cc3717e3a0395e03305e81b9a09be/.github/workflows/main.yml#L27) for example.
    To determine the permissions needed for your workflows, you may use [StepSecurity's online tool](https://app.stepsecurity.io/secureworkflow/) by ticking
    the "Pin actions to a full length commit SHA". You may also tick the "Restrict permissions for GITHUB_TOKEN" to fix issues found
    by the Token-Permissions check.
  - >-
    To help update your dependencies after pinning them, use tools such as those listed for the dependency update tool check.

SAST:
risk: Medium
tags: supply-chain, security, testing
repos: GitHub, Azure DevOps, local
short: Determines if the project uses static code analysis.
Risk: Medium (possible unknown bugs)

text
This check tries to determine if the project uses Static Application Security
  Testing (SAST), also known as [static code analysis](https://owasp.org/www-community/controls/Static_Code_Analysis).
  It is currently limited to repositories hosted on GitHub, and does not support
  other source hosting repositories (i.e., Forges).

  SAST is testing run on source code before the application is run. Using SAST
  tools can prevent known classes of bugs from being inadvertently introduced in the
  codebase.

  The checks currently looks for known GitHub apps such as
  [CodeQL](https://codeql.github.com/) (github-code-scanning) or
  [SonarCloud](https://sonarcloud.io/) in the recent (~30) merged PRs, or the use
  of "github/codeql-action" in a GitHub workflow. It also checks for the deprecated
  [LGTM](https://lgtm.com/) service until its forthcoming shutdown.

  Note: A project that fulfills this criterion with other tools may still receive
  a low score on this test. There are many ways to implement SAST, and it is
  challenging for an automated tool like Scorecard to detect them all. A low score
  is therefore not a definitive indication that the project is at risk.
remediation:
  - >-
    Run CodeQL checks in your CI/CD by following the instructions
    [here](https://github.com/github/codeql-action#usage).

SBOM:
risk: Medium
short: Determines if the project maintains a Software Bill of Materials.
repos: GitHub, Gitlab
tags: supply-chain, security, vulnerabilities, dependencies, SBOM
Risk: Medium (possible inaccurate reporting of dependencies/vulnerabilities)

text
This check tries to determine if the project maintains a Software Bill of Materials (SBOM)
  either as a file in the source or a release artifact. 

  An SBOM can give users information about what dependencies your project has which
  allows them to identify vulnerabilities in the software supply chain.

  Standards to be used during checks;
  - OSSF SBOM Everywhere SIG naming and directory conventions:
    - <https://github.com/ossf/SBOM-everywhere/blob/main/reference/sbom_naming.md#consistent-naming-conventions>

  This check currently looks for the existence of an SBOM in the
  source of a project and as a pipeline or release artifact.

  An SBOM Exists (one or more) (5/10 points):
    - Any SBOM found counts for this test either in source. pipeline or release. 
    - A SBOM stored with your source code is not ideal, but is a good first step.
    \* It is recommended to publish with your release artifacts.

  An SBOM is published as a release artifact (5/10 points):
    - This is the preferred way to store an SBOM, and will be awarded full points.
    - Checks release artifacts for an SBOM file matching established standards
remediation:
  - >-
    For Gitlab, see more information
    [here](https://docs.gitlab.com/ee/user/application_security/dependency_scanning/index.html#cyclonedx-software-bill-of-materials).
  - >-
    For GitHub, see more information
    [here](https://docs.github.com/en/code-security/supply-chain-security/understanding-your-software-supply-chain/about-supply-chain-security).
  - >-
    Alternatively, there are other tools available to generate [CycloneDX](https://cyclonedx.org/tool-center/) and [SPDX](https://spdx.dev/use/tools/) SBOMs.

Security-Policy:
risk: Medium
short: Determines if the project has published a security policy.
repos: GitHub, GitLab, Azure DevOps, local
tags: supply-chain, security, policy
Risk: Medium (possible insecure reporting of vulnerabilities)

text
This check tries to determine if the project has published a security policy. It
  works by looking for a file named `SECURITY.md` (case-insensitive) in a few
  well-known directories.

  A security policy (typically a `SECURITY.md` file) can give users information
  about what constitutes a vulnerability and how to report one securely so that
  information about a bug is not publicly visible.

  This check examines the contents of the security policy file awarding points
  for those policies that express vulnerability process(es), disclosure timelines,
  and have links (e.g., URL(s) and email(s)) to support the users.

  Linking Requirements (one or more) (6/10 points):
    - A valid form of an email address to contact for vulnerabilities
    - A valid form of a http/https address to support vulnerability reporting

  Free Form Text (3/10 points):
    - Free form text is present in the security policy file which is beyond
      simply having a http/https address and/or email in the file
    - The string length of any such links in the policy file do not count
      towards detecting free form text

  Security Policy Specific Text (1/10 points):
    - Specific text providing basic or general information about vulnerability
      and disclosure practices, expectations, and/or timelines
    - Text should include a total of 2 or more hits which match (case-insensitive)
      `vuln` and as in "Vulnerability" or "vulnerabilities";
      `disclos` as "Disclosure" or "disclose";
      and numbers which convey expectations of times, e.g., 30 days or 90 days

remediation:
  - >-
    Place a security policy file `SECURITY.md` in the root directory of your
    repository. This makes it easily discoverable by a vulnerability reporter.
  - >-
    The file should contain information on what constitutes a vulnerability
    and a way to report it securely (e.g. issue tracker with private issue
    support, encrypted email with a published public key). Follow the
    [coordinated vulnerability disclosure guidelines](https://github.com/ossf/oss-vulnerability-guide/blob/main/maintainer-guide.md)
    to respond to vulnerability disclosures.
  - >-
    For GitHub, see more information
    [here](https://docs.github.com/en/code-security/getting-started/adding-a-security-policy-to-your-repository).

Signed-Releases:
risk: High
tags: supply-chain, security, releases
repos: GitHub
short: Determines if the project cryptographically signs release artifacts.
Risk: High (possibility of installing malicious releases)

text
This check tries to determine if the project cryptographically signs release
  artifacts. It is currently limited to repositories hosted on GitHub, and does
  not support other source hosting repositories (i.e., Forges).

  Signed releases attest to the provenance of the artifact.

  This check looks for the following filenames in the project's last five
  [release assets](https://docs.github.com/en/repositories/releasing-projects-on-github/about-releases):
  [*.minisig](https://github.com/jedisct1/minisign), *.asc (pgp),
  *.sig, *.sign, *.sigstore, *.sigstore.json, [*.intoto.jsonl](https://slsa.dev).

  If a signature is found in the assets for each release, a score of 8 is given.
  If a [SLSA provenance file](https://slsa.dev/spec/v0.1/index) is found in the assets for each release (*.intoto.jsonl), the maximum score of 10 is given.

  This check looks for the 30 most recent releases associated with an artifact. It ignores the source code-only releases that are created automatically by GitHub.

  Note: The check does not verify the signatures.
remediation:
  - >-
    Publish the release.
  - >-
    Generate a signing key.
  - >-
    Download the release as an archive locally.
  - >-
    Sign the release archive with this key (should output a signature file).
  - >-
    Attach the signature file next to the release archive.
  - >-
    If the source is hosted on GitHub, check out the steps
    [here](https://wiki.debian.org/Creating%20signed%20GitHub%20releases).

Token-Permissions:
risk: High
tags: supply-chain, security, infrastructure
repos: GitHub, local
short: Determines if the project's workflows follow the principle of least privilege.
Risk: High (vulnerable to malicious code additions)

text
This check determines whether the project's automated workflows tokens follow the
  principle of least privilege. This is important because attackers may use a
  compromised token with write access to, for example, push malicious code into the
  project.

  It is currently limited to repositories hosted on GitHub, and does not support
  other source hosting repositories (i.e., Forges).

  The highest score is awarded when the permissions definitions in each workflow's
  yaml file are set as read-only at the
  [top level](https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#permissions)
  and the required write permissions are declared at the
  [run-level](https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idpermissions).
  One point is reduced from the score if all jobs have their permissions defined but the top level permissions are not defined.
  This configuration is secure, but there is a chance that when a new job is added to the workflow, its job permissions could be
  left undefined because of human error.

  Though a project's score won't be penalized, the check's details will include
  warnings for more sensitive run-level permissions, listed below:

  * `actions` - May allow an attacker to steal GitHub secrets by approving to run an action that needs approval.
  * `checks` - May allow an attacker to remove pre-submit checks and introduce a bug.
  * `contents` - Allows an attacker to commit unreviewed code. However, points are not reduced if the job utilizes a recognized packaging action or command.
  * `deployments` - May allow an attacker to charge repo owner by triggering VM runs, and tiny chance an attacker can trigger a remote service with code they own if server accepts code/location variables unsanitized.
  * `packages` - Allows an attacker to publish packages. However, points are not reduced if the job utilizes a recognized packaging action or command.
  * `security-events` - May allow an attacker to read vulnerability reports before a patch is available. However, points are not reduced if the job utilizes a recognized action for uploading SARIF results.
  * `statuses` - May allow an attacker to change the result of pre-submit checks and get a PR merged.

  This compromise makes it clear the maintainer has done what's possible to use those permissions safety,
  but allows users to identify that the permissions are used.

  The check cannot detect if the "read-only" GitHub permission setting is
  enabled, as there is no API available.

remediation:
  - >-
    Set top-level permissions as `read-all` or `contents: read` as described in
    GitHub's [documentation](https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#permissions).
  - >-
    Set any required write permissions at the job-level. Only set the permissions
    required for that job; do not set `permissions: write-all` at the job level.
  - >-
    To help determine the permissions needed for your workflows, you may use [StepSecurity's online tool](https://app.stepsecurity.io/secureworkflow/) by ticking
    the "Restrict permissions for GITHUB_TOKEN". You may also tick the "Pin actions to a full length commit SHA" to fix issues found
    by the Pinned-dependencies check.

Vulnerabilities:
risk: High
tags: supply-chain, security, vulnerabilities
repos: GitHub, GitLab, Azure DevOps, local
short: Determines if the project has open, known unfixed vulnerabilities.
Risk: High (known vulnerabilities)

text
This check determines whether the project has open, unfixed vulnerabilities 
  in its own codebase or its dependencies using the [OSV (Open Source Vulnerabilities)](https://osv.dev/) service.
  An open vulnerability is readily exploited by attackers and should be fixed as soon as
  possible.
remediation:
  - >-
    Fix the vulnerabilities in your own code base. The details of each vulnerability can be found
    on <https://osv.dev>.
  - >-
    If the vulnerability is in a dependency, update the dependency to a non-vulnerable version. If no update is available, consider whether to remove the dependency.
  - >-
    If you believe the vulnerability does not affect your project, the 
    vulnerability can be ignored. 
    To ignore, create an `osv-scanner.toml` file next to the dependency manifest (e.g. package-lock.json) and specify the ID to ignore and reason.
    Details on the structure of `osv-scanner.toml` can be found on 
    [OSV-Scanner repository](https://github.com/google/osv-scanner#ignore-vulnerabilities-by-id).

Dangerous-Workflow:
risk: Critical
tags: supply-chain, security, infrastructure
repos: GitHub, local
short: Determines if the project's GitHub Action workflows avoid dangerous patterns.
Risk: Critical (vulnerable to repository compromise)

text
This check determines whether the project's GitHub Action workflows has dangerous
  code patterns. Some examples of these patterns are untrusted code checkouts,
  logging github context and secrets, or use of potentially untrusted inputs in scripts.
  The following patterns are checked:

  Untrusted Code Checkout: This is the misuse of potentially dangerous triggers.
  This checks if a `pull_request_target` or `workflow_run` workflow trigger was used in conjunction
  with an explicit pull request checkout. Workflows triggered with `pull_request_target` / `workflow_run`
  have write permission to the target repository and access to target repository
  secrets. With the PR checkout, PR authors may compromise the repository, for
  example, by using build scripts controlled by the author of the PR or reading
  token in memory. This check does not detect whether untrusted code checkouts are
  used safely, for example, only on pull request that have been assigned a label.

  Script Injection with Untrusted Context Variables: This pattern detects whether a
  workflow's inline script may execute untrusted input from attackers. This occurs when
  an attacker adds malicious commands and scripts to a context. When a workflow runs,
  these strings may be interpreted as code that is executed on the runner. Attackers
  can add their own content to certain github context variables that are considered
  untrusted, for example, `github.event.issue.title`. These values should not flow
  directly into executable code.

  The highest score is awarded when all workflows avoid the dangerous code patterns.
remediation:
  - >-
    Avoid the dangerous workflow patterns.
    See this [post](https://securitylab.github.com/research/github-actions-preventing-pwn-requests/)
    for information on avoiding untrusted code checkouts.
    See this [document](https://docs.github.com/en/actions/security-guides/security-hardening-for-github-actions#understanding-the-risk-of-script-injections)
    for information on avoiding and mitigating the risk of script injections.

License:
risk: Low
tags: license
repos: GitHub, GitLab, Azure DevOps, local
short: Determines if the project has defined a license.
Risk: Low (possible impediment to security review)

text
This check tries to determine if the project has published a license. It
  works by using either hosting APIs or by checking standard locations
  for a file named according to common conventions for licenses.

  A license can give users information about how the source code may or may
  not be used. The lack of a license will impede any kind of security review
  or audit and creates a legal risk for potential users.

  Scorecard uses the
  [GitHub License API](https://docs.github.com/en/rest/licenses#get-the-license-for-a-repository)
  for GitHub hosted projects. Otherwise, Scorecard uses its own heuristics to
  detect a published license file.

  On its own, this check will detect files in the top-level directory with
  any combination of the following names and extensions:`LICENSE`, `LICENCE`,
  `COPYING`, `COPYRIGHT` and having common extensions such as `.html`, `.txt`,
  or `.md`. It will also detect these files in a directory named `LICENSES`.
  (Files in a `LICENSES` directory are typically named as their
  [SPDX](https://spdx.org/licenses/) license identifier followed by an
  appropriate file extension, as described in the [REUSE Specification](https://reuse.software/spec/).)

  License Requirements:
    - A detected `LICENSE`, `COPYRIGHT`, or `COPYING` filename, or license
      files in a `LICENSES` directory (6/10 points)
    - The detected file is at the top-level directory (3/10 points)
    - A [FSF or OSI](https://spdx.org/licenses/) license is specified (1/10 points)

remediation:
  - >-
    Determine [which license](https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/licensing-a-repository)
    to apply to your project. For GitHub hosted projects, follow those
    instructions to establish a license for your project.
  - >-
    For other hosting environments, create the license in a `.adoc`, `.asc`,
    `.docx`, `.doc`, `.ext`, `.html`, `.markdown`, `.md`, `.rst`, `.txt`,
    or `.xml`, named `LICENSE`, `COPYRIGHT`, or `COPYING`, and place it in the
    top-level directory. To identify a specific license, use an [SPDX license identifier](https://spdx.org/licenses/)
    in the filename. Examples include `LICENSE.md`, `Apache-2.0-LICENSE.md` or
    `LICENSE-Apache-2.0`.
  - >-
    Alternately, create a `LICENSES` directory and add a license file(s) with a name
    that matches your [SPDX license identifier](https://spdx.org/licenses/).
    such as `LICENSES/Apache-2.0.txt`.

Webhooks:
risk: Critical
tags: security, infrastructure
repos: GitHub
short: This check validates if the webhook defined in the repository has a token configured.
Risk: Critical (service possibly accessible to third parties)

text
This check determines whether the webhook defined in the repository has a token configured to authenticate the origins of requests.
remediation:
  - >-
    Check if the service your webhooks is configured with supports secrets.
  - >-
    If there is support for token authentication, set the secret in the webhook configuration. See [Setting up a webhook](https://docs.github.com/en/developers/webhooks-and-events/webhooks/creating-webhooks#setting-up-a-webhook).
  - >-
    If there is no support for token authentication, request the webhook service implement token authentication functionality by following [these directions](https://docs.github.com/en/developers/webhooks-and-events/webhooks/securing-your-webhooks).

--- METRICS ---