Secure software supply chains: good practices, at scale

Here at The Linux Foundation’s blog, we share content from our projects, such as this article from the Cloud Native Computing Foundation’s blog. The guest post was originally published on Contino Engineering’s blog by Dan Chernoff. 

Supply chain attacks rose by 42% in the first quarter of 2021 [1] and are becoming even more prevalent [2]. In response to secure software supply chain breaches like Solar Winds [3], Kaseya[4], and other less publicized compromises [5], the Biden administration issued an executive order that includes guidance designed to improve the federal government’s defense against cyber threats. With all of this comes the inevitable slew of blog posts that detail a software supply chain and how you would protect it. The Cloud Native Computing Foundation recently released a white paper regarding software supply chain security [7], an excellent summary of the current best practices for securing your software supply chain.

The genesis for the content in this article is work done to implement secure supply chain patterns and practices for a Contino customer. The core goals for the effort were; implement a pipeline agnostic solution that ensures the security of the pipelines and enables secure delivery for the enterprise. We’ll talk a little about why we chose the tools we did in each section and how they supported the end goal.

As we start our journey, we’ll first touch on what a secure software supply chain is and why you should have one to set the context for the rest of the blog post. But let’s assume that you have already decided that your software supply chains need to be secure, and you want to implement the capability for your enterprise. So let’s get into it!

Anteing Up

Before you embark upon the quest of establishing provenance for your software at scale, there are some table stakes elements that teams should already have in place. We won’t delve deeply into any of them here other than to list and briefly describe them.

Centralized Source Control, Git is by far the most popular choice. This ensures a single source of truth for development teams. Beyond just having source control, teams should also implement the signing of their Git commits.

Static Code Analysis. This identifies possible vulnerabilities within ‘static’ (non-running) source code by using techniques such as Taint Analysis and Data Flow Analysis. Analysis and results need to be incorporated into the cadence of development.

Vulnerability Scanning. Implement automated tools that scan the applications and containers that are built to identify potential vulnerabilities in the compiled and sometimes running applications.

Linting is a tool that analyzes source code to flag programming errors, bugs, and stylistic errors. Linting is important to reduce errors and improve the overall code quality. This in turn accelerates development.

CI/CD Pipelines. New code changes are automatically built, tested, versioned, and delivered to an artifact repository. A pipeline then automatically deploys the updated applications into your environments (e.g. test, staging, production, etc.).

Artifact Repositories. Provide management of the artifacts built by your CI/CD systems. An artifact repository can help with the version and access control of your artifacts.

Infrastructure as Code (IaC) is the process of managing and provisioning infrastructure (e.g. virtual machines, databases, load balancers, etc.) through code. As with applications, IaC provides a single source of truth for what the infrastructure should look like. It also provides the ability to test before deploying to production.

Automated…well, everything. Human-in-the-loop systems are not deterministic. They are prone to error which can and will cause outages and security gaps. Manual systems also inhibit the ability of platforms to scale quickly.

What is a Secure Software Supply Chain

A software supply chain consists of anything that goes into the creation of your end software product and the mechanisms you use to deliver the product to customers. This includes things like your source code, your build systems, the 3rd party libraries, deployment infrastructure, or delivery repositories.


  • Establishes Provenance — One part of establishing provenance is ensuring that any artifact that is created and accessed by the customer should be able to trace its lineage all the way back to the developer(s) that merged the latest commit. The other part is the ability to demonstrate (or attest) that for each step in the process, the software, components, and other materials that go into creating the final product are tamper-free.
  • Trust — Downstream systems and users need a mechanism to verify that the software that is being installed or deployed came from your systems and that the version being used is the correct version. This ensures that malicious artifacts have not been substituted or that older, vulnerable versions have not been relabeled as the current version.
  • Transparent — It should be easy to see the results and details for all steps that go into the creation of the final artifact. This can include things like test results, output from vulnerability scans, etc.

Key Elements of a Secure Software Supply Chain

Let’s take a closer look at the things that need to be layered into your pipelines to establish provenance, enable transparency, and ensure tamper resistance.

Here is what a typical pipeline might look like that creates a containerized application. We’ll use this simple pipeline and add elements as we discuss them.

Posted by Linux Admin