Is Your Supply Chain Haunted by CVEs?

For every piece of software written, there is a great team of people behind it! Thus, in order for us to trust software, we need to know that the software produced was created by trusted developers and using a set of trusted tools. In almost every organization today, every employee is provided with hardware and software which interacts with the IT services of an organization. Workforces today are largely mobile or remote — after all, we don’t expect every employee to badge into a data center to go about their day to day business. These endpoint devices operate in an uncontrolled environment — a developer in a café doesn’t know if someone’s looking over their shoulder, a laptop in a hotel room is left unattended and accessible by hotel staff, etc.

Developers, their equipment, and their practices are the foundation of securing your software supply chain. In a future post, I’ll talk about securing developer workstations. For now, let’s focus on three important questions:

• How do we know what’s written hasn’t been tampered with?
• We’re secure, but what about our dependencies?
• How do I know what’s produced?

How do we know what’s written hasn’t been tampered with? Rid your code of evil hexes.

It should go without saying that we need to make sure that the software that our developers create is secure. But that’s not the end of it. It doesn’t matter if we’ve written secure code if the code can be modified. Therefore, we need to ensure that when software is being written that it is secure. For example, attackers can modify code in insecure code repositories which result in injection of malware like in the case of Wordpress plugins hosted by AccessPress.

‍

Secure development is about ensuring that the process of developing software and the code produced in development is protected and not tampered with. This consists of ensuring that development tooling is secure, including source control management services, issue ticketing systems, etc. This includes:

• Ensuring that code does not live on developer workstations where possible through the use of remote code IDEs such as VS Code Server or Gitpod. This minimizes the risk of code compromise through developer workstations.
• Providing a secure deployment of a source control management system (e.g. GitHub) and ensuring only authorized developers have the access to repositories.
• Setting up a secure deployment of issue ticketing systems (e.g. JIRA) and ensuring only authorized developers and project/product managers have access.
• Providing a secure deployment of artifact repositories (e.g. Artifactory) and ensuring only authorized build systems, workload clusters, and developers have access.
• Ensuring that SSH or remote desktop connection configuration is secure to prevent person-in-the-middle attacks. This can include, but is not limited to, zero trust configurations or VPNs, with defense-in-depth in mind.

We’re secure, but what about our dependencies? Not all spirits are friendly.

Great — the software our organization wrote is safe! But our code is only a part of the overall system. Consider the writing of a payment application. This involves many things including cryptography routines, front-end frameworks, network proxies, HTTP servers, etc. Unless our hypothetical Secure Bank wants to rewrite the history of computing, it is inevitable that it relies on external software. 

‍

The functionality of software today is so much greater than it was 20 years ago. This is only possible through the ability to reuse code. However, the risks of using a simple library that reverses a list are very different compared to an entire container orchestration system like Kubernetes. Ecosystems have made it very convenient for developers to re-use functionality, but have not made it very easy to reason about the code being used. When a piece of open source software is used by a developer, the chances a developer understands the risk is low. This is not because of their competency, but because it takes a lot of time and effort to do so.

‍

Therefore, organizations need to be able to make policies and manage the risks of the software they use. If an organization needs to use open source code, it needs to evaluate, mitigate, and accept its risk. The same must be done with third-party software vendors, including the use of contracts to provide security assurances. Security is as strong as its weakest link, and the software development lifecycle of both open source and third-party software must be held to the same level of security in order to not degrade the overall security posture of an organization’s software.

‍

Secure Ingestion is about ensuring that only authorized and approved software and its metadata is used within an organization’s software supply chain ecosystem. This applies to software (through policies around what software is approved for use), but is equally important in ingesting the metadata around use of the software. For example, an unreliable source can provide false information about the security posture of a software leading to an undesirable policy decision of deploying an application. Secure ingestion includes:

• Establishing a policy on the use of third-party software — both open source and proprietary — within the organization
• Establishing a policy on usage of threat intelligence and vendor threat intelligence to be used within the organization
• Enforcing a proxy or secure mirror for open source packages used by the organization
• Creating an allow list for approved third-party code and package & code repositories
• Providing a mechanism to ingest software metadata and threat intelligence and use them in continuous integration and continuous deployment (CI/CD) checks

How do I know what’s produced? Verify the brew. 

The software supply chain has parallels to the food supply chain. One of key characteristics they share is that ingredients need to be processed so that they can be consumed. In software, our ingredients are code, and the produced output is compiled machine code or packages that can be executed to run the software. Because the process of compilation and code transformation is inherently lossy, it is not possible to know with certainty the composition of the output just by analyzing the output alone — the same way it is not possible to detect peanut particles by visual inspection.

‍

For example, imaging you are provided with a software executable, and you are asked under oath, “is this the approved latest version of the payment application?” As a CISO, you would not be able to give you a confident answer without knowing the executable’s provenance. Therefore, it is as important in software supply chains to, as the saying goes, know how the sausage was made — verifying that the code was processed securely.

‍

This is done through a Secure Software Factory. The Secure Software Factory is about being able to perform a secure build and perform the necessary checks to ensure that the produced software artifacts from builds are attested and reflect the specified organization policies. This includes:

• A build system that is able to create, orchestrate and perform secure build pipelines which are tamper-resistant and have the ability to create attested provenance
• Included in the build system:
  • Attested build compute provided with machine identity that has been authenticated and attested (preferably with a hardware root of trust)
  • Routines to be able to fetch and verify code, artifacts and metadata from trusted and approved sources
  • Mechanisms to securely build artifacts and produce non-falsifiable provenance
  • Ability to securely upload a produced artifact and its attestation of what steps were taken to produce the build
• Routines to carry out security posture checking of software being built including checking for known vulnerabilities and malware, SAST/DAST and fuzzing.

‍

The road to secure software is paved with both people and technology. With trusted developers and secure practices you better ensure that what you release into the world is free from curses.