How to close attack vectors for exposed secrets in Docker
Learn to protect your enterprise assets by detecting vulnerable container images and proactively following secure best practices.
With the advent of containerization, software developers could build, deploy, and scale applications in a transformative way. Docker quickly became the leading containerization platform and remains one of the most common container images used today. Docker Hub is one of the prominent locations for developers and enterprises to publish and distribute docker images.
With popularity comes greater attention from cybercriminals. Cybernews recently reported 5,500 out of 10,000 public docker images contained 48,000+ sensitive secrets - a combination of harmless and potentially vulnerable API keys. This report illustrates why it's imperative that security and platform teams know the most common attack vectors for their Docker containers and understand how to close them.
This post will provide a brief checklist of the various attack vectors into your Docker containers specifically originating from exposed secrets.
»Docker and exposed secrets
Let’s quickly examine the relationship between container runtime and registry. When we spin-up a container, an image is pulled from the registry via APIs and is deployed. This is visualized below:
The high number of secrets from the Cybernews report is attributed to developers re-using packages from a registry containing sensitive secrets. Secrets are commonly found in the container image metadata - the environment variables and filesystem. Also, source code leakage could allow attackers to generate newer valid tokens that could provide unauthorized system access.
»Attack surface
An attack surface is a collection of all vulnerable points an attacker can use to enter the target system. Attackers skillfully exploit these vulnerable points in technology and human behavior to access sensitive assets.
We need to understand two Docker concepts as we continue this discussion:
-
Filesystem: In Docker, each layer can contain directory changes. The most commonly used filesystem, OverlayFS, enables Docker to overlay these layers to create a unified filesystem for a container.
-
Layers: Docker images are created in layers - i.e., each command on the DockerFile corresponds to a layer.
With that context, let’s understand and analyze how exposed secrets can affect these Docker image attack vectors.
»Docker image layers
Secrets explicitly declared in the Dockerfile or build arguments can easily be accessed via the Docker image history command.
#terminal
docker image history <image>
This represents one of the simplest methods for an attacker to capitalize on a secret.
»Filesystem
This Dockerfile demonstrates a scenario where sensitive data like SSH private key and secrets.txt are added to the container's filesystem and later removed.
#Dockerfile
FROM nginx:latest
# Copy in SSH private key, then delete it; this is INSECURE,
# the secret will still be in the image.
COPY id_rsa .
RUN rm -r id_rsa
ARG DB_USERNAME
ENV DB_USERNAME =$DB_USERNAME
ARG DB_PASSWORD
ENV DB_PASSWORD =$DB_PASSWORD
ARG API_KEY
ENV API_KEY =$API_KEY
# Expose secrets via a publicly accessible endpoint (insecure practice)
RUN echo "DB_USERNAME=$DB_USERNAME" > /usr/share/nginx/html/secrets.txt
RUN echo "DB_PASSWORD=$DB_PASSWORD" >> /usr/share/nginx/html/secrets.txt
RUN echo "API_KEY=$API_KEY" >> /usr/share/nginx/html/secrets.txt
RUN rm /usr/share/nginx/html/secrets.txt
CMD ["nginx", "-g", "daemon off;"]
Docker uses layer caching - hence, the secret is still available in one of the layers. An internal attacker can also extract individual layers of a Docker image, stored as tar files in registries, which enables them to uncover hidden secrets.
After creating a Dockerfile, developers mistakenly use build arguments to create an image. For the above dockerfile, the secrets are input as arguments.
#terminal
docker build \
--build-arg DB_USERNAME=root \
--build-arg DB_PASSWORD=Xnkfnbgf \
--build-arg API_KEY=PvL4FjrrSXyT7qr \
-t myapp:1.0 .
While convenient, it is not secure since the arguments also get embedded in the image. A simple docker history --no-trunc <image> can expose the secret values. Developers should either use multi-stage builds or secret managers.
»Environment variables
Apart from Docker image access, unauthorized access to the source code of the docker image can provide additional attack vectors. The .env files are primarily used to store secrets such as API tokens, database credentials, and other forms of secrets that an application needs. When attackers have access to secrets in the .env, they can make unauthorized accesses that the secret allows.
»Dockerfile
DockerFile is a standard file that contains execution instructions to build an image while spinning up containers. Hard-coding secrets into DockerFile creates a significant attack surface. When attackers access the DockerFile, they can see hard-coded secrets, the base image, the list of dependencies, and critical file locations. Developers need to use appropriate secret managers to reference variables.
»Docker-compose.yml
Docker-compose defines networks, services, and storage volumes. When an attacker views the file, they can understand the application architecture and exploit or disrupt its operation.
docker-compose.yml
services:
web:
image: my-web-app:latest
ports:
- "80:80"
networks:
- app-network
db:
image: postgres:latest
volumes:
- db-data:/var/lib/postgresql/data
environment:
POSTGRES_PASSWORD: example_db_password
networks:
app-network:
volumes:
db-data:
In the above docker-compose.yml, the postgres database password is hardcoded. The password can easily be accessed with the docker exec command as shown below:
#terminal
docker exec -it <container id or name> /bin/bash
env
Apart from secrets, an attacker can also analyze the volume mappings and identify potential points of weakness. If they discover that the database volume (db-data) is also mounted to the host filesystem, they could exploit and perform a container breakout attack, gaining access to the underlying host system.
»CI/CD config files
CI/CD configuration files such as .gitlab-ci.yml, Azure-pipelines.yml , Jenkinsfile, etc., contain instructions for building, testing, and deploying applications. The logs generated in CI/CD pipeline can contain debugging and logging information. If a developer includes a logging statement that inadvertently prints a sensitive secret, it can lead to unauthorized exposure and compromise. Such secret leaks need to be detected so that developers can fix their source code.
Developers also tend to leave the registry login credentials in the CI CD configuration files. Consider the following gitlab-ci.yml.
.gitlab-ci.yml
variables:
DOCKER_IMAGE_TAG: latest
DOCKER_REGISTRY_URL: registry.example.com
DOCKER_IMAGE_NAME: my-docker-image
DOCKER_REGISTRY_USER: adminuser <-- should use $CI_REGISTRY_USER
DOCKER_REGISTRY_PASSWORD: secretpassword <-- should use $CI_REGISTRY_PASSWORD
# Jobs
build:
stage: build
image: image_name:stable
script:
- docker build -t $DOCKER_REGISTRY_URL/$DOCKER_IMAGE_NAME:$DOCKER_IMAGE_TAG .
- docker login -u $DOCKER_REGISTRY_USER -p $DOCKER_REGISTRY_PASSWORD $DOCKER_REGISTRY_URL
- docker push $DOCKER_REGISTRY_URL/$DOCKER_IMAGE_NAME:$DOCKER_IMAGE_TAG
In the configuration above, the developer makes a docker registry login using a hardcoded username and password, leading to unwarranted secret exposure. A good development practice is to integrate CI/CD environment with secret managers like Hashicorp Vault.
»Detecting secrets
Older or unused Docker images can contain unpatched vulnerabilities or outdated dependencies, posing security risks to the enterprise. Regularly scanning and removing unused images helps mitigate these risks by reducing the attack surface and ensuring that only secure images are deployed.
Enterprises also need to be actively using secret scanners to detect secrets in docker images, whether they’re stored in Docker Hub, JFrog Artifactory, AWS ECR, or any other repository. HCP Vault Radar can meet these requirements and is an excellent choice since it's an add-on to the most popular secrets manager: HashiCorp Vault. Vault Radar analyzes the contents of each layer described in this post to identify secrets in the software packages and dependencies.
»Learn more
Vault Radar can scan your container images and other destinations such as source code, productivity applications like Jira, Confluence, Slack, Terraform variables, server directories, and more. When it detects leaked secrets, it has options to remediate them and enhance your security posture.
You can sign up now for an HCP Vault Radar test run to detect secret sprawl in your enterprise, or learn more about HashiCorp Vault and Vault Radar on our homepage.
This post was originally published on Dev.to.
Sign up for the latest HashiCorp news
More blog posts like this one
Vault integrations with MongoDB, Private Machines, and walt.id strengthen customer security
Three new HashiCorp Vault ecosystem integrations extend security use cases for customers.
HashiCorp at re:Invent 2024: Security Lifecycle Management with AWS
A recap of HashiCorp security news and developments on AWS from the past year, for your security management playbook.
HCP Vault Dedicated adds secrets sync, cross-region DR, EST PKI, and more
The newest HCP Vault Dedicated 1.18 upgrade includes a range of new features that include expanding DR region coverage, syncing secrets across providers, and adding PKI EST among other key features.