Products & Technology

X.509 certificate management with Vault

In this blog post, we’ll look at practical public key certificate management in HashiCorp Vault using dynamic secrets rotation.

Feb 27 2024Christie Koehler, Rosemary Wang

This blog post was originally published in 2018 and updated on October 24, 2022 and February 27, 2024.

HashiCorp Vault provides secrets management and protection of sensitive data. It offers a central place to secure, store, and control access to tokens, passwords, certificates, and encryption keys. Users typically start by creating secrets and storing them in Vault’s static secrets engine. Applications would then retrieve and use secrets from Vault, restarting each time you manually revoke and rotate the secret in the Vault.

Alternatively, Vault can manage the revocation and rotation of secrets for you in the form of dynamic secrets. For example, Vault applies a dynamic secret approach to X.509 public key infrastructure (PKI) certificates, acting as a signing intermediary to generate short-lived certificates. This allows certificates to be generated on-demand and rotated automatically.

In this post, we’ll demonstrate how to configure Vault to manage PKI certificates with both self-signed and offline root certificate authorities (CAs). We’ll also use Vault Agent to write certificates to a file for applications to use.

»Enable the PKI secrets engine

Vault supports many secrets engines plugins that handle the storage and rotation of secrets. Secrets engines are enabled at a mount path. For Vault to manage and issue certificates, enable the PKI secrets engine at the pki/ path.

$ vault secrets enable pki
 
Success! Enabled the pki secrets engine at: pki/

By default, Vault mounts secrets engines at the path corresponding to their type. You can also enable secrets engines at their own unique path using the -path argument. Enabled secrets engines cannot access each other’s data, even if they are of the same type.

Certificates include a validity period with a start and end date before they expire. When enabling Vault’s PKI secrets engine, certificates have a default validity period of 30 days. However, most certificates have a validity period of up to one year. To configure the validity period of the certificate, adjust the global maximum time-to-live (TTL) for the secrets engine. For more information on the PKI secrets engine, refer to documentation on setup and usage.

»Configure a root CA

Each PKI secrets engine needs to reference a root CA, CA certificate, and private key. You have three methods for configuring a root CA in Vault:

Generate a self-signed root CA issued by Vault Bring your own offline root CA Import a CA certificate and private key bundle using the pki/config/ca endpoint.

For example, you can generate a self-signed root CA with a validity period of one year using the pki/root/generate/internal endpoint:

$ vault write pki/root/generate/internal \
   common_name=my-website.com \
   issuer_name="blog-root" \
   ttl=8760h
 
## omitted for clarity
 
Key                     Value
---                     -----
certificate             REDACTED
expiration              1697210094
issuer_id               5e240c32-47ce-8f9a-fac0-0c712e98c1e1
issuer_name             blog-root
issuing_ca              REDACTED
key_id                  b10be38f-4834-dd57-fb75-e742d92585a6
key_name                n/a
serial_number           REDACTED

In general, you will want to create a CA hierarchy in which a root CA issues intermediate CAs. Each intermediate CA will issue leaf certificates to applications and other services. A hierarchical CA protects the root CA by separating intermediate CAs depending on their purpose.

If you bring your own root CA hosted outside of Vault, avoid storing it in Vault. Issue short-lived intermediate CAs by creating mounts for each intermediate. Review PKI secrets engine documentation for additional security considerations.

Vault only allows one CA certificate per secrets engine. If you want to issue certificates from multiple CAs, mount the PKI secrets engine at multiple mount points with separate CA certificates in each.

»Set URL configuration

Each PKI secrets engine requires:

A URL configuration (for issuing certificate endpoints)
Certificate revocation list (CRL) distribution points
Online Certificate Status Protocol (OCSP) server endpoints The URLs usually point to Vault’s fully-qualified domain name (FQDN). This example uses a Vault instance running locally:

$ vault write pki/config/urls \
    issuing_certificates="http://127.0.0.1:8200/v1/pki/ca" \
    crl_distribution_points="http://127.0.0.1:8200/v1/pki/crl"
 
Success! Data written to: pki/config/urls

You can update the URLs at any time.

»Create a role to generate certificate

Generating certificates requires you to supply a Vault role. The role definition sets the conditions under which a certificate can be generated.

Use the /pki/roles/<name> endpoint to create and update roles.

$ vault write pki/roles/example-dot-com \
    allowed_domains=example.com \
    allow_subdomains=true max_ttl=72h
 
Success! Data written to: pki/roles/example-dot-com

Review the PKI HTTP API reference to learn about other attributes you can configure for roles, including allowed domains and IP Subject Alternative Names (IP SANs).

Once a role has been created, you can use it to generate certificates with the pki/issue endpoint:

$ vault write pki/issue/example-dot-com \
    common_name=my.example.com
 
Key                    Value
---                    -----
ca_chain               REDACTED
certificate            REDACTED
expiration             1665938529
issuing_ca             REDACTED
private_key            REDACTED
private_key_type       rsa
serial_number          REDACTED

Renew your certificate by providing an issuer with the same common name as an existing certificate. The original certificate will continue to be valid through its original time-to-live unless explicitly revoked.

»Revoke or manage expired certificates

When you revoke a certificate, you also regenerate the CRL. This removes any expired certificates from the list.

To revoke your certificate based on its serial number, type the following command:

$ vault write pki/revoke serial_number=<serial_number>
 
Key                        Value
---                        -----
revocation_time            1665679572
revocation_time_rfc3339    2022-10-13T16:46:12.169387969Z

To force a rotation of all certificates, read from the pki/crl/rotate endpoint:

$ vault read pki/crl/rotate
 
Key        Value
---        -----
success    true

Vault will maintain expired certificates for a certain buffer period. To optimize Vault’s storage backend and CRL, use the tidy endpoint to remove expired certificates from Vault.

$ vault write pki/tidy tidy_cert_store=true tidy_revoked_certs=true

You can also set up automatic tidying, which periodically removes expired certificates from Vault’s storage backend.

»Configure applications to use certificates from Vault

By setting up the PKI secrets engine, Vault automates the process of generating a private key, generating a certificate signing request (CSR), submitting to a CA, and then waiting for a verification and signing process to complete. How can applications and other services retrieve and use certificates from Vault?

Rather than refactor applications to call the Vault API, you can use Vault Agent to retrieve a certificate from Vault and write it to a file for the application to use.

»Create a Vault policy for the application’s certificates

The application needs sufficient access to retrieve a certificate from the PKI secrets engine. Create a Vault policy with create, read, and update permissions for the pki/ endpoint.

$ echo 'path "pki/*" {
  capabilities = ["read","create","update"]
}' | vault policy write certs -
 
Success! Uploaded policy: certs

»Set up authentication to Vault

For Vault Agent to authenticate to Vault, set up a compatible authentication method. Enable the AppRole auth method for this example:

$ vault auth enable approle

Create a named role for Vault Agent that matches the role for the PKI secrets engine.

$ vault write auth/approle/role/example-dot-com \
   role_id=example-dot-com \
   secret_id_ttl=30m \
   token_num_uses=0 \
   token_ttl=30m \
   token_max_ttl=60m \
   token_policies=certs \
   secret_id_num_uses=0
 
Success! Data written to: auth/approle/role/example-dot-com

Get the role ID and save it to a file for Vault Agent to reference.

$ vault read -field=role_id \
   auth/approle/role/example-dot-com/role-id > vault_agent_role_id

Get the secret ID and save it to a file for Vault Agent to reference.

$ vault write -f -field=secret_id \
   auth/approle/role/example-dot-com/secret-id > vault_agent_secret_id

»Create templates for certificate data

Vault Agent queries Vault at a supplied API endpoint (pki/issue/example-dot-com) with the given parameters (common_name=my.example.com). You previously issued the equivalent command as vault write pki/issue/example-dot-com. The API query returns a response with the following data:

{
...
  "data": {
    "certificate": "-----BEGIN CERTIFICATE-----",
    "issuing_ca": "-----BEGIN CERTIFICATE-----",
    "private_key": "-----BEGIN RSA PRIVATE KEY-----",
    ...
    },
}

Use Vault Agent’s template functionality to extract the values of each field into individual files.

Vault Agent reads a set of templates to create new files with the certificate contents, so create a directory for template files:

$ mkdir templates

Create a template file for the certificate, CA, and private key. The template reads information from pki/issue/example-dot-com using the pkiCert function and writes out the certificate, CA, and private key for Vault Agent to track. Using the writeToFile function, the certificate, CA, and private key each get written to their own files for a service to use.

$ cat << EOF > templates/certificate.tpl
{{ with pkiCert "pki/issue/example-dot-com" "common_name=my.example.com" }}
{{ .Cert }}{{ .CA }}{{ .Key }}
{{ .Key | writeToFile "examples/my-app.key" "root" "root" "0400" }}
{{ .CA | writeToFile "examples/ca.crt" "root" "root" "0644" }}
{{ .Cert | writeToFile "examples/my-app.crt" "root" "root" "0644" "append" }}
{{ end }}
EOF

»Use Vault agent to create certificate files

Vault Agent uses the role and secret ID to authenticate to Vault and retrieve certificate information. Then, it writes the certificate data based on each template file. Create a file for Vault Agent configuration using the code below:

$ cat << EOF > vault-agent.hcl
pid_file        = "./pidfile"
exit_after_auth = true
 
vault {
  address = "http://127.0.0.1:8200"
}
 
auto_auth {
 
  method {
    type = "approle"
    config = {
      role_id_file_path                   = "vault_agent_role_id"
      secret_id_file_path                 = "vault_agent_secret_id"
      remove_secret_id_file_after_reading = false
    }
  }
 
  sink {
    type = "file"
    config = {
      path = "vault_agent_token"
    }
  }
 
}
 
template {
  source      = "templates/certificate.tpl"
  destination = "examples/all-certs"
}
EOF

Use multiple template blocks to define multiple templates. The source directive indicates which source file on disk to use as the input template. The destination directive indicates the path on disk where the source template will render.

Run Vault Agent to generate the certificate files:

$ vault agent -config=vault-agent.hcl

Review the certificate files under the examples/ directory. It has four files: one for the certificate, CA, private key, and aggregated file for all certificate information.

$ ls examples/
 
all-certs   ca.crt  my-app.crt  my-app.key

Vault Agent renews the certificate at half the lease duration of the original. For example, if the certificate role has a maximum lease of 72 hours, Vault Agent generates a new certificate every 36 hours.

»Summary

Using Vault to manage certificate rotation and revocation, you can audit the issuance and expiration of certificates from one central location. To enable your application to use the certificates without refactoring the application, configure Vault Agent to retrieve the certificates each time they update and write them to a file.

Review our Vault PKI documentation for more attributes and configuration for the PKI secrets engine. For more information about Vault Agent, review its supported authentication methods and templating language.

This post was originally published in 2018 by HashiCorp Developer Advocate Christie Koeler. It was updated in 2022 by HashiCorp Developer Advocate Rosemary Wang.