PKI certificate metadata in Vault

Creating and using metadata with PKI certificates allows you to associate custom attributes to certificates beyond the basic data already available in X.509 format. For example, many larger organizations might want to associate additional business context or data, such as accounting codes, with a given certificate. Prior to Vault 1.17, customers could record such metadata only via manual offline processes, which is non-scalable and error-prone.

In HashiCorp Vault 1.17, custom metadata can now be created and submitted with the certificate signing request (CSR). The issued certificate and associated metadata is retained by Vault and thus completely automates the retention and association of metadata with the issued certificate.

This post explains the value of PKI certificate metadata and how it works in Vault.

»The value of PKI certificate metadata

The ability to create custom metadata and associate it with your PKI certificates is a powerful capability allowing you to associate custom attributes with your certificates beyond the standard data available on your certificates. By creating and using custom metadata, you can add information such as certificate owner, business unit, and billing or contact information. These attributes can then be used to create groups or collections of certificates to help you more effectively manage them at scale. For example, you can use metadata to monitor certificate usage, helping to ensure compliance with industry regulations and alignment with the policies and standards of your organization.

»General metadata categories

The type and amount of custom metadata you want to create varies depending on your needs and priorities, so it helps to take some time to think about what custom metadata is most useful for your business. While not exhaustive, there are three initial metadata categories you might consider

• Business information: Organizations might want to track data specific to business operations. This could be information such as billing codes or departments, which are used for accounting purposes. It could also include associated risk profiles or compliance categories for auditors.
• PKI administration: A PKI administrator may want to use metadata to tie the certificate to an application owner and location, which makes it easier to track down pertinent information such as application servers, DNS names, and contact details for every certificate issued. Having these tags makes it easier for a team to track down and take action when a certificate is about to expire.
• Operational information: Knowing who owns the application on a server is one thing, but it’s also important to capture operational details. The IT operations team may want to know which servers a certificate protects, the operating system being used, or what the various DNS or SANs are for the certificate. Metadata can even be created to track information such as the virtual host or Kubernetes cluster a particular certificate is used for.

Many other kinds of metadata could also be useful, so it makes sense to carefully think through and clearly define what metadata you want to track before you create it. Taking the time to plan up front will ensure you create, apply, and track useful metadata associated with your certificates to more efficiently operate at scale.

»How PKI certificate metadata works in Vault

The diagram below is an overview of a PKI certificate signing request which includes the association of custom metadata at the time the certificate is created and signed. When an application or user submits a certificate signing request (CSR), custom metadata can be automatically submitted with the certificate request. The custom metadata and the issued certificate are stored by the PKI secrets engine for later retrieval via the PKI API.

A typical example use case would be to leverage custom metadata associated with an application, such as billing code, and associate the metadata with the application’s certificate when the certificate is created and signed.

»Role-based configuration

All PKI secrets engine configurations are done via role. The same is true for creating custom metadata. Now, when you create a role you will see a new field called no_store_metadata that is set to “false” by default. With this setting, you can write custom metadata and store it in the PKI secrets engine.

The metadata is associated with the certificate, but not actually added or physically attached to the certificate. You can also set the no_store_metadata field to “true” if you do not want to store metadata. When the field is set to “false”, certificate metadata will be retained by the Vault PKI secrets engine using base-64 encoding.

You can use familiar PKI secrets engine commands to create and manage your custom metadata. These commands include write, list, read and tidy.

To create a new certificate and associate custom metadata for the application owner (“appowner” is the label here), your command structure would look like this:

Vault write pki/issue/myrole common_name=”company.example.com” cert_metadata=$(base 64 <<< “appowner”) TTL=30s

To see a list of all certificates with associated metadata, you can run the list command:

Vault list /pki/cert-metadata/

The resulting list will provide serial numbers for all certificates with associated metadata:

Keys

---

17:67:16:b0:b9:45:58:c0:3a:29:e3:cb:d6:98:33:7a:a6:3b:66:c1

26:0f:76:93:73:cb:3f:a0:7a:ff:97:85:42:48:3a:aa:e5:96:03:21

67:b4:f7:2c:aa:ef:b9:30:f6:ae:f5:12:21:79:ac:08:8a:86:89:72

Once you identify the serial number of the certificate for which you want to see the associated metadata, you can run the read command against that serial number to retrieve its metadata:

Vault read pki/cert-metadata/67:b4:f7:2c:aa:ef:b9:30:f6:ae:f5:12:21:79:ac:08:8a:86:89:72

Running the read command provides the following certificate data:

Key	Value

---	 ---- 

issuer_id 		e27bf456-51e1-d937-0001-4a609184fd9b

expiration 	2022-11-02T14:41:47.327515Z

cert_metadata 	dXNlci1wcm92aWRlZC1tZXRhZGF0YQ==

Role 	MyRole

serial_number 	67:b4:f7:2c:aa:ef:b9:30:f6:ae:f5:12:21:79:ac:08:8a:86:89:72

The following command will fetch and decode the custom metadata associated with the certificate:

base64 --decode <<< $(vault read --field=cert_metadata pki/cert-metadata/67:b4:f7:2c:aa:ef:b9:30:f6:ae:f5:12:21:79:ac:08:8a:86:89:72 )

You cannot edit or update the metadata after creating it, but you can use the tidy command to delete it once the associated certificate has expired or been revoked. The metadata information is typically useful only as long as the certificate it describes is useful. Therefore, metadata information should be cleaned up in the same manner that certificates are cleaned up, by using the tidy endpoint(s).

Because the expiration date of the certificate is stored on the metadata entry itself, the tidy operation simply compares that date with the current date and deletes the entry if the certificate it describes has expired.

This does require you to enable metadata individually on each role. We do not expect this to be a concern, even with a large number of roles, as they would usually be generated using Terraform templates.

»Other metadata creation methods

Role-based configuration does not cover every endpoint that can be used to generate a certificate. For instance, certificate authority (CA) generation and CIEPS do not use roles. These methods have less obvious value than role-based issuance because:

• CAs are usually generated very rarely, and therefore are much easier to track relevant metadata manually.
• Because ACME and EST are independent protocols, they cannot easily accept a “metadata” field.
• CIEPS, by design, requires an external policy service, and by integrating such a service you already have an integration point you can use to create and associate metadata with a certificate if desired.

»Learn more

Creating custom metadata for PKI certificates further enhances HashiCorp Vault’s PKI and secret lifecycle management capabilities to help enterprises reduce risks and improve efficiency. The ability to centralize secrets management along with certificate lifecycle management further differentiates Vault’s Security Lifecycle Management portfolio.

To learn more, check out these resources:

• What is PKI?
• PKI secrets engine
• Automated PKI infrastructure