HashiCorp Vault 0.2
We are proud to announce the release of Vault 0.2. Vault is a tool for managing secrets. From storing credentials and API keys to encrypting sensitive data, Vault is meant to be a solution for all secret management needs.
The initial public release of Vault was almost two months ago and we have been busy extending the core capabilities, adding new secret and storage backends, improving user experience, and fixing bugs.
Vault 0.2 brings many new features including key rotation, rekeying, a PKI secret backend for dynamic certificate generation, a Cassandra secret backend, many new storage backends, and derived unique per-transaction keys for the transit backend. There are so many amazing changes that we cannot possibly list them here, so please see the full Vault 0.2 CHANGELOG for more details.
You can download Vault 0.2 from the project website.
Read on to learn more about the major new features in Vault 0.2.
Key Rotation and Rekeying
When a Vault server is first started, the vault init
command is used to initialize
the system. This creates the encryption key which is used to protect data at rest,
the master key which is used to protect the encryption key, and key shares which
are used to split the master key.
In Vault 0.1, all of those were static and could not be changed after they
were created. Vault 0.2 allows these to all be updated. This is done by
adding two new commands: rotate
and rekey
.
The first command rotate
is used to change the encryption key that is used
to protect data at rest. When invoked (either through the CLI or directly
with the API), a new random key is generated and installed in the keyring. All
new data that is persisted uses the new key, while keys encrypted under older
keys can still be decrypted. As of Vault 0.2, the old keys are not automatically
re-encrypted but this is a planned future enhancement.
$ vault key-status
Key Term: 1
Installation Time: 2015-07-14 20:46:31 +1000 AEST
$ vault rotate
Key Term: 2
Installation Time: 2015-07-14 20:46:36 +1000 AEST
$ vault key-status
Key Term: 2
Installation Time: 2015-07-14 20:46:36 +1000 AEST
The second command rekey
is used to change the master key or the parameters
of the key shares. The init
command allows the number of shares and the
threshold required to be set. Using the rekey
command allows those parameters
to be changed. This also provides a way to change the key shares if a previous
key holder leaves an organization.
The rekey
process is a bit more complex, as it must involve the existing
key holders. The process is started by specifying the new number of shares
and threshold (which could be the same). Once the existing threshold is met,
the master key will be regenerated and the new key shares provided.
$ vault rekey -init
Started: true
Key Shares: 5
Key Threshold: 3
Rekey Progress: 0
Required Keys: 1
$ vault rekey
Rekey already in progress
Key Shares: 5
Key Threshold: 3
Key (will be hidden):
Key 1: 9de4b5732ad06ca5c982ebc189d100763e0dfa88afc44ebfe2d707237cde17c001
Key 2: 54cc5de89f2731e46eaac690b3b017fa6173acab9375285bfb360c6b6c6bc15902
Key 3: 2f216b05195ae4b454f98c23ed0aa9a966cb4e187f54a71a1253ab975b0f1d6403
Key 4: 9288da985a7956255b7e8ee75220e27fcaa0dc99b4bb953314822aeb3a674dfb04
Key 5: e965ec75dc048375612dc4540c9a5c2ccd183e2a589a1a72fde78d170d0391c605
Vault rekeyed with 5 keys and a key threshold of 3. Please
securely distribute the above keys. When the Vault is re-sealed,
restarted, or stopped, you must provide at least 3 of these keys
to unseal it again.
Vault does not store the master key. Without at least 3 keys,
your Vault will remain permanently sealed.
Both the rekey
and rotate
commands can be done online without any
disruption to service. They also work in HA environments with multiple
Vault instances.
PKI Secret Backend
The new Vault pki
secret backend
can be used to turn Vault into an internal certificate authority. The backend makes use
of the dynamic secret features of Vault to generate x509 certificates on demand for
clients.
Running an internal PKI setup has typically been a challenging process due to the automation required to generate certificates, distribute them, and properly manage updating and revocation. This has caused many organizations to issue very long-lived certificates and distribute them with configuration management systems. This creates a large exposure surface and in the case of a compromise the long-lived certificates tend to be usable due to limited use of CRL and OCSP.
The pki
backend to Vault makes this process much simpler and more secure.
Operators can simply provide Vault with a root or intermediary certificate and
define various roles which require certificates. Clients of Vault can requests
certificates as needed, and have them generated on demand. Instead of using very
long-lived certificates, Vault can generate certificates that last only minutes,
and instead are frequently updated. This limits the exposure in case of a compromise,
and combined with automatic CRL generation makes it much safer to issue certificates.
$ vault write pki/issue/common common_name=www\.hashicorp\.com
Key Value
lease_id pki/issue/common/819393b5-e1a1-9efd-b72f-4dc3a1972e31
lease_duration 259200
lease_renewable false
certificate -----BEGIN CERTIFICATE-----
MIIECDCCAvKgAwIBAgIUXmLrLkTdBIOOIYg2/BXO7docKfUwCwYJKoZIhvcNAQEL
...
az3gfwlOqVTdgi/ZVAtIzhSEJ0OY136bq4NOaw==
-----END CERTIFICATE-----
issuing_ca -----BEGIN CERTIFICATE-----
MIIDUTCCAjmgAwIBAgIJAKM+z4MSfw2mMA0GCSqGSIb3DQEBCwUAMBsxGTAXBgNV
...
-----END CERTIFICATE-----
private_key -----BEGIN RSA PRIVATE KEY-----
MIIEowIBAAKCAQEA0cczc7Y2yIu7aD/IaDi23Io+tvvDS9XaXXDUFW1kqd58P83r
...
3xhCNnZ3CMQaM2I48sloVK/XoikMLb5MZwOUQn/V+TrhWP4Lu7qD
-----END RSA PRIVATE KEY-----
serial 5e:62:eb:2e:44:dd:04:83:8e:21:88:36:fc:15:ce:ed:da:1c:29:f5
The pki
backend is another example of how Vault can be used as a security
foundation, to go beyond just storing secret data. Using the pki
backend makes
it simpler for organizations to embrace mutual TLS and begin to migrate to a
zero trust datacenter.
A major thanks to Jeff Mitchell for contributing this backend and to Akamai for sponsoring his efforts.
Per-Transaction Derived Keys
The transit
secret backend
has been available since the initial release of Vault. The backend is used to
help encrypt and decrypt data in transit through Vault. We previously blogged
about how HashiCorp uses Vault's transit backend to protect the sensitive data
stored by Atlas.
At the core, the transit
backend manages a set of named keys. The keys are held
within Vault, and data can be sent in plaintext to be encrypted within Vault or
ciphertext sent to be decrypted. This allows clients to manipulate data without
ever having access to the encryption keys. This makes it easier to manage
PII information in web servers, while still using RDBMS systems for scalable storage.
With Vault 0.2, the transit
backend now supports setting the derived
property
of a named key. When a key is created with this property, any encrypt or decrypt
operation must also provide a context
along with the plaintext or ciphertext.
This context
is used with the raw encryption key to generate a derived key.
This transaction key is derived using the secured high entropy key, and is used
to encrypt or decrypt the provided data. This allows each transaction using the
same named key to use a different encryption key. A compromise of any transaction
does not reveal any data about any other transactions with a different context
or the raw encryption key.
$ vault write transit/keys/foo derived=true
Success! Data written to: transit/keys/foo
$ CTX=`echo -n foo | base64`
$ vault write transit/encrypt/foo context=$CTX plaintext=`echo -n bar | base64`
Key Value
ciphertext vault:v0:DykggPIDCMz+vqrWsqa309HtQs2c3Y2BxedNCTlQQQ==
$ vault write transit/decrypt/foo context=$CTX ciphertext="vault:v0:DykggPIDCMz+vqrWsqa309HtQs2c3Y2BxedNCTlQQQ=="
Key Value
plaintext YmFy
$ echo YmFy | base64 -D
bar
The actual implementation of derived keys is based on the NIST SP 800-108 recommendations.
ACL Policy Improvements
The policy language for Vault was inspired by the Consul ACL policy language. In Vault 0.1, the policies looked like:
path "sys/" {
policy = "read"
}
path "secret/" {
policy = "write"
}
In this original version, all path
stanza's would glob and greedily
match on the request path by default. This made it impossible to specify
an exact-match policy. With Vault 0.2, this is now made explicit with the
*
glob wildcard:
path "sys/*" {
policy = "read"
}
path "secret/*" {
policy = "write"
}
# Only permit the 'foo' key exactly
path "transit/encrypt/foo" {
policy = "write"
}
Second, Vault is always default deny system, meaning unless permission
is explicitly given it will be denied by the ACLs. However, with Vault 0.1
the policy language gave the deny
policy the lowest precedence, this made
it hard to black list specific paths. Vault 0.2 now gives it the highest
precedence. This enables the following;
path "secret/*" {
policy = "read"
}
path "secret/super-secret" {
policy = "deny"
}
With Vault 0.2, any operation to the secret/super-secret
path would
now be denied where it was previously allowed by the first rule. Please note
that this may be a breaking change for existing policy files.
Upgrade Details
Vault 0.2 introduces some major internal changes to support the rekey
and rotate
commands. The policy language changes similarly change the
way policies are specified and enforced.
All of these upgrades however are done online when the new version of Vault is unsealed. The existing representation of the encryption key will be upgraded to the keyring in place. Any existing policies will be updated to ensure the implicit glob behavior continues to work.
All policy files should be upgraded, since any new policies written are assumed to conform to the Vault 0.2 specification. There is no way to use the old specification as an attempt to deprecate it while the project is still young.
Roadmap
Vault 0.2 is a major release that extends the functionality and fixes many major usability and stability issues. As a result, we expect that there will be some new issues which will be address in point released following. We are expecting to start a full external code audit of Vault shortly.
Following that, Vault 0.2 will be focused on adding support for vault ssh
, which
will use one-time-keys and one-time-passwords for solving the shared SSH key problem,
adding support for new secret, auth, and credential backends, improving usability
and fixing bugs.
As always, we recommend upgrading and testing this release in an isolated environment. If you experience any issues, please report them on the Vault GitHub issue tracker or post to the Vault mailing list.
Sign up for the latest HashiCorp news
More blog posts like this one
Fix the developers vs. security conflict by shifting further left
Resolve the friction between dev and security teams with platform-led workflows that make cloud security seamless and scalable.
HashiCorp at AWS re:Invent: Your blueprint to cloud success
If you’re attending AWS re:Invent in Las Vegas, Dec. 2 - Dec. 6th, visit us for breakout sessions, expert talks, and product demos to learn how to take a unified approach to Infrastructure and Security Lifecycle Management.
HCP Vault Secrets adds enterprise capabilities for auto-rotation, dynamic secrets, and more
HCP Vault Secrets focuses on making a fast and easy path for secure development with key new features including auto-rotation (GA), dynamic secrets (beta), a new secret sync destination, and more.