HashiCorp Vault 0.6

We are proud to announce the release of Vault 0.6. Vault is a tool for managing secrets. From API keys and encrypting sensitive data to being a complete internal CA, Vault is meant to be a solution for all secret management needs.

This release contains major new features, some new secure workflow enhancements, and many improvements and bug fixes. A major focus was token management and token/authentication workflows.

In Detail:

• Token Accessors
• Token Authentication Backend Roles
• Response Wrapping
• AWS EC2 Authentication Backend

In Brief:

• Codebase Audit
• Integrated Consul Health Checks
• Listener Certificate Reloading
• MSSQL Credential Generation
• Azure Data Store
• Swift Data Store

(Some of these features appeared in 0.5.1 and 0.5.2, however they were not discussed in previous blog posts.)

Please see the full Vault 0.6 CHANGELOG for more details. Additionally, please be sure to read the upgrade information at the end of this post.

As always, a big thanks to our community for their ideas, bug reports, and pull requests.

Read on to learn more about the major new features in Vault 0.6.

»Token Accessors

When creating tokens, two values are now returned: the token itself and an accessor.

$ vault token-create -policy=default
Key             Value
---             -----
token           82c5fb97-da1b-1d2c-cfd5-23fa1dca7c85
token_accessor  dd256e17-b9d9-172d-981b-a70422e12cb8
token_duration  2592000
token_renewable true
token_policies  [default]

The accessor can be used to perform lookup and revocation on a token without knowledge of the token itself, either via auth/token/lookup-accessor/auth/token/revoke-accessor or via the -accessor flag to the respective CLI commands. The lookup function, when using an accessor, does not return the token ID.

This allows a service generating tokens for other clients to store identifying information about a client along with the client's accessor in order to perform lookup and, if necessary, revocation of the token later on. For instance, if an employee is given a token and later leaves the company, the accessor can be used to revoke the token without needing the token ID itself to be stored.

To facilitate this, as well as make this workflow useful when not creating tokens directly against the token store, there is also now an option to the audit backends to disable HMAC'ing of the accessor value. (It is HMAC'd by default since someone with access to the accessor values could implement denial-of-service attacks by revoking tokens without authorization, and logs are often not tightly controlled.) When enabling an audit backend, pass hmac_accessor=false as one of the configuration options to enable this behavior.

»Token Authentication Backend Roles

The Token Authentication Backend has gained a concept of roles. These provide a degree of administrative flexibility to allow authorized users to create tokens, including some with unique properties, on behalf of others.

Together with token accessors, these roles enable much better workflows when using direct token creation. An administrator can configure a role to provide certain limits or capabilities and delegate token creation for the role to a trusted user or service. As that service issues tokens, the accessor values can be written in plaintext into the audit logs; if revocation is needed, specific tokens can be revoked via the accessors, or all tokens issued from a certain role can be revoked.

One extra note: tokens created directly via auth/token/create can now also have their renewability and explicit maximum TTL controlled.

»Response Wrapping

Token authentication backend roles and token accessors provide better degrees of management granularity, but do not solve the problem of securely distributing generated tokens. Secure introduction is a difficult problem, and one of the reasons is that it can be very hard to establish a secure channel in advance.

In Vault 0.6 we are introducing response wrapping. This is a security primitive that can be used not only for token distribution but to facilitate secure distribution of Vault-derived secrets generally.

Before going into detail about the mechanism, interested readers may want to review the Cubbyhole authentication paradigm post -- this is completely optional as the mechanism will be discussed in brief in the next paragraph. However, response wrapping is a formalized implementation of a part of that paradigm that provides better security than can be achieved from a standalone process managing temp and perm tokens. However, this implementation goes beyond what is described in the post, in that, with the exception of /sys paths, response wrapping can be used for any response coming from Vault.

Response wrapping works as follows, given a scenario in which a client wants to retrieve a secret from Vault but wants to ensure that only the intended target can see the secret:

1. The client requests that the response be wrapped by providing a duration for the wrapping token. On the CLI, this can be specified with the global client flag -wrap-ttl.
2. Vault processes the request normally. However, once the response has been generated, rather than JSON-serialize the response and return it, it JSON-serializes the response and places it into a new token's cubbyhole in a well-known location. The new token is limited-use with a single use remaining, has the TTL specified by the client, and cannot be renewed.
3. Vault generates and returns a new response containing wrapping information: the wrapping token ID, and the wrapping token's duration.
4. The client passes on the wrapping token to the intended target.
5. The intended target performs an unwrap operation using the token. Under the hood, this simply fetches the original response from the well-known location in the token's cubbyhole, parses the JSON, and returns it as normal. From the perspective of the API or CLI, there is no difference between a response directly returned by Vault and one parsed and read from a wrapping token.
6. If the intended target is told that permission is denied, it can check the token's TTL against its creation time to see if it has expired (perhaps it took too long to get to the intended target). However, if the token has not expired, the target can raise a security alert with the assumption that the response has already been unwrapped and the secret has been divulged to an unintended party.

To illustrate this, the following shows two calls to the token-create CLI command. One behaves normally, and the other uses wrapping followed by an unwrap command.

#
# Normal token-create command
#

$ vault token-create -policy=default
Key             Value
---             -----
token           ff999148-077e-2629-8d9a-cb0a7b97e811
token_accessor  8c243823-5820-ff8f-f641-6999290c60c0
token_duration  2592000
token_renewable true
token_policies  [default]

#
# token-create command with response wrapping
#

$ vault token-create -policy=default -wrap-ttl=60s
Key                             Value
---                             -----
wrapping_token:                 7a39125a-2001-be9b-e363-3ba85a16c311
wrapping_token_ttl:             60
wrapping_token_creation_time:   2016-06-14 06:02:11.171558903 +0000 UTC
wrapped_accessor:               ed96a7ae-dfdb-3c09-0a60-a02b53dfe6b2

#
# Unwrapping the wrapped response
#

$ vault unwrap 7a39125a-2001-be9b-e363-3ba85a16c311
Key             Value
---             -----
token           d878b2ed-564e-0790-4154-67bcf7386268
token_accessor  ed96a7ae-dfdb-3c09-0a60-a02b53dfe6b2
token_duration  2592000
token_renewable true
token_policies  [default]

#
# Detection of invalid (previously used) token
#

$ vault unwrap 7a39125a-2001-be9b-e363-3ba85a16c311
error reading cubbyhole/response: Error making API request.

URL: GET https://127.0.0.1:8200/v1/cubbyhole/response
Code: 400. Errors:

* permission denied

As can be seen from the example, when the wrapped response contains a token, the token's accessor is provided with the wrapping information. This allows a privileged caller generating tokens to easily keep track of which accessor corresponds to which client, in case the token must be revoked.

Because response wrapping can be used with almost any Vault request, it can be used to provide extra transport security for any secret:

#
# K/V (generic) backend
#

$ vault write secret/foo bar=baz
Success! Data written to: secret/foo

$ vault read -wrap-ttl=60s secret/foo
Key                             Value
---                             -----
wrapping_token:                 3a63ff9f-1c38-af43-0a85-dc1155f2469d
wrapping_token_ttl:             60
wrapping_token_creation_time:   2016-06-10 13:46:14.155857566 -0400 EDT

$ vault unwrap 3a63ff9f-1c38-af43-0a85-dc1155f2469d
Key                     Value
---                     -----
refresh_interval        2592000
bar                     baz

#
# Transit backend
#

$ echo "bar" | base64 | vault write transit/encrypt/foo plaintext=-
Key             Value
---             -----
ciphertext      vault:v1:j4Z1/6WLK+snlhIyooxa1zW0yEmBqFSfZTL88bN/Qt4=

$ vault write -wrap-ttl=60s transit/decrypt/foo ciphertext="vault:v1:j4Z1/6WLK+snlhIyooxa1zW0yEmBqFSfZTL88bN/Qt4="
Key                             Value
---                             -----
wrapping_token:                 e93a78ec-cced-dc94-d1f8-d71e84faddde
wrapping_token_ttl:             60
wrapping_token_creation_time:   2016-06-07 15:59:18.251657838 -0400 EDT

$ vault unwrap -field=plaintext e93a78ec-cced-dc94-d1f8-d71e84faddde | base64 -d
bar

»AWS EC2 Authentication Backend

The AWS EC2 Authentication Backend provides a way for EC2 instances to automatically authenticate to Vault and retrieve a Vault token. An AMI baked with a supporting client can authenticate without any user intervention or configuration management required.

The backend works by using AWS as a trusted third party. Specifically, AWS provides instance metadata signed with their private key and verifiable with their published public key. At a high level, the backend verifies the given metadata's authenticity, checks that the instance ID has not been used before (a principle known as TOFU, or Trust On First Use), and verifies that the instance is in a running state. The backend also accepts a client-specified nonce value that can be used by the client to fetch new tokens when the initial token expires, even while other clients attempting to reuse the metadata are denied access.

At a low level, various modifications are supported to fit into the security policy and workflow of a wide variety of organizations. Please see the documentation for details.

In order for an instance to authenticate, a client running in the instance must actually contact Vault and provide the necessary information through Vault's API. For Vault Enterprise customers, HashiCorp has built a standalone binary that does exactly this, allowing for easy deployment of this workflow. The client keeps the provided token renewed and, upon token expiration, reauthenticates to fetch a new token.

The Vault Enterprise client binary is simply a consumer of the backend API and does not rely on any behavior within Vault that is not open source. Organizations that are not Vault Enterprise customers can build their own client if they desire, or can contact HashiCorp for information on Vault Enterprise.

»Other Features

There are too many new features and improvements in this release to describe all of them in depth, so a few more are covered below in brief:

• Codebase Audit: Vault's 0.5 codebase was audited by iSEC Partners. (The terms of the audit contract do not allow us to make the results public.)
• Consul Data Store Health Checks: The Consul data store will automatically register a vault service and perform its own health checking. By default the active node can be found at active.vault.service.consul and all with standby nodes are standby.vault.service.consul. Sealed vaults are marked critical and are not listed by default in Consul's service discovery. See the documentation for details.
• Listener Certificate Reloading: Vault's configured listeners now reload their TLS certificates and private key when Vault is sent a SIGHUP, allowing for online rotation of Vault's certificates without needing to shut down Vault and unseal it again.
• MSSQL Secrets Backend: Similar to Vault's existing Postgres/MySQL support, there is now a backend to generate credentials for MSSQL.
• Azure Physical Data Store: Similar to Vault's existing S3 support, you can now use Azure blob object storage as your Vault physical data store.
• Swift Physical Data Store: Similar to Vault's existing S3 support, you can now use Swift blob object storage as your Vault physical data store.

»Upgrade Details

Vault 0.6 has behavioral changes that must be understood before upgrading. As such, we provide both general upgrade instructions as well as a version-specific guide. Note that this guide covers upgrading from 0.5.x; if upgrading from earlier versions, please take note of the upgrade instructions in the corresponding release blog posts.

As always, we recommend upgrading and testing this release in an isolated environment. If you experience any issues, please report them on the Vault GitHub issue tracker or post to the Vault mailing list.

We hope you enjoy Vault 0.6!