Building a Vault Token Helper

Vault is an open source tool for managing secrets. Earlier we showcased how Vault provides Encryption as a Service and how build custom secure Vault plugins. This post explores a lesser-known feature of Vault Open Source and Vault Enterprise: token helpers.

»Token Helpers

In many ways, Vault is conceptually similar to a web application. Users authenticate to applications via username and password, OAuth (login with GitHub, login with Twitter, etc), or tokens through an API. During that process, the web application takes the user-supplied information, verifies it, and maps the verified user's identity onto policies and permissions in the application. The user's "identity" is commonly returned in the form of a "session" which is stored in a secure cookie, managed by the web browser. Future requests to the website include the signed cookie, which the application uses to verify identity. Without this functionality, users would need to enter their username and password each time they clicked a link.

In Vault, the most equivalent thing to a "session" is the Vault Token. Additionally, since it does not run in the browser, Vault makes use of token helpers for the local vault command line tool. A token helper shares similar responsibilities for tokens as a browser does for cookies. Token helpers are responsible for storing, updating, retrieving, and deleting authentication data for the local CLI.

When making an authenticated API request, the Vault CLI queries the configured token helper, and the token helper returns the stored token. The Vault CLI uses this token to make authenticated API requests to the Vault server. Additionally, the Vault CLI may request the token helper store authentication data, perhaps as a byproduct of a vault auth command.

»Default Token Helper

Vault includes a default token helper, which persists the provided token in the current user's home directory at ~/.vault-token. For example:

$ vault auth 11413f3f-ec5b-cd25-92cb-96970d76bbe1
# ...

$ cat ~/.vault-token
11413f3f-ec5b-cd25-92cb-96970d76bbe1

Users on shared systems or users that do not trust the filesystem for storing sensitive data may want to leverage a different tool like 1Password, LastPass, or Keychain Access to store local Vault authentication information. Vault's extensible token helper model makes it easy to build custom integrations into these third party tools.

»Writing a Custom Token Helper

Token helpers are local binaries or scripts that receive shell commands from Vault to store, get, or erase a token:

store -> my-helper store (the token to store is passed on stdin)
get -> my-helper get (return the token on stdout)
erase -> my-helper erase

»Storing

Vault will invoke the token helper with the argument store. The token to store will be provided on standard in.

$ my-helper store 

For local testing, pipe the token to the binary:

$ echo "super-secret-token" | my-helper store

The token helper should:

1. Strip any leading or trailing whitespace from the provided token on standard in. Vault tokens will never have leading or trailing whitespace, but some shells can inadvertently add it.

2. Verify that the provided token is not the empty string. If the value is the empty string, call erase instead.

3. Persist the token in a durable manner. Because the binary is executed as a command (not a server), most token helpers cannot persist the token in memory; they will need to write the token to disk, a keychain, or some external tool.

4. Exit with status code 0 (ok).

In general, the store operation should only return errors that occur with persisting the provided token.

»Getting

Vault will invoke the token helper with the argument get. The token helper should return the stored token, if any, on standard out.

$ my-helper get
super-secret-token

If no value is stored, the token helper must return an empty value. A token helper should never return an error if a value is not stored.

The token helper should:

1. Retrieve the token from the durable storage.

2. Print the token on standard out.

3. Exit with status code 0 (ok).

»Erasing

Vault will invoke the token helper with the argument erase. The token helper should scrub the stored token, if any.

$ my-helper erase

If no value is stored, the token helper should simply return. A token helper should never return an error if a value is not stored.

The token helper should:

1. Purge or scrub any trace of the token.

2. Exit with status code 0 (ok).

»Example

Here is an example token helper written in Ruby which behaves similarly to the default token helper. Instead of storing the token in ~/.vault-token, it stores the token in /tmp/vault-token:

#!/usr/bin/env ruby

PATH = "/tmp/vault-token".freeze

def main(args)
  case args[0]
  when "get" then get
  when "erase" then erase
  when "store" then store
  else
    puts "Unknown method #{args[0]}"
    exit 1
  end
end

def get
  print File.read(PATH).strip if File.exist?(PATH)
end

def erase
  File.delete(PATH) if File.exist?(PATH)
end

def store
  File.open(PATH, "wb+") { |f| f.write($stdin.read.strip) }
end

main(ARGV.dup)

There are a few important things to note, namely:

• The program only responds to get, erase, and store.
• The get operation does not include any leader or trailing whitespace.
• The erase command gracefully handles if the file does not exist.
• The store command upserts the creation of the file if it does not already exist, and overwrites any existing content in the file if it does.

For a more advanced token helper that integrates with Mac OS Keychain Access, check out sethvargo/vault-token-helper-osx-keychain on GitHub.

Token helpers can be written in almost any language, and thus can perform almost any function, including integrating with third-party services and APIs. The next section details how to install this token helper and configure the local Vault CLI to use it.

»Installing and Configuring

With no custom configuration, the local Vault CLI tool will use the default token helper. To configure the local CLI tool, create a configuration file for the Vault CLI at ~/.vault:

# ~/.vault
token_helper = "/full/path/to/token-helper.rb"

Note: In this example, we are configuring the local Vault CLI tool. These options will apply for every vault command run on this system for this user. This is different than the Vault server configuration file, which is given to a server at startup and details the server's configuration options.

The Vault CLI tool automatically searches for ~/.vault for configuration options, parses, and loads them for each invocation of the command. It is possible to specify a different path for this configuration file using the VAULT_CONFIG_PATH environment variable, but this post assumes the default location.

The path to the token helper must be the full path to the binary, and the binary must have executable permissions. The token helper cannot be a relative path, and it cannot be anything other than the path to a binary.

That is all the required configuration! All future vault commands that require a token will automatically use the configured token helper.

»Conclusion

Extending a security tool like Vault may seem intimidating, but hopefully this post has convinced you otherwise. By leveraging Vault's flexible token helpers, we are able to customize the way local tokens are stored and retrieved. For a more advanced token helper that integrates with Mac OS Keychain Access, check out sethvargo/vault-token-helper-osx-keychain on GitHub. We look forward to seeing what custom Vault token helpers you will build!