Getting Vault Enterprise Installed and Running

WARNING: This guide is from 2018 and its instructions are now OUT OF DATE. We strongly recommend you read through HashiCorp Learn's guide for Vault Enterprise Installation, as well as the Vault Operations track, which begins by outlining a Vault Reference Architecture.

After you sign up for a Vault Enterprise trial, you will receive an email with links to download the Vault Enterprise binary. Vault is distributed as a binary package for all supported platforms and architectures and must first be installed on your machine. This page will not cover how to compile Vault from source, but compiling from source is covered in the documentation for those who want to be sure they're compiling source they trust into the final binary.

Install Vault

To install Vault, find the appropriate package for your system and download it. Vault is packaged as a zip archive.

After downloading Vault, unzip the package. Vault runs as a single binary named vault. Any other files in the package can be safely removed and Vault will still function.

The final step is to make sure that the vault binary is available on the PATH. See this page for instructions on setting the PATH on Linux and Mac. This page contains instructions for setting the PATH on Windows.

Verify installation

After installing Vault, verify the installation worked by opening a new terminal session and checking that the vault binary is available. By executingvault, you should see help output similar to the following:

$ vault
Usage: vault <command> [args]

Common commands:
    read        Read data and retrieves secrets
    write       Write data, configuration, and secrets
    delete      Delete secrets and configuration
    list        List data or secrets
    login       Authenticate locally
    server      Start a Vault server
    status      Print seal and HA status
    unwrap      Unwrap a wrapped secret

Other commands:
    audit          Interact with audit devices
    auth           Interact with auth methods
    lease          Interact with leases
    operator       Perform operator-specific tasks
    path-help      Retrieve API help for paths
    policy         Interact with policies
    secrets        Interact with secrets engines
    ssh            Initiate an SSH session
    token          Interact with tokens

If you get an error that the binary could not be found, then your PATH environment variable was not setup properly. Please go back and ensure that your PATH variable contains the directory where Vault was installed.

Otherwise, Vault is installed and ready to go!

Install command line completion

Vault also includes command-line completion for subcommands, flags, and path arguments where supported. To install command-line completion, you must be using Bash or ZSH or Fish. Unfortunately other shells are not supported at this time.

To install completions, run:

$ vault -autocomplete-install

This will install the helpers in your ~/.bashrc or ~/.zshrc, or to ~/.config/fish/completions/vault.fish for Fish users. Then restart your terminal or reload your shell:

$ exec $SHELL

Now when you type vault <tab>, Vault will suggest options. This is very helpful for beginners and advanced Vault users.

Intro to the Vault server

With Vault installed, the next step is to start a Vault server.

Vault operates as a client/server application. The Vault server is the only piece of the Vault architecture that interacts with the data storage and backends. All operations done via the Vault CLI interact with the server over a TLS connection.

In this page, we'll start and interact with the Vault server to understand how the server is started.

Start the dev server

First, we're going to start a Vault dev server. The dev server is a built-in, pre-configured server that is not very secure but useful for playing with Vault locally. Later in this guide we'll configure and start a real server.

To start the Vault dev server, run:

$ vault server -dev
==> Vault server configuration:

                     Cgo: disabled
         Cluster Address: https://127.0.0.1:8201
              Listener 1: tcp (addr: "127.0.0.1:8200", cluster address: "127.0.0.1:8201", tls: "disabled")
               Log Level: info
                   Mlock: supported: false, enabled: false
        Redirect Address: http://127.0.0.1:8200
                 Storage: inmem
                 Version: Vault v1.2.3
             Version Sha: ...

WARNING! dev mode is enabled! In this mode, Vault runs entirely in-memory
and starts unsealed with a single unseal key. The root token is already
authenticated to the CLI, so you can immediately begin using Vault.

You may need to set the following environment variable:

    $ export VAULT_ADDR='http://127.0.0.1:8200'

The unseal key and initial root token are displayed below in case you want to seal/unseal the Vault or re-authenticate.

Unseal Key: 1aKM7rNnyW+7Jx1XDAXFswgkRVe+78JB28k/bel90jY=
Root Token: root

Development mode should NOT be used in production installations!

==> Vault server started! Log data will stream in below:

# ...

You should see output similar to that above. Vault does not fork, so it will continue to run in the foreground. Open another shell or terminal tab to run the remaining commands.

The dev server stores all its data in-memory (but still encrypted), listens on localhost without TLS, and automatically unseals and shows you the unseal key and root access key. Do not run a dev server in production!

With the dev server running, do the following three things before anything else:

1. Launch a new terminal session.

2. Copy and run the export VAULT_ADDR ... command from the terminal output. This will configure the Vault client to talk to our dev server.

3. Save the unseal key somewhere. Don't worry about how to save this securely. For now, just save it anywhere.

4. Do the same as step 3, but with the root token. We'll use this later.

Verify the server is running

As instructed, copy and execute export VAULT_ADDR='http://127.0.0.1:8200'.

Verify the server is running by running the vault status command. This should succeed and exit with exit code 0.

If it ran successfully, the output should look like the below:

$ vault status
Key             Value
---             -----
Sealed          false
Total Shares    1
Version         (version unknown)
Cluster Name    vault-cluster-81109a1a
Cluster ID      f6e0aa8a-700e-38b8-5dc5-4265c880b2a1
HA Enabled      false

If the output looks different, especially if the numbers are different or the Vault is sealed, then restart the dev server and try again. The only reason these would ever be different is if you're running a dev server from going through this guide previously.

We'll cover what this output means later in the guide.

Intro to secrets

Now that the dev server is up and running, let's get straight to it and read and write our first secret.

One of the core features of Vault is the ability to read and write arbitrary secrets securely. On this page, we'll do this using the CLI, but there is also a complete HTTP API that can be used to programmatically do anything with Vault.

Secrets written to Vault are encrypted and then written to backend storage. For our dev server, backend storage is in-memory, but in production this would more likely be on disk or in Consul. Vault encrypts the value before it is ever handed to the storage driver. The backend storage mechanism never sees the unencrypted value and doesn't have the means necessary to decrypt it without Vault.

Write secret

Let's start by writing a secret. This is done very simply with the vault kv command, as shown below:

$ vault kv put secret/hello foo=world
Success! Data written to: secret/hello

This writes the pair foo=world to the path secret/hello. We'll cover paths in more detail later, but for now it is important that the path is prefixed with secret/, otherwise this example won't work. The secret/ prefix is where arbitrary secrets can be read and written.

You can even write multiple pieces of data, if you want:

$ vault kv put secret/hello foo=world excited=yes
Success! Data written to: secret/hello

vault kv is a very powerful command. In addition to writing data directly from the command-line, it can read values and key pairs from STDIN as well as files. For more information, see the command documentation.

~> Warning: The documentation uses the key=value based entry throughout, but it is more secure to use files if possible. Sending data via the CLI is often logged in shell history. For real secrets, please use files. See the link above about reading in from STDIN for more information.

Read secret

As you might expect, secrets can be read with vault get:

$ vault kv get secret/hello
Key                 Value
---                 -----
refresh_interval    768h
excited             yes
foo                world

As you can see, the values we wrote are given back to us. Vault reads the data from storage and decrypts it.

The output format is purposefully whitespace separated to make it easy to pipe into a tool like awk.

This contains some extra information. Many secrets engines create leases for secrets that allow time-limited access to other systems, and in those cases lease_id would contain a lease identifier and lease_duration would contain the length of time for which the lease is valid, in seconds.

Optional JSON output is very useful for scripts. For example below we use the jq tool to extract the value of the excited secret:

$ vault kv get -format=json secret/hello | jq -r .data.data.excited
yes

When supported, you can also get a field directly:

$ vault kv get -field=excited secret/hello
yes

Delete a secret

Now that we've learned how to read and write a secret, let's go ahead and delete it. We can do this with vault delete:

$ vault kv delete secret/hello
Success! Data deleted (if it existed) at: secret/hello

Intro to the secrets engine

Previously, we saw how to read and write arbitrary secrets to Vault. You may have noticed all requests started with secret/. Try using a different prefix - Vault will return an error:

$ vault write foo/bar a=b
# ...
* no handler for route 'foo/bar'

The path prefix tells Vault which secrets engine to which it should route traffic. When a request comes to Vault, it matches the initial path part using a longest prefix match and then passes the request to the corresponding secrets engine enabled at that path.

By default, Vault enables a secrets engine called kv at the path secret/. The kv secrets engine reads and writes raw data to the backend storage.

Vault supports many other secrets engines besides kv, and this feature makes Vault flexible and unique. For example, the aws secrets engine generates AWS IAM access keys on demand. The database secrets engine generates on-demand, time-limited database credentials. These are just a few examples of the many available secrets engines.

For simplicity and familiarity, Vault presents these secrets engines similar to a filesystem or virtual filesystem. A secrets engine is enabled at a path. Vault itself performs prefix routing on incoming requests and routes the request to the correct secrets engine based on the path at which they were enabled.

The read/write/delete/list operations are forwarded to the corresponding secrets engine, and the secrets engine decides who to react to those operations.

This abstraction is incredibly powerful. It enables Vault to interface directly with physical systems, databases, HSMs, etc. But in addition to these physical systems, Vault can interact with more unique environments like AWS IAM, dynamic SQL user creation, etc. all while using the same read/write interface.

This page discusses secrets engines and the operations they support. This information is important to both operators who will configure Vault and users who will interact with Vault.

Start secrets engine

To get started, enable another instance of the kv secrets engine at a different path. Just like a filesystem, Vault can enable a secrets engine at many different paths. Each path is completely isolated and cannot talk to other paths. For example, a kv secrets engine enabled at foo has no ability to communicate with a kv secrets engine enabled at bar.

$ vault secrets enable -path=kv kv
Success! Enabled the kv secrets engine at: kv/

The path where the secrets engine is enabled defaults to the name of the secrets engine. Thus, the following commands are actually equivalent:

$ vault secrets enable -path=kv kv
$ vault secrets enable kv

To verify our success and get more information about the secrets engine, use the vault secrets list command:

$ vault secrets list
Path          Type         Description
----          ----         -----------
cubbyhole/    cubbyhole    per-token private secret storage
kv/           kv           n/a
secret/       kv           key/value secret storage
sys/          system       system endpoints used for control, policy and debugging

This shows there are 4 enabled secrets engines on this Vault server. You can see the type of the secrets engine, the corresponding path, and an optional description (or "n/a" if none was given).

~> The sys/ path corresponds to the system backend. While the system backend is not specifically discussed in this guide, there is plentiful documentation on the system backend. Many of these operations interact with Vault's core system and is not required for beginners.

Take a few moments to read and write some data to the new kv secrets engine enabled at kv/. Here are a few ideas to get started:

$ vault write kv/my-secret value="s3c(eT"

$ vault write kv/hello target=world

$ vault write kv/airplane type=boeing class=787

$ vault list kv

Disable secrets engine

When a secrets engine is no longer needed, it can be disabled. When a secrets engine is disabled, all secrets are revoked and the corresponding Vault data and configuration is removed. Any requests to route data to the original path would result in an error, but another secrets engine could now be enabled at that path.

If, for some reason, Vault is unable to delete the data or revoke the leases, the disabling operation will fail. If this happens, the secrets engine will remain enabled and available, but the request will return an error.

$ vault secrets disable kv/
Success! Disabled the secrets engine (if it existed) at: kv/

Note that this command takes a PATH to the secrets engine as an argument, not the TYPE of the secrets engine.

In addition to disabling a secrets engine, it is also possible to "move" a secrets engine to a new path. This is still a disruptive command. All configuration data is retained, but any secrets are revoked, since secrets are closely tied to their engine's paths.

What is a secrets engine?

Now that you've successfully enabled and disabled a secrets engine... what is it? What is the point of a secrets engine?

As mentioned above, Vault behaves similarly to a virtual filesystem. The read/write/delete/list operations are forwarded to the corresponding secrets engine, and the secrets engine decides how to react to those operations.

This abstraction is incredibly powerful. It enables Vault to interface directly with physical systems, databases, HSMs, etc. But in addition to these physical systems, Vault can interact with more unique environments like AWS IAM, dynamic SQL user creation, etc. all while using the same read/write interface.

Intro to dynamic secrets

Now that you've experimented with the kv secrets engine, it is time to explore another feature of Vault: dynamic secrets.

Unlike the kv secrets where you had to put data into the store yourself, dynamic secrets are generated when they are accessed. Dynamic secrets do not exist until they are read, so there is no risk of someone stealing them or another client using the same secrets. Because Vault has built-in revocation mechanisms, dynamic secrets can be revoked immediately after use, minimizing the amount of time the secret existed.

Note: Before starting this page, please register for an AWS account. We won't be using any features that cost money, so you shouldn't be charged for anything. However, we are not responsible for any charges you may incur.

Enable the AWS secrets engine

Unlike the kv secrets engine which is enabled by default, the AWS secrets engine must be enabled before use. This step is usually done via configuration management.

$ vault secrets enable -path=aws aws
Success! Enabled the aws secrets engine at: aws/

The AWS secrets engine is now enabled at aws/. As we covered in the previous sections, different secrets engines allow for different behavior. In this case, the AWS secrets engine generates dynamic, on-demand AWS access credentials.

Configure the AWS secrets engine

After enabling the AWS secrets engine, you must configure it to authenticate and communicate with AWS. This requires privileged account credentials. If you are unfamiliar with AWS, use your root account keys.

Do not use your root account keys in production. This is a getting started guide and is not "best practices" for production installations.

$ vault write aws/config/root \
    access_key=AKIAI4SGLQPBX6CSENIQ \
    secret_key=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
Success! Data written to: aws/config/root

These credentials are now stored in this AWS secrets engine. The engine will use these credentials when communicating with AWS in future requests.

Creating a role

The next step is to configure a "role". A "role" in Vault is a human-friendly identifier to an action. Think of it like a symlink.

Vault knows how to create an IAM user via the AWS API, but it does not know what permissions, groups, and policies you want to attach to that user. This is where roles come in - roles map your configuration options to those API calls.

For example, here is an IAM policy that enables all actions on EC2. When Vault generates an access key, it will automatically attach this policy. The generated access key will have full access to EC2 (as dictated by this policy), but not IAM or other AWS services. If you are not familiar with AWS' IAM policies, that is okay - just use this one for now.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt1426528957000",
      "Effect": "Allow",
      "Action": [
        "ec2:*"
      ],
      "Resource": [
        "*"
      ]
    }
  ]
}

As mentioned above, we need to map this policy document to a named role. To do that, write to aws/roles/:name:

$ vault write aws/roles/my-role policy=-<<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt1426528957000",
      "Effect": "Allow",
      "Action": [
        "ec2:*"
      ],
      "Resource": [
        "*"
      ]
    }
  ]
}
EOF
Success! Data written to: aws/roles/my-role

Again, we're using a special path here aws/roles/:name to write an IAM policy to Vault. We just told Vault:

When I ask for a credential for "my-role", create it and attach the IAM policy { "Version": "2012..." }.

Generate secrets

Now that the AWS secrets engine is enabled and configured with a role, we can ask Vault to generate an access key pair for that role by reading from aws/creds/:name where :name corresponds to the name of an existing role:

$ vault read aws/creds/my-role
Key                Value
---                -----
lease_id           aws/creds/my-role/0bce0782-32aa-25ec-f61d-c026ff22106e
lease_duration     768h
lease_renewable    true
access_key         AKIAJELUDIANQGRXCTZQ
secret_key         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
security_token     <nil>

Success! The access and secret key can now be used to perform any EC2 operations within AWS. Notice that these keys are new, they are not the keys you entered earlier. If you were to run the command a second time, you would get a new access key pair. Each time you read from aws/creds/:name, Vault will connect to AWS and generate a new IAM user and key pair.

Take careful note of the lease_id field in the output. This value is used for renewal, revocation, and inspection. Copy this lease_id to your clipboard. Note that the lease_id is the full path, not just the UUID at the end.

Revoke secrets

Vault will automatically revoke this credential after 768 hours (see lease_duration in the output), but perhaps we want to revoke it early. Once the secret is revoked, the access keys are no longer valid.

To revoke the secret, use vault revoke with the lease ID that was outputted from vault read when you ran it:

$ vault lease revoke aws/creds/my-role/0bce0782-32aa-25ec-f61d-c026ff22106
Success! Revoked lease: aws/creds/my-role/0bce0782-32aa-25ec-f61d-c026ff22106e

Done! If you login to your AWS account, you will see that no IAM users exist. If you try to use the access keys that were generated, you will find that they no longer work.

With such easy dynamic creation and revocation, you can hopefully begin to see how easy it is to work with dynamic secrets and ensure they only exist for the duration that they are needed.

Intro to built-in help

You've now worked with vault write and vault read for multiple paths: the kv secrets engine with kv/ and dynamic AWS credentials with the AWS secrets engine provider at aws/. In both cases, the structure and usage of each secrets engines differed, for example the AWS backend has special paths like aws/config.

Instead of having to memorize or reference documentation constantly to determine what paths to use, Vault has a built-in help system. This help system can be accessed via the API or the command-line and generates human-readable help for any path.

Configure built-in help

This section assumes you have the AWS secrets engine enabled at aws/. If you do not, enable it before continuing:

$ vault secrets enable -path=aws aws

With the secrets engine enabled, learn about it with the vault path-help command:

$ vault path-help aws
## DESCRIPTION

The AWS backend dynamically generates AWS access keys for a set of
IAM policies. The AWS access keys have a configurable lease set and
are automatically revoked at the end of the lease.

After mounting this backend, credentials to generate IAM keys must
be configured with the "root" path and policies must be written using
the "roles/" endpoints before any access keys can be generated.

## PATHS

The following paths are supported by this backend. To view help for
any of the paths below, use the help command with any route matching
the path pattern. Note that depending on the policy of your auth token,
you may or may not be able to access certain paths.

    ^config/lease$
        Configure the default lease information for generated credentials.

    ^config/root$
        Configure the root credentials that are used to manage IAM.

    ^creds/(?P<name>\w+)$
        Generate an access key pair for a specific role.

    ^roles/(?P<name>\w+)$
        Read and write IAM policies that access keys can be made for.

The vault path-help command takes a path. By specifying a root path, it will give us the overview of that secrets engine. Notice how the help not only contains a description, but also the exact regular expressions used to match routes for this backend along with a brief description of what the route is for.

Enable path help

After seeing the overview, we can continue to dive deeper by getting help for an individual path. For this, just use vault path-help with a path that would match the regular expression for that path. Note that the path doesn't need to actually work. For example, we'll get the help below for accessing aws/creds/my-non-existent-role, even though we never created the role:

$ vault path-help aws/creds/my-non-existent-role
Request:        creds/my-non-existent-role
Matching Route: ^creds/(?P<name>\w(([\w-.]+)?\w)?)$

Generate an access key pair for a specific role.

## PARAMETERS

    name (string)
        Name of the role

## DESCRIPTION

This path will generate a new, never before used key pair for
accessing AWS. The IAM policy used to back this key pair will be
the "name" parameter. For example, if this backend is mounted at "aws",
then "aws/creds/deploy" would generate access keys for the "deploy" role.

The access keys will have a lease associated with them. The access keys
can be revoked by using the lease ID.

Within a path, we are given the parameters that this path requires. Some parameters come from the route itself. In this case, the name parameter is a named capture from the route regular expression. There is also a description of what that path does.

Go ahead and explore more paths! Enable other secrets engines, traverse their help systems, and learn about what they do.

Intro to authentication

Now that we know how to use the basics of Vault, it is important to understand how to authenticate to Vault itself. Up to this point, we have not "logged in" to Vault. When starting the Vault server in dev mode, it automatically logs you in as the root user with admin permissions. In a non-dev setup, you would have had to authenticate first.

Authentication is the process by which user or machine-supplied information is verified and converted into a Vault token with matching policies attached. The easiest way to think about Vault's authentication is to compare it to a website.

When a user authenticates to a website, they enter their username, password, and maybe 2FA code. That information is verified against external sources (a database most likely), and the website responds with a success or failure. On success, the website also returns a signed cookie that contains a session id which uniquely identifies that user for this session. That cookie and session id are automatically carried by the browser to future requests so the user is authenticated. Can you imagine how terrible it would be to require a user to enter their login credentials on each page?

Vault behaves very similarly, but it is much more flexible and pluggable than a standard website. Vault supports many different authentication mechanisms, but they all funnel into a single "session token", which we call the "Vault token".

Vault has pluggable auth methods, making it easy to authenticate with Vault using whatever form works best for your organization. On this page we will use the token auth method and the GitHub auth method.

Authenticate tokens

Token authentication is enabled by default in Vault and cannot be disabled. When you start a dev server with vault server -dev, it outputs your root token. The root token is the initial access token to configure Vault. It has root privileges, so it can perform any operation within Vault.

You can create more tokens:

$ vault token create
Key                Value
---                -----
token              463763ae-0c3b-ff77-e137-af668941465c
token_accessor     57b6b540-57c8-64c4-e9c6-0b18ab058144
token_duration     ∞
token_renewable    false
token_policies     [root]

By default, this will create a child token of your current token that inherits all the same policies. The "child" concept here is important: tokens always have a parent, and when that parent token is revoked, children can also be revoked all in one operation. This makes it easy when removing access for a user, to remove access for all sub-tokens that user created as well.

After a token is created, you can revoke it:

$ vault token revoke 463763ae-0c3b-ff77-e137-af668941465c
Success! Revoked token (if it existed)

In a previous section, we use the vault lease revoke command. This command is only used for revoking leases. For revoking tokens, use vault token revoke.

To authenticate with a token:

$ vault login a402d075-6d59-6129-1ac7-3718796d4346
Success! You are now authenticated. The token information displayed below
is already stored in the token helper. You do NOT need to run "vault login"
again. Future Vault requests will automatically use this token.

Key                Value
---                -----
token              a402d075-6d59-6129-1ac7-3718796d4346
token_accessor     7636b2f8-0cf1-e110-9b18-8f8b5ecf8351
token_duration     ∞
token_renewable    false
token_policies     [root]

This authenticates with Vault. It will verify your token and let you know what access policies the token is associated with. If you want to test the vault login command, create a new token first.

~> Note: In practice, operators should not use the token create command to generate Vault tokens for users or machines. Instead, those users or machines should authenticate to Vault using any of Vault's configured auth methods such as GitHub, LDAP, AppRole, etc. For legacy applications which cannot generate their own token, operators may need to create a token in advance. Auth methods are discussed in more detail in the next section.

Enable auth methods

Vault supports many auth methods, but then must be enabled before use. Auth methods give you flexibility. Enabling and configuration are typically performed by a Vault operator or security team. As an example of a human-focused auth method, let's authenticate via GitHub.

First, enable the GitHub auth method:

$ vault auth enable -path=github github
Success! Enabled github auth method at: github/

Just like secrets engines, auth methods default to their TYPE as the PATH, so the following commands are equivalent:

$ vault auth enable -path=github github
$ vault auth enable github

Unlike secrets engines which are enabled at the root router, auth methods are always prefixed with auth/ in their path. So the GitHub auth method we just enabled is accessible at auth/github. As another example:

$ vault auth enable -path=my-github github
Success! Enabled github auth method at: my-github/

This would make the GitHub auth method accessible at auth/my-github. You can use vault path-help to learn more about the paths.

Next, configure the GitHub auth method. Each auth method has different configuration options, so please see the documentation for the full details. In this case, the minimal set of configuration is to map teams to policies.

$ vault write auth/github/config organization=hashicorp

With the GitHub auth method enabled, we first have to configure it. For GitHub, we tell it what organization users must be a part of, and map a team to a policy:

$ vault write auth/github/config organization=hashicorp
Success! Data written to: auth/github/config

$ vault write auth/github/map/teams/my-team value=default,my-policy
Success! Data written to: auth/github/map/teams/my-team

The first command configures Vault to pull authentication data from the "hashicorp" organization on GitHub. The next command tells Vault to map any users who are members of the team "my-team" (in the hashicorp organization) to map to the policies "default" and "my-policy". These policies do not have to exist in the system yet - Vault will just produce a warning when we login.

As a user, you may want to find which auth methods are enabled and available:

$ vault auth list
Path       Type      Description
----       ----      -----------
github/    github    n/a
token/     token     token based credentials

The vault auth list command will list all enabled auth methods. To learn more about how to authenticate to a particular auth method via the CLI, use the vault auth help command with the PATH or TYPE of an auth method:

$ vault auth help github
Usage: vault login -method=github [CONFIG K=V...]

  The GitHub auth method allows users to authenticate using a GitHub
  personal access token. Users can generate a personal access token from the
  settings page on their GitHub account.

  Authenticate using a GitHub token:

      $ vault login -method=github token=abcd1234

Configuration:

  mount=<string>
      Path where the GitHub credential method is mounted. This is usually
      provided via the -path flag in the "vault login" command, but it can be
      specified here as well. If specified here, it takes precedence over the
      value for -path. The default value is "github".

  token=<string>
      GitHub personal access token to use for authentication.

Similarly, you can ask for help information about any CLI auth method, even if it is not enabled:

$ vault auth help aws
$ vault auth help userpass
$ vault auth help token

As per the help output, authenticate to GitHub using the vault login command. Enter your GitHub personal access token and Vault will authenticate you.

$ vault login -method=github
GitHub Personal Access Token (will be hidden):
Success! You are now authenticated. The token information displayed below
is already stored in the token helper. You do NOT need to run "vault login"
again. Future Vault requests will automatically use this token.

Key                    Value
---                    -----
token                  7efb3969-8743-880f-e234-afca6e12d790
token_accessor         f7bfb6a3-c41e-eb87-5317-88a0aad200ae
token_duration         768h
token_renewable        true
token_policies         [default my-policy]
token_meta_org         hashicorp
token_meta_username    my-user

Success! As the output indicates, Vault has already saved the resulting token in its token helper, so you do not need to run vault login again. However, this new user we just created does not have many permissions in Vault. To continue, re-authenticate as the root token:

$ vault login <initial-root-token>

You can revoke any logins from an auth method using vault token revoke with the -mode argument. For example:

$ vault token revoke -mode path auth/github

Alternatively, if you want to completely disable the GitHub auth method:

$ vault auth disable github
Success! Disabled the auth method (if it existed) at: github/

This will also revoke any logins for that auth method.

Intro to policies

Policies in Vault control what a user can access. In the last section, we learned about authentication. This section is about authorization.

For authentication Vault has multiple options or methods that can be enabled and used. For authorization and policies Vault always uses the same format. All auth methods map identities back to the core policies that are configured with Vault.

There are some built-in policies that cannot be removed. For example, the root and default policies are required policies and cannot be deleted. The default policy provides a common set of permissions and is included on all tokens by default. The root policy gives a token super admin permissions, similar to a root user on a linux machine.

Write policies

Policies are authored in HCL, but it is JSON compatible. Here is an example policy:

# Normal servers have version 1 of KV mounted by default, so will need these
# paths:
path "secret/*" {
  capabilities = ["create"]
}
path "secret/foo" {
  capabilities = ["read"]
}

# Dev servers have version 2 of KV mounted by default, so will need these
# paths:
path "secret/data/*" {
  capabilities = ["create"]
}
path "secret/data/foo" {
  capabilities = ["read"]
}

With this policy, a user could write any secret to secret/, except to secret/foo, where only read access is allowed. Policies default to deny, so any access to an unspecified path is not allowed.

Do not worry about getting the exact policy format correct. Vault includes a command that will format the policy automatically according to specification. It also reports on any syntax errors.

$ vault policy fmt my-policy.hcl

The policy format uses a prefix matching system on the API path to determine access control. The most specific defined policy is used, either an exact match or the longest-prefix glob match. Since everything in Vault must be accessed via the API, this gives strict control over every aspect of Vault, including enabling secrets engines, enabling auth methods, authenticating, as well as secret access.

Write policies with command line

To write a policy using the command line, specify the path to a policy file to upload.

$ vault policy write my-policy my-policy.hcl
Success! Uploaded policy: my-policy

Here is an example you can copy-paste in the terminal:

$ vault policy write my-policy -<<EOF
# Normal servers have version 1 of KV mounted by default, so will need these
# paths:
path "secret/*" {
  capabilities = ["create"]
}
path "secret/foo" {
  capabilities = ["read"]
}

# Dev servers have version 2 of KV mounted by default, so will need these
# paths:
path "secret/data/*" {
  capabilities = ["create"]
}
path "secret/data/foo" {
  capabilities = ["read"]
}
EOF

To see the list of policies, run:

$ vault policy list
default
my-policy
root

To view the contents of a policy, run:

$ vault policy read my-policy
# Normal servers have version 1 of KV mounted by default, so will need these
# paths:
path "secret/*" {
  capabilities = ["create"]
}
...

Test policies

To use the policy, create a token and assign it to that policy:

$ vault token create -policy=my-policy
Key                Value
---                -----
token              a4ebda12-23bf-5cf4-f80e-803ee2f37aab
token_accessor     aba6256e-401e-9591-31b2-a27048cb15ed
token_duration     768h
token_renewable    true
token_policies     [default my-policy]

$ vault login a4ebda12-23bf-5cf4-f80e-803ee2f37aab
Success! You are now authenticated. The token information displayed below
is already stored in the token helper. You do NOT need to run "vault login"
again. Future Vault requests will automatically use this token.

Key                Value
---                -----
token              a4ebda12-23bf-5cf4-f80e-803ee2f37aab
token_accessor     aba6256e-401e-9591-31b2-a27048cb15ed
token_duration     767h59m18s
token_renewable    true
token_policies     [default my-policy]

Verify that you can write any data to secret/, but only read from secret/foo:

Dev Servers

$ vault kv put secret/bar robot=beepboop
Key              Value
---              -----
created_time     2018-05-22T18:05:42.537496856Z
deletion_time    n/a
destroyed        false
version          1

$ vault kv put secret/foo robot=beepboop
Error writing data to secret/data/foo: Error making API request.

URL: PUT http://127.0.0.1:8200/v1/secret/data/foo
Code: 403. Errors:

* permission denied
»

Non-dev Servers

$ vault kv put secret/bar robot=beepboop
Success! Data written to: secret/bar

$ vault kv put secret/foo robot=beepboop
Error writing data to secret/foo: Error making API request.

URL: PUT http://127.0.0.1:8200/v1/secret/foo
Code: 403. Errors:

* permission denied

You also do not have access to sys according to the policy, so commands like vault policy list or vault secrets list will not work. Re-authenticate as the initial root token to continue:

$ vault login <initial-root-token>

Map policies to auth methods

Vault is the single policy authority, unlike auth where you can enable multiple auth methods. Any enabled auth method must map identities to these core policies.

We use the vault path-help system with your auth method to determine how the mapping is done, since it is specific to each auth method. For example, with GitHub, it is done by team using the map/teams/<team> path:

$ vault write auth/github/map/teams/default value=my-policy
Success! Data written to: auth/github/map/teams/default

For GitHub, the default team is the default policy set that everyone is assigned to no matter what team they're on.

Other auth methods use alternate, but likely similar mechanisms for mapping policies to identity.

Vault Enterprise supports opt-in automatic unsealing via cloud technologies such Amazon KMS or Google Cloud KMS. This feature enables operators to delegate the unsealing process to trusted cloud providers to ease operations in the event of partial failure and to aid in the creation of new or ephemeral clusters.

Automatic unsealing is not enabled by default. To enable automatic unsealing, specify the seal stanza in your Vault configuration file:

seal "awskms" {
  region = "us-east-1"
  access_key = "..."
  secret_key = "..."
  kms_key_id = "..."
}

Notes on the seal stanza

The seal stanza configures the seal type to use for additional data protection, such as using HSM or Cloud KMS solutions to encrypt and decrypt the master key. This stanza is optional, and in the case of the master key, Vault will use the Shamir algorithm to cryptographically split the master key if this is not configured.

The seal can also be used for seal wrapping to add an extra layer of protection and to satisfy compliance and regulatory requirements.

For more examples, please refer to the below: - AWS KMS - GCP Cloud KMS - HSM PKCS11

Seal configuration

Seal configuration can be done through the Vault configuration file using the seal stanza:

seal [NAME] {
  # ...
}

For example:

seal "pkcs11" {
  # ...
}

For configuration options which also read an environment variable, the environment variable will take precedence over values in the configuration file.

Seal and unseal

Every initialized Vault server starts in the sealed state. From the configuration, Vault can access the physical storage, but it can't read any of it because it doesn't know how to decrypt it. The process of teaching Vault how to decrypt the data is known as unsealing the Vault.

Unsealing has to happen every time Vault starts. It can be done via the API and via the command line. To unseal the Vault, you must have the threshold number of unseal keys. In the output above, notice that the "key threshold" is 3. This means that to unseal the Vault, you need 3 of the 5 keys that were generated.

-> Note: Vault does not store any of the unseal key shards. Vault uses an algorithm known as Shamir's Secret Sharing to split the master key into shards. Only with the threshold number of keys can it be reconstructed and your data finally accessed.

Begin unsealing the Vault:

$ vault operator unseal
Unseal Key (will be hidden):
Key                Value
---                -----
Sealed             true
Total Shares       5
Unseal Progress    1/3
Unseal Nonce       786e7190-d1e2-84d2-520c-022efee5b71e
Version            (version unknown)
HA Enabled         true
HA Mode            sealed

After pasting in a valid key and confirming, you'll see that the Vault is still sealed, but progress is made. Vault knows it has 1 key out of 3. Due to the nature of the algorithm, Vault doesn't know if it has the correct key until the threshold is reached.

Also notice that the unseal process is stateful. You can go to another computer, use vault unseal, and as long as it's pointing to the same server, that other computer can continue the unseal process.

This is incredibly important to the design of the unseal process: multiple people with multiple keys are required to unseal the Vault. The Vault can be unsealed from multiple computers and the keys should never be together. A single malicious operator does not have enough keys to be malicious.

Continue with vault unseal to complete unsealing the Vault. To unseal the vault you must use three different keys, the same key repeated will not work. As you use keys, as long as they are correct, you should soon see output like this:

$ vault operator unseal
Unseal Key (will be hidden):
# ...

$ vault operator unseal
Unseal Key (will be hidden):
# ...

When the value for Sealed changes to false, the Vault is unsealed:

Key             Value
---             -----
Sealed          false <--
Total Shares    5
Version         (version unknown)
Cluster Name    vault-cluster-8a8b2c36
Cluster ID      34e94a2e-2d8f-c7cc-271d-96fd438ccc6d
HA Enabled      true
HA Mode         standby
HA Cluster      n/a

Feel free to play around with entering invalid keys, keys in different orders, etc. in order to understand the unseal process.

Finally, authenticate as the initial root token (it was included in the output with the unseal keys):

$ vault login 14d1316e-78f6-910b-a4cc-9ba6697ec814
Success! You are now authenticated. The token information displayed below
is already stored in the token helper. You do NOT need to run "vault login"
again. Future Vault requests will automatically use this token.

Key                Value
---                -----
token              14d1316e-78f6-910b-a4cc-9ba6697ec814
token_accessor     a8bbcc57-9be6-6584-a7a6-46290962fd33
token_duration     ∞
token_renewable    false
token_policies     [root]

As a root user, you can reseal the Vault with vault seal. A single operator is allowed to do this. This lets a single operator lock down the Vault in an emergency without consulting other operators.

When the Vault is sealed again, it clears all of its state (including the encryption key) from memory. The Vault is secure and locked down from access.

Intro to deployment

Up to this point, we have been working with the "dev" server, which automatically authenticated us, setup in-memory storage, etc. Now that you know the basics of Vault, it is important to learn how to deploy Vault into a real environment.

Configure Vault for deployment

Vault is configured using HCL files. The configuration file for Vault is relatively simple:

storage "consul" {
  address = "127.0.0.1:8500"
  path    = "vault/"
}

listener "tcp" {
 address     = "127.0.0.1:8200"
 tls_disable = 1
}

Within the configuration file, there are two primary configurations:

• storage - This is the physical backend that Vault uses for storage. Up to this point the dev server has used "inmem" (in memory), but the example above uses Consul, a much more production-ready backend.

• listener - One or more listeners determine how Vault listens for API requests. The example above listens on localhost port 8200 without TLS. In your environment set VAULT_ADDR=http://127.0.0.1:8200 so the Vault client will connect without TLS.

For now, copy and paste the configuration above to a file called config.hcl. It will configure Vault to expect an instance of Consul running locally.

Starting a local Consul instance takes only a few minutes. Just follow the Consul Getting Started Guide up to the point where you have installed Consul and started it with this command:

$ consul agent -dev

Start server

With the configuration in place, starting the server is simple, as shown below. Modify the -config flag to point to the proper path where you saved the configuration above.

$ vault server -config=config.hcl
==> Vault server configuration:

         Log Level: info
           Storage: consul
        Listener 1: tcp (addr: "127.0.0.1:8200", tls: "disabled")

==> Vault server started! Log data will stream in below:

-> If you get a warning message about mlock not begin supported, that is okay.However, you should run Vault on a system that supports mlock for maximum security.

Vault outputs some information about its configuration, and then blocks. This process should be run using a resource manager such as systemd or upstart.

You'll notice that you can't execute any commands. We don't have any auth information! When you first setup a Vault server, you have to start by initializing it.

On Linux, Vault may fail to start with the following error:

$ vault server -config=example.hcl
Error initializing core: Failed to lock memory: cannot allocate memory

This usually means that the mlock syscall is not available.
Vault uses mlock to prevent memory from being swapped to
disk. This requires root privileges as well as a machine
that supports mlock. Please enable mlock on your system or
disable Vault from using it. To disable Vault from using it,
set the `disable_mlock` configuration option in your configuration
file.

For guidance on dealing with this issue, see the discussion of disable_mlock in Server Configuration.

Initialize

Initialization is the process configuring the Vault. This only happens once when the server is started against a new backend that has never been used with Vault before. When running in HA mode, this happens once per cluster, not per server.

During initialization, the encryption keys are generated, unseal keys are created, and the initial root token is setup. To initialize Vault use vault operator init. This is an unauthenticated request, but it only works on brand new Vaults with no data:

$ vault operator init
Unseal Key 1: E4GnjX+VP9G50uWQNcwpCflzGAMKGR38BbQywgq4I6L8
Unseal Key 2: PYMxcCOswEYMNz7N6UW53Up6nu6y+SjAPwTJOTtkju3d
Unseal Key 3: yuJ5cSxC7tSBR5mMVJ/WJ9bfhhfGb+uwWw9FQR0JKILh
Unseal Key 4: 0vdvEFHM9PHEGMctJrl2ylHqoKQK8DLkfMU6ntmDz6jv
Unseal Key 5: cI8yglWJX+jPf/yQG7Sg6SPWzy0WyrBPvaFTOAYkPJTx

Initial Root Token: 62421926-81b9-b202-86f8-8850176c0cf3

Vault initialized with 5 key shares and a key threshold of 3. Please securely
distribute the key shares printed above. When the Vault is re-sealed,
restarted, or stopped, you must supply at least 3 of these keys to unseal it
before it can start servicing requests.

Vault does not store the generated master key. Without at least 3 key to
reconstruct the master key, Vault will remain permanently sealed!

It is possible to generate new unseal keys, provided you have a quorum of
existing unseal keys shares. See "vault operator rekey" for more information.

Initialization outputs two incredibly important pieces of information: the unseal keys and the initial root token. This is the only time ever that all of this data is known by Vault, and also the only time that the unseal keys should ever be so close together.

For the purpose of this getting started guide, save all of these keys somewhere, and continue. In a real deployment scenario, you would never save these keys together. Instead, you would likely use Vault's PGP and Keybase.io support to encrypt each of these keys with the users' PGP keys. This prevents one single person from having all the unseal keys. Please see the documentation on using PGP, GPG, and Keybase for more information.

Intro to using HTTP APIs

All of Vault's capabilities are accessible via the HTTP API in addition to the CLI. In fact, most calls from the CLI actually invoke the HTTP API. In some cases, Vault features are not available via the CLI and can only be accessed via the HTTP API.

Once you have started the Vault server, you can use curl or any other http client to make API calls. For example, if you started the Vault server in dev mode, you could validate the initialization status like this:

$ curl http://127.0.0.1:8200/v1/sys/init

This will return a JSON response:

{
  "initialized": true
}

Accessing secrets via the REST APIs

Machines that need access to information stored in Vault will most likely access Vault via its REST API. For example, if a machine were using AppRole for authentication, the application would first authenticate to Vault which would return a Vault API token. The application would use that token for future communication with Vault.

For the purpose of this guide, we will use the following configuration which disables TLS and uses a file-based backend. TLS is disabled here only for exemplary purposes; it should never be disabled in production.

backend "file" {
  path = "vault"
}

listener "tcp" {
  tls_disable = 1
}

Save this file on disk as config.hcl and then start the Vault server:

$ vault server -config=config.hcl

At this point, we can use Vault's API for all our interactions. For example, we can initialize Vault like this:

$ curl \
    --request POST \
    --data '{"secret_shares": 1, "secret_threshold": 1}' \
    http://127.0.0.1:8200/v1/sys/init

The response should be JSON and looks something like this:

{
  "keys": [
    "373d500274dd8eb95271cb0f868e4ded27d9afa205d1741d60bb97cd7ce2fe41"
  ],
  "keys_base64": [
    "Nz1QAnTdjrlSccsPho5N7SfZr6IF0XQdYLuXzXzi/kE="
  ],
  "root_token": "6fa4128e-8bd2-fd02-0ea8-a5e020d9b766"
}

This response contains our initial root token. It also includes the unseal key. You can use the unseal key to unseal the Vault and use the root token perform other requests in Vault that require authentication.

To make this guide easy to copy-and-paste, we will be using the environment variable $VAULT_TOKEN to store the root token. You can export this Vault token in your current shell like this:

$ export VAULT_TOKEN=6fa4128e-8bd2-fd02-0ea8-a5e020d9b766

Using the unseal key (not the root token) from above, you can unseal the Vault via the HTTP API:

$ curl \
    --request POST \
    --data '{"key": "Nz1QAnTdjrlSccsPho5N7SfZr6IF0XQdYLuXzXzi/kE="}' \
    http://127.0.0.1:8200/v1/sys/unseal

Note that you should replace Nz1QAnT... with the generated key from your output. This will return a JSON response:

{
  "sealed": false,
  "t": 1,
  "n": 1,
  "progress": 0,
  "nonce": "",
  "version": "1.2.3",
  "cluster_name": "vault-cluster-9d524900",
  "cluster_id": "d69ab1b0-7e9a-2523-0d05-b0bfd09caeea"
}

Now any of the available auth methods can be enabled and configured. For the purposes of this guide lets enable AppRole authentication.

Start by enabling the AppRole authentication.

$ curl \
    --header "X-Vault-Token: $VAULT_TOKEN" \
    --request POST \
    --data '{"type": "approle"}' \
    http://127.0.0.1:8200/v1/sys/auth/approle

Notice that the request to enable the AppRole endpoint needed an authentication token. In this case we are passing the root token generated when we started the Vault server. We could also generate tokens using any other authentication mechanisms, but we will use the root token for simplicity.

Now create an AppRole with desired set of ACL policies. In the following command, it is being specified that the tokens issued under the AppRole my-role, should be associated with dev-policy and the my-policy.

$ curl \
    --header "X-Vault-Token: $VAULT_TOKEN" \
    --request POST \
    --data '{"policies": ["dev-policy", "my-policy"]}' \
    http://127.0.0.1:8200/v1/auth/approle/role/my-role

The AppRole backend, in its default configuration expects two hard to guess credentials, a role ID and a secret ID. This command fetches the role ID of the my-role.

$ curl \
    --header "X-Vault-Token: $VAULT_TOKEN" \
     http://127.0.0.1:8200/v1/auth/approle/role/my-role/role-id

The response will include a data key with the role_id:

{
   "data": {
    "role_id": "86a32a73-1f2b-05e0-113a-dfa930145d72"
  }
}

This command creates a new secret ID under the my-role.

$ curl \
    --header "X-Vault-Token: $VAULT_TOKEN" \
    --request POST \
    http://127.0.0.1:8200/v1/auth/approle/role/my-role/secret-id

The response will include the secret_id in the data key:

{
   "data": {
    "secret_id": "cd4b2002-3e3b-aceb-378d-5caa84dffd14",
    "secret_id_accessor": "6b9b58f6-d11a-c73c-ffa8-04a47d42716b"
  }
}

These two credentials can be supplied to the login endpoint to fetch a new Vault token.

$ curl \
    --request POST \
    --data '{"role_id": "86a32a73-1f2b-05e0-113a-dfa930145d72", "secret_id": "cd4b2002-3e3b-aceb-378d-5caa84dffd14"}' \
    http://127.0.0.1:8200/v1/auth/approle/login

The response will be JSON, under the key auth:

{
  "auth": {
    "client_token": "50617721-dfb5-1916-7b13-4091e169d28c",
    "accessor": "ada8d354-47c0-5d9e-50f9-d74e6de2df9b",
    "policies": [
      "default",
      "dev-policy",
      "my-policy"
    ],
    "metadata": {
      "role_name": "my-role"
    },
    "lease_duration": 2764800,
    "renewable": true
  }
}

The returned client token (50617721...) can be used to authenticate with Vault. This token will be authorized with specific capabilities on all the resources encompassed by the policies default, dev-policy and my-policy.

The newly acquired token can be exported as a new VAULT_TOKEN and use it to authenticate Vault requests.

$ export VAULT_TOKEN="50617721-dfb5-1916-7b13-4091e169d28c"

$ curl \
    --header "X-Vault-Token: $VAULT_TOKEN" \
    --request POST \
    --data '{"bar": "baz"}' \
    http://127.0.0.1:8200/v1/secret/foo

This will create a new secret named "foo" with the given JSON contents. We can read this value back with the same token:

$ curl \
    --header "X-Vault-Token: $VAULT_TOKEN" \
    http://127.0.0.1:8200/v1/secret/foo

This should return a response like this:

{
 "data": {
    "bar": "baz"
  },
  "lease_duration": 2764800,
  "renewable": false,
  "request_id": "5e246671-ec05-6fc8-9f93-4fe4512f34ab"
}

You can see the documentation on the HTTP APIs for more details on other available endpoints.

Congratulations! You now know all the basics to get started with Vault.

That concludes the getting started guide for Vault. Hopefully you're excited about the possibilities of Vault and ready to put this knowledge to use to improve your environment.

We've covered the basics of all the core features of Vault in this guide. Due to the importance of securing secrets, we recommend reading the following as next steps.

• Introduction to HashiCorp Vault with Armon Dadgar
• Getting Started with HashiCorp Vault on Sweetcode.io