Running Vault as a Service on HashiCorp Cloud Platform

Transcript

Rand Fitzpatrick: Welcome to HashiConf Digital 2020, and welcome to the Vault on HashiCorp Cloud Platform presentation. I'm Rand Fitzpatrick on the product management team here at HashiCorp. I'll be joined a little bit later by Thor Hansen on the engineering team to walk you through a bit of HashiCorp Cloud Platform’s Vault offering.

We're all relatively familiar with Vault. We've got a world-class tool for helping you manage that transition from static infrastructure and dynamic infrastructure and provide the identity-based access and security that's necessitated to secure that transition and the changing ephemeral workloads that we demand.

HashiCorp delivers an entire stack of tools to help with these workloads across the cloud. Everything from Terraform, Vault, Consul and Nomad, and these all compose extremely well across the various workflows ­— wherever you might use them in whichever cloud. Vault, however, has been able to offer us a huge degree of flexibility and capability in making sure that you can secure your applications as you scale out and build your environment in a more robust and secure way.

Secrets Management

HashiCorp Vault is the world-leading solution for secrets management, which is its most well-known capability — for managing all of your secrets, credentials, everything from PKI and other secure data that you would want to distribute across your application stack.

Data Encryption

It's also one of the world's leading solutions for data encryption, with transit and transform. All of your encryption workloads, both format-preserving and otherwise enabled by Vault within your application path — and also, broadly, identity-based access to help secure all of your machine-to-machine connections using trusted identities and secure and auditable relationships. Generally extending this, it's the one machine authorization and authentication point to connect all of your systems in a zero trust world through identity-driven control.

Using Vault has always been a huge boon, both on the open source and enterprise sides. But sometimes it can take a little bit of effort to get up and running and get it customized for your specific use case. HCP Vault — or HashiCorp Cloud Platform Vault — is entering private beta today to help get you directly to day zero of realizing value, letting HashiCorp manage the operations and complexity to help you scale this out and get it used in the topologies to suit your application best.

What is HashiCorp Cloud Platform?

I know that we've heard about this earlier today as well as earlier in the year when HCP Consul was first announced. But the true vision is that it takes care of the push-button deployment, infrastructure management, and multi-cloud workflows that are going to help you extend HashiCorp's tools, wherever you are in terms of your cloud deployment journey and scale with you as you grow.

Push-Button Deployment

All we need to do to instantiate a Vault cluster for use at this point is come in to HCP, once we've got an HVN — which is the HashiCorp Virtual Network — just instantiate a cluster. Within 10 minutes — usually faster — we will have spun up a full production-scale Vault cluster, ready for your use.

Fully-Managed Infrastructure

Not only will we have its current state — we'll have its logs, we will have its status. The entire SRE teams for HCP will be monitoring and making sure that it's up to date and keeping it up with patches and version progression, so you're always using the healthiest and most well-secured version of Vault.

Multi-Cloud Workflows

And in terms of supporting those multi-cloud workflows, HCP is going to have this same set of extensible capabilities and well-managed primitives that work wherever you are. Right now, HCP is primarily on AWS, but we're rapidly expanding, and you'll be able to deploy this wherever you need your workloads.

On top of that HCP platform, we're bringing you HCP Vault, which is our best attempt to offer you Vault ready-to-go — ready to securely support your dynamic infrastructure. Before I hand it off to Thor we're going to get into the specifics about how we're bringing this to you in a way that's going to support your infrastructure.

Secure and Scalable Workflows

I mentioned earlier that we've got HVNs. These are the HashiCorp wrapper over an isolated network and compute environment. We support a fully replicated Vault cluster with integrated storage and wrapped up replication within that environment. That environment is owned entirely by the end user — the only things running in there end up belonging under an account for that end user, and it's a very secure space.

The control plane for the HashiCorp Cloud Platform will exercise command actions that can help reconverge and recover — and always make sure that your systems are in a well-running state. We can scale out from here to make sure that as your capacity needs grow we can increase the horizontal and vertical scalability of these clusters to meet your storage needs, your throughput needs, and your processing needs; depending on how your workloads are rated across static secrets, dynamic secrets, and encryption type workloads.

Also, in these environments are secure snapshots and restore capabilities — and audit logs to make sure that as you go through your Vault lifecycle, you have the data that you need to both recover as well as maintain your audibility and data retention requirements.

As you connect this with your infrastructure, we offer a number of secure pathways. Everything from current modality of VPC peering to more direct connections through VPNs and other network topologies. This is to make sure that wherever you use your infrastructure, HCP Vault can be there to support you.

And as I mentioned, we want to focus on getting you to day 0 value. Instead of working to right-size your cluster — working to make sure that your storage and backup workflows are well-executed — you can just instantiate a Vault cluster, export some environment variables, and through the Vault client immediately get up and running and connected to your Vault cluster and start using it.

Focus on Value Over Ops

What does this allow us to do?

Jump Directly into Production

Instead of worrying about all the operations, you can jump directly to your production needs. You can start focusing on what it means to develop against Vault and get it integrated into your environment. It can help you focus on understanding what auth methods, secrets engines, and policies are going to be useful in supporting your application environment — rather than focusing on the operations work upfront.

Support the Vault Lifecycle

Likewise, we'll offer lightweight developer services to help you test with Vault, learn Vault, integrate into applications without having to impact your full-scale production clusters. All of these available dynamically within your HVN on HCP.

Leverage HCP for Reliability

Additionally, you'll be able to rely on HCP for all of that resilience and scale, as well as best of breed hardening to help you keep your systems orchestrated and healthy. You can worry about what your application is meant to do rather than the care and feeding of Vault itself.

Enable the Future

And finally, we help enable the future with secure integration. HCP makes it easier to stay up to date with new versions of Vault — as they go — updating seamlessly behind your application facade, as well as being able to take advantage of additional capabilities that we build onto the platform as HCP grows and develops over time.

To dive into more of the detailed usage of HCP Vault as we're rolling it out today, I'm going to hand it off to Thor Hansen.

Thor Hansen: Hi, I'm Thor Hansen. I'm an engineer here at HashiCorp. I'm going to give you a demo today on Vault on HCP. We're going to create a virtual network. We're going to create a Vault cluster inside that network, then we're going to use an EKS cluster on AWS, and we are going to inject secrets into a pod that we create on that cluster. Let's dive in.

HCP Vault Demo

Here we have HashiCorp's Cloud Platform. We'll be welcomed with an overview page where we can see set up steps as well as other things that we can create on the platform — like Consul. We're going to create a virtual network. We can select a name, a region, a CIDR Block, and then we'll create the network. This will take a couple of minutes to initialize, but once it is, we'll have our HVN, as we call it.

Creating a Vault Cluster

The next step we can do with our HVN is create a Vault cluster inside the network. Let's click that. Then we can name our cluster; we can put it in a region, and then we can click the blue button to simply create the cluster inside our HVN. That'll take a couple of minutes, but then we'll be presented with our Vault cluster.

We have our Vault configuration where we can generate a token, get the address of the cluster. We can see the version, the configuration options, we'll have a getting started with Vault section, where people who are newer to Vault can learn how to use it. We can see the cluster's table and then the assigned network that we created the cluster inside.

Establishing Peering Connections

Next, we'll go back to our HVN and select peering connections. This allows us to peer the HVN with an AWS VPC. We'll copy in our AWS VPC information into this peering connection helper. We'll grab the account ID, the VPC ID, the region that the VPC is in, and then we'll also copy in the VPC CIDR block. Then we'll click create connection. You'll see that this goes from creating to pending acceptance.

That means we have to hop on over to our AWS control panel into the peerings tab, and we can click accept request. We can check to make sure that all of the settings are correct and that this is the connection that we established — and we can click yes accept. Once that's been created, our VPC and the HVN are now peered. We can go back in and see that the connection is now active.

Creating a Route Table

The last step is to go back to our VPC and create a route table. This route table will allow us to map a CIDR Block from the VPC into our HVN. We'll create one of those, give it a name, and then we'll go back into the panel and create a route for it.

Let's go into the routes tab, we'll hit edit routes. Then we can add a new route, which will be the HVN VPC CIDR Block. I'll copy out that CIDR block, use that as our destination. Then as the target we'll use the peering connection that we've created. This should auto-populate by AWS… there it is. We'll click accept. Now we've created a route from our VPC to our HVN and vice versa. This will allow us to access the Vault cluster from our VPC.

Generating an Admin Token

Last thing is to go back into our Vault cluster and generate an admin token. This will generate a token. We copy that out, and this will allow us to access the Vault cluster. We'll go back to our EKS cluster in our VPC, we'll create a Vault namespace, and then we'll create a couple of Kubernetes objects, like a service account, a ClusterRole, and a ClusterRoleBinding. These will allow us to configure the Kubernetes auth backend in Vault to access Vault from Kubernetes. Create those things.

Next, we're going to copy out the access token that's created under the surface account. We'll use this access token when we configure the Kubernetes auth endpoint. You can see we've successfully grabbed the JWT auth token.

Next we're going to create a pod that we can exec into and to run the Vault CLI. If you recall, we had to peer our Vault — our HVN — with this VPC, so that's the only way that we're able to access the Vault cluster. We're going to create a container that we can exec into, and then we'll be able to set up our Vault CLI to access our Vault cluster that's been peered with this VPC.

You'll see here, we'll set up the Vault address. We'll set up the Vault namespace. We're going to import some of the information that we copied out from Kubernetes earlier — like the JWT, as well as the CA certificate from EKS. These things aren't for the Vault CLI, but they’ll help us when we set up our Kubernetes auth backend.

Now we're going to log in using the token that we copied from the control plane webpage to begin with. Now that we're logged in, we can enable the Kubernetes auth backend. Now that's enabled, we need to configure it — and we'll use the JWT and the CA cert that we talked about before. Then we'll create a role that gets assigned when our future app will log in under that auth method. Then we'll enable the secrets engine, and we'll write a little HashiConf secret to that secrets engine that our app can get injected into it later. Lastly, we'll create a role that will allow us to access that secret and Vault.

Setting up Vault Agent Injector

Now we've done all those things, the last thing is to set up our Vault agent’s injector. We’ll use helm to do this —we'll use the HashiCorp helm resource. We'll install via helm, setting our Vault external address to be in our HCP Vault cluster.

Then, last, we have our application. In the annotations of this application, you'll notice that it has directives on how to grab secrets, where to grab them from, and where to store them. Once we create that, we can see that the app is now running, which means a secret has successfully been injected into that pod. Then we can exec into that pod and look to see if our secret has been successfully copied into a file there. We'll cat it out at vault/secrets/kv-secret — and there we can see HashiConf rocks.

Summary and Conclusion

That's been a quick demo on how we can quickly inject secrets into an EKS cluster by simply peering a VPN connection and then treating the Vault on HCP like any other Vault that we had running elsewhere.

This can be really powerful because it doesn't require you — as the end user — to set up Vault, manage it, and deal with scaling it out or scaling it down. Our whole team constantly monitors this. We have long-running workflows that keep it constantly updated — keep constantly going. We're excited for you guys to try this out and to tell us what you think. I'm going to pass it over, back to Rand, so he can wrap up. Thanks.

Rand Fitzpatrick: And as I mentioned, HCP Vault is launching in private beta today. While this is a private beta — and we're making sure that it's ready for broader use extremely soon — we'd love to have those of you who would like to get on board, please reach out. We've got the link there at hashi.co/cloud-platform. Let us know, and we'll love to get you set up. Thank you again.