Bring Your Own Key to the Cloud With HashiCorp Vault

How do you protect data in the cloud, comply with regulatory frameworks, or adhere to security requirements while still maintaining a root of trust with HashiCorp Vault? Azure calls them "BYOK" or bring your own key. AWS calls them customer managed encryption keys. But in essence, with the Key Management Secret Engine, Vault can now become your root of trust outside of the cloud providers’ control. See how it works.

Speakers: Austin Gebauer, Narayan Iyengar

»Transcript

Narayan Iyengar:

Hi there. Welcome to HashiConf Europe. 

My name is Narayan Iyengar. I'm a product manager on the Vault ecosystem team, and along with me is my friend, Austin Gebauer, who's a software engineer on the Vault ecosystem as well. Today I want to talk to you about something that we've all experienced, and that is that our infrastructure is moving. Our infrastructure that we have traditionally kept on-premises, self-managed in a static space, between 4 walls and a firewall, is now moving, and it's moving to the cloud. It is becoming very dynamic. It is spread across multiple clouds, and it is deployed all over the world. 

With that in mind, I also want to bring your attention to the fact that not only is the infrastructure moving, but our data that also used to be behind those 4 walls, in static VMs or static infrastructure, is moving.

And our data is also moving to the cloud. For instance, today we use Gmail or Office 365 for collaboration, so our data is there. A PowerPoint or a Google Slides, our data resides either in Azure or Google. We use Snowflake or MongoDB, for data analytics and data storage. So not only is our infrastructure moving; our data is moving with it as well. 

Which brings us to the topic that I want to chat about today: protecting your data in the cloud using Vault. 

»Securing Access in the Cloud

We know that infrastructure is moving to the cloud. How do you secure access to this infrastructure? You can use Vault today as a centralized secrets management tool, and you can provide fine-grained access control to all of this infrastructure that's already been deployed in various clouds.

If you're not familiar with Vault, Vault is a cloud-agnostic, centralized secrets management and data protection platform. It allows you to bring your own identity to Vault. You can broker your identity, provide fine-grained access controls to various system backends, whether it's AWS, VMware, whether it's Snowflake or MongoDB. Any backend that you want, Vault has an integration built for it. But what about the data? How would you protect your data in the cloud today? Let's imagine that your data is already in the cloud. 

You have operators that now have to build very customized solutions in order to generate and create and distribute encryption keys to various cloud options, whether it's Azure Key Vault, AWS KMS, or other platforms. What does this mean? This means that these are very customized solutions, and your operators now have to learn different APIs, adhere to different API contracts. And this all adds up to a workflow that just seems suboptimal. 

Furthermore, if you're in highly regulated industries and you want to adhere to regulatory frameworks, such as GDPR, or adhere to guidelines provided by the Monetary Authority of Singapore or FINMA, the Swiss banking guidelines, this may not necessarily work for you. Furthermore, you may want to maintain that root of trust of your encryption keys for your data outside of the cloud provider's control. The cloud providers definitely realize that.

So how can Vault help you protect your data in the cloud? 

This slide shows an actual customer workflow. Today you can create custom solutions where you would independently go talk to Vault, generate keys, figure out how to securely export those keys out of Vault and then how to import those keys again securely into various platforms where your data resides.

This solves definitely the problem of maintaining the root of trust outside of your control. But again, not necessarily super-automatable. 

There are a lot of manual steps. Your operators are still having to adhere to different API contracts, whether it's Azure or Google or AWS or other software providers. 

But I believe that there's a better way. 

»The Key Management Secrets Engine

Today in Vault version 1.7, we have the Key Management Secrets Engine generally available for Azure. In the next release of Vault, which will be Vault 1.8, we will have the Key Management Secrets Engine generally available for AWS KMS. So what does the Key Management Secrets Engine do? It allows for operators to interact with Vault, just like any other secrets engine within Vault, and provides destinations that you can distribute these keys to.

Vault then takes care of generating keys and securely distributing them to various cloud providers. All of this is very simple to use, as Austin will demonstrate. 

»BYOK with Vault

This allows you to bring your own key into various cloud providers. Azure calls them "BYOK," or bring your own key. AWS calls them "customer-managed encryption keys." Various cloud providers and various software services call them something different. But in essence, Vault now generally becomes your root of trust outside of the cloud provider's control. 

This allows you fine-grained access control to who can do what with those keys and who can push those keys. Now you have a complete audit trail of who generated these keys, who accessed these keys, and now you have the ability to adhere to certain regulatory frameworks such as GDPR or follow the guidelines set forth by FINMA. 

Now Austin will walk you through the demo of the Key Management Secrets Engine.

»A Key Management Demo

Austin Gebauer:

I'm Austin Gebauer, and I'm a software engineer on the Vault ecosystem team at HashiCorp. I'm excited to demonstrate how Vault can be used to enable some of these key management workflows that Narayan has laid out for us. 

Just a quick overview of what I'll be showing here. On the right of this slide is the Microsoft Azure portal, and in it an existing Key Vault instance.

I'll be using the Key Management Secrets Engine to distribute and manage a key in this instance. And on the left, I'll be using the Vault CLI to exercise the key management workflow. So with that, I'll get started by enabling the secrets engine.

The first thing that I'll do is generate a named cryptographic key called "demo key." You can see that it's an RSA key type of a specific bit length. Next, I'll read the key just to show some more properties of it. You can see that each key may have many versions, and the public key can be obtained for asymmetric key types. We'll use this public key later to perform an encryption operation. 

Next, we will create a KMS, provide a resource in the secrets engine. This resource will represent the named Key Vault instance by way of specifying the Azure Key Vault provider, the name of the Key Vault instance, and by providing credentials that are used to authenticate and manage keys in it. 

I've provided those credentials ahead of time, via environment variables. With that set up, we're ready to distribute a copy of our RSA key that we created earlier to the Azure Key Vault instance. As part of doing so, we're able to define a more specific purpose for the key, in other words, the cryptographic capabilities that the key will have in the external providers. 

Some examples are encryption, decryption, signing, or verification. In this case, we chose decryption. We're also able to define a protection, in other words, where cryptographic operations will take place in the external provider. Some examples are in software or an HSM. In this case, we chose HSM.

We should now see that the key has been securely distributed to the Key Vault instance. Drilling into the key, we can see that the key has a single version, and we can see its properties, for example, the key type and the permitted operations. 

The secrets engine securely imports key material following the bring your own key specifications of supported cloud providers. This diagram shows some technical details for how our key is securely wrapped for transit to the Azure Key Vault instance. The first step, when we want to distribute our target key to the Azure Key Vault instance, is generating a set of wrapping keys that we use to securely wrap the key and import it into the Azure Key Vault instance. 

For the first step, we will generate an AES key, and we will also generate an RSA key in Azure, and we will obtain the public key. 

Those keys will be wrapped, and we'll use the AES key to wrap our target key and use the RSA key to wrap our wrapping key. The output of those wrapping operations will be concatenated and sent to Azure. 

Azure will break those pieces apart, and then will unwrap the wrapping key with the private key and it will eventually unwrap the target key with the wrapping key. Upon success, Vault will delete the key exchange key, and the key is successfully imported into the Azure Key Vault instance. 

When we distribute our key to the Azure Key Vault instance, there are a few steps along the way. We'll generate a set of wrapping keys and a key exchange key, we'll wrap that target key with these keys, we'll send that to Azure. and Azure will take care of unwrapping those keys and securely importing them. 

Finally, Vault will delete our key exchange key, and that successfully imports the key. 

»Performing Cryptographic Operations

Next, I want to show how we can use this key to perform cryptographic operations. I mentioned earlier that we can obtain the public key by reading the key. We're going to use the public key to encrypt some local texts and decrypt it using the private key in Azure. 

Here's the public key, and here's some plaintext we'll encrypt with it. We'll use OpenSSL with the public key to perform the encryption, and this is what the encoded ciphertext looks like as a result. So next, we're going to use the Azure CLI targeting the key that we distributed earlier to decrypt our encoded ciphertext. 

From that, you can see the result, which I'll decode to show that our original plaintext came back out from the decryption operation. 

»Key Rotation

Next, an important part of the key lifecycle is rotation. I'll show what it looks like by rotating the key. The operation rotates the key in both the secrets engine and in the Key Vault instance. Now you can see that we have a new key version that is set as the current version, and it has the same properties as the prior version. 

»Disabling Key Versions

Next, I'll show that we can enable and disable sequences of key versions. This is useful if we want to keep a key version around, but disable it from performing cryptographic operations. 

Our first key version is now disabled. I'll delete this key. This revokes the key from the Key Vault instance. Afterwards, we can see that this key is no longer in the instance. I also want to highlight that after revoking the key from Azure, the key still exists in Vault.

What this means is that a copy of the key was securely distributed to Azure, and we can redistribute the key to the same Key Vault instance or another instance at a later time. This provides additional means for disaster recovery. 

Finally, I want to highlight that this key management workflow is consistent across cloud providers. The general user experience that I demonstrated will be very similar for managing keys in AWS, for example. That's all for the demo, back to you, Narayan.

Narayan Iyengar:

Thanks, Austin, for the demo. 

As you can see, the Key Management Secrets Engine allows you to do not only key creation and distribution, but also manage the lifecycle of the key from Vault. If you're familiar with Vault concepts, the Key Management Secrets Engine is just another secrets engine and allows you to have a very consistent workflow for key management across various cloud providers and other platforms. 

Today, with Vault version 1.7, the Key Management Secrets Engine works with Azure and has been tested against Snowflake and MongoDB Atlas. With Vault 1.8, which will come out in a few months, Key Management Secrets Engine will also support AWS KMS. Other platforms are coming soon. Thank you for listening to this talk, and I hope you have a great HashiConf.