Vault as a Platform for Enterprise Blockchain

Blockchain wallets are used to secure the private keys that serve as the identity and ownership mechanism in blockchain ecosystems: Access to a private key is equivalent to access to cryptocurrency assets. Since most blockchain ecosystems are decentralized by design—Bitcoin and Ethereum being preeminent examples—wallets are typically designed to be local to an individual's device and mainly protected by the access controls to the device. In marked contrast, most enterprises require redundancy and shared access to resources—in order to scale and provide resiliency.

In other words, the enterprise operational model doesn't align with the design of most blockchain wallets. This talk and live demo will show how HashiCorpVault and its plugin architecture provide a framework to build wallets that can bridge this gap and connect the enterprise to blockchain.

Transcript

Hello, everyone! Thanks for coming. Thanks to HashiCorp for having me.

My name is Jeff. I dev the sec and I sec the ops. My company, Immutability LLC, is developing a set of tools and protocols to curate and provision: credentials, access controls, and secrets-orientated infrastructure.

This talk will be an interactive narrative about using Vault plugins to help bring blockchain to the enterprise. I only have about 35 minutes, so there’ll be no intro to blockchain or to Vault. But, if you see me anytime during the conference, please feel free to reach out.

But first, two things: If you are at all like me, you might wanna play along while I’m talking, and to that end, you can go to my GitHub, Immutability-io,and, if you clone the Vault Ethereum plugin, clone this repo, and go into the helper directory, there’s a set of scripts that will install Vault and the plugin exactly the way I’m demoing it here. If you Keybase and Jenkins, you can use these scripts as is. If not, you’ll have to muck about.

The second thing is, take note of my Twitter handle. There will be a quiz during this, and that quiz may involve much wow, or at least, some wow.

Vault as a platform

All right, so what will we talk about? Vault as a platform—what does that mean?

Well, a long time ago in a galaxy far, far away, the web was just a place to share markup files with embedded hyperlinks. However, it took only a few enabling extensions for web servers to become platforms for web applications, and the world changed. Last year, the Vault team introduced the ability for us to craft custom plugins. I believe that this is the enabling extension that elevates Vault from a secrets management tool to a platform for secrets applications.

Blockchain is a secrets-oriented application. Every blockchain transaction requires access to a private key. Now, I’m gonna take it as a given, without any attempt at justification, that we wanna use blockchain in the enterprise. Despite all the hype, every large financial company that I know of has at least one, if not many, blockchain projects in the works. Even JPMorgan, whose CEO has contributed much FUD to this space, built and maintains Quorum, which is an enterprise-orientated version of Ethereum.

While this talk focuses specifically on a Vault plugin for public Ethereum, there’s nothing about the blockchain problems we’re solving for that are specific to public blockchain or to Ethereum.

I mentioned problems. What are the problems?

Well, when an enterprise makes financial transactions, it does so within a regulatory framework that has many controls to prevent the illegitimate transfer of funds. You may have heard of the fin-syn kill chain. If your credit card is stolen, transactions can be rolled back. If wire fraud occurs, you can call the FBI or the Secret Service. And complying with anti-money laundering AML laws is not optional.

In contrast to this, blockchains are immutable. In this world, a loss of your private keys can mean a total loss of all funds. There is no regulatory body or institutional apparatus that can roll back the blockchain, and there are no AML controls. Some would call that a feature. So there are a few problems, and ultimately, the first step in designing anything is to think about the problem that you wanna solve.

Making blockchain safe for the enterprise

Our problem is: How do we construct a set of controls that make it safe-ish to use blockchain in the enterprise? But are we begging the question as regards a problem? Can’t we just run an Ethereum node and use one of the many native wallets? Well, let’s take a look at Ethereum.

It’s your standard-issue, decentralized peer-to-peer network. Every node is the same, the code is the same, the blockchain is the same, with one important exception: Your private keys only live on your local node. So let’s say you’ve spent a few days, weeks, months syncing a full node to the blockchain and you wanna use that node as an enterprise wallet. What do you need to do?

Well, the first thing you’re gonna need to do is open up that node to remote traffic. Well, there is no authentication, there is no TLS. Ethereum is trustless, that means we don’t trust DNS, we don’t trust certificate authorities, so you won’t be sending any passphrases or credentials over the wire. So we have to get onto this node in order to use these private keys. We do that by unlocking them, which makes them available to any subsequent transactions that hit that node, which is unfortunate, because, we just opened up that node to remote unauthenticated traffic.

Now, we also have no audit trail of access to these private keys. There are no controls limiting to whom or how much we can transact. And the passphrase is pretty damn precious too. Loss of a passphrase is equivalent to a loss of keys. In light of these facts, our problem is narrowed. How do we build a multi-user wallet with fine-grain access controls and transaction governance? How do we do this?

Designing a plugin

I contend Vault can help. It does robust multi-user, fine-grain access control with auditability out of the box. But why do we need a plugin? Can’t we just use Vault to store private keys? Well, you’d have to make your keys outside of Vault and put them in, then pull them out of Vault again, in order to sign transactions, generate public keys and addresses. Which contradicts our goal of preventing the exposure of private keys. So I think we need a plugin. And how in the world do we do that?

Well, the first thing to think about when you design a plugin is what kind of operational environments you’re gonna need to support. How are you gonna support the development, testing, and operation of production systems? This discovery process is gonna influence the type of configuration options your plugin needs. So here’s an example:

We have isolated dev, SIT, and prod environments. In our dev environment, we’re pointing to a private Ethereum testnet. In our SIT environment, we’re pointing to the public Rinkeby Testnet. In prod, of course, we’re pointing to the Ethereum mainnet. This may be a familiar deployment model to companies who must comply with regulations governing access to production data; network segmentation may even be involved.

In other scenarios, a single Vault cluster may support multiple environments. This is the model we’re gonna use for our demo. Now, we’re gonna wanna make Vault do what this picture shows. So without further ado, let’s do that.

First thing we’re gonna do is check to see if Vault is running. And indeed, it is. We’re gonna check the Vault status, and we see that Vault is unsealed. Our process status shows that we’re using this as our Vault configuration, so let’s take a look at it. And we see that there is a plugin directory that’s configured, and it points here. And there is our Vault Ethereum plugin sitting fat and happy.

A trust relationship

However, the mere fact that Vault has a reference to this plugins directory, and there’s a plugin there, means nothing. We have to establish a trust relationship between Vault and this plugin. How do we do that?

This is how:

First thing we’re gonna do is we’re gonna write an entry to the Vault plugin catalog. We’re gonna tell Vault to trust the bytes on disc via a trusted checksum. All right. So, we’re basically saying, this is the checksum for these bytes. Then we’re going to provide some TLS material so that we can establish a mutually authenticated TLS connection between Vault and the plugin.

Next, as the picture showed, we’re gonna mount the plugin at two different paths, a prod mount and a dev mount.

And lastly, we’re gonna configure the prod mount to point to the Ethereum mainnet, and we’re gonna configure the dev mount to point to the Rinkeby Testnet.

So let’s do this.

So let me take a peek at the process status, after this. You notice, we have two more Vault processes running. These are the plugins: dev mount and prod mount. These processes are isolated from Vault proper, so if one of them borks, it’s not gonna take down the Vault server. Furthermore, if you’re running on a real operating system, not like this MacBook, you can lock memory, so that process memory is isolated from each other.

When we configured these two mounts, we saw a curious property scroll by. This bound sider list is a typical network-oriented control, where you whitelist a set of hosts that you want to be allowed to reach your Vault Ethereum plugin, in my case, the Vault Ethereum plugin. Now, I added this code to the plugin to do this. This is not just a given in the ecosystem.

Now, let’s play around with this. Let’s log in as the mount administrator. This guy’s only allowed to set configuration options on the mount.

Now, what we’ve done is we’ve said that, any address—it’s basically no address on this network, I hope—is allowed to reach our plugin. And to confirm this, we’re gonna try and reach an endpoint, a resource in our plugin and that would be the convert resource. And sure enough, we see that the loop-back address is unauthorized because it’s not 10.2.3.4.

Now let’s play around a bit. I’m going to FUBAR this Vault session. Now this Vault session should not be permitted to do anything. We’ll demonstrate. And sure enough, we can’t read the configuration at the mount because we’re not authorized.

However, our IP constraints still take hold. Why is this? It’s because the convert endpoint, the resource that we’re addressing, is unauthenticated. Why in the world would we have an unauthenticated endpoint in a Vault secrets plugin?

To prove it’s unauthenticated, let’s go to the prod mount, which we did not put an IP constraint on. Sure enough, it works.

The Ethereum domain

Let’s talk about the domain for a second, the Ethereum domain. Ethereum’s denominated into several units of measure. There’s ETH, which is typically correlated to that hyper-vacillating exchange rate in U.S. dollars or euros or what have you. But every Ethereum transaction is transacted in Wei. And there are this many Wei in an ETH. I don’t trust myself not to drop a decimal point when I’m dealing with that many zeros. I don’t trust myself with a calculator to not drop a decimal point.

Why not have a trusted service that can do that conversion for us? And since the service isn’t revealing any secrets, we don’t need it to be authenticated. This is hinting at the idea of Vault as a platform, which is way to provide assistance to developers so that they can build applications. And this kind of a service helps us build applications that can be safe and easy to use.

Before I go back to the slides, I’m going to unset by FUBARed session, and I’m going to take off that IP constraint, because we’ll use this mount later.

We’ve discussed mounts and configurations in an unauthenticated path, and this brings us to what I think is the most important part of designing a Vault plugin, and that’s how you shape the resource tree that represents the secrets and services that you’re plugin is going to provide.

This will almost certainly be an iterative process. So let’s take a look at what I mean by a resource tree. This model’s our domain. Here you’ll see accounts, which represent the private keys; operations on accounts like debits; signing transactions; transferring ARC20 tokens. You’ll see addresses. You’ll see the convert endpoint. You’ll see export, import, deploying of smart contracts.

These paths will map directly to the code in our plugin. And this is where we make the first decision, which is: Which paths will be protected by Vault policies—these guys here, which are the default—and which paths are special and we can make them unauthenticated? These are paths that we just want to trust that we’re getting a good answer back, but there are no secrets revealed.

These protected paths are protected with Vault policies, and here are a couple of examples of Vault policies. To avoid any possible confusion, I just want to let you know that Vault policies only take effect after an actor is authenticated. They map permissions to a resource for an authenticated actor.

And I talk about our mission, which is to build a set of controls. We usually want fine-grained access controls. And I’ve heard this a couple of times in talks today. But these are very complicated to administer at any kind of scale. These access controls should be reflective of common use cases. If your plugin is poorly designed, meaning the operations on your resource tree don’t map organically to the policies you want to enforce, the demands of accommodating your user base will lead to one of two bad outcomes: Either you will have overly permissive policies, or you’ll have burdensome policy provisioning workflows.

Tweeting for cash

Now, I’m going to contrive a use case and play around, but first, this is our pop quiz. If you remember my address, my Twitter handle, this is a tweet race. In the next couple of moments, I’m going to show you how to create an Ethereum account with a Vault plugin. If you have an Ethereum account from a wallet, or if you are following along, paste an address that you have control of from the mainnet into a tweet, mention me, and add to the tweet #HashiConfRocks.

Like I said, much wow will happen after that.

Separation of duties

Now we’re going to demonstrate separation of duties in the enterprise. We’re going to demonstrate how Vault plugins can help us do this. I’m gonna create a role for an account administrator, and this is somebody that would be governing an Ethereum account, and then I’m going to create some roles for users of an account. And there is no intersection between these two.So we have a set of parallel policies for dev and prod. That admin HTL policy is our account administrator. “Cypherhat” and “Immutability” represent two users of Ethereum accounts.

We’ve just logged in as the account administrator, and what I’m going to do is I’m going to create two accounts, and then I’ll clarify what the remainder of the use case is. We create the Immutability account. We create the Cypherhat account, and this particular user cannot create any other accounts.

Now let’s push the boundaries of our demo here. When I look at this Cypherhat account that I just created, I see some interesting properties. And these are what I’m calling “domain-specific controls.” This is the last layer of control that we’re going to talk about.

We have a blacklist, which is a typical AML-style control that prohibits transacting with a set of addresses. We have a spending limit total, which caps how much spend we have over the lifetime of an account involved. We have a spending limit transactional, which caps spend on a transactional basis. And we have a whitelist, which is a list of our trusted transaction partners.

Notice, however, for the next part of the demo, I’m going to want to try to send some ether, and we don’t have any balance here. And this is where it’s going to get a little curious, because I want to talk to you about how we fund this in this dev account. If this were prod, we’d have to mine ether right now, and I think that would put you all to sleep, because we’d probably have to wait for a couple years, if possible. Or we’d have to go to an exchange and exchange fiat currency, a.k.a. real money, for cryptocurrency and then have it wired back to this address.

But fortunately for us, we’re on Rinkeby and I can do this: I can say, “Make it rain.” I’ll paste that address, and I’m going to say, “Bound HashiConf.” Then I take this tweet and then I go over to what’s called the Rinkeby Faucet, which is a place to get free money.

Now what’s going to happen is that they’re going to throw a captcha up here, and you’re going to have to help me with this, because this screen is really tiny, and I can’t see worth crap. Captchas are the worst.

All right. And sure enough, they gave us some free ether. Of course it’s not real, but it still makes you feel good.

Setting policies

What we want to do is we want to say, “The Cypherhat account holder, or account user, is only allowed to send $20 at a time, USD, $200 total, and only to the Immutability account. And the Immutability account is not allowed to send the ether back to Cypherhat. So, I’ll show you how we do that. This command here says we’re gonna configure the Cypherhat account by setting a spending limit total of $200 USD, denominated in Wei, transaction total $20, denominated in Wei. And only allowed to send to the Immutability account. So let’s go ahead and grab that.

And we have governed that account. Now we’re gonna set a blacklist on the Immutability account. It’s a contrived use case, right? But he can’t send this back.

So now we know the Cypherhat account is funded, so let’s just go ahead and send it back. Uh-oh. We’re the administrator. This is the separation of duties part: We’re not allowed to spend the ETH. We’re only allowed to govern the account. So, we have to log in as the Cypherhat user.

And now we can spend it. Now, if we tried to send it back, of course, we get a 403 because we’re not Immutability, so we’re gonna log in as Immutability and try and send it back.

But of course, we can’t, because he’s blacklisted, for using counterfeit ether probably.

And the winner is...

So we’re approaching the end of the talk. I’m gonna check to see if anybody actually gets any free ether here. So, anybody mention me? Let’s see notifications. I see one. Oh, I see a bunch of them. This is awesome. Let’s pick the first one. Don’t you think that’s fair?

I think that’s fair. There’s Taylor Becker. There’s Root. Root. The Root. I think what we have is a winner! Winner, winner, chicken dinner.

We don’t have any real cash, do we? Or real ether.

I’m gonna take our existing Vault, which is only the demo Vault, but we’re gonna take it offline. And if I had more USB connections, I’d have two separate flash drives, one for unsealed keys, and one for all the Vault data. But I don’t, so just understand that you should never keep your keys and your Vault data in the same place.

So what I’m gonna do is I’m gonna take this existing Vault offline. We’re gonna drop the keys into the Vault data, onto the flash drive. And all this does is kill Vault, drop some files on the flash drive, and then rip them off your hard drive. So it’s a pretty risky operation.

Now, I’m gonna do the opposite. I’m gonna take my corporate Ethereum wallet online. I’m going to authenticate. Because this is real money, so I’ve added an additional control, which is MFA on my wallet. So as soon as I authenticate here, Duo will push to me.

Now we’re gonna say, “Vault, read,” and we’re gonna look at the prod account. And look at that, we have a whopping $400 USD there. So I am going to take Taylor’s address. I’m going to say, “Cat policies/winner.”

So I’m sending to Taylor $20 USD denominated in Wei. And permission is denied. All right, what did I do wrong? I might have screwed up these policies. Let’s try the other one.

This is where the testing matters, right?

Let’s try again, see if we can send from the Immutability account. Yes! Taylor, you are now $20 richer.

Now what I’m gonna do is take this thing offline because I don’t trust any of you.

The importance of Vault to the Blockchain

So we’ve talked about trusted execution, network constraints, fine-grain access controls, and we’ve talked about domain-specific controls. I think we’ve barely scratched the surface. But I think we made the case that Vault is a platform for enterprise blockchain. Now, how important will Vault be to the blockchain? Well, I mentioned that large financial companies are in the process of embracing blockchain. This embrace will bring regulatory operations that govern these institutions.

For example, this popped up yesterday. We’re gonna have AML controls that are gonna be required to be implemented on the blockchain. How are you gonna do that? Well, of course, you’re gonna use my Vault plugin as a prototype.

I think Vault will be the best platform for delivering the controls that enterprises need to adopt blockchain, which is why so many blockchain startups have it baked into their stacks. I’ve talked to several of these startups, and the one common denominator that I’ve seen in their stacks is not Ethereum, it’s not bitcoin, it’s Vault. Pretty damn slick.

Now if you ask a blockchain aficionado—and I believe the term is “futurist”—about what the decentralized web will look like in five years, they’ll tell you that a cryptographically manifest identity will be the focal point of all applications. Which is not too dissimilar from what enterprise experts are saying. Applications are becoming more and more identity-centric, and identity is becoming more aligned with self-administered cryptography. Regardless of which version of the future you subscribe to, I think Vault will be part of it.

Thank you.