What are dynamic secrets, and why do I need them?

Transcript

I think this starts is by defining what a secret is. When we talk about a secret, it's really any piece of information that can either authorize or authenticate us.

This could be a username or password. It could be an API token. It could be a TLS certificate. All of these are secrets, and at the end of the day our applications need these to be able to communicate with databases, communicate with cloud providers, consume external services. So these things are sort of a necessary evil. We have to have them and our applications need them to function.

When we talk about a system like Vault, the first-level thing we talk about is how do we move it from a world where these secrets are sprawled everywhere to a place where these secrets are centrally managed in a system like Vault. This is the idea behind secret management—we put them in a secret place, we wrap that with encryption and access control, and fine-grained audit logging so we know who's doing what and only giving access to people and applications that need it.

What is a dynamic secret?

This brings us to an interesting question, which is, "What is a dynamic secret?" In Vault's parlance, a dynamic secret is a credential that we're not statically putting into Vault.

We're not creating a database password and then putting it into Vault, and Vault manages it for a long term. Instead, Vault is creating that database password on demand, just in time.

So when a web server talks to Vault and says, "I need the database password," Vault talks to the database, creates that password on demand, and provides that back to the application.

Why do we need dynamic secrets?

Why is this useful? Why do we actually need this? It turns out, applications do a completely terrible job keeping anything secret.

Even if we perfectly manage all of our credentials and put them in Vault, and distribute it to our application which needs that credential, it turns out the first thing the application will do is log the credential out to disk and say, 'here's my configuration.' And when there's an error or an exception, it'll capture that exception and send that credential out to our external monitoring system. Then if there's a 500 error in rendering a request, usually it'll render a diagnostic page, and it'll put its configuration including our username and password inside that diagnostic page.

It turns out that these applications actually do a very bad job keeping things secret. If what we're doing is generating a static credential, a database using a password, putting it in Vault and managing that for a long period of time, it turns out if you give it to enough applications, over time it's not really a secret anymore. The applications have effectively leaked it out to so many different people and so many different systems that it's not just being centrally managed anymore by Vault. It's now once again sprawled into many different logging and monitoring and alerting systems.

Mitigating secret leaks

The idea behind dynamic secrets is: How do we mitigate the risk of this? The way we do that is by generating these dynamic credentials and then time-bounding them.

When we generate that new dynamic database credential, we give it to the web server, but we automatically expire it after some period of time; 24 hours, 7 days, 30 days, it's configurable. But the goal is, even though our application is now going to leak via a log or send it to a monitoring system, how do we minimize how long that thing is valid for? How do we ensure that even though it's there, that if an attacker gets to it that that credential's already been revoked and is no longer valid. As opposed to if it's a static long-lived secret, an attacker can come to it months, weeks, years later and it's still valid.

So the goal is minimizing our surface area of exposure by focusing on managing intent, saying our web server should be allowed to talk to the database, instead of managing a mechanism such as a username and password that once leaked, there's really no way to revoke it automatically unless we're using a mechanism like dynamic secrets.