Best Practices for Using Terraform with Vault

One of the HashiCorp tools' greatest strengths is their modularity—the way in which all of the tools tightly integrate to produce elegant DevOps workflows that span security, provisioning, networking, and development. Maybe you use HashiCorp Vault for security and secrets management. Maybe you use HashiCorp Terraform for rapid, safe provisioning. But if you start using both, your teams are empowered with even more capabilities for protecting things like database credentials, certs, and cloud API keys in the provisioning workflow.

HashiCorp Vault engineer Becca Petrin shows how powerful the combination of Terraform and Vault can be. Many developers have made the mistake of publishing their cloud keys into a public GitHub repository.

But by using Terraform to spin up and connect with a Vault cluster backed by Consul and running on AWS, Petrin shows how you can easily harness dynamic secrets and never publish any plaintext secrets into configuration files or version control while provisioning infrastructure. And using dynamic secrets means that no long-lived secrets live in your Terraform configuration files.

So if a malicious actor finds your Terraform state file, or you accidentally publish it to GitHub, those credentials are useless in a few hours or minutes if your setup cycles secrets quickly.

Watch this live demo to see how a real-world Terraform + Vault workflow looks.

Outline

0:00 — Agenda and introduction to the HashiCorp suite

4:30 — How to use Terraform to spin up a 3-node Vault cluster backed by Consul and running on AWS

9:42 — Reviewing the recommended, single-datacenter architecture for Vault

14:50 — Unsealing Vault

20:33 — How dynamic secrets work in a Terraform + Vault setup

24:57 — Setting up Vault user roles and configuring Vault to generate dynamic AWS secrets

30:00 — Setting up Terraform to pull secrets from Vault

35:00 — Q&A

Q&A

• Is there a module like the AWS one used in this webinar, but for Azure?
• In order for Terraform to pull things, do you need access to the internet?
• Is the terraform apply step running in a single thread?
• Can we use the Terraform best practices to deploy Vault in a production environment?
• How does the Auto-Unseal feature work?
• For application-specific TF scripts, do you recommend we keep it in a separate repo along with other TF resources or in the application-specific repo?
• Does Terraform handle cases where an operation in AWS takes longer than 5 minutes?
• Is Vault becoming a single point of failure?
• Do the master passwords for the database example expire after 5 mins? For resources that do not allow you to change passwords, how would that work?
• Can the aws/config/root be rotated?
• Is it possible to try Vault and Terraform without access to a cloud provider?
• Can I have 3 Vault servers with each one on a different cloud provider?
• Would you ever expose Vault on the open internet? Or should it only be available on the company's network?
• Can Vault get secrets if we have multiple AWS accounts and use role-based access to switch accounts?
• For secrets that are passed as ENV variables (e.g., the Vault token), are these captured in the Terraform statefile?
• To execute the vault login command, you would first need to ssh into the bastion and then to the Vault instance, correct?
• Does Consul get installed along with Vault to manage the Vault cluster? Is Consul licensed separately?
• Can we see the user who creates/manages the AWS resources in CloudTrail since the AWS keys are generated from Vault?
• What version of Terraform was used in this Demo?

Additional resources

• Vault Reference Architecture
• Best Practices Vault Cluster on AWS
• Production Hardening Guide
• How to Pull AWS Keys from Vault

Slides