Terraform 1.0: We Have Arrived at Our Destination

Get a deep dive into the new features of Terraform 1.0 from a pure open source workflow.

Speaker: Kristin Laemmert

»Transcript

Hi. I'm Kristin Laemmert. I'm an engineer at HashiCorp, and I work on Terraform Core — specifically the open source CLI component. You might remember me from such previous HashiConf talks as, “What is a Terraform even?” “Why isn't Terraform 1.0 out yet?” And, “What is a Terraform Even? Part Two, I'm Still Not Sure.” 

No, these are not actual talk titles, but they are things I say fairly frequently at work. While there's more to me than my cats, my past few talks have shown me that the cats are all that you're likely to remember and even care about. Yes, I am too into my cats. I was raised by cats. Please enjoy pictures of my cats. 

I am absolutely delighted to be here today in celebration of this important milestone. Terraform 1.0 is released. It's been a long and winding path, but we have arrived at our destination, and I'm proud to be here today.

I'd also like to introduce the rest of my immediate team. I get to work with James, Martin, Alisdair, and Pam. Pam has since moved on to another company, but we all worked together on Terraform 1.0. And when it's time to celebrate our work, we enjoy doing so while admiring farm animals over Zoom. I cannot overstate how delightful it is to watch a Zoom full of adults grinning unabashedly at farm animals. If you aren't already, I highly recommend finding something you can do to celebrate your own accomplishments.

»Requirements for 1.0 at HashiCorp

Last year, I gave a talk called The Path to Terraform 1.0, and I spent a lot of time talking about HashiCorp's very high standards for a 1.0 release. One thing HashiCorp is well-known for — dare I say notorious — is that our widely used, production-ready, and critical path tools stay at zero-dot-something releases for years. 

During that last talk, I said that Terraform was close to meeting these criteria: Definitely deployed broadly. We understand the use cases and the UX, but we still had a lot of work to do specifically on that last bullet point — the stable and mature architecture.

Well, we did the work — like a lot of work. So much that we needed to break up our giant change logs into one per release, and each of those releases had a long list of changes. This last one is just the bug fixes in a single release. 

»The Last Year Wasn't Exactly Easy!

Before I spend too much time bragging about all the hard work me and my team did, there's something else I want to talk about — another detour on the road to 1.0. It has been a heck of a year — and some — year and a half for everyone. And while there's nothing new about humans living through difficult situations, that's just life. There are always natural disasters, political unrest, public health crises, heartbreaking injustices. 

I still think most people are going to agree that 2020 was especially anxiety-inducing. I'm bringing this up because I think it's very important that we say aloud I am not up here bragging about Terraform 1.0 because my team doubled down and focused on our productivity and gave 120% every day. Although if that was your coping strategy, that is fine and valid.

I am up here because my team members are kind and patient, and empathetic. We give each other space to feel what we're feeling. We support each other by listening, being there for each other, doing what we can to lift each other up. A coworker sent me this image here. “You can even”. On a particularly rough day and, as silly as you know these memes are, it is exactly what I needed to hear. Please don't let anyone tell you that empathy and kindness aren't among the most important skills to cultivate and look for in others.

»What’s New in Terraform 1.0

We are here to talk about what is new in Terraform 1.0. I got a really good feeling about this next slide. You're going to love it. So, what is new in Terraform 1.0 is.... absolutely nothing. 

Please contain your shock. At this point, you're probably wanting to know either why there aren't any new features in Terraform 1.0 or why they let me speak at the conference when I clearly have nothing to say. Well, let's recap on that previous slide.

This last criteria, which is where I said Terraform needed the most work is a stable and mature architecture. We do a lot of experimentation in Core. So, when we release new features, we depend heavily on user feedback to find out how you're using those features and what changes we might need to make. 

New features frequently change after the initial release and sometimes in ways that aren't backward-compatible. To ensure that we don't do that — we don't have to make any breaking changes to features in 1.0 — we didn't put anything new in 1.0. Don't worry. You're going to see new features starting in 1.1, and I will talk about that in a little bit.

But our primary focus for 1.0 was stability. Really Terraform 0.15 was 1.0 beta, and 0.15 dot six-ish is 1.0. 1.0 is really a promise. We couldn't release 1.0 until we could make these promises. So, barring any critical bugs or security issues, we will not make breaking changes to the configuration language and workflows. You won't be required to run an update tool to go from one dot anything to one dot anything else. Although we may still write an upgrade command for new features we add to help you adopt them. We also promise that the plugin protocols and a bunch of our public interfaces are stable for all of the 1.0 series —1.X.

To make those promises, we needed to complete a bunch of big projects — refactoring, bug fixes, removing deprecated components, and stabilizing the protocol and features so that we can promise that long-term support. I'm going to dig deeper into a couple of these topics on the next slides. 

»Secret Refactoring

We did a lot of refactoring, and most of this is work that's going to be completely invisible to practitioners. We removed a lot of unused code. I'd like to give a shout out to Alex, another HashiCorp engineer, who kicked off that work for us with his early PR removing some 3,000 lines of code. I got to do a very large and very satisfying refactor of Terraform's internals. I'd honestly love to give a talk on this project, but again, it's only interesting to other core developers. 

We started refactoring the CLI command package. We've split out the argument processing and UI rendering into their own packages. This is another thing that I have wanted to see done for the three years I've worked on Terraform — and I loved the work my team did on this. And oh my, did we write a lot of tests. We'd spend a lot of time fixing and extending our test suite. We also needed to completely remove deprecated features before releasing Terraform 1.0. We're stuck with anything we didn't remove in time for 1.0 until 2.0, so it was important we get this right. 

We removed the unmaintained built-in vendor provisioners. Don't worry; local exec, remote exec, and the file provisioners are here to stay, and you can still use third-party provisioners as external plugins for all of 1.0. CLI commands now have a -chdir flag, which is replacing that very inconsistently implemented directory argument. Then some language features like the list and map functions or the older ignore changes syntax have been replaced and updated with more modern functions in syntax.

»Removing Deprecated Components

I hope you notice something on this slide. We're not removing things for the sake of removing things. We're only removing features that have already been communicated as deprecated, first of all, so no surprises. 

And we focused on features where we have either a direct replacement or a more general better way of doing things. We've tried our hardest to do this thoughtfully to make the Terraform experience better for 1.0 and beyond. 

»Features!

We have been adding new features in the last couple of releases. These are all critical features that we needed to get into Terraform 1.0 — specifically, like I said earlier — to give us time to make any breaking changes once we got those features into users' hands. 

We made a lot of changes to the provider installer, and I'm going to go into that in more detail on the next slide. We improved the mechanism for passing providers into modules by adding in explicit configuration aliases field to the required providers, which is inside the Terraform block. This should make things easier for both module authors and module consumers. And it's in keeping with Terraform's main principle that explicit is better than implicit. 

We relaxed the remote state file compatibility. It's no longer limited to only a minor release. What I mean by this is that Terraform is no longer going to complain that it can't possibly read a state file from 1.1 or 0.15, so that's been made a lot more consistent. 

We've added sensitive input variables and extended the capabilities of provider-defined sensitive attributes. This gives both provider developers and Terraform practitioners better control over what values get output in the UI. 

This is the next feature I'm going to go a little deeper into. I'm highlighting these features specifically because they're examples of larger projects that we had to roll out gradually over a series of multiple releases. I don't have time for a real deep dive, but my goal is to give you a look at what's available. And that way, if something here sparks your interest, you can go check out our documentation and the HashiCorp Learn website for more information. 

»Provider Installer Enhancements

The provider installer has changed a lot over the last couple of releases. This started in version 0.13. We've added the concept of provider source. This enables automatic installation of community providers when you run Terraform init. 

Previously, you could automatically download all of HashiCorp's providers. But last time I checked, there are around 1,000 community providers available on the registry. You can now download all of these automatically when you run Terraform init. We've also added a lot of options for configuring provider installation — and a dependency lock file that lets practitioners ensure Terraform will always use the same version of a provider, regardless of what workstation you're running Terraform from. Then there are some new supporting CLI commands to go along with this work. 

»Provider Source

Let's get into some examples. An interesting side effect of the provider source work is that it's valid to have two providers with effectively the same name. In this example configuration, I have two random providers. 

This configuration’s required provider block shows that I'm using both the HashiCorp brand and provider, as well as this custom klaemmity mildwonky random provider. This is a fairly real example. For another project, I had to fork the random provider code. Part of that project was proving that Terraform could still use both versions of random in a single configuration. So this is a neat thing, I think. 

You can see two random pet resources in this snippet, and one of them is using HashiCorp random. That's the default behavior. That top resource that doesn't have any special configuration is going to use the default provider. Then that second resource is using my custom version of random. And you see that would be provider = myrandom. 

I'm going to give a glimpse of the installation options. This snippet is from your Terraform CLI configuration file. In this one, I've defined both a file system mirror and a network mirror. In the file system mirror configuration, the include directive tells Terraform that all providers whose source address is the Terraform registry — and that's the default source if you haven't set anything specifically — should be installed from this local file system mirror I've configured. My custom fork provider is an exception. That is hosted by this made-up network mirror that I've set.

Then, on the days I'm wearing my provider developer hat, I use this dev overrides configuration option. That option tells Terraform to look in the directory — well, in my option — tells Terraform to look in the directory where the provider binary gets compiled. 

A neat detail you can’t see in the file is when I use dev overrides, I don't need to run Terraform init. I can recompile that binary over and over and over again, and Terraform will pick it up automatically. If you're someone who has to recompile your binary every five minutes because you don't know what you're doing, that is quite a time saver. 

»Sensitivity Enhancements

I'm going to switch gears and talk about the sensitivity enhancements. These are really our first steps towards an even more secure Terraform experience. Earlier versions of Terraform had the concept of provider-defined sensitive attributes and sensitive outputs. In Terraform version 0.14, we added user-defined sensitive input variables. In the next several releases, we did a lot of work to incorporate those changes throughout Terraform. Any expression that is derived from a sensitive value will also be marked sensitive and redacted in the UI. Then, like before, we added some new functions that help us support this work. 

The examples I have with sensitivity are very basic. I'm trying to give you a taste of the cool things that are available. Here I have a variable name password, which I have declared as sensitive by adding sensitive=true. There's also a corresponding sensitive output, which prints the value of this password. It's worth noting that Terraform would omit an error if I use that output without writing sensitive = true. That's one of Terraform’s few are you sure you mean to share this value checks.

So with this configuration, I can run plan and apply. In both cases, Terraform redacts that sensitive password from the UI. If I need to see that sensitive value with my own eyeballs, for whatever reason, I can use this nonsensitive function to override the value sensitivity. 

Now when I run Terraform apply, it displays my secret password in the UI. Incredibly basic example, but it’s interesting that Terraform can track those expressions derived from the sensitive values. So, if I took that password and built it into a longer string or passed it to a template, or referred to the output from an entirely different module, Terraform keeps track of that sensitivity and will continue to redact any expression built up from my password. 

»What’s Next? 1.1 and Beyond!

We've got a few big projects in sight for our 1.X releases. Now please keep in mind, this is not a roadmap. I'm not making promises about what's coming when, but I want to show you what we've been talking about inside the team. We talk a lot about Everything is a Plan — where we're rethinking all of Terraform's operations. Not just plan and apply as operations that can be planned and applied. 

We're supporting that idea with some research into representing very specific commands, such as import and state moves in configuration directly — either instead of or as an alternative — to the bespoke command-line operations. 

We're always working on improving Terraform core's integration with both Terraform Cloud and our enterprise offering — doing some lovely work, making sure that there is parity — so that anything you can do in the CLI, you can do in cloud. That's directly related to my Everything is a Plan work and improving the UI output and how things look.

I already introduced the Terraform core team. But we're only one small part of the whole Terraform 1.0 effort at HashiCorp. There have been so many teams and individuals that helped us out here. We've had designers, technical writers, product managers, release engineering – They are the people who work on the registry and on our cloud and enterprise tools. These are all friends and coworkers that contributed and helped us get to Terraform 1.0. I'm grateful to all of my fellow HashiCorp-orials. I could not have done this without all of your combined efforts. Thank you.