Terraform 1.0 Launch Keynote

HashiCorp Co-founders Mitchell Hashimoto and Armon Dadgar open HashiCorp Europe 2021 with the announcement of Terraform 1.0.

Speakers: Dave McJannet, Armon Dadgar, Mitchell Hashimoto

This keynote will take a broad look at the new features, the promises of 1.0, and the history of Terraform. We'll also talk about recent updates to Terraform Cloud and HashiCorp CEO Dave McJannet will take a look at the state of HashiCorp.

»Transcript

Dave McJannet:

I'd like to offer a very warm welcome to our HashiConf Europe digital event. I'm here to make a few opening comments. Then I'm going to hand things over to Mitchell and Armon to share the specifics of what we're going to talk about today. 

Over the next four days, nearly 10,000 cloud engineers will be participating in HashiConf. Although we've organized this to run during European business hours, we have registrants from over 75 countries around the world. Thank you to all of you for joining us. 

As you will see, we designed this event platform to give you opportunities to connect with each other, and the experts in the HashiCorp community, and the HashiCorp employees. In case you miss any of these sessions, just know that they will be available on demand — and ultimately on YouTube on our channel in the coming weeks. 

It's been an unprecedented year for all of us. Not just in Europe but throughout the globe as the pandemic changed — not just where we work — but how we work. If it were not for tools like Slack and Zoom and their ability to scale infinitely as 10x more people started using them, we would have had a very different experience. And it is cloud infrastructure which underpinned all the applications we've used for the last 18 months that has been the silent hero behind it all. And you all — the practitioners — are at the heart of this. You all leaned on HashiCorp to make the infrastructure go in a relatively unseen way. 

For the first time, we saw a hundred million downloads of our open source projects in a single year, as the operators of cloud in the world were extraordinarily busy. But you also contributed to our products. We saw contributions from more than 12,000 different people in the last 12 months, totaling an astounding 280,000 code commits. This ecosystem — this community — that we participate in is truly astounding to watch as the scale of it starts to be seen more broadly.

This also manifested in participation in our user groups, where we had 143 HashiCorp user groups in 52 different countries around the globe, totaling more than the 36,000 community members that collaborate and participate to run infrastructure using the HashiCorp approach. 

And HashiCorp, as a company, has also been investing over that period. We've added well over 400 employees in the last 12 months, and we're totaling 1,300+ employees. Many of whom started like you and our community as contributors to our projects before becoming full-time employees to work on these products full-time. 

I'd be remiss if I did not acknowledge the extraordinary work of the ecosystem partners that have been doing this with us — Amazon, Azure, Google, Alibaba, Oracle, VMware — have all done enormous engineering efforts to make our products work, better together. 

This incorporates the additional features in the Terraform providers. Each time they introduced new projects, this incorporates the new services and backend systems supported in Vault. This incorporates integrations with Consul — and things like ECS and Fargate — that allow that experience for our collective customers to be better at a deeply engineering level. For our bigger customers, we have a heavy reliance on system integrators to do the heavy lifting of bringing this cloud operating model into some traditional companies in the world that are perhaps not as familiar with cloud.

So a huge shout out — and thank you — to all of those participating with us to make our customers successful each and every day. We've seen over the last 12 months — which has been amazing to me — this growing breadth of our tech partner ecosystem. We have Terraform providers for things like Cisco ACI or Palo Alto Networks to allow you to use Terraform to configure networking gear and security gear. We've seen plugins in the Vault ecosystem expand enormously. Things like system backend support for MongoDB, system support for Snowflake, as a core element supported within Vault. This is all only possible because of the work of our tech partners, as we aspire to drive a standard and consistent workflow across any infrastructure element. Thank you to all those in our tech ecosystem.

Probably the most fun thing for me to watch has been observing this cloud operating model for infrastructure  has been developed by the Zooms, Robloxes, Snowflakes, and the cloud-native companies really make its way into the mainstream global 2000. This blueprint for how to run infrastructure using Terraform, how to think about security using identity with Vault, how to think about networking in a consistent way across different platforms in a dynamic service-oriented way using Consul. That is how these cloud-native companies are built. And that blueprint is making its way into the biggest companies on the planet as they adopt cloud. 

Speaking to those large enterprises, we count more than 150 fortune 500 and nearly 300 global 2,000 as customers. You see some of those logos here on the slide because our growth in Europe and around the globe has been amazing. Our customers include in European companies like Barclays in the UK, Ubisoft in France, Lufthansa in Germany, LV in Norway, ABN AMRO, and booking.com in the Netherlands. There are more than 1,700 commercial customers of ours around the world that rely on HashiCorp products — with an ever-growing set also adopting the cloud-managed versions of those products that we offer through Terraform Cloud on the HashiCorp Cloud Platform.

If you haven't taken a look at the HashiCorp Cloud Platform, I strongly encourage you to take a look because we have the ability to run Vault as a service on your behalf. We think that's going to massively increase the surface area of adoption for our products across the globe. A big thank you to everybody that's been part of our ecosystem and community over the last 12 months. We wouldn't be here without all of you — and your commitment to making our products better for everybody. 

I'm going to hand things over to Mitchell. He and Armon have some exciting things to talk about that we'll build on what we've created so far — and help you all take your infrastructure into the future. Please welcome Mitchell, our co-founder and CTO.

Mitchell Hashimoto:

Thanks, Dave, and welcome everyone to HashiConf Europe. Our product vision at HashiCorp has always been based on recognizing a shift from static to dynamic infrastructure. As part of the shift, we're seeing fundamental changes across the way you run infrastructure, do security, do networking, and how you run applications on top of all that infrastructure. 

We call the changes necessary to adapt to the Cloud Operating Model. Today, I want to focus in on the infrastructure layer, which is all about the shifts around provisioning infrastructure. In the dynamic model, there's a lot more scale and differences in how frequently you're provisioning infrastructure. There's a lot more heterogeneity in both what you're provisioning and where you're provisioning — VMs versus containers or cloud versus private data center. Our solution to this infrastructure layer is Terraform. I want to take a step back and talk about the history of Terraform before we talk about where we're going with it.

»The History of Terraform 

Terraform begins in 2011. This is three years before we wrote any lines of code for Terraform. In 2011, Cloud Formation was released by AWS. I wrote a blog post praising it, but I felt that it fell a little short of what I wanted to see. I wanted to see something more generic for infrastructure as code — something that worked with more providers than AWS. I also proposed that the only way to do this efficiently was probably open source. 

Fast forward to 2014, and no one had addressed this challenge. HashiCorp was starting to hit this challenge in the real world. So we decided to solve it ourselves, and we started writing the first version of Terraform. This first version of Terraform was released in the summer of 2014 — and supported only AWS and DigitalOcean. We wanted to showcase the initial functionality that we could support multiple providers and focus on that core feature set out of the gate.

Following the 0.1 release, our focus was on building an open source community, improving the workflow, and adding new infrastructure providers. The years 2014 to 2017 were relatively quiet for Terraform. Terraform wasn't an overnight success, and we weren't seeing a ton of growth in terms of downloads or activity. We focused on making it easier to write providers and adding more providers ourselves. Our follow-on releases quickly added support for new providers, such as Google Cloud, Microsoft Azure, Docker, OpenStack, and more. 

By the end of 2016, beginning of 2017, we had about eight major releases since that 0.1. We had grown slowly and surely to over a thousand total contributors over a dozen providers — and downloads were starting to double every month. We were starting to see real growth about three years in. 

By 2017, things were beginning to take off for Terraform. In that year, three key things came into play. First, we announced the Terraform provider program. This let our partners and community members publish and support their own providers with Terraform. We saw huge growth in the ecosystem of what Terraform could support. The second was that we announced an official partnership with Microsoft and Microsoft Azure. 

This was the first official cloud provider partnership, and this was a huge deal. This was the first time that a major cloud provider was very publicly supporting Terraform. And this was a huge milestone for Terraform's maturity and ecosystem. 

Lastly, we announced the Terraform registry. The Terraform registry is a place where community members could submit modules and create blueprints for reusable infrastructure that made Terraform both reusable for experienced users and more approachable for new users. 

Then from 2018 to 2020, we put a lot of focus on building on this foundation — of making it more stable — and expanding functionality. We released Terraform 0.12, the biggest open source release we've ever done. This laid the foundation for a stable Terraform. We also released Terraform Enterprise and Terraform Cloud which provide workflows for Terraform in organizational settings. This was also the first foray of Terraform into a commercialization setting. 

»Automation, Workflows, and Ecosystem

Terraform has come a long way. It's been amazing to see all the success it's had in the past ten years. Something that's important to note about Terraform is that our vision wasn't just to build infrastructure as code or make infrastructure as code as a concept succeed. We're trying to address three major parts of infrastructure using infrastructure as code. One is automation. The second is workflows. The third is ecosystem. 

»Automation 

When we talk about automation, this is perhaps the most obvious part of Terraform, the idea of using infrastructure as code to provision any infrastructure in the world automatically. The way we measure success is: how much is Terraform used?

We can see that it's used a lot. By our measure, Terraform has been downloaded over a hundred million times. There are over 100,000 monthly HashiCorp Learn users, and there have been over 10,000 certifications for Terraform. 

»Workflow

Next we wanted to focus on workflows. We wanted Terraform to provide a standard workflow that provided composition, collaboration, security, compliance, and management functionality. The way we could look at the success of the workflow is by looking at how much Terraform is being used in an organizational setting. The best place for that is Terraform Cloud. Terraform Cloud today has over 120,000, and we're adding over 5,000 new users per month. All of those users are doing over 30,000 Terraform runs a day. That's a lot of infrastructure being changed per day via Terraform Cloud.

»Ecosystem

And lastly, we wanted to build a large and healthy ecosystem. Ecosystem for us is about Terraform supporting every provider and every resource out there that our community and customers need to have access to. 

For us, we could see the success of the community through how many providers and how many modules there are. There are over a thousand providers in the Terraform registry, and over 5,000 modules. The Terraform ecosystem supports pretty much everything. 

Today, Terraform is seven years old. It's been quite a journey. It had a slow humble beginning, and certainly was not an overnight success. But then we saw it rapidly begin to grow in late 2016 and into 2017. Today it's a very mature product with thousands of members in our community. 

»Terraform 1.0 

With that, we're excited to finally announce Terraform 1.0. Before we jump into what Terraform 1.0 means we'd be remiss not to say thank you. Terraform would not be what it is today without the community.

This includes our users, contributors, HashiCorp ambassadors, and our partners, who all helped us get Terraform to where it is today. Thank you very, very much. 

Before we get into 1.0 — and what you can expect — this has been a long time coming. We wanted to share some of our favorite things our community has been saying prior to us getting here. This sort of tweet shows that, once you start using Terraform, you tend to want to use Terraform for everything. I certainly felt this way before. But folks are generally having a ton of success with Terraform. We constantly see tweets like this, where Terraform is saving people a lot of time and pain, and we love to see it.

It usually works so well that with almost every release we get our question of when is 1.0? We're happy to finally have gotten there. And this tweet— I'm not sure whether to laugh or cry — but Terraform was first, I think. 

»Interoperability, Upgrades, and Maintenance 

I want to take a moment and talk about what Terraform 1.0 means for our community and our customers. 1.0 is a major milestone in delivering infrastructure as code. We're focusing on interoperability, upgrades, and maintenance as the highlight things for 1.0 and beyond. 

»Interoperability 

In terms of interoperability, we've seen since version 0.12, every release after that has focused on laying the foundations that we could promise interoperability between releases. Up to this date, we know we've made breaking changes. We know we've made Terraform incompatible with other versions, and this is something we want to stop.

So as of Terraform 1.0 and the 1.x series, any upgrades in the Terraform 1.x from 1.0, will no longer require rewriting any existing code. All 1.x configs will be backward compatible. 

»Upgrades

For upgrades related to interoperability, all the 0.x upgrades typically had some sort of complex upgrade guide associated with it. But in the 1.0 series, all upgrades will be straightforward and effectively a drop-in. 

»Maintenance 

In terms of maintenance, we've always encouraged organizations to use the latest version of Terraform. We still encourage people to do that today, especially since upgrades are getting so much easier. However, if you feel more comfortable staying on a single release, organizations can confidently and safely stick with 1.0 for up to 18 months — we'll be back boarding fixes this whole time because 1.0 is a stable and primary release for us.

»What’s Next for Terraform? 

That's what Terraform 1.0 means in terms of how you work with Terraform. But because we're at 1.0, it doesn't mean we're done iterating on Terraform. Far from it. We have a lot planned. So, what's next? Where does Terraform go from here? 

»Testing

The first thing we're looking at is testing. Testing is something that our community has asked for — for years. We felt that it was the perfect feature to focus on for the 1.x series. The first phase of what we're thinking has been released in Terraform 0.15 as an experiment. We'd like to ask you to take a look at that and provide feedback on how you think we should iterate on testing from here.

»Terraform CDK 

Next, we're going to look at the Terraform CDK. This is something we released last year as a way to write Terraform in languages other than HCL and JSON. Today, we support TypeScript, Python, Java, C#, and we're looking to continue to add more languages. The Terraform CDK allows Terraform to act as this middle layer between the providers and resources it supports — and the frontend languages that are used to interface with it. With the Terraform CDK, we'd like to continue adding support and developing more features. 

»Open Roadmap and Community Feedback

Finally, we're leaving it open to you. We thought the 1.0 was a perfect opportunity to leave the roadmap relatively open, spend some more time listening to the community, and seeing what the community has to say about where they want Terraform to go. Please go to the community forum, start talking to us, and let us know what you want to see in Terraform next. 

Again, thank you all for joining us on this journey Terraform 1.0 and continuing to build. To continue talking about the workflow enhancements around Terraform Cloud and our ecosystem, I'd like to hand it off to Armon.

Armon Dadgar:

Thanks so much, Mitchell. I'm super excited about Terraform 1.0. Can't wait to hear more about what feedback the community has for us on that. I want to pick up from where Mitchell left off. He spent a lot of time talking about the problems around Terraform that we've spent time trying to solve as well. It's not just the core heart of the infrastructure as code pieces It's some of the workflow challenges around it as well. 

And as he mentioned earlier, our first foray into this was launching Terraform Enterprise at the end of 2017 and starting to gain adoption 2018. Then with Terraform Cloud in early 2019. I think the first step in what we've realized is, in this journey, it's first about adopting the tool — adopting the practices of infrastructure as code. But very quickly you move beyond that into figuring out, well, how do you do this at scale? How do you do this with teams?

»Building at Scale

I think that next challenge is around how you build at scale. And when we talk about building at scale, it's about realizing it's not going to be a single giant Terraform configuration for all of your infrastructure. It's going to be many different teams that own parts of the overall infrastructure — and that they need to be able to compose those pieces together. 

The composability of infrastructure — allowing these small pieces to work together to form a greater whole — is an important part of that workflow. At the same time, teams start to realize there are going to be many reusable patterns. We don't want every team to reinvent the wheel on these common patterns — instead, it's about enabling reuse. There has been a big focus on Terraform modules — the public module registry, the private module registry — allowing you to define these reusable components of infrastructure and share them easily.

Together, these pieces formed the heart of collaboration, allowing teams to each define their own workspace, allowing those workspaces to compose with one another, and allowing you to share and reuse common bits of infrastructure in the form of Terraform modules. 

»Standardization 

Now, as we go a little bit larger than that scale, we start to move beyond the challenges of collaboration, composability, and reuse into thinking about standardization. How do we apply security best practice? How do we apply approaches for our compliance challenges? There, we've made a bunch of investment into policy as code, as a workflow — and into Sentinel as a key technology. The idea is to implement and define these controls — whether they're security controls, compliance controls, operational best practices — define those as code in a Sentinel policy and then implement those through Terraform Enterprise and Terraform Cloud. So that every single Terraform execution that takes place is being validated against. This gives us a high bar of knowing all of the changes we were making — those have already been vetted — and they're in compliance with all of our policies. This is an important piece as we think about the standardization. And this goes above and beyond standardizing on these reusable modules — these reusable components of infrastructure. 

»Innovation and Self-Service 

These pieces together are in service of: how do we get to a true self-service infrastructure? We want to have composability, so each team can define their own infrastructure and run as fast as they want —but compose with everybody else. 

We want reusable blueprints, so everyone doesn't have to redefine the same things over and over. Then we want to layer on top of that, the standardization of policy — this approach to doing policy as code — so that we can make sure while everyone's running fast, they're doing it in compliance. They're doing it within the best practices and security guard rails. And all of that leads to this self-service infrastructure as code that allows you to go quickly and innovate. 

»Curate Modules from Public to Private Registry 

I'm super excited today to talk about two new enhancements that we're announcing. The first of these is the ability to curate modules from the public registry into an organization's private registry. Today, thousands of modules exist in the public registry — some of them have been downloaded tens of millions of times./ But how do we go from thousands of these modules into a handful that makes sense for your organization that you can curate and manage in a more fine-grained way? This allows you to take these modules and import them and share them within your organization’s private registry in more of a curated experience.

»Workspace Interaction

The next big announcement is rethinking how users interact with a workspace. At the core of how Terraform Cloud, Terraform Enterprise organize is the idea that you have a workspace which represents a set of resources. These workspaces might be owned by teams. Teams might own many of them, but it's the unit around which we organize. There have been a number of enhancements we've made. 

First of all, is bringing in a rich readme. You can have much more details around: what does that workspace do? What is its purpose? How does it work? How does it fit in? You can have metrics. There are a lot more highlighted details when you land on the overview page that shows the important bits. And as you can see — as we're going through this video — we can queue a run. And then from there — you can see when we go to the new run page — there's a lot more detail and a lot more visual elements here.

So you can see visually as an apply is happening, what resources are impacted, what percent are being changed versus created versus modified. There's a lot more information. And once that apply happens, and we go back to the home screen, we can see these two new tables — one that shows us the critical outputs of that workspace. As we're talking about composing these workspaces and consuming those outputs, we can easily see what outputs that workspace emits as well as the resources under management. Really making it a lot richer, providing that at-a-glance information as you consume these workflows. We're super excited about both of these changes. They're available today. You can go check them out on app.terraform.io.

»The Terraform Ecosystem

Now, when we talk about the Terraform workflow, we understand that it's not just Terraform in isolation. Mitchell spent a lot of time talking about the ecosystem of providers — and the integrations around Terraform and how there are thousands of those today. 

But it's not just about Terraform and those providers. There's a whole bunch of other tooling like version control systems, CI/CD tooling, CMDBs, etc., that it's important to integrate into the provisioning lifecycle. As we talk about it, Terraform Cloud — Terraform Enterprise — is also designed to enable the extensibility of that whole workflow to bring these different pieces in. That's why there's an API. That's why there are webhook mechanisms. It’s designed to allow these different components to all be integrated. One way we've allowed that type of integration to take place in the past is in-between a plan and apply. 

We were able to apply policy with Sentinel — as we talked about — to do any number of different kinds of checks. As part of a Sentinel check, we could do a webhook out to an external system to query information from it. We can use the plugin SDK to write custom Sentinel logic. This gave us some amount of extensibility, but it might still represent having to write Sentinel policy or write custom plugins to do it. Well, we might want a more seamless way of integrating. 

»Terraform Run Checks

Today I'm excited to show the preview of a new capability that we're calling Terraform run checks. This is designed to make third-party integrations with the system much simpler. We can define a run check. And we can see that within the context of doing a run — in between that plan and apply — there are now multiple of these different run checks executing. 

This is going to make a call out to an external system that is going to apply some other logic. That logic could be around security policy, costing, change management — anything you want it to be. 

If that policy fails, we'll get a link to click through and see more details. In this case, we're going to click through to Bridgecrew. And in the Bridgecrew system, we're going to be able to see why my policy fails. There are these lines of Terraform code that triggered us to fail this apply. The user is able to go back, make a change in their code, commit that to GitHub and say, great — I fixed these issues that were in policy, the new Terraform run can now apply.

We can see that policy has passed. We're able to apply our change successfully and everything is passing. This is starting to showcase that ability to extend Terraform — take that extensibility to the next level where it's not just the extensibility around Terraform providers, but the extensibility of Terraform Cloud and the overall provisioning workflow around it. Here we can see the apply is successfully completed and we can go back and see which policy checks were involved with that. 

We're super excited to announce this new capability. It's in beta, and it's going to be available more broadly later in the summer, we're working with a number of partners. You saw the integration with Bridgecrew. We're working with folks like CheckPoint, Snyk, McAfee, Infracost, Accurics, and others. We're super excited to have people participate in this, give us feedback. If you're interested in learning more, please click through and sign up to participate as part of the beta. 

»Keynote Summary 

We've announced a lot today. I wanted to quickly summarize. Of course, the big tent pole today was Terraform 1.0. I know we're super excited about it. Our community is super excited about it. And we can't wait to hear more about what you all have to say. The other big set of updates has been around Terraform Cloud. We announced a number of new functions and capabilities. I'm excited personally for the Terraform run checks work. Those will be available later on this summer.

For folks that don't yet have a Terraform Cloud account. It's free to sign up. I recommend going to app.terraform.io, and you can register and get started with all of this today. Right after this, we have a number of other sessions coming up. There'll be an infrastructure keynote. That's going to go deeper into network infrastructure automation and how to apply some of this to the networking world. We're also going to have a session around Terraform 1.0 and Terraform Cloud and the updates there. And then we'll have a deep dive into all things 1.0. Lots more content. Stick around. I hope you all enjoy the rest of HashiConf.