AWS and HashiCorp's Journey to 1 Billion Terraform AWS Provider Downloads

AWS re:Invent is one of the largest IT conferences in the world, and it’s right around the corner. AWS re:Invent 2022 will be in Las Vegas, November 28 - December 2, and HashiCorp will be featured heavily with breakout sessions, expert talks, and product demos.

Before AWS re:Invent kicks off, we want to reflect on Terraform’s history, our current project collaboration with AWS, and the state of the Terraform AWS provider, which topped one billion downloads this year.

»It Started with Infrastructure as Code

The rise of hardware virtualization in the mid-2000s spawned new opportunities for infrastructure hosting. Cloud hosting providers began offering access to dynamic infrastructure as a service (IaaS) platforms. As these platforms grew and began offering more sophisticated infrastructure services, the complexity of a traditional systems administration role also grew. The need to rapidly configure and manage expanding cloud infrastructures quickly became a challenge.

In 2011, AWS introduced CloudFormation, a service that helps model and set up AWS resources so that you can spend less time managing those resources and more time focusing on your applications that run on AWS. With CloudFormation you could create a template that describes all the AWS resources you want to use (like Amazon EC2 instances or Amazon RDS database instances), and CloudFormation takes care of provisioning and configuring those resources for you. You no longer have to individually create and configure AWS resources and figure out what's dependent on what; CloudFormation handles that for you.

»HashiCorp Terraform 0.1

Over time, though, a need grew for an environment-agnostic solution; a tool that could provide the same workflows, no matter what application you were using. The best, and maybe the only way to do this, was through the power of an open source ecosystem. In 2014, HashiCorp released Terraform 0.1, an open source, cloud-agnostic infrastructure as code solution. Terraform 0.1 supported AWS resources at release. The idea was to start there and extend this automated provisioning workflow to any infrastructure simply by adding a new provider for that specific piece of infrastructure.

»Terraform’s Growth

Terraform was far from an overnight success. Downloads were mostly stagnant for the first 18 months, and at one point shutting down the project was considered. The team at HashiCorp believed that Terraform would succeed primarily based on the ecosystem created around it; that is, the providers it would support. Building this ecosystem required an active and vibrant open source community. So Terraform was built to be easy to write and easy to use, and continued to iterate on the core workflow to ensure it solved real world problems.

By the end of 2016 Terraform had more than 750 contributors. In 2017, HashiCorp launched the first provider program to help partners and communities build, post, and maintain providers. In that same year, the Terraform Registry launched to enable partners and community to compose and share modules. Downloads began to double every month as new modules and providers were added.

»Scaling Terraform

From 2018 to 2020, HashiCorp began to put more focus on commercializing Terraform. This was an important step to ensure the open source project and the company behind it were sustainable. HashiCorp released Terraform Enterprise and Terraform Cloud and began adding large organizations as paying customers. Meanwhile, HashiCorp continued to iterate on Terraform and released the largest, most impactful open source release, Terraform 0.12, in 2018.

»Terraform: Beyond the Basics with AWS

AWS supported Terraform in 2016 by publishing one of the most popular AWS Partner Network (APN) blog posts of all time — Terraform: Beyond the Basics with AWS. The post showed AWS users how to create, update, and version AWS infrastructure using Terraform. It also defined many best practices for working in Terraform and AWS, including:

• How to keep secrets
• How to structure Terraform files
• How to organize modules

As of last year, it still remains in the top 10 all-time APN blog posts:

1. The 5 Pillars of the AWS Well-Architected Framework
2. Terraform: Beyond the Basics with AWS
3. Using Terraform to Manage AWS Programmable Infrastructures
4. Now You Can Take the AWS Certified Cloud Practitioner Exam at Your Home or Office 24/7
5. AWS Lambda Custom Runtime for PHP: A Practical Example

»Terraform Today

With more than one billion downloads, thousands of contributions, and an incredible community, Terraform is the world’s most widely adopted infrastructure as code tool. Today there are more than 2,500 Terraform providers and 10,000 modules. Terraform has achieved this success because of the community around it: the open source contributors, the ambassadors, our users, and our partners.

As the community adopted Terraform for infrastructure as code, many large organizations also realized that Terraform had what they needed for successful infrastructure automation. As they scale, these organizations need a standardized approach to compose, reuse, and collaborate around infrastructure as code to provision infrastructure. Today, there are thousands of Terraform commercial customers. World class organizations are standardizing on Terraform Enterprise to help drive their digital transformations. Terraform is boosting their operation’s productivity, reducing risks, and increasing the velocity of their application teams.

»The State of Terraform and AWS

In 2022, the Terraform AWS provider surpassed one billion downloads — as of publication the count stands at 1.3 billion downloads, half of which occurred in this year:

While one billion downloads represents a major milestone, AWS and HashiCorp continue to develop new integrations to help customers work faster, use more services and features, and provide developer-friendly ways to deploy cloud infrastructure. Our ongoing collaborations include:

»CDK for Terraform

Over the past two years, HashiCorp and the AWS Cloud Development Kit (AWS CDK) team partnered to develop and launch Cloud Development Kit for Terraform (CDKTF), an open source tool that lets you write Terraform configurations in your choice of C#, Python, TypeScript, Java, or Go. This year, the project reached general availability status.

With CDKTF, developers can set up their infrastructure as code without context switching from their familiar programming language, using the same tooling and syntax to provision infrastructure resources as they are using to define the application business logic. Teams can collaborate in familiar syntax, while still leveraging the power of the Terraform ecosystem and deploying their infrastructure configurations via established Terraform deployment pipelines.

»AWS Cloud Control Provider

The AWS Cloud Control (AWSCC) provider for Terraform is a collaboration between AWS and HashiCorp currently in technical preview. This provider is built around the AWS Cloud Control API and is designed to bring new services to Terraform faster. The new provider is automatically generated, which means new features and services on AWS can be supported right away. The AWS Cloud Control provider supports hundreds of AWS resources, with more support being added as AWS service teams adopt the Cloud Control API standard.

For Terraform users managing infrastructure on AWS, this new provider will be used alongside the existing AWS provider, which will continue to be the primary interface for AWS resources. Given the ability to automatically support new features and services, this provider will increase the resource coverage and significantly reduce the time it takes to support new capabilities.

»Learn More About AWS and HashiCorp

AWS and HashiCorp continue to partner, building new integrations to help customers work faster, take advantage of more services and features, and provide developer-friendly ways to deploy cloud infrastructure.

Developers can use the Terraform AWS provider to interact with the many resources supported by AWS. To learn the basics of Terraform using this provider, follow the hands-on tutorials for getting started with Terraform on AWS on our developer education platform. Interact with AWS services, including Lambda, RDS, and IAM by following the AWS services tutorials.

If you’re going to be attending AWS re:Invent, please reach out to your account team to arrange a meeting, and stop by our booth (#3410) to chat with our technical experts, take in a product demo, and learn how companies are accelerating their cloud journey with HashiCorp and AWS.