CDK for Terraform: Enabling Python & TypeScript Support

Developers are accustomed to using their own tooling and writing in familiar languages. For several years, HashiCorp Configuration Language (HCL) has been one of the fastest growing languages at GitHub, and yet we have often heard the desire to provision infrastructure with familiar programming languages. Today, we are pleased to announce the community preview of the Cloud Development Kit for HashiCorp Terraform which allows users to define infrastructure using TypeScript and Python while leveraging the hundreds of providers and thousands of module definitions provided by Terraform and the Terraform ecosystem.

We collaborated with AWS and their Cloud Development Kit (CDK) team to support this new project leveraging two key components of the AWS CDK: a set of language native frameworks for defining infrastructure and adaptors to an underlying provisioning tool. Using these libraries, the AWS CDK generates CloudFormation configuration. Similarly, the CDK for Terraform generates Terraform configuration to enable provisioning with Terraform. The adaptor works with any existing provider and modules hosted in the Terraform Registry. The core Terraform workflow remains the same, with the ability to plan changes before applying.

You can use this project today with our initial release. We've created step-by-step examples with a Docker quick start tutorial, AWS, and other examples in Python and Typescript.

»Terraform as a Platform

The goal of Terraform is to provide a consistent workflow for provisioning infrastructure. Terraform follows an infrastructure as code approach and is extensible to support many providers of cloud infrastructure and software services. The advantages of codification are the ability to automate the creation of infrastructure, use a version control-driven workflow, collaboratively review changes, and use best practices of software development.

With the introduction of CDK support for Terraform, there are more ways than ever to provision and manage infrastructure with Terraform. Historically we have supported HCL and JSON. While HCL was meant to be read and written by people, JSON was meant to be machine-generated and consumed. Earlier this year, we announced experimental support for Kubernetes Custom Resource Definitions (CRDs) using a Terraform Operator. This enables Kubernetes users to configure Terraform resources with a Kubernetes native workflow. The Custom Resource Definition gets translated into Terraform, and executed remotely by Terraform Cloud. Traditional businesses that use ITSM software like ServiceNow can integrate with Terraform Enterprise to support a more traditional ticketing based approach. Finally, with the CDK support, programming languages such as Python and TypeScript can be used today. Additional language support for Javascript, Java, and C# could be used in the future.

This enables Terraform to act as a common platform for infrastructure provisioning and lifecycle management. Today there are hundreds of Terraform providers, spanning public cloud, private cloud, networking and storage hardware, cloud services, and more. Users of Terraform can compose their infrastructure and extend it to support their needs easily. Our goal is to make it as accessible as possible to use Terraform, and we do that by providing the interfaces users find most convenient. There is a broad range of interfaces available today, and we continue to learn from our community and customers about where we can improve.

While Terraform provides the core platform for provisioning, we’ve built Terraform Cloud on top to provide the core platform for collaboration as well. By providing a central place to store state, manage permissions, enable webhooks, and enforce policy we make it easy for teams of tens, hundreds, or thousands to collaborate on infrastructure. By supporting APIs and webhooks, we make it easy to integrate Terraform Cloud with other systems, such as GitHub, GitLab, CircleCI, and more. This allows you to use Terraform Cloud as part of a more complex delivery pipeline, but still maintain visibility and control.

»CDK for Terraform

With the CDK for Terraform project you can define infrastructure resources using the supported programming languages. The tool generates a Terraform configuration in JSON that can be applied with terraform apply or using the CDK for Terraform CLI with cdktf deploy. It supports polyglot languages by using the foundational libraries that the AWS CDK relies upon to generate and synthesize infrastructure configuration.

The CDK for Terraform project includes two packages:

• “cdktf-cli” - A CLI that allows users to run commands to initialize, import, and synthesize CDK for Terraform applications.
• “cdktf” - A library for defining Terraform resources using programming constructs.

»Getting Started

The community preview for CDK for Terraform includes features that allow a user familiar with Terraform to write programming constructs and generate Terraform configuration in JSON files. Use the CDK for Terraform CLI to initialize a project in either Typescript or Python.

The example for getting started uses TypeScript. Begin by installing Node.js, yarn.

Next, install the CDK for Terraform globally.

$ npm install -g cdktf-cli

»Initialize a New Project

Create a directory to store TypeScript files for creating an AWS VPC. Initialize a set of TypeScript templates using cdktf init.

$ mkdir vpc-example
$ cd vpc-example
$ cdktf init --template=typescript

Enter details regarding the project including Terraform Cloud for storing the project state. You can use the --local option to continue without using Terraform Cloud for state management.

We will now setup the project. Please enter the details for your project.
If you want to exit, press ^C.

Project Name: (default: 'vpc-example')
Project Description: (default: 'A simple getting started project for cdktf.')

Several steps occur when running cdktf init. The command:

• Downloads a sample project with the template specified, defined in main.ts.
• Configures Terraform Cloud organization and workspace if specified.
• Runs terraform init to download the appropriate Terraform providers and modules. In this example, it downloads the AWS Provider specified in cdktf.json.
• Generates language-specific objects for resources and modules under the .gen folder based on the provider and module schemas.

After running cdktf init, the example directory contains the TypeScript AWS resources for use.

$ tree
├── .gen
│   └── providers
│       └── aws
│        ├── accessanalyzer-analyzer.ts
│        ├── # omitted for clarity
│        └── xray-sampling-rule.ts
│   └── modules
├── .terraform
├── cdktf.json
├── help
├── main.d.ts
├── main.js
├── main.ts
├── package.json
└── tsconfig.json

In main.ts, import the Vpc object from ./.gen/providers/aws/vpc. The CDK for Terraform generates the Vpc definition in .gen/providers/aws/vpc.ts. Next, define the VPC and its CIDR block. We use Visual Studio Code’s auto-completion to fill in the resource definition.

To create a subnet based on the VPC identifier, pass the vpc.id to a Subnet object. The main.ts demonstrates the creation of the VPC and a subnet based on its identifier. The example uses a Token to cast the VPC ID as a string type and enables the CDK for Terraform to resolve the value of the VPC ID later in synthesis.

import { Construct } from 'constructs';
import { App, TerraformStack, Token } from 'cdktf';
import { Vpc } from './.gen/providers/aws/vpc';
import { Subnet } from './.gen/providers/aws/subnet';
import { AwsProvider } from './.gen/providers/aws'
 
class MyStack extends TerraformStack {
 constructor(scope: Construct, name: string) {
   super(scope, name);
 
   new AwsProvider(this, 'aws', {
     region: 'us-east-1'
   });
 
   const vpc = new Vpc(this, 'my-vpc', {
     cidrBlock: '10.0.0.0/16'
   });
    new Subnet(this, 'my-subnet', {
     vpcId: Token.asString(vpc.id),
     cidrBlock: '10.0.0.0/24'
   });
 }
}
 
const app = new App();
new MyStack(app, 'vpc-example');
app.synth();

»Synthesize TypeScript to Terraform Configuration

Next, synthesize TypeScript to Terraform configuration by running cdktf synth. The command generates Terraform JSON configuration files in the cdktf.out directory.

$ cdktf synth
Generated Terraform code in the output directory: cdktf.out

$ tree cdktf.out
cdktf.out
├── .terraform
└── cdk.tf.json

Inspect the generated Terraform JSON file by examining cdktf.out/cdk.tf.json. It includes the Terraform configuration for the VPC and interpolates the VPC ID into the subnet resource.

{
 "//": {
   "metadata": {
     "version": "0.0.10",
     "stackName": "vpcexample"
   }
 },
 "terraform": {
   "required_providers": {
     "aws": "~> 2.0"
   }
 },
 "provider": {
   "aws": [
     {
       "region": "us-east-1"
     }
   ]
 },
 "resource": {
   "aws_vpc": {
     "vpcexample_myvpc_80A1790F": {
       "cidr_block": "10.0.0.0/16",
       "//": {
           .....
       }
     }
   },
   "aws_subnet": {
     "vpcexample_mysubnet_3769B309": {
       "cidr_block": "10.0.0.0/24",
       "vpc_id": "${aws_vpc.vpcexample_myvpc_80A1790F.id}",
       "//": {
           .....
       }
     }
   }
 }
}

Users can also print Terraform JSON configuration in their terminal using cdktf synth --json command.

After synthesis, users can use the Terraform workflow of initializing, planning, and applying changes within the cdktf.out working directory or use the CDK for Terraform CLI to run cdktf deploy.

Terraform workflow is as follows:

$ cd cdktf.out
$ terraform init
$ terraform plan                             
Refreshing Terraform state in-memory prior to plan..
# omitted for clarity
$ terraform apply
# omitted for clarity
aws_vpc.vpcexamplemyvpc80A1790F: Creating...
aws_vpc.vpcexamplemyvpc80A1790F: Creation complete after 2s [id=vpc-09e8fffce46a61e3d]
aws_subnet.vpcexamplemysubnet3769B309: Creating...
aws_subnet.vpcexamplemysubnet3769B309: Creation complete after 0s [id=subnet-0b5b4c407444472e6]

Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

# destroy resources
$ terraform destroy

CDK for Terraform CLI is as follows:

$ cdktf deploy
Stack: vpcexample
Resources
 + AWS_SUBNET       mysubnet 	aws_subnet.vpcexample_mysubnet_3769B309
 + AWS_VPC              myvpc    	aws_vpc.vpcexample_myvpc_80A1790F

Diff: 2 to create, 0 to update, 0 to delete.
Do you want to continue (Y/n)?

To destroy, from the root directory run cdktf destroy.

$ cdktf destroy

The example demonstrates the synthesis of two AWS resources. To include resources from other providers, such as the Chef Provider, add the provider and its version to cdktf.json.

{
 "language": "typescript",
 "app": "npm run --silent compile && node main.js",
 "terraformProviders": ["aws@~> 2.0", "chef"]
}

To generate the TypeScript objects for the Chef Provider, run cdktf get.

$ cdktf get

In main.ts import the ChefProvider, DataBag, DataBagItem object from ./.gen/providers/chef. Next define the following configuration.

import { Construct } from 'constructs';
import { App, TerraformStack, Token } from 'cdktf';
import { Vpc } from './.gen/providers/aws/vpc';
import { Subnet } from './.gen/providers/aws/subnet';
import { AwsProvider } from './.gen/providers/aws';
import { ChefProvider, DataBag, DataBagItem } from './.gen/providers/chef';
 
class MyStack extends TerraformStack {
 constructor(scope: Construct, name: string) {
   super(scope, name);
 
   new AwsProvider(this, 'aws', {
     region: 'us-east-1'
   });
 
   const vpc = new Vpc(this, 'my-vpc', {
     cidrBlock: '10.0.0.0/16'
   });
   const subnet = new Subnet(this, 'my-subnet', {
     vpcId: Token.asString(vpc.id),
     cidrBlock: '10.0.0.0/24'
   });
  
  new ChefProvider(this, 'chef', {
    serverUrl: 'http://hello.hashicorp.com',
    clientName: 'terraform'
  });
 
  const subnetsDataBag = new DataBag(this, 'subnets', {
    name: 'subnet_cidrs'
  });
 
  new DataBagItem(this, 'cidrs', {
    dataBagName: subnetsDataBag.name,
    contentJson: JSON.stringify({id: vpc.id, subnet_cidr: subnet.cidrBlock})
  });
 }
}
 
const app = new App();
new MyStack(app, 'vpc-example');
app.synth();

The same workflow applies to the synthesis of resources for other Terraform providers and modules within the Terraform Registry. If you would like to use Python to generate Terraform configurations, check out the Getting Started guide for Python.

»Community Collaboration

We would like to thank Sebastian Korfmann, maintainer of the Terrastack project, who helped to deliver the CDK for Terraform project. Talking to Sebastian we came to realize that the Terrastack project had a similar vision for providing support for native programming language for Terraform resources. We are pleased to share that we have Sebastian as a maintainer of the CDK for Terraform project and will continue to contribute to the project going forward.

Additionally, we would like to recognize the AWS CDK team for helping us kick off the project and providing recommendations on patterns and practices for CDK constructs and jsii libraries.

»What’s Next

CDK for Terraform is in alpha and is in the early stages of development. This project is another way to interface with Terraform using languages like TypeScript and Python. We are working to support additional languages such as JavaScript, Java, and C#. In addition to language support, we plan on expanding the scope of CDK for Terraform project to support other first-class commands to provide a user experience similar to the AWS CDK.

Learn more about the CDK for Terraform at hashicorp/terraform-cdk. For more information about AWS CDK and its frameworks, check out its FAQ page. Additional ecosystem projects for CDK include the CDK for Kubernetes.

Try it yourself with our step-by-step examples using a Docker quick start tutorial, AWS, and other examples in Python and Typescript.

Since CDK for Terraform is an experimental project, we would like to ask the community for feedback. If you would like to see a feature for the CDK for Terraform, please review existing GitHub issues and upvote. If a feature does not exist in a GitHub issue, feel free to open a new issue. In addition to opening issues, you can contribute to the project by opening a pull request. Also, use the HashiCorp Discuss form to ask questions and share your feedback.