Products & Technology

CDK for Terraform 0.13 Significantly Improves Performance

Cloud Development Kit for Terraform (CDKTF) 0.13 vastly reduces synth time across all supported languages.

Oct 10 2022Nara Kasbergen Kwon

Back in August, we were excited to announce the first generally available (GA) release of Cloud Development Kit for Terraform, with version 0.12. With CDKTF, you can write Terraform configurations in your choice of TypeScript, Python, C#, Java, or Go, and still benefit from the full ecosystem of HashiCorp Terraform providers and modules. Among the changes in CDKTF release 0.12, we announced that Go is now fully supported, after having been classified as experimental previously.

For CDKTF release 0.13 we added namespaces in order to make performance improvements across all supported languages. In this post we’ll explain why we made this change and how the upgrade process will work. We’ll also preview some of the improvements we’re planning for CDKTF 0.14.

»Performance Gains in CDKTF 0.13

After the first GA release, we eagerly took in feedback from the community, and one common theme predominated: performance. Go users in particular reported long build times and frequent out-of-memory errors. Synthesizing your infrastructure in Java and C# was significantly slower than our TypeScript benchmarks. And Python users reported a related issue where the generated classes for some providers were so big that they were crashing IDEs.

While our metrics indicate that CDKTF is working well for the majority of users, the team prioritized addressing performance issues because we recognize that speed and stability are a core part of the user experience, and one of our North Star goals is a first-class developer experience across all five languages that we support.

We experimented with different ways to increase performance in a language-agnostic way, culminating in our 0.13 release. With this update, we're excited to announce significant improvements across the board, including (based on benchmark tests):

a 96.8% reduction in cdktf synth time when using Go with the Azure provider
an 83% reduction in cdktf synth time when using Java with the Google Cloud provider
a 36.8% reduction in cdktf synth time when using C# with the AWS provider
a 61.5% reduction in cdktf synth time when using TypeScript with the Kubernetes provider

In addition, the same changes that led to these performance enhancements also fixed the issue that was causing IDEs to crash for Python users, so we truly did see an improved developer experience across the board.

The downside is that in order to achieve these incredible performance gains, we must introduce a breaking change in CDKTF that will affect all existing users.

»Breaking Change: Namespaces

With CDKTF 0.13, we are introducing namespaces in every class in our generated provider bindings, with namespaces automatically derived from the Terraform resource or data source it originates from.

Currently, each provider’s exports are a flat list of constructs (resources, data sources, etc.). While that’s really convenient at times, it also results in large packages that compilers and language server protocols find taxing to handle. Not only that, but generating providers also takes longer for CDKTF. With the move to namespacing, the providers are divided into logical groups based on the kinds of constructs being exported. These numerous small packages are faster for the compiler to process than one large package, resulting in the significant speedups highlighted above.

»What You Need To Do

The introduction of namespaces means that the import paths for all of your provider-related classes will need to be updated. As an example, instead of:

import { AwsProvider, ec2 } from "@cdktf/provider-aws";
 
class MyStack extends TerraformStack {
  constructor(scope: Construct, id: string) {
    super(scope, id);
 
    new AwsProvider(this, "AWS", {
      region: "us-west-1",
    });
 
    const instance = new ec2.Instance(this, "compute", {
      ami: "ami-01456a894f71116f2",
      instanceType: "t2.micro",
    });
  }
}

In CDKTF 0.13, you would write:

import { AwsProvider } from "@cdktf/provider-aws/lib/provider";
import { Instance } from "@cdktf/provider-aws/lib/instance";
 
class MyStack extends TerraformStack {
  constructor(scope: Construct, id: string) {
    super(scope, id);
 
    new AwsProvider(this, "AWS", {
      region: "us-west-1",
    });
 
    const instance = new Instance(this, "compute", {
      ami: "ami-01456a894f71116f2",
      instanceType: "t2.micro",
    });
  }
}

For more upgrade information, check out our guide to Upgrading to CDK for Terraform Version 0.13.

»When Do You Need To Update?

If you're currently using CDKTF 0.12 and it's working well for you, you don't have to update to 0.13 right away. The only change introduced in 0.13 was the performance enhancement, so you're not missing out on any critical features.

If you'd like to take advantage of the performance improvements in 0.13 but aren't ready to change the import paths for all of your existing providers, we have good news: we've made the 0.13 release backwards-compatible with 0.12 provider bindings. This allows you to have two versions of the same provider in your code (one locally generated and one pre-built) and to mix namespaced and non-namespaced providers as you like. Our goal here is to reduce the disruption of this breaking change and give users more time to gradually migrate their code to the new syntax.

Once you're ready to fully embrace our newly namespaced world in CDKTF 0.13, then you can go ahead and update your code as described in the "What You Need To Do" section above.

No matter what, you will need to update your existing code to the new namespaced syntax in order to upgrade to CDKTF 0.14. We are only maintaining the backwards-compatible provider bindings for this one release, meaning the 0.12 (non-namespaced) provider bindings will no longer work with CDKTF 0.14 and above. We are currently targeting mid-November 2022 for the release of 0.14.

»What's Next for CDKTF

The CDKTF 0.14 release will focus primarily on quality-of-life improvements that make it easier to use pre-built providers. One key discovery we made as we researched ways to improve performance was that pre-built providers give a vastly better developer experience than locally generating them using cdktf get, so we want to encourage users to take advantage of these as much as possible.

In the coming months, we'll be looking to make more pre-built providers available as installable packages. Until then we'll be making improvements to the cdktf provider add command introduced in CDKTF 0.11 and the cdktf init workflow to make it as easy as possible to find and add pre-built providers to your project.

»Special Thanks to AWS

We would like to extend our deepest gratitude to the AWS CDK team, particularly Romain Marcadier, for working closely with us to explore all the options for improving the performance of code generation through jsii and helping us validate namespaces as our best path forward.

»Try CDK for Terraform

If you’re new to the project, these tutorials for CDK for Terraform are the best way to get started. You can dive deeper into our documentation beginning with this overview of CDKTF.

Whether you’re experimenting or actively using CDK for Terraform, we’d love to hear from you. Please file any bugs you encounter, let us know about your feature requests, and share other questions, thoughts, and experiences in the CDK for Terraform discussion forum.

CDK for Terraform 0.13 Significantly Improves Performance

»Performance Gains in CDKTF 0.13

»Breaking Change: Namespaces

»What You Need To Do

»When Do You Need To Update?

»What's Next for CDKTF

»Special Thanks to AWS

»Try CDK for Terraform

Sign up for the latest HashiCorp news

More blog posts like this one

Fix the developers vs. security conflict by shifting further left

HashiCorp at AWS re:Invent: Your blueprint to cloud success

Speed up app delivery with automated cancellation of plan-only Terraform runs