CDK for Terraform in Production: Learning from Pocket

Kelvin Yeboah and Mathijs Miermans from Pocket’s engineering team joined us in a recent Terraform Community Office Hours to share a live demo of how they define and provision infrastructure using CDK for Terraform. Check out the recording to watch the live demo, and read on for a summary of how their team is using CDK for Terraform.

»CDK for Terraform

The Cloud Development Kit for Terraform (CDKTF) allows you to use familiar programming languages to define cloud infrastructure and provision it through Terraform. This gives you access to the entire Terraform ecosystem while allowing you to leverage the power of your existing toolchain.

CDKTF is under active development and is not officially recommended for production use cases. However, some early adopters, like Pocket, are already using CDK for Terraform in production, and we are excited to work with them to validate and improve workflows.

»Provisioning with Terraform Using the Familiarity of TypeScript

Pocket — a Mozilla product — is a website and app that finds the most interesting, thought-provoking and entertaining articles from trusted sources around the internet and puts them all in one place. Pocket also lets you save articles — as well as anything else you find online (videos, recipes, shopping pages, etc.) — to your personal Pocket for digging into later.

To support the user experience, the Pocket engineering team is tasked with building, maintaining, and deploying a wide variety of services, with the same type of infrastructure being deployed frequently. Over time, the team transitioned from managing infrastructure in a UI to working with Terraform, where they loved having version control and being able to manage state. But HCL was a new syntax and framework for the team to learn.

Transitioning to writing infrastructure code in their preferred programming language— TypeScript — using CDK for Terraform has allowed them to spin up new services faster and get new features in front of users with less delay. Managing infrastructure in a familiar language has also empowered their developers to engage closely with the infrastructure, giving them a better understanding of the services they are building and removing blockers.

“When CDKTF came out, that was like a game changer for us. We are mostly application developers, we are not an SRE team … so if we could write our infrastructure in code in the language that we already use, that was a game changer for us. Because then we can empower ourselves to work faster and more efficiently and build services faster.” — Kelvin Yeboah, Senior Software Engineer, Mozilla, Pocket

Another practical improvement that they’ve noticed is that they have fewer files to manage and they are able to use their existing code pipeline with CircleCI to automate the plan/apply deployment workflow. For those interested in seeing a CDK for Terraform project in action, the team’s open source GitHub repos are an excellent reference for structuring and writing code with CDKTF: https://github.com/Pocket.

»The Demo

During the live demo, Mathijs and Kelvin walked us through how to deploy a “Hello world” Apache web server in a production-ready environment using Amazon ECS in fewer than 150 lines of readable TypeScript code. In addition to provisioning infrastructure for the web server, this demo application also includes other production-ready necessities, including an ECS cluster, application load balancer, defined security groups, alarms via PagerDuty, and snapshot testing of the output JSON configuration file.

You can reference the code used for this demo in the repos below, and even follow along as you watch the recording:

• Demo source code
• Reusable constructs

And be sure to check out the second demo recording that Pocket did after that, also covering more of their CDK for Terraform production usage.

»Simplifying Application Code with Reusable Constructs

CDK for Terraform allows you to manage complexity and reduce code duplication by creating custom abstraction layers, referred to as constructs. During the demo, Kelvin references the custom construct libraries that their team has built for their CDK for Terraform TypeScript applications. These are opinionated libraries that their team has built to customize the CDKTF interface to the developer team’s needs. For example, in the demo, Kelvin uses two of these custom packages to add a production-ready ECS cluster, a load balancer, and to create alarms using PagerDuty. You can find these libraries in an open source repo of Node.js packages that define their infrastructure patterns in CDKTF.

For more examples of how the Pocket team uses custom constructs to simplify their development process and ensure best practices, check out these open source repos:

• Pocket/firefox-android-home-recommendations: TypeScript REST API
• Pocket/collection-api: TypeScript GraphQL API with a relational database
• Pocket/recommendation-api: Python GraphQL API with a key-value store and AWS Lambda functions

»Get Started with CDK for Terraform

If you’re new to CDK for Terraform, the best place to get started is with the hands-on tutorials on HashiCorp Learn.

There's also a new follow up office hours, where the Pocket team does a more thorough walkthrough of their codebase, recommended architecture, and best practices.