Announcing Consul-Terraform-Sync Tech Preview

Application and operation teams are striving to improve application deployments times and enable self-service capabilities by adopting new practices and technologies such as DevOps, infrastructure as code, Kubernetes, and service mesh. However, network and security teams are severely hampered by manual ticket-driven processes. This has created an impedance mismatch with the application teams and overall slower delivery of the solutions to the customers. This challenge is most evident in Day 2 operations related to applications scaling up and down in dynamic networking environments. Additionally, these manual processes present a risk to the organization by increasing the likelihood of network outages from misconfiguration of multiple network devices.  

Network and security teams can solve these challenges by enabling dynamic service-driven network automation with self-service capabilities using an automation tool which supports multiple networking technologies. 

To help these teams address these challenges, we are pleased to announce the tech preview of Consul-Terraform-Sync, which enables Network Infrastructure Automation (NIA) using HashiCorp Terraform. These capabilities provide operational consistency across teams using a shared source of truth for service discovery, which enables a publisher-subscriber paradigm when an application service scales up or down. 

Consul-Terraform-Sync enables a service update-driven push-based approach to configure network devices. This approach consists of three components:

1. HashiCorp Consul — HashiCorp's service networking platform
2. Consul-Terraform-Sync —  A new daemon that subscribes to Consul and manages the automation of target network devices through Terraform
3. Consul-Terraform-Sync-compatible Terraform modules — Modules built by our ecosystem partners leveraging their existing Terraform providers

The interaction is illustrated in the figure below.

Illustration of the interaction between the Consul, Consul-Terraform-Sync, and the underlying network infrastructure devices

Consul-Terraform-Sync leverages Terraform as the underlying automation tool and utilizes the Terraform provider ecosystem to drive relevant changes to the network infrastructure. 

Managing and automating network infrastructure requires expertise on not only operating the network infrastructure devices, it also requires an understanding of frequent workflow on those devices. With this in mind, HashiCorp has partnered with leaders in the networking and security industry on this solution.

A10 Networks, Checkpoint, Cisco, F5, and Palo Alto Networks are the launch partners for Consul-Terraform-Sync. These technology partners have identified workflows related to application scale up/down on their devices, built Terraform modules compatible with Consul-Terraform-Sync and, upstreamed those modules on the Terraform Registry. You can get more information on the use cases enabled by these modules here. 

Consul-Terraform-Sync introduces a key construct Task, which enables users to subscribe to the desired services in the Consul catalog and trigger the execution of the specified automation runbook when those subscribed services are updated. Details on “Task” is described below 

• The automation runbook used by Task is a Terraform module built using the target infrastructure’s resources, data sources, and service level variables generated by Consul-Terraform-Sync.
• Each Task is executed locally in its own workspace created by Consul-Terraform-Sync and corresponding Terraform state for each task is stored in Consul K/V store.  

The service updates that would trigger a Task, and the Terraform workflow of plan and apply for the module, can include the addition or removal of service instances on a node, change of service address or port number, updates to service tags, meta or health, etc. 

Illustration of Task for Consul-Terraform-Sync in HashiCorp Configuration Language (HCL)

You will find a detailed usage example for Consul-Terraform-Sync available here.

For networking and security technology partners interested in developing their own modules for Consul-Terraform-Sync, they can connect with us through the Network Infrastructure Automation Integration Program.

The Consul-Terraform-Sync is available on GitHub. This repo includes instructions for building and running the Consul-Terraform-Sync, as well as example usage. You can also download a pre-built binary for Consul-Terraform-Sync here. We are excited to release this new architecture to the community and gather feedback. Feel free to try it out and give us feedback in the issue tracker. 

For more information about HashiCorp Consul, please visit the Consul product page.