Consul-Terraform-Sync 0.5 Adds New APIs and UX Improvements
HashiCorp Consul-Terraform-Sync (CTS) 0.5 adds improvements to task lifecycle management and new ecosystem integrations.
We are pleased to announce the general availability of Consul-Terraform-Sync (CTS) 0.5. This release marks another step in the maturity of our larger Network Infrastructure Automation (NIA) solution.
CTS combines Terraform and Consul functionality to eliminate manual ticket-based systems across on-premises and cloud environments. It can be broken down into two parts. For Day 0 and 1, teams use Terraform to quickly deploy network devices and infrastructure in a consistent and reproducible manner. Once established, teams manage Day 2 networking tasks by integrating Consul’s catalog to register services into the system via CTS. Whenever a change is recorded to the service catalog, CTS triggers a Terraform run that uses partner ecosystem integrations to automate updates and deployments for load balancers, firewall policies, and other service-defined networking components.
This post highlights the new features in CTS 0.5, including new APIs to manage the lifecycle of tasks and user experience enhancements. We’ll also discuss the new integrations with Cisco Secure Firewall Management Center (FMC) and Fortinet FortiManager. CTS 0.5 is available for download on the CTS release page, but before we dive into what’s new in CTS 0.5, let’s look at how CTS has progressed over the past few releases.
»CTS 0.1 Through 0.4
- The 0.1 and 0.2 CTS releases enabled a publisher-subscriber (pub-sub) paradigm for updating your network infrastructure based on changes in the HashiCorp Consul catalog, leveraging the Terraform CLI on the local node. These releases focused on establishing core networking use cases for practitioners, including applying firewall policies, updating load balancer member pools, and more.
- The 0.3 release marked a major milestone by adding support for Consul-Terraform-Sync Enterprise. This release featured regex support for service triggers and Terraform Enterprise integration.
- The 0.4 release of Consul-Terraform-Sync enabled integration with Terraform Cloud and support to trigger a Terraform workflow based on Consul key-value (KV) changes.
- The integration with Terraform Enterprise and Terraform Cloud allowed organizations to dynamically manage their Day 2 applications’ networking delivery lifecycle and oversee infrastructure with governance and security functionality.
»Task Lifecycle APIs and CLIs
Tasks are at the core of CTS’ automation capabilities. It ensures CTS can monitor Consul for updates and trigger a Terraform workflow. But we learned from our users that managing the lifecycle of a task post-CTS initialization required them to stop the CTS process, make changes to the config file, and restart the CTS process. To address these challenges, we are adding support for four new secure API endpoints and a CLI, which allow users to create, update, delete, and read tasks.
Payload:
{
"task": {
"description": "Writes the service name, id, and IP address to a file",
"enabled": true,
"name": "task_a",
"providers": ["my-provider"],
"condition": {
"services": {
"names": ["web", "api"]
}
},
"module": "path/to/module"
}
}
Request:
curl --header "Content-Type: application/json" \
--request POST \
--data @payload.json \
localhost:8558/v1/tasks
»Streamlined Configuration
Our goal has been to provide our users with a smooth and consistent experience for configuring and operating CTS. As we add more capabilities into CTS to trigger workflows — not just based on service changes, but other updates from Consul as well, like key-value changes — we wanted to streamline the configuration for monitoring these changes into the core task functionality.
In CTS 0.4, we introduced module_input blocks to the task configuration, which allowed users to monitor services by matching regexes in a task definition to services within the Consul catalog. At that time, a module_input block could be configured only with scheduled tasks. With this release, the module_input block can now be used with all task types. Users can also configure more than one module_input block per task so long as module input types are unique.
With this capability, users can leverage CTS for application config change management workflows that rely on Consul KV. As a part of this effort, we are deprecating the services field. The full list of deprecations in CTS 0.5 can be found in the release notes.
Previously, a configuration with a catalog-services condition that monitors some services might have looked like this:
task {
name = "service_registration_task"
module = "path/to/module"
condition "catalog-services" {
regexp = ".*"
}
services = ["api-1", "api-2", "api-3"]
}
With the expanded usage of module_input blocks, the services field can be replaced with module_input "services". Users also have the option to configure monitoring for consul-kv.
task {
name = "service_registration_task"
module = "path/to/module"
condition "catalog-services" {
regexp = ".*"
}
module_input "services" {
// option to list the services by name or regexp
regexp = "api-"
}
module_input "consul-kv" {
// new option to add other types of module inputs
path = "data/api"
}
}
»CTS Ecosystem Integrations
Our partner ecosystem plays a critical role in helping users automate manual networking tasks. That’s why we are pleased to announce new integrations with Cisco Secure Firewall Management Center (FMC) and Fortinet FortiManager.
»Cisco Secure Firewall Management Center (FMC) Integration
Cisco FMC provides complete and unified management over firewalls, application control, intrusion prevention, URL filtering, and advanced malware protection. The integration with Cisco FMC is achieved through a module that updates dynamic objects in the Cisco Secure Firewall Management Center (FMC) based on service changes within Consul and applies the dynamic objects as access rules to the firewall directly.
»Fortinet FortiManager Integration
FortiManager supports network operations use cases for centralized management, best-practices compliance, and workflow automation to provide better protection against breaches. FortiManager integration with CTS enables management of firewall address groups based on service changes detected within Consul. This integration is delivered through a Terraform module.
»Learn More
There are several HashiCorp Learn tutorials available to help you get started with CTS:
- Network Infrastructure Automation with Consul-Terraform-Sync Intro
- Consul-Terraform-Sync and Terraform Enterprise/Cloud Integration
- Build a Custom Consul-Terraform-Sync Module
You can find more CTS tutorials by visiting the CTS tutorial collection at learn.hashicorp.com.
Feel free to try out CTS 0.5 and give us feedback in the issue tracker. You can also stay up to date on CTS by following our public roadmap or checking the changelog. For more information about Consul, please visit our Consul product page.
Sign up for the latest HashiCorp news
More blog posts like this one
Fix the developers vs. security conflict by shifting further left
Resolve the friction between dev and security teams with platform-led workflows that make cloud security seamless and scalable.
HashiCorp at AWS re:Invent: Your blueprint to cloud success
If you’re attending AWS re:Invent in Las Vegas, Dec. 2 - Dec. 6th, visit us for breakout sessions, expert talks, and product demos to learn how to take a unified approach to Infrastructure and Security Lifecycle Management.
Speed up app delivery with automated cancellation of plan-only Terraform runs
Automatic cancellation of plan-only runs allows customers to easily cancel any unfinished runs for outdated commits to speed up application delivery.