Consul 1.16 enhances service mesh reliability, user experience, and security

We're excited to announce that HashiCorp Consul 1.16 is now generally available. Consul uses service identities and traditional networking practices to help organizations securely discover and connect services across any runtime or platform.

Organizations constantly aim to improve their application's scale, resilience, performance, and security while ensuring quick deployment and troubleshooting. The latest Consul releases further these goals by enhancing enterprise readiness, simplifying the user experience, speeding app deployments, and bolstering application security in the mesh.

Consul 1.16 adds new features for reliability, scalability, security, service mesh UX, extensibility, and service discovery.

»Reliability and scale

Consul 1.16 debuts several new reliability and scalability features for Consul Enterprise:

»Sameness groups (Consul Enterprise - beta)

When implementing service failover between Consul clusters under common administration, operators often need to seamlessly handle failover between active-standby clusters. Peered clusters and administrative partitions within Consul are typically treated as separate autonomous meshes, which can sometimes create complexity in achieving uniform service failover.

To address this, we’ve introduced sameness groups to logically group Consul peers and partitions that share common administration. This logical grouping enables efficient service failover between clusters, ensuring high availability and resiliency for services deployed across multiple regions and Consul clusters.

By enabling sameness groups, organizations can ensure that even in the event of issues in one cluster or region, the failover mechanism redirects traffic to an active standby cluster, maintaining uninterrupted service delivery. This capability significantly enhances the reliability and resilience of services in distributed environments. Refer to the sameness groups overview documentation for more information.

»Enhanced control plane protection (Enterprise)

In Consul 1.15, remote procedure call (RPC) rate limits were applied globally, allowing any client to potentially overwhelm the available query capacity. This could lead to resource exhaustion and service disruptions.

To mitigate this, Consul now supports applying rate limits on a per-IP basis. This means that requests from each client are individually controlled and limited based on their source IP address. Operators can set different limits depending on the resource being accessed, providing more precise control over resource utilization.

With per-IP rate limits, operators can allocate server resources to support normal traffic patterns without worrying about being overwhelmed by a single client or experiencing service disruptions. It provides a tailored approach to resource allocation, allowing for better management of system resources.

Refer to the traffic rate limiting overview documentation for more information.

»Security

Consul 1.16 adds a number of new security features:

»FIPS 140-2 compliant Consul Enterprise

Consul Enterprise now offers FIPS 140-2 compliant builds that meet the security needs of federal agencies protecting sensitive, unclassified information with approved cryptographic measures. These builds use a certified cryptographic module and include specific safeguards to comply with FIPS 140-2 Level 1 requirements, enabling compliant Consul deployments.

Refer to the Consul Enterprise FIPS documentation for more information.

»JWT auth for service-to-service traffic

Consul’s service intentions can now authorize connections based on claims in JSON Web Tokens (JWT), in addition to service identity. Operators can configure one or more JWT providers to authenticate tokens and authorize connections to services and specific HTTP paths based on claims embedded in JWT. By offloading JWT validation and authorization to the service mesh, organizations have a uniform method to validate and authorize communication between services in a diverse service-oriented architecture.

Refer to the JWT authorization with service intentions documentation for more information.

»Service mesh UX

Consul 1.16 adds several user experience improvements for transparent proxies, external mTLS connections, and simplified API Gateway installation:

»Transparent proxy for failover and virtual services

Consul has enhanced transparent proxy mode to support failover between service instances deployed across clusters and the ability to route to virtual services. Prior to this enhancement, there were operational complexities when it came to supporting cross-cluster failover.

With the new transparent proxy enhancements, Consul now provides a seamless solution for failover between services across clusters. This means that if one service becomes unavailable, Consul will automatically route the traffic to a backup service, helping to ensure continuity and reliability.

Additionally, transparent proxy also extends its functionality to handle virtual services, which are services defined using the ServiceResolver config entry. Previously, explicit configuration was required for services to communicate with upstream services. With this update, virtual services can now be addressed efficiently using KubeDNS within a datacenter and Consul DNS between datacenters. This enables consistent and reliable communication between services, regardless of their location.

Refer to the documentation below for additional information:

• Onboard services while in transparent proxy mode
• Route traffic to virtual services
• Configure failover services while in transparent proxy mode

»Permissive mTLS

In brownfield environments, migrating services to a service mesh can be challenging because the mesh requires all communication to be secured with mutual TLS (mTLS). This can pose a problem as downstream applications that haven't been migrated yet will experience connectivity issues to upstream applications that have been onboarded to the mesh.

To address this challenge, Consul 1.16 introduces the permissive mTLS feature. With permissive mTLS, an upstream service can be configured to accept both mTLS traffic from mesh-enabled services and non-Consul mTLS traffic from external downstream applications. This means that services within the mesh can communicate securely with each other using mTLS, while maintaining connectivity with applications outside the mesh that haven't undergone migration. This approach allows for a gradual onboarding of services to the service mesh without disrupting the existing connectivity and functionality of downstream applications.

Refer to the Onboard services while in transparent proxy mode documentation for more information on how to use permissive mTLS to onboard services to Consul.

»Simplified API Gateway installation for Consul on Kubernetes

API Gateway is now built into Consul on Kubernetes. Previously, the API Gateway for Kubernetes was a separately released software package compatible with specific versions of Consul and the Consul Helm chart. Now, users don’t need to manage the version of API Gateway they install because it is just another feature in Consul and Consul for Kubernetes. This simplifies installation and configuration for Consul on Kubernetes.

»Extensibility

Consul 1.16 includes several extensions for Envoy:

»External AuthZ Envoy extension

The external AuthZ Envoy extension enhances Consul's capabilities by enabling configuration of Envoy external authorization filters for TCP and HTTP listeners. This extension allows you to augment Consul’s service intentions with custom authorization solutions for the data-plane traffic within your service mesh. By configuring the external authorization service through Consul config entries, requests between services can be authorized based on the policies embedded in an external auth service such as Open Policy Agent (OPA). This feature helps organizations maintain better control over access and security leveraging a wider variety of tools, so that requests adhere to your organization’s specified policies and regulations.

Refer to the Delegate authorization to an external service documentation for more information on how to use this extension.

»Property override Envoy extension

The property override extension allows operators to modify small configuration settings in Envoy that are not directly exposed in Consul configuration entries. With this extension, users can easily fine-tune their Envoy configuration by defining configuration entries in Consul. They can dynamically adjust the properties without manually writing large portions of Envoy configuration files with escape hatches.

This extension simplifies the configuration management process by providing a reliable, straightforward way to make small configuration changes to Envoy. It also provides better compatibility and reliability during Consul and Envoy version upgrades, because the extension is designed to work seamlessly with the evolving Consul and Envoy ecosystem.

Overall, the Envoy property override extension in Consul improves the flexibility and manageability of Envoy configuration, enabling users to easily make fine-grained adjustments while maintaining compatibility and stability in their service mesh environment.

Refer to the Configure Envoy proxy properties documentation for more information on how to use this extension.

»Wasm Envoy extension

The WebAssembly (Wasm) extension lets operators easily configure and run custom Wasm plugins on TCP and HTTP listeners in Envoy. Wasm plugins can be enabled on a per-service basis using the service-defaults configuration entry, or on a mesh-wide basis using the proxy-defaults configuration entry. By supporting Wasm, Consul enables users to enhance their service mesh architecture, enabling innovative use cases and customization. Wasm simplifies integration, reduces complexities, and improves the overall functionality and flexibility of the service mesh.

Refer to the Run WebAssembly plug-ins in Envoy proxy documentation for more information on how to use this extension.

»Service discovery

Consul 1.16 also enhances service discovery:

»Catalog sync support for health checks and ingress

Catalog sync for Consul now supports health checks and ingress resources, enhancing the functionality and benefits of the Consul catalog. With this update, Kubernetes pod health status is seamlessly synchronized with service instances, enabling more accurate traffic routing decisions. Additionally, ingress resource metadata is registered with services, allowing for easy discovery of services based on ingress routing rules. This provides a more comprehensive view of Kubernetes resources, empowering efficient and reliable service discovery within Kubernetes environments.

»Next steps for Consul

Our goal with Consul is to enable a consistent, enterprise-ready control plane to discover and securely connect any application. We are excited for users to try these new Consul updates and further expand their service mesh implementations. Consul 1.16 includes enhanced scalability, reliability, and security for service networking. With improved failover, security measures, user experience, and extensibility, Consul empowers organizations to build resilient and efficient applications in distributed environments.

• Learn more in the Consul documentation.
• Get started with Consul 1.16 by installing the latest Helm chart, provided in our Consul Kubernetes documentation.
• Try Consul Enterprise by starting a free trial.

This post was originally published on June 13 for the beta release. The sameness groups feature remains in beta.