Consul 1.15 adds Envoy extensions and enhances Envoy access logging

We’re pleased to announce that HashiCorp Consul 1.15 is now generally available to all users. This release represents yet another step forward in our effort to help organizations simplify onboarding to service mesh, improve their troubleshooting workflow, and reduce operational complexity.

Important new features in Consul 1.15 include:

• Envoy access logging
• Consul Envoy extensions
• Service-to-service troubleshooting
• Linux VM runtime support in Consul-native API gateway (Beta)
• Consul server rate limiting
• Raft write-ahead log (Experimental)

Let’s run through what’s new.

»Envoy access logging

Envoy currently provides access logging to understand incoming traffic patterns to the proxy. Prior versions of Consul allowed the operator to adjust bootstrap configuration for Envoy using escape hatches to enable access logs, which proved hard to manage at scale. With Consul 1.15, Envoy access logs are now centrally managed via config entries and CRDs to allow operators to easily turn on access logs for all proxies within the service mesh.

Access log configuration can be fine tuned to output to stdout pipe or to a file, and can also be configured to output a variety of different request properties such as response codes and request headers. This allows the service mesh operator to quickly audit incoming traffic to proxies within the mesh for security purposes, and also allows them to troubleshoot requests made to workloads within the mesh. Please visit Proxy default configuration entry for more details on how to enable access logs.

»Consul Envoy extensions

Consul 1.15 now adds an extension system to Consul that allows the service mesh operators to modify Consul-generated Envoy resources without customizing the Consul binary. This will allow extensions to add, delete, and modify Envoy listeners, routes, clusters, and endpoints, enabling support for additional Envoy features without changes to the Consul codebase.

Prior work for modifying existing Envoy config involved utilizing escape hatches to rewrite the Envoy resources so that they are compatible with Consul, a task that also requires understanding how Consul names Envoy resources and enforces intentions. With Consul Envoy extensions you can now configure your services to use extensions through the EnvoyExtensions field with the Proxy Defaults or Service Defaults configuration entries or CRDs. Current supported extensions include the Lua and AWS Lambda extensions. Please visit the Envoy extensions overview for more information on how to leverage these extensions for Consul service mesh.

»Service-to-service troubleshooting

Identifying communication issues between services is challenging due to the complexity of configurations and lack of standard methodology for troubleshooting. Ambiguous error outputs and overwhelming amounts of logs and telemetry data add to the difficulty, leading to a challenging user experience.

Consul now includes a built-in tool for troubleshooting communication between services in a service mesh. A new consul troubleshoot command is provided to validate Envoy configurations on both upstream and downstream Envoy proxies deployed on VM and Kubernetes deployments.

The consul troubleshoot command performs several checks in sequence that enable you to discover issues that impede service-to-service communication. The process systematically queries the Envoy administration interface API to determine the cause of the communication failure.

The troubleshooting command validates service-to-service communication by checking for the following common issues:

• Upstream service does not exist

• One or both hosts are unhealthy

• A filter affects the upstream service

• The CA has expired mTLS certificates

• The services have expired mTLS certificates

Please visit our Service-to-service troubleshooting overview for more details on how to use the new troubleshooting commands.

»Linux VM runtime support in Consul-native API gateway (Beta)

Prior to Consul 1.15, Consul API gateway was only supported on Kubernetes. VM support is now available, allowing operators to stand up and configure API gateway for VM-based deployments using native Consul config entries. The following workflows are now available for Consul API gateway in a VM environment:

1. Create an API gateway configuration entry. The configuration entry includes listener configurations and references to TLS certificates.
2. Deploy the API gateway configuration entry to create the listeners.
3. Create and deploy routes to bind to the gateway.

VM support is currently in beta and will be generally available in a future release. Please read API gateways on virtual machines for more details.

»Consul server rate limiting

Consul servers do not have means to protect themselves from excessive usage or a load spike. They rely fully on Consul client rate limiting and operators dimensioning the servers to handle all the required load. With the release of a new architecture to use Consul Dataplane for establishing connections back to Consul servers within the mesh, Consul clients are no longer available to perform rate limiting and protect the servers.

In a scenario where a single Consul user is misbehaving by sending an excessive amount of load to the servers, the entire Cluster is impacted, which could potentially lead to outages in some Consul environments.

With Consul 1.15, an operator will now be able to configure rate limiting for each Consul server. To do so, an operator can add a set of runtime configurations in the server configuration which will configure:

• A set of global limits for read and write operations for each Consul server
• A mode to apply when the global limit is reached (e.g. permissive, enforcing) for a given Consul server

This provides a critical way to protect Consul servers from cascading failures when they reach capacity or a traffic spike is observed, allowing a graceful degradation. For more details on how to utilize the new server rate limiting features on Consul, please visit Limit traffic rates overview.

»Raft write-ahead log (Experimental)

Consul now provides an experimental storage backend called write-ahead log (WAL). WAL implements a traditional log with rotating, append-only log files. The current LogStore uses BoltDB, which is a copy-on-write BTree, which is less optimized for append-only workloads. The storage backend now resolves a number of performance issues with the current BoltDB storage backend, which was not originally designed for retaining a large number of logs. The new raft write-ahead log storage backend is not recommended for production use cases yet, but is ready for testing by the general community. Please read our Experimental WAL LogStore backend overview for more details.

»Next steps for Consul

We are excited for users to try these new Consul updates and further expand their service mesh implementations. The Consul 1.15 includes enhancements for all types of Consul users leveraging the product for service discovery and service mesh across multiple environments, including serverless functions. Our goal with Consul is to enable a consistent enterprise-ready control plane to discover and securely connect any application.

• Learn more in the Consul documentation.
• Get started with Consul 1.15 by installing the latest Helm chart provided in our Consul Kubernetes documentation.