Announcing HashiCorp Consul 1.9

We are pleased to announce the public beta availability of HashiCorp Consul 1.9. Consul is a multi-cloud service networking platform to discover, connect, and secure services across any runtime platform and public or private cloud.

Consul 1.9 focuses on providing more granular control for service mesh users, improving observability into services within the service mesh, and delivering tighter integrations with more runtime platforms in a way that is more Kubernetes-native.

This release includes the following features:

• Application-Aware Intentions: Extends Consul’s intentions model to support Layer 7 constructs and allow/disallow service communication based on HTTP request headers and URL paths.

• Service Mesh Visualization: Provides a new topology tab in the Consul UI with topology diagrams and key service mesh metrics like request, error rates, and timing metrics. These new features will assist developers and operators in verifying configuration and troubleshooting issues within the service mesh.

• Custom Resources for Kubernetes: Provide a way to configure Consul service mesh configuration via Kubernetes Native API extensions known as Custom Resource Definitions (CRDs).

• Deploy Consul in OpenShift: Enable installing Consul via Helm chart in OpenShift.

• Active Health Checks for Consul on Kubernetes (available at GA): Integrate Kubernetes health checks into Consul to ensure that traffic is not routed to pods with readiness or health failures.

• Streaming: Reduce CPU and network bandwidth usage on Consul servers very significantly in large scale deployments via streaming. This is a major architectural enhancement in how update notifications for blocking queries are delivered.

Consul 1.9 is available for public beta now, with general availability to follow. Please review the Consul 1.9 changelog for a detailed list of changes. The binaries can be downloaded here.

NOTE: This beta release does not have a forward compatibility guarantee, and certain functionality may change that might cause it to be incompatible with the general availability release. We recommend that you only use the beta releases to test functionality and upgrades with clusters that can be lost.

»Application-Aware Intentions

Enterprises adopting service mesh desire to implement least privilege, zero trust networking using granular application-based access policies.

Consul’s intentions in previous versions are connection-based (Layer 4). They do not provide the ability to utilize higher level application protocol information when evaluating authorization policies.

Application-aware (Layer 7) intentions provide the ability for operators to construct policies which, in addition to service identity, evaluate application-layer information such as HTTP Path, Headers, or Method when authorizing HTTP-based (HTTP/1.1, HTTP/2, gRPC) service-to-service communication.

»Service Mesh Visualization

A service mesh is a configurable network layer that controls how an application’s services interact with each other. Configuring and managing a service mesh requires setting up proxies and upstreams correctly. Consul users have relied on error logs and command line output, as well as sending data plane metrics to systems like Datadog or Prometheus to be able to troubleshoot the Consul service mesh.

In Consul 1.9, we introduce a new topology tab in the Consul UI that shows an overview of how services connect with one other, with connection-specific metrics like error rates, success rates, and timing overlaid on top. The UI will also include deep linking into your external metrics dashboards.

Besides metrics specific to services registered in Consul, users also need to monitor Consul performance overall. Our metrics and monitoring guide shares best practices for operators to use to make sure their Consul cluster is healthy. Consul 1.9 additionally updates several metrics to reduce cardinality in metric names in favor of the best practice of using labels instead and will provide dashboard templates that expose the most important metrics associated with the Consul datacenter for operators.

»Custom Resource Definitions for Kubernetes

Kubernetes natively provides an API that allows you to describe the “desired state” for objects within the Kubernetes system. When you use kubectl, behind the scenes you invoke the Kubernetes API and send requests to the Kubernetes API server as JSON. After an object is created, the Kubernetes control plane continually and actively manages the object's state until it matches the desired state that you specified via Kubernetes YAML.

As third-party software has adopted Kubernetes as a target platform, software vendors have started to leverage Custom Resource Definitions (CRDs) within Kubernetes as an effective way to integrate their software with the Kubernetes API and extend the functionality that users’ Kubernetes clusters can provide. Since its release, the Consul Kubernetes integration has aimed to provide a production-ready experience of Consul on Kubernetes infrastructure. The installation and configuration of Consul is currently managed via Helm, and the configuration of Layer 7 routing and the creation of intentions is performed via the Consul CLI or API.

In this release, we are providing a Kubernetes-first experience through CRDs to allow practitioners to easily configure Consul via Kubernetes-style objects. Specifically, we are providing the capability to manage Consul service mesh configuration for services via CRDs.

»Deploy Consul in OpenShift

OpenShift is IBM RedHat’s distribution of Kubernetes. Its main value proposition is that it’s secure by default, making it very popular among enterprise users.

Previously, deploying Consul on OpenShift required modifications in our Helm chart due to the default permissions that provide restrictions on which pod capabilities are available. We’ve now ensured that Consul Kubernetes runs on OpenShift securely by ensuring that Consul runs as non-root and also provided a set of SecurityContextConstraints to deploy Consul securely.

Installing Consul on OpenShift should now be as simple as running a Helm install with global.openshift.enabled set to true.

»Active Health Checks for Consul on Kubernetes

Health checks currently exist in Consul today to provide practitioners a way of defining application-level checks for services that are registered within Consul. Consul service mesh currently provides the ability to direct traffic to only instances of a service that are healthy (see resolvers); however, this relies on the existence of Consul health checks to direct traffic to healthy service instances.

Like Consul, Kubernetes implements its own set of health checks, namely the Liveness and Readiness probes. Unlike Consul, Consul on Kubernetes does not understand the health of deployed service instances because Consul health checks are not available on Kubernetes. By monitoring the health of the pods (i.e., service instances) on Kubernetes, we can direct traffic to service instances and we can prevent a pod from receiving traffic and failing to respond to a request.

This release will provide Consul service mesh integration with Kubernetes Readiness probes. This ability to natively detect health status from Kubernetes via Readiness probe is then used for directing service mesh traffic.

»Streaming

Several Consul users and customers that run Consul clusters at scale manage large fleets of Consul clients and monitoring changes to tens of thousands of service instances. Typically, changes to the health status of a service are monitored through blocking queries.

In this release, we’ve introduced streaming, a major architectural enhancement in how update notifications for blocking queries are delivered within the cluster. Streaming results in very significant reduction of CPU and network bandwidth usage on Consul servers in large-scale deployments. Streaming is particularly helpful in scaling blocking queries in Consul clusters that have rapid changes in service state.

Streaming is now available for the service health HTTP endpoint. It can be enabled through configuration flags and adding the cached URL parameter. In subsequent releases, we will continue to enable streaming in more endpoints.

»Getting Started

The Consul 1.9 release provides increased control over and observability to services within the service mesh and enables you to adopt a service mesh in more environments and via more developer-friendly ways. We would like to thank our active community members who have been invaluable in adding new features, reporting bugs, and improving the documentation for Consul in this release. To get started with the public beta of Consul 1.9, please refer to one of the HashiCorp Learn tutorials:

• Manage Traffic across services with Consul and Envoy
• Manage service intentions in Consul service mesh using Kubernetes Custom Resource Definitions (CRDs)