Automate networking across Google Cloud runtimes with Consul and Apigee

HashiCorp Consul is a service networking platform that can be used to automate networking across multiple cloud runtimes, including Google Cloud. Consul now includes Envoy extension capabilities that allow operators to offload service-to-service authorization (AuthZ) to external tools and platforms. This unlocks more options to authorize traffic based on more conditions such as allow/deny based on business hours.

The Apigee adapter for Envoy with Consul is a prime example of Consul’s Envoy extension capabilities. In this post, you’ll learn about the benefits you can gain from this integration along with how it works and how to set it up. Apigee’s implementation requires an API key, JWT, or a third-party JWT to be passed between services in order to allow traffic. Once you have one of those, you’re ready to get started.

»HashiCorp Consul

HashiCorp Consul is a service networking solution that provides global service discovery and service mesh capabilities. With Consul, platform operators can ensure the least privileged access between services based on a service identity rather than relying on firewalls to authorize communication based on IP addresses.

One of the primary service mesh capabilities is service authorization: the process of allowing or denying service-to-service communication. In Consul 1.16 and above, this authorization capability can also be offloaded onto an external authority, like Apigee.

»Apigee

Apigee is Google Cloud's native API management platform that can be used to build, manage, and secure APIs for any use case, environment, or scale. Apigee offers high-performance API proxies to create a consistent, reliable interface for your backend services. The proxy layer gives you granular control over security, rate limiting, quotas, analytics, and more for all of your services.

»Manage service-to-service communication across environments

Consul’s integration with Apigee combines Apigee’s extensive API lifecycle management features with Consul’s ability to authenticate, encrypt, and control traffic across any public or private cloud and across multi-runtime environments. In addition, Consul provides Apigee users with the ability to automate and offload mTLS authentication and encryption between network services, while still controlling authorization from a centralized cloud control plane.

While you can install Apigee onto your own infrastructure, a full deployment of Apigee may be undesirable due to the size and complexity required to support its full feature set. Deploying the Apigee API management component close to target applications reduces the network latency.

The solution proposed here is to install the Apigee remote Envoy service and onboard that service onto the Consul service mesh to enforce authorization policies provided by the central Apigee authority residing in Google Cloud.

Benefits of the integration include:

• Availability of the full suite of Apigee Analytics metrics, dashboards, and APIs.
• Keeping API traffic within the enterprise-approved boundaries for security or compliance purposes
• Asynchronous communication with Apigee, allowing API traffic data to be captured and sent to Apigee without affecting latency
• Resilience against temporary communication interruptions with the management plane

Over time, however, an interruption can lead to a loss of functionality. The adapter requires communication with the management plane for:

• Retrieving fresh tokens from Apigee
• Verifying API keys (connection is required for the first verification, after which it is cached)
• Enforcing quotas
• Sending analytics data to the management plane

For more information on the adapter, check out Google’s documentation.

»How does it work?

The Consul Envoy extension system allows modification of Consul-generated Envoy resources without customizing the Consul binary, enabling additional Envoy features for service mesh traffic that passes through an Envoy proxy. You can learn more about Envoy extension and supported extensions here.

Apigee integrates with Consul using the external authorization extension type. The diagram below illustrates the sequence of six events when a service attempts to communicate with another upstream service. The Apigee Remote Service asynchronously polls the external AuthZ management plane and downloads proxy, API product, and other configurations that it needs to respond to authorization requests.

1. Service A (client app) tries to access Service B (service backend) in the Consul service mesh. Communication is routed through the local Envoy proxy on Service A.
2. The Envoy proxy on Service A sends traffic to the Envoy proxy on Service B along with the authorization key.
3. The Envoy proxy on Service B passes the security context (using HTTP headers) to the Apigee Remote Service.
4. The Apigee Remote Service acts as a policy decision point (PDP) and advises the Envoy proxy to allow or deny access to the API consumer for the request.
5. A response is sent back to the Envoy proxy on Service B.
6. If the request call is allowed, the Envoy proxy on Service B forwards the traffic to Service B (client backend).

For a more detailed demonstration please see the demo below covering how to use Consul to deploy Apigee using external authorization.

»Configuration requirements

For a smooth integration of Apigee with Consul, complete the following prerequisites:

1. Install Apigee: Apigee can be provisioned in Google Cloud. For more information please refer to the Google Cloud documentation.
2. Install the Envoy adapter for Apigee: The Envoy adapter enables the remote service. Find more information in the Apigee Envoy adapter documentation.
3. Install HashiCorp Consul: For more information, please refer to the Consul installation guide.

Once you've completed these prerequisites, integrating Apigee with your Consul deployment you’ll need to use a Consul configuration entry called a service default. These entries encapsulate essential configuration parameters and dictate the behavior of each service within the service mesh.

A service default should be created for both the apigee-remote-service-envoy service and for any additional service that requires authorization from Apigee. Below are some examples of the service default files.

Service default for the apigee-remote-service-envoy service

The service default below is used to configure the apigee-remote-service as a grpc service.

Service default with external authorization extension for each destination service

As mentioned above, a service default should also be created for each service that has inbound traffic and requires authorization from Apigee. These service defaults include an EnvoyExtensions section that allows the Envoy filters to route requests to the apigee-remote-service-envoy, which acts as the external authorization system.

Each request is authorized by Apigee, which makes access control decisions based on the metadata extracted from the requests. The example service default below will ensure authorization is given by Apigee before inbound communication is allowed to the api service.

»Conclusion

In today’s modern cloud world, most customers have standardized their platform of choice for different market segments. This best-of-breed approach allows organizations to standardize around their preferred provider for cloud, identity, secrets management, observability, etc. This makes it mandatory that all vendors integrate well with their counterparts in the ecosystem.

Consul supports this model and integrates well to extend capabilities to many existing platforms. Whether customers are looking to integrate Consul with their existing secrets management and PKI platform like HashiCorp Vault, centralize observability with Datadog, or offload service-to-service authorization to Apigee, Consul has the flexibility to accommodate many environments.

For a full tutorial on how HashiCorp Consul integrates with Apigee follow the learn guide here.