HashiCorp Consul 0.4

Today we are proudly releasing Consul 0.4. Consul is a solution for service discovery, configuration, and orchestration. Consul is completely distributed, highly available, and scales to thousands of nodes and services across multiple datacenters.

Consul 0.3 was released two months ago and we've been busy fixing bugs, making improvements, and adding new features.

The major new features added include a fine-grained ACL system, a watch mechanism to invoke callbacks on changes, an event system for custom events such as deploys or service restarts, and remote execution for simplified administration.

Consul 0.4 can be downloaded here, and the full changelog can be viewed here.

Read on to learn more about the major new features in 0.4.

»ACL System

One of the most highly requested features for Consul has been an easy to use ACL system for restricting access to the KV space. The approach taken by Consul is similar to IAM in AWS, and is very simple to use and integrate.

Consul ACLs can act in a blacklist mode, meaning all actions are permitted except those specifically denied, or in a whitelist mode, meaning all actions are denied except those specifically permitted.

Policy is set per ACL token, using either HCL or JSON. A policy might look like:

# Default all keys to read-only
key "" {
    policy = "read"
}

# Allow write access to the service folder
key "service/my-app/" {
    policy = "write"
}

# Deny all access to secrets!
key "secrets/" {
    policy = "deny"
}

Additionally, the Web UI has been updated to support ACL management:

This is only a brief look at ACLs in Consul, but there is detailed documentation available, which should be referenced for a more in-depth look.

»Watches

A common use case for Consul is to watch a list of nodes, services, keys, etc and then update an external application when there is a change. While this has always been possible using the APIs, Consul 0.4 introduces "watches", which simplify this use case.

A watch couples a "view" of data in Consul with a handler that is invoked on changes. Watches can be used to handle updates to KV data, services, nodes, health checks, user events and more. Watches can either be used by providing the proper configuration to an agent, or by using the new consul watch command.

As an example, to capture a list of nodes, the following watch could be used:

$ consul watch -type nodes /usr/local/bin/nodes-handler.sh

This will start a new watch for any changes to the list of nodes and automatically invoke the /usr/local/bin/nodes-handler.sh script on changes.

The use cases are far reaching: dynamic load balancer setup, DNS configuration, application configuration updates, deploy hooks, etc. The watch system makes it simple to interact with Consul in real time without writing code.

»Event System

Consul is built on top of Serf which provides a mechanism for reliably propagating events to a cluster. This feature is finally now available in Consul as well using the event command.

By using the peer-to-peer gossip layer, events can quickly be propogated to clusters of any size in a scalable and reliable way. Using the new watch features, any number of event handlers can be setup to handle these events.

For example, an application deploy handler could be setup using a watch:

$ consul watch -type event -name web-deploy /usr/local/bin/web-deploy.sh

Then using the event command the event can be fired:

$ consul event -name web-deploy sha:24323e0

Events also support filtering delivery by node name, services, and tags. This simplifies orchestration, as you can filter to the specific nodes that should handle the event.

»Remote Execution

The last major new feature of 0.4 is the exec command. This command allows for remote execution using the new event system.

Here is an example of calling uptime on a few nodes:

$ consul exec uptime
    node1:  16:40:22 up 2 days, 18:46,  1 user,  load average: 0.12, 0.10, 0.09
    node1:
==> node1: finished with exit code 0
    node5:  16:40:22 up 2 days, 18:48,  0 users,  load average: 0.01, 0.03, 0.05
    node5:
==> node5: finished with exit code 0
    node2:  16:40:22 up 2 days, 18:47,  0 users,  load average: 0.12, 0.07, 0.06
    node2:
==> node2: finished with exit code 0
    node3:  16:40:22 up 2 days, 18:47,  0 users,  load average: 0.08, 0.08, 0.06
    node3:
==> node3: finished with exit code 0
    node6:  16:40:22 up 2 days, 18:48,  0 users,  load average: 0.00, 0.01, 0.05
    node6:
==> node6: finished with exit code 0
    node4:  16:40:22 up 2 days, 18:47,  0 users,  load average: 0.00, 0.01, 0.05
    node4:
==> node4: finished with exit code 0
6 / 6 node(s) completed / acknowledged

Using the remote execution feature makes it easy to orchestrate a common action across an entire fleet, or any subset. Remote execution can filter by node name, services, or tags as well. This enables targeted actions like restarting apache only on the web servers, or bouncing the memcache nodes, etc.

»Roadmap

Work will continue on Consul as we march towards a 0.5 release. In the meantime, we will focus on stability and correctness of 0.4 and expect to have a few minor point releases to resolve any issues that are found.

Consul 0.5 will be focused on improving the service abstraction, deeper integration between the catalog and KV store, and richer locking and session primitives. In the meantime, we hope you enjoy Consul 0.4!