Message received

Faster, smarter, better service delivery

Criteo adopted HashiCorp Consul to automate and streamline its service discovery operations. Specifically, the company chose Consul for the tool's ability to connect and secure services across any runtime platform and any public or private cloud.

The lightweight service-based networking product provides a real-time, multi-platform directory of all running services to improve application inventory management. More importantly, it dynamically locates applications and infrastructure services to accelerate discovery, and automates network configuration to simplify connectivity without human intervention.

Along with HashiCorp Vault for database secrets management and HashiCorp Terraform for deploying AWS and Microsoft Azure resources for newly acquired cloud services, Consul forms the backbone of Criteo's entire operation. Souchay says that without Consul the rest of Criteo's operations won't work. In particular, he cites how the tool has freed his team to pursue higher-value activities that support the company's long-term business objectives.

"One of the biggest benefits of Consul is that it tells us where our services are, whether they're healthy or not, and shows us the shortest network path to reach them regardless of whether they're running on virtual machines or bare metal," Souchay says. "Consul has fully replaced our manual service discovery activities with automated workflows and we've repurposed most of our Consul operations staff to other projects because the tool is so reliable, efficient, and intelligent. We don't even work on discovery anymore unless there's a specific reason."

Resolving a potential mess with service mesh

Beyond automating discovery and connectivity among various microservices, Souchay says that Consul has also played a pivotal role in helping the organization enhance its service monitoring and overall performance across the enterprise as well.

In particular, the team uses Consul to support its open-source monitoring tool, Prometheus. The custom-built tooling exports Consul metrics into a Prometheus format that centralizes all infrastructure health metrics into a single tool for greater visibility across the environment. The solution enables Criteo engineers to observe Consul and make sure it's functioning at peak performance and capable of scaling horizontally to manage its other services spread across more than 45,000 bare metal servers and approximately 1,000 virtual machines.

A properly configured and fully functional Consul is key to Criteo's application of IoC — Inversion of Control — in which the service discovery solution interconnects all services with their probes and links services with their owners to automate discovery. This replaces a much more complicated process of having to assign individual services to run within a monitoring tool.

"Using Consul to align all our services and Prometheus to monitor Consul's performance creates a widespread service mesh that automates basic monitoring of production services and the availability of Service-Level Objectives," Souchay explains. "We're able to register and de-register services instantly, regularly check service health and availability, and automatically back up any upgrades or changes we've made."

Criteo is already planning for their future with service mesh using Consul and HAProxy. The Criteo team has developed an in-house tool that enables them to use HAProxy as a sidecar proxy for Consul. This would allow them to incorporate intentions, Consul's service mesh traffic routing capability, into their datacenters for automating TLS-encrypted connections between services. This is in the early stages of development for Criteo, but highlights their continued focus on innovation.

Pushing the envelope of efficiency, transparency, and performance

Automating infrastructure deployment and service discovery has paid huge dividends for Criteo. The increased business agility helped the company radically accelerate the development of new services from weeks to minutes and now boasts more than 4,000 service types running on its servers to meet virtually any and every customer need around the globe.

At the same time, the company has also used HashiCorp tools to replace a wide range of other infrastructure components, radically reducing its infrastructure footprint and helping save millions of dollars per year in infrastructure maintenance, licensing, and upgrade costs.

"HashiCorp tools have been instrumental in helping us gain the flexibility, speed, and scale our business needs to keep up with the changing demands of our customers," Souchay says. "We're eager to add more features to our Consul instance and push the envelope of what's possible with this level of automation and efficiency."