The producer-consumer infrastructure model

Multi-cloud adoption continues to grow in the enterprise, and according to the 2022 HashiCorp State of Cloud Strategy Survey, that’s a good thing. In fact, 90% of survey respondents say multi-cloud is helping them achieve their business goals. At the same time, however, the complexity of managing multiple cloud-based technologies, applications and APIs, not to mention developing standardized processes around them, breeds deep operational challenges.

In that environment, manually managing IT infrastructure simply doesn’t scale. For one thing, the HashiCorp Survey also confirmed the ongoing shortage of cloud professionals. In fact, skills shortages were the No. 1 factor complicating the ability to operationalize multi-cloud. But while the skills shortage is a well known problem, there’s another related issue that doesn’t get the attention it deserves: Different infrastructure users face different challenges.

You don’t have to dig too deep to see two distinct personas that interact with the provisioning workflow — infrastructure producers and infrastructure consumers have very different concerns and need very different functionality from their infrastructure automation solutions.

»Infrastructure producers vs. infrastructure consumers

First, infrastructure producers or publishers are the folks who create infrastructure for others to use. An infrastructure producer might be a site reliability engineer (SRE) or an operator, primarily concerned with how to most efficiently provision and manage infrastructure. They are responsible for provisioning cloud resources, with repeatability for several teams, and must have a way to manage provisioned resources during their entire life cycle.

Second, infrastructure consumers are the people responsible for actually building applications. These folks are usually developers, whose main concern is deploying applications. Infrastructure consumers do not typically care about the infrastructure, they just want to consume infrastructure easily so they can focus on deploying their applications.

As the two main stakeholders in the infrastructure provisioning process, these two personas have different requirements and responsibilities. At the same time, tight collaboration between the two groups is integral to an optimal DevOps process.

»The producer-consumer infrastructure model

Infrastructure producers need to provision and manage infrastructure, so they need a reusable infrastructure to be easily accessible for the infrastructure consumer.

Infrastructure consumers are concerned with tying in the workflow that the ops team has provided. They just want to be able to self-serve resources created and published by operators to build a workspace of resources for development, testing and production stages.

So, how can the needs of both groups be met at the same time? The answer is the producer-consumer model, which looks like this:

1. The infrastructure producer defines and publishes infrastructure (typically best-practice modules), then makes it discoverable for infrastructure consumers in a central repository.
2. Infrastructure consumers find and provision infrastructure.

More specifically, infrastructure producers need a language that can be codified, set up in a reusable fashion and consumed by infrastructure developers. They don’t want to be manually configuring individual servers and dealing with complications that come with that.

Infrastructure consumers, meanwhile, need to be able to build and maintain the applications the business is built on. They require reliable infrastructure to execute on, flexibility to program in a language that they like and they should be able to provision without having to code at all.

To meet the needs of both groups, vendors must provide a flexible infrastructure as code solution. That means serving an operator who is not traditionally a coder, but also taking a programmatic approach to creating and managing infrastructure. It also means enabling an infrastructure consumer who needs to deploy infrastructure but may not be familiar with coding.

HashiCorp Terraform offers a single workflow that seamlessly unifies the producer-consumer model to automate infrastructure across the whole organization. Terraform delivers everything that both infrastructure producers and consumers need to get started with infrastructure as code. More importantly, Terraform provides an infrastructure as code solution that addresses the needs of both personas, taking into account their different levels of expertise.

• Infrastructure producers can use HashiCorp Configuration Language (HCL) to create and publish approved infrastructure.
• Infrastructure consumers can use Cloud Development Kit for Terraform if they prefer to codify their own infrastructure as code.
• Infrastructure consumers also have access to a no-code provisioning workflow via the Terraform Cloud/Enterprise private module registry.

Clearly, different personas have different needs when it comes to provisioning infrastructure, and the producer-consumer model addresses the needs of both types of users at their respective stages of the infrastructure automation journey.

A version of this article was originally published on The New Stack.