Using Infrastructure as Code to Automate VMware Deployments

Over a decade ago, public cloud infrastructure was introduced to the market revolutionizing the speed and efficiency of software application delivery. Yet, some of the largest enterprises in the world still maintain the same practices for provisioning and managing infrastructure in their private data center as they did before the cloud revolution— operators closing developer created tickets by pointing-and-clicking in software GUI’s to provision virtual machines.

The tools that are now pervasively used to provision in cloud, such as HashiCorp Terraform, are extensible to be used in private data centers. For users of VMware vSphere and NSX-T, this blog will cover how Terraform can be used to improve operator workflows to take advantage of infrastructure as code for provisioning and managing VMware infrastructure.

»The Shift in Provisioning is Not Only for Cloud

Infrastructure as code is at the heart of provisioning for cloud infrastructure marking a significant shift away from monolithic point-and-click management tools. Infrastructure as code allows users to address concerns around scale, heterogeneity, and efficiency by automating processes through codification. Regardless of the type of infrastructure, infrastructure as code enables operators to take a programmatic approach to provisioning. With the logging, auditing, and versioning made possible with infrastructure as code, organizations have better insight into the exact, current state of their infrastructure. This removes reliance on error-prone manual practices and creates a single workflow for provisioning both cloud-based and private data center infrastructure.

Terraform has the advantage of being able to manage every major cloud and many SaaS offerings. This provides a single workflow to provision and maintain infrastructure and services from all of your vendors, making it not only easier to switch providers (e.g. from on premises VMware to cloud-based AWS), but also dramatically reduces the learning curve required for a new vendor or service.

This transition to infrastructure as code can seem daunting to those operators who have worked hard to understand the monolithic user-interfaces and abstractions of point-and-click GUI-based tools. Terraform aims to simplify this transition for operators with support for common VMware infrastructure through Terraform providers, codified infrastructure with a human-readable language, and pre-defined VMware modules to help operators get started quickly and easily.

»Terraform Providers for VMware

A Terraform Provider is responsible for understanding API interactions between and exposing the resources from a given Infrastructure, Platform, or SaaS offering to Terraform. For VMware, there are providers for vSphere, NSX-T, and vCloud Director, which can be used to manage many aspects of a VMware-based environment. Using the vSphere Provider with Terraform, for example, you can write a Terraform file that describes the Virtual Machine that you want, apply that file with Terraform and create that VM as you described without ever needing to log into the vSphere dashboard.

»Defining VMware Infrastructure with Code

You can write infrastructure as code for Terraform in JSON or HashiCorp Configuration Language (HCL). HCL is an easy to learn and write configuration language with a simple syntax that is both human and machine readable. HCL offers the benefit of being a single language to manage all of your infrastructure providers and tools.You can define infrastructure with HCL in a Terraform file, and then use Terraform to create and destroy that infrastructure as needed.

As an operator, when creating a vSphere configuration using Terraform, you would leverage the vSphere provider to define the desired topology of resources. Likewise, here is an example using the NSX-T provider.

The following HCL script is for an example vSphere environment:

provider "vsphere" {
  user           = "${var.vsphere_user}"
  password       = "${var.vsphere_password}"
  vsphere_server = "${var.vsphere_server}"

  # If you have a self-signed cert
  allow_unverified_ssl = true
}

data "vsphere_datacenter" "dc" {
  name = "dc1"
}

data "vsphere_datastore" "datastore" {
  name          = "datastore1"
  datacenter_id = "${data.vsphere_datacenter.dc.id}"
}

data "vsphere_resource_pool" "pool" {
  name          = "cluster1/Resources"
  datacenter_id = "${data.vsphere_datacenter.dc.id}"
}

data "vsphere_network" "network" {
  name          = "public"
  datacenter_id = "${data.vsphere_datacenter.dc.id}"
}

resource "vsphere_virtual_machine" "vm" {
  name             = "terraform-test"
  resource_pool_id = "${data.vsphere_resource_pool.pool.id}"
  datastore_id     = "${data.vsphere_datastore.datastore.id}"

  num_cpus = 2
  memory   = 1024
  guest_id = "other3xLinux64Guest"

  network_interface {
    network_id = "${data.vsphere_network.network.id}"
  }

  disk {
    label = "disk0"
    size  = 20
  }
}

You can see the provider credentials are passed in at the top of the script to connect to the vSphere account. Then, names and ID’s are given to relevant parts of the infrastructure such as the datacenter and datastore. Lastly, the virtual machine resource is defined.

Imagine you wanted to provision a vSphere virtual machine. You could either provision this within vSphere, pointing and clicking to create your desired topology, or you could describe your desired topology in code— expressing names, networking, disk sizes, and capacities in hardcoded values or to-be-defined variables. The advantage is that once codified this configuration can be treated the same way application developers treat code— it can be reviewed, versioned, tested, updated, and executed (when needed).

»Creating and using a Terraform Module for vSphere

Terraform has the concept of modules— a way to encapsulate infrastructure resources into a reusable format. Having a module for each environment and resource your organization uses creates a set of predefined resources to allow others to rapidly create infrastructure they need. To make this simple for those just getting started, we have created a standard Terraform module for vSphere. Here’s an example of creating a VMware virtual machine, but instead of creating it from scratch (like the first example), we’ve used the VMware virtual machine module:

module "virtual_machines" {
  source                     = "vancluever/terraform-vsphere-virtual-machine"
  version                    = "1.0.0"
  datacenter                 = "dc1"
  datastore                  = "datastore1"
  disk_size                  = "10"
  guest_id                   = "otherLinuxGuest"
  memory                     = "2048"
  network                    = "network1"
  resource_pool              = "cluster1/Resources"
  vm_count                   = "3"
  vm_name_prefix             = "srv"
}

The set of variables for the module allow for customization of the code without having to codify any other required components. This module takes in similar variables to the infrastructure as code example above, allowing for the same level of detail in defining your infrastructure, but doesn’t require the same overhead.

»Terraform workflow

Terraform open source provides a workflow for practitioners to get started with Infrastructure as code using a broad set of available providers. The Terraform module registry is available to share, store and take advantage of modules created by the community. You can fork publicly available modules into your private repository if you’re working on a closed source project, and you can customize both the code of the module itself or variables provided within the module.

Here is a simple workflow for using the vSphere module with Terraform OSS. It starts with grabbing a module from the public Terraform module registry, then editing that module, and finally, provisioning it from the CLI:

»Terraform Enterprise Workflow

As operations teams begin collaborating on common vSphere infrastructure, the transition to Terraform Enterprise (TFE) works well to suit those needs. TFE enables teams to delegate ownership of creating modules and sharing them within the organization. The below example is how team members can leverage the remote runs, variable and state storage, and private module registry to provision and manage a vSphere environment by many operators within the organization and ultimately be able to provide self-service configurations to developers and avoid a backlog of queued up requests.

TFE users have access to the same public module registry, with the ability to fork these public modules into private repositories, however, TFE users also have access to a private module registry. The private Terraform module registry offers an organization a centralized location for sharing, storing, and discovering modules only for use within that organization. Then the Configuration Designer offers a single graphical interface for discovering modules, combining them, and setting the variables within them to rapidly provision infrastructure without needing to understand the deeper details therein.

TFE users enjoy the same plan and apply phases of Terraform Open Source, with the addition of rich detail and logging of those phases available in the dashboard.

»Conclusion

Terraform enables a single workflow for managing your VMware environment, major clouds, and SaaS offerings. Moving away from point-and-click GUI’s to infrastructure as code enables a new agility for your development teams to bring applications to production without sacrificing the oversight of your operations team. To learn more about Terraform or get started today, go to hashicorp.com/terraform.