Unlocking the Cloud Operating Model: Cloud Compliance & Management

Overview

To thrive in an era of multi-cloud architecture, driven by digital transformation, Enterprise IT must evolve from ITIL-based gatekeeping to enabling shared self-service processes for DevOps excellence.

For most enterprises, the goals of digital transformation efforts mean delivering new business and customer value more quickly, and at a very large scale. The implication for Enterprise IT then is a shift from cost optimization to speed optimization. The cloud is an inevitable part of this shift as it presents the opportunity to rapidly deploy on-demand services with limitless scale.

To unlock the fastest path to value of the cloud, enterprises must consider how to industrialize the application delivery process across each layer of the cloud: embracing the cloud operating model, and tuning people, process, and tools to it.

In this white paper, we look at the implications of the cloud operating model, and present solutions for IT teams to adopt this model across the infrastructure provisioning layer.

• Have dynamic usage characteristics, needing to scale loads up and down by orders of magnitude during short time periods.
• Be under pressure to quickly build and iterate. Many of these new systems may be ephemeral in nature, delivering a specific user experience around an event or campaign.

For most enterprises though, these systems of engagement must connect to existing “systems of record”—the core business databases and internal applications, which often continue to reside on infrastructure in existing data centers. As a result, enterprises end up with a hybrid—a mix of multiple public and private cloud environments.

The essential implications of the transition to the cloud is the shift from “static” infrastructure to “dynamic” infrastructure: from a focus on configuration, and management of a static fleet of IT resources, to provisioning, securing, connecting, and running dynamic resources on demand.

To address these challenges, then those teams must ask the following questions:

• People. How can we enable a team for a multi-cloud reality, where skills can be applied consistently regardless of target environment?
• Process. How do we position central IT services as a self-service enabler of speed, versus a ticket-based gatekeeper of control, while retaining compliance and governance?
• Tools. How do we best unlock the value of the available capabilities of the cloud providers in pursuit of better customer and business value?

A common starting point for the Cloud Operating Model is to enable the operations team to shift their focus away from provisioning only dedicated servers based on homogenous sets of infrastructure to workflows that enable shift left IT with capacity on-demand from a variety of Cloud (and service) providers.

To address the core tenets affecting provisioning (volume and distribution of services, ephemerality and immutability, and deploying to multiple target environments), organizations are moving to an automation based operating model for cloud infrastructure.

As the implications of the cloud operating model impact teams across infrastructure, security, networking, and applications, we see a repeating pattern amongst enterprises of establishing central shared services–centers of excellence–to deliver the dynamic infrastructure necessary at each layer for successful application delivery.

As teams deliver on each shared service to for the cloud operating model, IT velocity increases. The greater cloud maturity an organization has, the faster its velocity.

Multi-Cloud Infrastructure Provisioning

The foundation for adopting the cloud is infrastructure provisioning. HashiCorp Terraform is the world’s most widely used cloud provisioning product and can be used to provision infrastructure for any application using an array of providers for any target platform.

To achieve shared services for infrastructure provisioning, IT teams should start by implementing reproducible infrastructure as code practices, and then layering compliance and governance workflows to ensure appropriate controls.

Reproducible Infrastructure as Code

The first goal of a shared service for infrastructure provisioning is to enable the delivery of reproducible infrastructure as code, providing DevOps teams a way to plan and provision resources inside CI/CD workflows using familiar tools throughout.

DevOps teams can create Terraform templates that express the configuration of services from one or more cloud platforms. Terraform integrates with all major configuration management tools to allow fine grained provisioning to be handled following the provisioning of the underlying resources. Finally, templates can be extended with services from many other ISV providers to include monitoring agents, application performance monitoring (APM) systems, security tooling, DNS, and Content Delivery Networks, and more. Once defined, the templates can be provisioned as required in an automated way. In doing so, Terraform becomes the lingua franca and common workflow for teams provisioning resources across public and private cloud.

For self-service IT, the decoupling of the template-creation process and the provisioning process greatly reduces the time taken for any application to go live since developers no longer need to wait for operations approval, as long as they use a pre-approved template.

Compliance and Management

For most teams, there is also a need to enforce policies on the type of infrastructure created, how it is used, and which teams get to use it. HashiCorp’s Sentinel policy as code framework provides compliance and governance without requiring a shift in the overall team workflow, and is defined as code too, enabling collaboration and comprehension for DevSecOps.

Without policy as code, organizations resort to using a ticket based review process to approve changes. This results in developers waiting weeks or longer to provision infrastructure and becomes a bottleneck. Policy as code allows us to solve this by splitting the definition of the policy from the execution of the policy.

Centralized teams codify policies enforcing security, compliance, and operational best practices across all cloud provisioning. Automated enforcement of policies ensures changes are in compliance without creating a manual review bottleneck.

Decomposing these implications, various changes of approach are implied: The infrastructure layer transitions from running dedicated servers at limited scale to a dynamic environment where organizations can easily adjust to increased demand by spinning up thousands of servers and scaling them down when not in use. As architectures and services become more distributed, the sheer volume of compute nodes increases significantly.

Additionally, each cloud provider has its own solution to these challenges. For enterprise IT teams, these shifts in approach are compounded by the realities of running on hybrid- and multicloud infrastructures and the varying tools each technology provides.

When you push changes to a connected VCS repository, Terraform will automatically trigger a plan in any workspace connected to that repository. This plan can be reviewed for safety and accuracy in the Terraform UI, then it can be applied to provision the specified infrastructure.

Continuous Integration, Continuous Delivery (CI/CD) Pipeline

Terraform can be called from within most CI/CD pipelines such as Jenkins, Circle, Travis, and Gitlab. Many users leverage the programmability of Terraform to automate as much of their provisioning workflow as possible, while enforcing guardrails through policy as code. Terraform’s API-driven run provides flexible provisioning workflows using an infrastructure as code approach that any organization can manage. A continuous integration (CI) system monitors changes in Terraform code and drives provisioning using Terraform Cloud's REST API. This approach allows organizations to implement a range of actions in their CI pipeline as part of an infrastructure provisioning workflow and still benefit from Terraform Cloud's capabilities such as private modules, state management, policy as code (Sentinel) and more.

IT Service Management (ITSM)

Terraform Enterprise also includes a first class ServiceNow integration. ServiceNow provides digital workflow management, helping teams work quickly and efficiently with one another by offering a straightforward workflow for their interactions. The ServiceNow Service Catalog offers a storefront of services that can be ordered by different people in the organization. One common request between teams is for Cloud resources: a developer needs a fleet of machines to test out a codebase or the IT team in finance has a request for infrastructure to run their new accounting software. For organizations who use the ServiceNow Service Catalog, the requests can be submitted through ServiceNow and routed to the right team for Cloud Infrastructure.

Terraform Enterprise provides provisioning automation through infrastructure as code and security, compliance, and cost-sensitive policy enforcement against all resources as they are provisioned.

Our newest integration connects the human workflow power of ServiceNow with the infrastructure workflow capabilities of Terraform Enterprise. This enables teams that are not code-centric to safely adopt best-in-class provisioning workflows and tooling, while still getting developing competency with infrastructure as code.

As organizations begin to adopt the cloud operating model, they first face the challenge of provisioning cloud infrastructure. Many organizations believe that even if the cloud unlocks new speed, it also opens up new security risks, which can only be solved through processes that reduce that unlocked speed. However, Terraform offers a powerful alternative to that legacy model by combining an infrastructure as code approach to provisioning with a policy as code approach to compliance and management. This enables organizations to have both high agility and high control as they develop competency in infrastructure provisioning, compliance and management. This paper provides an overview of how organizations can leverage Terraform Enterprise to implement multi-cloud compliance and management ensuring effective, unencumbered adoption of the cloud operating model.