New Survey Shows Broad Cloud Waste — HashiCorp Terraform Can Help

The premise of the cloud is simple enough: it offers infrastructure on demand so you can grow as needed with nearly unlimited capacity. However, unlike with a private datacenter, many users in an organization may have the ability to provision cloud infrastructure, which can cause costs to quickly spiral out of control without governance and oversight. To solve this, operations teams can create a system that mimics private datacenters — where IT is the gatekeeper and must approve tickets for infrastructure — but that creates bottlenecks that diminish the cloud’s promises of speed and agility.

As more organizations move to the cloud or advance further into their cloud journeys, we’re seeing that cost continues to be a problem. In our 2022 State of Cloud Strategy Survey (commissioned by HashiCorp and conducted by Forrester Consulting) we found that 94% of respondents noted avoidable cloud spend, most commonly due to some combination of idle or underused resources (66%), overprovisioned resources (59%), and lack of needed skills (47%). This shouldn’t be surprising, other recent research also shows that anywhere from 32% to 70% of cloud budgets are wasted.

Source: a commissioned study conducted by Forrester Consulting on behalf of HashiCorp, August 2022

»What Causes Cloud Waste?

A wide variety of factors contribute to overspending in the cloud:

»Paying for Unutilized Cloud Infrastructure

A lack of oversight, visibility, and tracking around what developers can provision can lead to idle resources, overprovisioning, and orphaned resources. These costs can quickly inflate your cloud bill, without providing any real benefits.

Many companies pay for cloud resources that sit idle most of the time. For example, resources are typically paid for as if they were needed to run 24/7 — even in a dev/test environment. In reality, these resources are often consumed mostly during a 40-hour work week — and then sit idle for the other 128 hours of the week.

In addition, operators are often able to provision premium instances whether or not they are actually needed. Organizations may have no way to ensure that teams use a smaller instance during the development phase. Similarly, teams may be able to provision resources they end up never using.

Developers often fail to de-provision software or notify operations once they finish a project. In some cases they might simply forget, or perhaps the organization requires a long, complex process to get ops to de-provision unneeded resources, and other tasks take priority.

»Loss of Developer Productivity

Some cloud costs may not be immediately visible. There’s a huge but not immediately obvious impact when inefficient cloud processes mean that highly paid developers are unable to work efficiently in a multi-cloud environment. That can be caused by having to deal with nuances specific to multiple cloud service providers and hybrid cloud environments. For example, each cloud service provides different tooling, forcing teams to build expertise in multiple systems. To create effective multi-cloud teams that maximize productivity and minimize waste, they need to apply their skills consistently regardless of the environment.

»Security Risks Can Lead to Wasted Spend

Unnecessary cloud costs also accumulate from risks. While the old-school, ticket-based gatekeeper IT approach can create bottlenecks and limit developer productivity as noted above, its built-in controls can also help reduce risk. Compare that to a modern multi-cloud environment, where many proprietary tools and workflows align to a single cloud vendor — so there’s no single control plane or workflow. That makes it difficult to secure every aspect of your infrastructure; for example, multiple workflows and limited visibility into infrastructure can enable cloud sprawl and shadow IT deployments, making it difficult to know the current state of your infrastructure.

These security issues can take many developer hours to correct and may even lead to significant fines. In addition, negative cyber incidents such as security breaches can result in loss of reputation — all adding up to lost revenue.

»How HashiCorp Terraform Helps Cut Cloud Waste

Managing cloud costs is not easy, but there are tools and approaches that can help optimize cloud spending. HashiCorp Terraform helps organizations minimize cloud waste by providing efficient provisioning and infrastructure lifecycle management throughout the infrastructure automation journey.

In the first phase of the journey — adopting and establishing a provisioning workflow — Terraform enables infrastructure as code provisioning so organizations get automated provisioning when they need it and how they want it. That enables team collaboration and offers the ability to provision any infrastructure via APIs. Unified provisioning across providers solves the multiple workflow issues that emerge in multi-cloud environments that create inefficiencies, limit developer productivity, and ultimately drive up cloud costs. Unified provisioning also creates the foundation for better visibility into the sources of cloud spending — including idle or overprovisioned resources.

In the second phase — standardizing the workflow — Terraform minimizes redundant work with modules and provides ways to create the proper guardrails for security, compliance, and operational consistency. By allowing organizations to publish reusable, validated infrastructure as code components while HashiCorp’s Sentinel policy as code capabilities and Run Tasks define policies, Terraform helps organizations stay productive and enforce policies to limit unneeded infrastructure. And with Terraform Cloud’s new Drift Detection functionality, operators can be notified of any changes that occur in their infrastructure state. All of this reduces risk while maintaining best cost practices.

In the third and final phase — optimizing and operating at scale — Terraform helps to safely extend self-service provisioning to developers while keeping the proper guardrails in place. Cloud costs can be reined in using Terraform's management capabilities, including enabling proper policies and guardrails and creating ways to remediate policy violations.

Terraform also lets organizations use data to gather insights to help optimize their infrastructure. For example, operators can view an entire organization’s cloud spend to uncover overprovisioning, quickly deprovision un- or under-utilized resources, and create policies to enforce provisioning best practices going forward. Terraform‘s single source of truth makes it easier to see infrastructure changes and understand cloud spend, and also provides continuous management and governance.

»Conclusion

Unnecessary cloud costs are generated in a variety of ways, but one thing is clear — organizations need to carefully manage their cloud spending. According to the HashiCorp 2022 State of Cloud Strategy Survey, 90% of organizations believe that a multi-cloud strategy has helped advance or achieve their business goals. With that kind of momentum, a single platform to automate infrastructure is becoming critical to controlling your cloud bill.

Using Terraform to codify infrastructure and increase infrastructure visibility and tracking can help organizations reduce their cloud spend by more than 20% via:

• Real-time control and proactive policy enforcement
• Reduction of unnecessary cloud resources such as idle resources, overprovisioning, and orphaned resources
• Elimination of manual processes and bottlenecks
• Centralized management and control across technologies

To learn more about how to control cloud waste, see our page on how to Optimize Cloud Spending.

Finally, this post is one of several we’re publishing to highlight key takeaways from the 2022 HashiCorp State of Cloud Strategy Survey. You can also find deep dives into the essential role of platform teams, security in a multi-cloud world, how to address skills shortages, and more. For the full story, all the numbers, and the charts and graphs that make it all clear, visit the HashiCorp State of Cloud Strategy Survey site. That’s also where you'll find links to the Forrester Consulting Unlocking Multicloud’s Operational Potential study, free to download.