Wait Conditions in the Kubernetes Provider for HashiCorp Terraform
Discover how the Kubernetes and Kubernetes-alpha providers for HashiCorp Terraform allow users to specify whether they must meet certain conditions before they successfully apply a resource, giving users more control over their Kubernetes infrastructure.
The HashiCorp Terraform Strategic Integrations team is working hard to bring you a Kubernetes provider for HashiCorp Terraform with a better-than-native experience. We recently improved the wait_for configurations on several resources, as well as introduced an entirely new generic waiter. The new and improved configuration options allow you to specify whether Terraform should wait for a specific condition, or not, before continuing to apply your configuration or complete successfully.
»Wait For Rollout
The wait_for_rollout attribute is available on both the kubernetes_deployment and the kubernetes_stateful_set resources. The default value for wait_for_rollout is true, so if that’s the behavior you want, you don’t need to do anything. However, there are cases where you may not expect a rollout to complete before you’re finished applying your Kubernetes configuration. In these cases, you can set wait_for_rollout to false and the Kubernetes provider will move on after the Deployment or StatefulSet have been successfully submitted to Kubernetes. Below is an example configuration that demonstrates the usage of wait_for in a deployment resource.
terraform {
required_providers {
kubernetes = {
source = "hashicorp/kubernetes"
}
}
}
provider "kubernetes" {
config_path = "~/.kube/config"
}
resource "kubernetes_deployment" "nginx" {
metadata {
name = "example-nginx"
labels = {
App = "ExampleNGINX"
}
}
spec {
replicas = 2
selector {
match_labels = {
App = "ExampleNGINX"
}
}
template {
metadata {
labels = {
App = "ExampleNGINX"
}
}
spec {
container {
image = "nginx:1.7.8"
name = "example"
port {
container_port = 80
}
}
}
}
}
wait_for_rollout = false
}
»Wait For Load Balancer
The wait_for_load_balancer attribute is an option on the kubernetes_ingress resource. An Ingress controls access to services in the Kubernetes cluster. By default, wait_for_loadbalancer is set to false, which means the Kubernetes provider will move on after Kubernetes has accepted the Ingress object API request. When creating or modifying an Ingress with a load balancer configuration, some users want to make sure at least one endpoint is available through the load balancer before the Terraform apply succeeds, or moves on. When wait_for_load_balancer is set to true, that’s exactly what the Kubernetes provider will do.
terraform {
required_providers {
kubernetes = {
source = "hashicorp/kubernetes"
}
}
}
provider "kubernetes" {
config_path = "~/.kube/config"
}
resource "kubernetes_ingress" "example" {
metadata {
name = "example"
annotations = {
"ingress.kubernetes.io/rewrite-target" = "/"
}
}
spec {
backend {
service_name = "example"
service_port = 8080
}
rule {
host = "myminikube.info"
http {
path {
path = "/"
backend {
service_name = "exampleserver"
service_port = 8080
}
}
}
}
}
wait_for_load_balancer = true
}
»Wait For Completion
The last wait for attribute on the Kubernetes provider is wait_for_completion, which is available on the kubernetes_job resource. Jobs create Pods that run until a specific task either completes successfully or fails. This attribute ensures Terraform waits until the job has completed or failed before continuing.
terraform {
required_providers {
kubernetes = {
source = "hashicorp/kubernetes"
}
}
}
provider "kubernetes" {
config_path = "~/.kube/config"
}
resource "kubernetes_job" "test-pr" {
metadata {
name = "job-with-wait"
namespace = "default"
}
spec {
completions = 1
template {
metadata {}
spec {
container {
name = "sleep"
image = "busybox:latest"
command = ["sleep", "30"]
}
restart_policy = "Never"
}
}
}
wait_for_completion = true
timeouts {
create = "40s"
}
}
»The New Generic Wait
The Kubernetes-alpha provider introduces a new, generic kubernetes_manifest resource and with it, a new generic wait_for attribute. Unlike the previous wait_for_x attributes, the new generic wait_for configuration gives users more control over wait conditions. This new attribute instructs the Terraform provider to wait on a condition based on any attribute of a Kubernetes object.
For example, you can specify that Terraform should consider a deployment with 10 replicas complete when two of the pods enter the ready state, and the ingress has an IP address assigned. Keep in mind that the wait_for attribute waits for all conditions to be met simultaneously. So, in this example, if the deployment’s readyReplicas value is greater than two before the ingress has an IP address assigned, the conditions won’t be met and the provider will eventually timeout. Currently, the Kubernetes-alpha provider doesn’t support a configurable timeout. To address this, you can also use regular expressions in your conditions.
terraform {
required_providers {
kubernetes = {
source = "hashicorp/kubernetes-alpha"
}
}
}
provider "kubernetes-alpha" {
config_path = "~/.kube/config"
}
resource "kubernetes_manifest" "example" {
provider = kubernetes-alpha
manifest = {
apiVersion = "v1"
kind = "Pod"
metadata = {
name = "example-pod"
namespace = "default"
annotations = {
"test.terraform.io" = "test"
}
labels = {
app = "nginx"
}
}
spec = {
containers = [
{
name = "nginx"
image = "nginx:1.19"
readinessProbe = {
initialDelaySeconds = 10
httpGet = {
path = "/"
port = 80
}
}
}
]
}
}
wait_for = {
fields = {
"metadata.annotations[\"test.terraform.io\"]" = "test",
"status.containerStatuses[0].restartCount" = "0",
"status.containerStatuses[0].ready" = "true",
"status.podIP" = "^(\\d+(\\.|$)){4}",
"status.phase" = "Running",
}
}
}
For more information on the new Kubernetes Provider, visit the Terraform Registry. To discover more about managing Kubernetes with Terraform, review the guides on HashiCorp Learn.
We would love to hear your feedback and expand on these projects! You can post bugs and feature requests for the Kubernetes-alpha provider by opening an issue at hashicorp/terraform-provider-kubernetes-alpha, or the Kubernetes provider at hashicorp/terraform-provider-kubernetes.
You can also engage with us and the community on HashiCorp Discuss and in #terraform-providers on the Kubernetes Slack (Sign up here).
Sign up for the latest HashiCorp news
More blog posts like this one
Fix the developers vs. security conflict by shifting further left
Resolve the friction between dev and security teams with platform-led workflows that make cloud security seamless and scalable.
HashiCorp at AWS re:Invent: Your blueprint to cloud success
If you’re attending AWS re:Invent in Las Vegas, Dec. 2 - Dec. 6th, visit us for breakout sessions, expert talks, and product demos to learn how to take a unified approach to Infrastructure and Security Lifecycle Management.
Speed up app delivery with automated cancellation of plan-only Terraform runs
Automatic cancellation of plan-only runs allows customers to easily cancel any unfinished runs for outdated commits to speed up application delivery.