Wait Conditions in the Kubernetes Provider for HashiCorp Terraform
Discover how the Kubernetes and Kubernetes-alpha providers for HashiCorp Terraform allow users to specify whether they must meet certain conditions before they successfully apply a resource, giving users more control over their Kubernetes infrastructure.
The HashiCorp Terraform Strategic Integrations team is working hard to bring you a Kubernetes provider for HashiCorp Terraform with a better-than-native experience. We recently improved the wait_for configurations on several resources, as well as introduced an entirely new generic waiter. The new and improved configuration options allow you to specify whether Terraform should wait for a specific condition, or not, before continuing to apply your configuration or complete successfully.
»Wait For Rollout
The wait_for_rollout attribute is available on both the kubernetes_deployment and the kubernetes_stateful_set resources. The default value for wait_for_rollout is true, so if that’s the behavior you want, you don’t need to do anything. However, there are cases where you may not expect a rollout to complete before you’re finished applying your Kubernetes configuration. In these cases, you can set wait_for_rollout to false and the Kubernetes provider will move on after the Deployment or StatefulSet have been successfully submitted to Kubernetes. Below is an example configuration that demonstrates the usage of wait_for in a deployment resource.
terraform {
required_providers {
kubernetes = {
source = "hashicorp/kubernetes"
}
}
}
provider "kubernetes" {
config_path = "~/.kube/config"
}
resource "kubernetes_deployment" "nginx" {
metadata {
name = "example-nginx"
labels = {
App = "ExampleNGINX"
}
}
spec {
replicas = 2
selector {
match_labels = {
App = "ExampleNGINX"
}
}
template {
metadata {
labels = {
App = "ExampleNGINX"
}
}
spec {
container {
image = "nginx:1.7.8"
name = "example"
port {
container_port = 80
}
}
}
}
}
wait_for_rollout = false
}
»Wait For Load Balancer
The wait_for_load_balancer attribute is an option on the kubernetes_ingress resource. An Ingress controls access to services in the Kubernetes cluster. By default, wait_for_loadbalancer is set to false, which means the Kubernetes provider will move on after Kubernetes has accepted the Ingress object API request. When creating or modifying an Ingress with a load balancer configuration, some users want to make sure at least one endpoint is available through the load balancer before the Terraform apply succeeds, or moves on. When wait_for_load_balancer is set to true, that’s exactly what the Kubernetes provider will do.
terraform {
required_providers {
kubernetes = {
source = "hashicorp/kubernetes"
}
}
}
provider "kubernetes" {
config_path = "~/.kube/config"
}
resource "kubernetes_ingress" "example" {
metadata {
name = "example"
annotations = {
"ingress.kubernetes.io/rewrite-target" = "/"
}
}
spec {
backend {
service_name = "example"
service_port = 8080
}
rule {
host = "myminikube.info"
http {
path {
path = "/"
backend {
service_name = "exampleserver"
service_port = 8080
}
}
}
}
}
wait_for_load_balancer = true
}
»Wait For Completion
The last wait for attribute on the Kubernetes provider is wait_for_completion, which is available on the kubernetes_job resource. Jobs create Pods that run until a specific task either completes successfully or fails. This attribute ensures Terraform waits until the job has completed or failed before continuing.
terraform {
required_providers {
kubernetes = {
source = "hashicorp/kubernetes"
}
}
}
provider "kubernetes" {
config_path = "~/.kube/config"
}
resource "kubernetes_job" "test-pr" {
metadata {
name = "job-with-wait"
namespace = "default"
}
spec {
completions = 1
template {
metadata {}
spec {
container {
name = "sleep"
image = "busybox:latest"
command = ["sleep", "30"]
}
restart_policy = "Never"
}
}
}
wait_for_completion = true
timeouts {
create = "40s"
}
}
»The New Generic Wait
The Kubernetes-alpha provider introduces a new, generic kubernetes_manifest resource and with it, a new generic wait_for attribute. Unlike the previous wait_for_x attributes, the new generic wait_for configuration gives users more control over wait conditions. This new attribute instructs the Terraform provider to wait on a condition based on any attribute of a Kubernetes object.
For example, you can specify that Terraform should consider a deployment with 10 replicas complete when two of the pods enter the ready state, and the ingress has an IP address assigned. Keep in mind that the wait_for attribute waits for all conditions to be met simultaneously. So, in this example, if the deployment’s readyReplicas value is greater than two before the ingress has an IP address assigned, the conditions won’t be met and the provider will eventually timeout. Currently, the Kubernetes-alpha provider doesn’t support a configurable timeout. To address this, you can also use regular expressions in your conditions.
terraform {
required_providers {
kubernetes = {
source = "hashicorp/kubernetes-alpha"
}
}
}
provider "kubernetes-alpha" {
config_path = "~/.kube/config"
}
resource "kubernetes_manifest" "example" {
provider = kubernetes-alpha
manifest = {
apiVersion = "v1"
kind = "Pod"
metadata = {
name = "example-pod"
namespace = "default"
annotations = {
"test.terraform.io" = "test"
}
labels = {
app = "nginx"
}
}
spec = {
containers = [
{
name = "nginx"
image = "nginx:1.19"
readinessProbe = {
initialDelaySeconds = 10
httpGet = {
path = "/"
port = 80
}
}
}
]
}
}
wait_for = {
fields = {
"metadata.annotations[\"test.terraform.io\"]" = "test",
"status.containerStatuses[0].restartCount" = "0",
"status.containerStatuses[0].ready" = "true",
"status.podIP" = "^(\\d+(\\.|$)){4}",
"status.phase" = "Running",
}
}
}
For more information on the new Kubernetes Provider, visit the Terraform Registry. To discover more about managing Kubernetes with Terraform, review the guides on HashiCorp Learn.
We would love to hear your feedback and expand on these projects! You can post bugs and feature requests for the Kubernetes-alpha provider by opening an issue at hashicorp/terraform-provider-kubernetes-alpha, or the Kubernetes provider at hashicorp/terraform-provider-kubernetes.
You can also engage with us and the community on HashiCorp Discuss and in #terraform-providers on the Kubernetes Slack (Sign up here).
Sign up for the latest HashiCorp news
More blog posts like this one
New Terraform integrations with Crowdstrike, Datadog, JFrog, Red Hat, and more
12 new Terraform integrations from 9 partners provide more options to automate and secure cloud infrastructure management.
Terraform delivers launch-day support for Amazon S3 Tables, EKS Hybrid Nodes, and more at re:Invent
The Terraform provider for AWS now enables users to manage a variety of new services just announced at re:Invent.
HashiCorp at re:Invent 2024: Infrastructure Lifecycle Management with AWS
A recap of HashiCorp infrastructure news and developments on AWS from the past year, from a new provider launch to simplifying infrastructure provisioning and more.