Products & Technology

Automate AWS deployments with HCP Terraform and GitHub Actions

Learn how to use GitHub Actions to automate HCP Terraform operations.

Sep 09 2024Saravanan Gnanaguru

Saravanan Gnanaguru is a HashiCorp Ambassador

Using GitHub Actions with HashiCorp Terraform to automate infrastructure as code workflows directly from version control is a popular early path for many developer teams. However, this setup can make it difficult to stop configuration drift as your infrastructure codebase grows.

Rather than running Terraform on the GitHub Actions instance runner, it’s much easier and safer to run configurations remotely via HCP Terraform. This ensures that the creation, modification, and deletion of Terraform resources is handled on a managed cloud platform rather than on the GitHub Actions runner. HCP Terraform has many more systems and safeguards for team Terraform management and drift prevention.

This post shows how to use HCP Terraform to define AWS infrastructure and GitHub Actions to automate infrastructure changes. You’ll learn how to set up a GitHub Actions workflow that interacts with HCP Terraform to automate the deployment of AWS infrastructure, such as Amazon EC2 instances.

»Workflow overview

For this tutorial you can use your own Terraform configuration in a GitHub repository, or use this example repository. The example repository’s GitHub Actions include a workflow that creates the AWS resources defined in the repository. Whenever the repository trigger event happens on the main branch, it runs the workflow defined in the .github/workflows directory. It then performs the infrastructure creation or management in AWS. The figure below outlines the interaction between the GitHub repository, Actions, HCP Terraform, and AWS.

HCP Terraform and GitHub Actions workflow

Here’s how to implement this workflow.

»Prerequisites

Ensure you have the following:

An AWS account with necessary permissions to create resources.
A HCP Terraform account, with a workspace set up for this tutorial.
A GitHub account, with a repository for Terraform configuration files.
The Terraform CLI installed on your local machine for testing purposes.

Follow the following steps below to add AWS credentials in the HCP Terraform workspace and the TF_API_TOKEN in GitHub Actions secrets.

»Securely access AWS from HCP Terraform

HCP Terraform’s dynamic provider credentials allow Terraform runs to assume an IAM role through native OpenID Connect (OIDC) integration and obtain temporary security credentials for each run. These AWS credentials allow you to call AWS APIs that the IAM role has access to at runtime. These credentials are usable for only one hour by default, so their usefulness to an attacker is limited.

For more on how to securely access AWS from HCP Terraform with OIDC federation, check out the Access AWS from HCP Terraform with OIDC federation blog.

»Add HCP Terraform token to GitHub Actions

Fetch the TF_API_TOKEN by following instructions available in the HCP Terraform documentation. This example creates a user API token for the GitHub Action workflow.

Open the GitHub repository with the Terraform configuration.

Click on "Settings" in the repository menu. From the left sidebar, select "Secrets" and then choose "Actions".

To add a new repository secret, click on "New repository secret". Name the secret TF_API_TOKEN and add the HCP Terraform API token to the “Value” field. Click "Add secret" to save the new secret.

Naming the secret

By following these steps, you will securely provide your AWS credentials to HCP Terraform and also provide the HCP Terraform API token to GitHub Actions, enabling automated infrastructure deployment through a GitHub Actions workflow.

»Create the GitHub Actions workflow

After setting up the credentials, add a GitHub Actions workflow to your repository. The example repository uses a workflow YAML file defining one job with four steps to initialize, plan, apply, and destroy Terraform. This workflow uses the HashiCorp official marketplace actions for performing the Terraform command operations.

# This workflow will create AWS resource using HCP Terraform
# It is reusable workflow that can be called in other workflows
 
name: AWS Infra Creation Using in HCP Terraform
 
on:
 workflow_call:
   secrets:
       TF_API_TOKEN:
           required: true
 push:
   branches: [ "main" ]
 pull_request:
   branches: [ "main" ]
 workflow_dispatch:
 
env:
 tfcode_path: tfcloud_samples/amazon_ec2
 tfc_organisation: demo-tf-org # Replace it with your TFC Org
 tfc_hostname: app.terraform.io
 tfc_workspace: demo-tf-workspace # Replace it with your TFC Workspace
 
jobs:
 aws_tfc_job:
   name: Create AWS Infra Using TFC
 
   runs-on: ubuntu-latest
 
   steps:
   - name: Checkout tf code in runner environment
     uses: actions/checkout@v3.5.2
 
   # Configure HCP Terraform API token, since we are using remote backend option of HCP Terraform in AWS code
   - name: Setup Terraform CLI
     uses: hashicorp/setup-terraform@v2.0.2
     with:
       cli_config_credentials_token: ${{ secrets.TF_API_TOKEN }}
 
   # Add the AWS Creds as ENV variable in HCP Terraform workspace, since the tf run happens in HCP Terraform environment
 
   # Invoke the Terraform commands
   - name: Terraform init and validate
     run: |
       echo `pwd`
       echo "** Running Terraform Init**"
       terraform init
        
       echo "** Running Terraform Validate**"
       terraform validate
     working-directory: ${{ env.tfcode_path }}
 
   - name: Terraform Plan
     uses: hashicorp/tfc-workflows-github/actions/create-run@v1.3.0
     id: run
     with:
       workspace: ${{ env.tfc_workspace }}
       plan_only: true
       message: "Plan Run from GitHub Actions"
       ## Can specify hostname,token,organization as direct inputs
       hostname: ${{ env.tfc_hostname }}
       token: ${{ secrets.TF_API_TOKEN }}
       organization: ${{ env.tfc_organisation }}
 
   - name: Terraform Plan Output
     uses: hashicorp/tfc-workflows-github/actions/plan-output@v1.3.0
     id: plan-output
     with:
       hostname: ${{ env.tfc_hostname }}
       token: ${{ secrets.TF_API_TOKEN }}
       organization: ${{ env.tfc_organisation }}
       plan: ${{ steps.run.outputs.plan_id }}
  
   - name: Reference Plan Output
     run: |
       echo "Plan status: ${{ steps.plan-output.outputs.plan_status }}"
       echo "Resources to Add: ${{ steps.plan-output.outputs.add }}"
       echo "Resources to Change: ${{ steps.plan-output.outputs.change }}"
       echo "Resources to Destroy: ${{ steps.plan-output.outputs.destroy }}"
 
 # Once the user verifies the Terraform Plan, the user can run the Terraform Apply and Destroy commands
 apply_terraform_plan:
     needs: aws_tfc_job
     if: github.event_name == 'workflow_dispatch'
     runs-on: ubuntu-latest
     steps:
     - name: Checkout
       uses: actions/checkout@v3.5.2
     - name: Setup Terraform CLI
       uses: hashicorp/setup-terraform@v2.0.2
       with:
         cli_config_credentials_token: ${{ secrets.TF_API_TOKEN }}
 
     # Invoke the Terraform commands
     - name: Terraform init and validate
       run: |
         echo `pwd`
         echo "** Running Terraform Init**"
         terraform init
      
         echo "** Running Terraform Validate**"
         terraform validate
       working-directory: ${{ env.tfcode_path }}
    
     - name: Terraform Apply
       run: echo "** Running Terraform Apply**"; terraform apply -auto-approve
       working-directory: ${{ env.tfcode_path }}
      - name: Terraform Destroy
       run: echo "** Running Terraform Destroy**"; terraform destroy -auto-approve
       working-directory: ${{ env.tfcode_path }}

Let’s review each section of the workflow.

»Define the triggers

When you push commits or open a pull request on the main branch, the workflow initializes, plans, and applies Terraform. This workflow can be triggered by a:

workflow_call: This allows the workflow to be reused in other workflows. It requires the TF_API_TOKEN secret.
push: Triggers the workflow when there is a push to the main branch.
pull_request: Triggers the workflow when a pull request is made to the main branch.
workflow_dispatch: Allows the GitHub Actions interface to manually trigger the workflow.

# This workflow will create AWS resource using HCP Terraform
# It is reusable workflow that can be called in other workflows
 
name: AWS Infra Creation Using in HCP Terraform
 
on:
 workflow_call:
   secrets:
       TF_API_TOKEN:
           required: true
 push:
   branches: [ "main" ]
 pull_request:
   branches: [ "main" ]
 workflow_dispatch:

»Configure environment variables

Set the tfcode_path environment variable to specify the location of your Terraform configuration files within the repository. Include the HCP Terraform organization, workspace, and hostname for future jobs.

env:
 tfcode_path: tfcloud_samples/amazon_ec2 # Directory in which the tf files are stored
 tfc_organisation: demo-tf-org # Replace it with your HCP Terraform Org
 tfc_hostname: app.terraform.io
 tfc_workspace: demo-tf-workspace # Replace it with your HCP Terraform Workspace

»Define jobs

In the GitHub Actions workflow, define each automation step inside the jobs block. The job consists of job name and workflow runner instance definition in the runs-on block. This workflow uses the ubuntu-latest instance runner.

The first step in the apply_terraform_plan is the actions/checkout entry, which clones the repository into the GitHub Actions runner. This makes the Terraform configuration files available for subsequent steps.

jobs:
 aws_tfc_job:
   name: Create AWS Infra Using HCPTF
 
   runs-on: ubuntu-latest
 
   steps:
   - name: Checkout tf code in runner environment
     uses: actions/checkout@v3.5.2

In the second step, use the hashicorp/setup-terraform pre-built action to configure the Terraform CLI in the runner environment. It sets up the HCP Terraform API token (TF_API_TOKEN) for authentication. This token allows the Terraform CLI to communicate with HCP Terraform, enabling it to manage state, perform operations, and apply configurations in the context of the HCP Terraform workspace.

    - name: Setup Terraform CLI
      uses: hashicorp/setup-terraform@v2.0.2
      with:
        cli_config_credentials_token: ${{ secrets.TF_API_TOKEN }}

Next, initialize and validate Terraform. terraform init initializes the Terraform working directory by downloading necessary providers and initializing backends. If you’re using HCP Terraform as the backend, this command configures the workspace and prepares it for operations.

terraform validate checks the syntax and validity of the Terraform files, ensuring they are correctly formatted and logically sound. These commands run inside the working directory defined by env.tfcode_path, which contains the Terraform configuration.

    - name: Terraform init and validate
      run: |
        echo `pwd`
        echo "** Running Terraform Init**"
        terraform init
          
        echo "** Running Terraform Validate**"
        terraform validate
      working-directory: ${{ env.tfcode_path }}

In general, you will want to review the terraform plan before applying to verify the changes Terraform will make. Use the hashicorp/tfc-workflows-github/actions/create-run action to run a plan in HCP Terraform and export the plan using the plan_only attribute. Then, use the hashicorp/tfc-workflows-github/actions/plan-output action to get the output of the Terraform plan using the plan_id from the previous step's output. Finally, print the plan status and resource changes in the workflow’s output.

   - name: Terraform Plan
     uses: hashicorp/tfc-workflows-github/actions/create-run@v1.3.0
     id: run
     with:
       workspace: ${{ env.tfc_workspace }}
       plan_only: true
       message: "Plan Run from GitHub Actions"
       hostname: ${{ env.tfc_hostname }}
       token: ${{ secrets.TF_API_TOKEN }}
       organization: ${{ env.tfc_organisation }}
 
   - name: Terraform Plan Output
     uses: hashicorp/tfc-workflows-github/actions/plan-output@v1.3.0
     id: plan-output
     with:
       hostname: ${{ env.tfc_hostname }}
       token: ${{ secrets.TF_API_TOKEN }}
       organization: ${{ env.tfc_organisation }}
       plan: ${{ steps.run.outputs.plan_id }}
  
   - name: Reference Plan Output
     run: |
       echo "Plan status: ${{ steps.plan-output.outputs.plan_status }}"
       echo "Resources to Add: ${{ steps.plan-output.outputs.add }}"
       echo "Resources to Change: ${{ steps.plan-output.outputs.change }}"
       echo "Resources to Destroy: ${{ steps.plan-output.outputs.destroy }}"

The workflow outputs the plan status and any resources to add, change, or destroy in its log.

Workflow log output for plan

If the plan does not reflect the correct changes, fix the Terraform configuration to achieve the expected output. After verifying the plan in the previous job, manually trigger the workflow to run terraform apply. This command starts a run in the HCP Terraform workspace, where the actual infrastructure changes are made.

 apply_terraform_plan:
     needs: aws_tfc_job
     if: github.event_name == 'workflow_dispatch'
     runs-on: ubuntu-latest
     steps:
     - name: Checkout
       uses: actions/checkout@v3.5.2
     - name: Setup Terraform CLI
       uses: hashicorp/setup-terraform@v2.0.2
       with:
         cli_config_credentials_token: ${{ secrets.TF_API_TOKEN }}
 
     # Invoke the Terraform commands
     - name: Terraform init and validate
       run: |
         echo `pwd`
         echo "** Running Terraform Init**"
         terraform init
      
         echo "** Running Terraform Validate**"
         terraform validate
       working-directory: ${{ env.tfcode_path }}
    
     - name: Terraform Apply
       run: echo "** Running Terraform Apply**"; terraform apply -auto-approve
       working-directory: ${{ env.tfcode_path }}

Apply after reviewing plan

Optionally, you can add a terraform destroy step to clean up resources and avoid unnecessary costs. You can add this step in non-production or testing environments.

    - name: Terraform Destroy
      run: |
        echo "** Running Terraform Destroy**"
        terraform destroy -auto-approve
      working-directory: ${{ env.tfcode_path }}

»Setup review and further learning

This setup leverages the strengths of both platforms: GitHub Actions for CI/CD automation and HCP Terraform for secure, collaborative infrastructure management. Integrating HCP Terraform with GitHub Actions provides a powerful, automated pipeline for deploying and managing AWS infrastructure. By leveraging these tools, teams can achieve more reliable and efficient infrastructure management, reduce manual errors, and ensure consistency across environments.

HCP Terraform facilitates collaboration among team members ensuring that infrastructure changes are managed safely and efficiently. Platform teams can audit the runs in HCP Terraform while development teams can review runs in GitHub Actions.

From a security perspective, HCP Terraform workspaces can be configured with environment variables, such as AWS credentials, or dynamic credentials. The GitHub Actions workflow does not directly handle the credentials, which minimizes the blast radius of compromised credentials through the workflow. HCP Terraform provides additional features like access controls, private module registry, and policy enforcement to ensure that infrastructure changes are secure and compliant with organizational policies.

This guide has walked you through setting up a basic workflow, but the flexibility of both platforms allows for customization to fit your specific needs.

For further questions on best practices, please refer to the GitHub Actions and HCP Terraform FAQs available in this repository. As mentioned before, this repository includes the full code example used in this post. For more information on GitHub Actions, review GitHub’s documentation. To learn more about automating Terraform with GitHub Actions, review the official tutorial on the HashiCorp Developer portal and the starter workflow templates to use HCP Terraform with GitHub Actions.

For alternatives to this workflow, you can also use HCP Vault Secrets to sync the TF_API_TOKEN to the GitHub Actions secrets. With this method, you only need to update the token in one place, rather than every GitHub repo. HashiCorp also has integration templates for HCP Terraform and GitHub Actions that you should check out.

Automate AWS deployments with HCP Terraform and GitHub Actions

»Workflow overview

»Prerequisites

»Securely access AWS from HCP Terraform

»Add HCP Terraform token to GitHub Actions

»Create the GitHub Actions workflow

»Define the triggers

»Configure environment variables

»Define jobs

»Setup review and further learning

Sign up for the latest HashiCorp news

More blog posts like this one

Access AWS from HCP Terraform with OIDC federation

New infrastructure integrations with GitHub, Illumio, Palo Alto Networks, Tessell, and more

Terraform Enterprise improves deployment flexibility with Nomad and OpenShift