Managing GitHub with Terraform

For a more up-to-date tutorial, read our HashiCorp Learn tutorial how to manage GitHub users, teams, and repository permissions in the GitHub Terraform provider

Terraform is an open source tool for managing infrastructure as code. Earlier I authored a blog post on leveraging version-controlled infrastructure with Terraform, and Terraform continues to push the boundaries on the definition of "infrastructure". Terraform is able to manage almost anything with an API, including Consul, Nomad, and GitHub. This blog post showcases using Terraform to manage GitHub organizations, repositories, teams, and permissions.

»Why Terraform?

The use case for managing cloud resources with Terraform is fairly straightforward - codify, version, automate, audit, reuse, and release. Managing GitHub organizations, repositories, teams, and permissions with Terraform provides the same benefits. You have immediate insight and a complete view of all memberships, repositories, and permissions inside all of your GitHub organizations.

Imagine a new employee onboarding process in which the employee adds their GitHub account to a team inside a Terraform configuration and submits a Pull Request. The hiring manager verifies the changes and merges the Pull Request. On the next Terraform run, the changes propagate out to GitHub, granting the new permissions. Not only does this happens in complete visibility of the company, but it also ensures consistency. Instead of relying on a human to click around in GitHub's web interface, we rely on a machine to push out policy and access. Whether you are managing a massive enterprise with hundreds of GitHub users or implementing a consistent labeling scheme across your personal projects, Terraform is the right tool for the job.

»Provider Setup

In order for Terraform to communicate with GitHub's API, we need to configure the GitHub Terraform provider. A Terraform provider is an abstraction of an API. Just like APIs require authentication, so do Terraform providers. In this case, the GitHub Terraform provider requires a token and organization. Here is a sample Terraform configuration:

provider "github" {
  token        = "a2bcl5..."
  organization = "terraform-example"
}

The token is a personal access token for your account. GitHub has excellent documentation on generating a personal access token. The organization is the human-friendly name of the organization. It is also possible to source these values from environment variables, but that is not discussed in this post. For this post, the token must have repo, admin:org, and delete_repo permissions.

GitHub Enterprise users may also specify the base_url option to point to their GitHub Enterprise installation. The default value points to the public GitHub.com.

»Create a Repository

Terraform can manage the creation and lifecycle of all your GitHub repositories. Terraform will not touch existing GitHub repositories, so it is safe to adopt gradually. Here is an example configuration to create a new repository named "example-repo". This repository will be created in the organization specified in the provider.

resource "github_repository" "example-repo" {
  name        = "example-repo"
  description = "My new repository for use with Terraform"
}

Next, run terraform plan to see what changes Terraform plans to make on GitHub.

$ terraform plan

+ github_repository.example-repo
    default_branch: "<computed>"
    description:    "My new repository for use with Terraform"
    full_name:      "<computed>"
    git_clone_url:  "<computed>"
    http_clone_url: "<computed>"
    name:           "example-repo"
    ssh_clone_url:  "<computed>"
    svn_url:        "<computed>"

Plan: 1 to add, 0 to change, 0 to destroy.

Now run terraform apply to apply the changes. This will create a real repository on GitHub.

$ terraform apply
# ... 

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

You can verify the operation was successful by visiting your organization on GitHub and searching for the repository named "example-repo".

Because Terraform's syntax is declarative, any changes to the configuration result in a computed changeset. To demonstrate this behavior, change the description of the repository in the Terraform configuration. When you run terraform plan, Terraform will report the resource has changed. When you run terraform apply, Terraform will update the description of the repository, but not the other attributes.

Once the resource is under management with Terraform, all its attributes are controlled by the configuration. As an exercise, edit the "description" field for the newly-created repository on GitHub.com, and run terraform apply. Terraform will detect the discrepancy and make an API call to GitHub to force the description to match the value in the Terraform configuration. The Terraform configuration becomes the single source of truth and policy.

»Creating Teams

Terraform supports more than just the management of GitHub repositories - it can also create GitHub teams and manage the members of those teams. Here is a sample Terraform configuration for creating a team. We can include this code in the same file as we created the GitHub repository resource. Terraform will intelligently handle both resources in the same file.

resource "github_team" "example-team" {
  name        = "example-team"
  description = "My new team for use with Terraform"
  privacy     = "closed"
}

Just like before, run terraform plan and terraform apply:

$ terraform plan
github_repository.example-repo: Refreshing state... (ID: example-repo)

+ github_team.example-team
    description: "My new team for use with Terraform"
    name:        "example-team"
    privacy:     "closed"

Plan: 1 to add, 0 to change, 0 to destroy.

$ terraform apply
github_repository.example-repo: Refreshing state... (ID: example-repo)
github_team.example-team: Creating...
  description: "" => "My new team for use with Terraform"
  name:        "" => "example-team"
  privacy:     "" => "closed"
github_team.example-team: Creation complete (ID: 2326575)

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Terraform created a team named "example-team" in the organization. You can login to GitHub and verify the team was created successfully, but it will have no members.

Terraform can add members to the team using the github_team_membership resource:

resource "github_team_membership" "example-team-membership" {
  team_id  = "${github_team.example-team.id}"
  username = "mitchellh"
  role     = "member"
}

This will add the GitHub user with the username "mitchellh" to the team we just created. Instead of hardcoding the team_id, we can use Terraform's interpolation syntax to reference the output from the previous resource. Internally, this builds a dependency graph and tells Terraform to create the team before it creates the team membership. Because our team already exists, the terraform plan will fill in the team_id. If the resources did not exist, that argument would be marked as <computed>.

$ terraform plan
Refreshing Terraform state in-memory prior to plan...

github_repository.example-repo: Refreshing state... (ID: example-repo)
github_team.example-team: Refreshing state... (ID: 2326575)

+ github_team_membership.example-team-membership
    role:     "member"
    team_id:  "2326575"
    username: "mitchellh"

Plan: 1 to add, 0 to change, 0 to destroy.

Next apply the changes:

$ terraform apply
github_repository.example-repo: Refreshing state... (ID: example-repo)
github_team.example-team: Refreshing state... (ID: 2326575)
github_team_membership.example-team-membership: Creating...
  role:     "" => "member"
  team_id:  "" => "2326575"
  username: "" => "mitchellh"
github_team_membership.example-team-membership: Creation complete (ID: 2326575:mitchellh)

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

We can verify the team was created by looking in the GitHub web interface.

»Adding Teams to Repositories

Thus far, we have created a GitHub repository, GitHub team, and added a member to that GitHub team. To bring the journey full-circle, we can grant the team permission on the newly-created repository using the Terraform github_team_repository resource.

resource "github_team_repository" "example-team-repo" {
  team_id    = "${github_team.example-team.id}"
  repository = "${github_repository.example-repo.name}"
  permission = "push"
}

Just as before, run terraform plan and terraform apply.

$ terraform plan
Refreshing Terraform state in-memory prior to plan...

github_team.example-team: Refreshing state... (ID: 2326575)
github_repository.example-repo: Refreshing state... (ID: example-repo)
github_team_membership.example-team-membership: Refreshing state... (ID: 2326575:mitchellh)
+ github_team_repository.example-team-repo
    permission: "push"
    repository: "example-repo"
    team_id:    "2326575"

Plan: 1 to add, 0 to change, 0 to destroy.

$ terraform apply
github_team.example-team: Refreshing state... (ID: 2326575)
github_repository.example-repo: Refreshing state... (ID: example-repo)
github_team_membership.example-team-membership: Refreshing state... (ID: 2326575:mitchellh)
github_team_repository.example-team-repo: Creating...
  permission: "" => "push"
  repository: "" => "example-repo"
  team_id:    "" => "2326575"
github_team_repository.example-team-repo: Creation complete (ID: 2326575:example-repo)

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Now members of the team "example-team" have push and pull access to the "example-repo" repository.

»Bonus: Setting Repository Labels

Many organizations have a common set of repository labels they like to apply to all projects. This helps ensure consistency and parity across projects. These labels may tie into internal systems that measure issue progress or metrics. In the past, managing these labels across repository has been a manual process or involved building a tool using the GitHub API. The challenge with both of these approaches is that they require the user to think about idempotency, change, and rollout effect. With Terraform, it is easy to manage issue labels and colors across all GitHub repositories. Even better, these labels are managed declaratively in Terraform configuration, so any changes are visible to the organization. This example also showcases a more advanced use of utilizing maps and lookups to build a more dynamic Terraform configuration.

First, create a map of the project label name to the hex color code. Remember that labels are case-sensitive, and the color code should not include the leading "#" character.

variable "issue_labels" {
  default = {
    "custom-label"  = "533D99"
    "documentation" = "FFB340"
    "waiting-reply" = "CC6A14"
  }
}

Next, use this variable with the github_issue_label resource in the Terraform configuration:

resource "github_issue_label" "test_repo" {
  repository = "${github_repository.example-repo.id}"
  count      = "${length(var.issue_labels)}"
  name       = "${element(keys(var.issue_labels), count.index)}"
  color      = "${element(values(var.issue_labels), count.index)}"
}

»Conclusion

Terraform is a powerful tool for codifying your organization's services. Whether you are provisioning instances on Amazon EC2, configuring monitoring with Datadog, or managing your GitHub teams and permissions, Terraform's declarative syntax can assist in managing the complexity of modern computing.