Announcing HashiCorp Packer 1.5 With HCL2 Support

HashiCorp Packer 1.5 brings two major new features and a long list of smaller improvements. The most exciting changes are that Packer now supports basic HCL2 templates, and we now can share some special information between builders and provisioners, like host IP and port for the build instance. We hope that having access to these details will help simplify your templates and your operations efforts. Let's dig into some more detail on these big changes:

»HCL2

HCL2 templates can now be used in place of a single json template. Users can still use json templates if they prefer; we are not going to be deprecating json support any time soon.

Here's an example of a folder containing Packer HCL2 template files for a basic AWS build:

# folder/sources.pkr.hcl
source "amazon-ebs" "example" {
    ami_name = "packer-partyparrot-{{timestamp}}"
    region = "us-east-1"
    instance_type = "t2.micro"
    source_ami_filter {
        filters {
          virtualization-type = "hvm"
          name =  "ubuntu/images/*ubuntu-xenial-16.04-amd64-server-*"
          root-device-type = "ebs"
        }
        owners = ["amazon"]
        most_recent = true
    }
    communicator = "ssh"
    ssh_username = "ubuntu"
}

# folder/build.pkr.hcl
# A build starts sources and runs provisioning steps on those sources.
build {
  sources = [
    # there can be multiple sources per build */
    "source.amazon-ebs.example"
  ]
 # All provisioners and post-processors have a 1:1 correspondence to their
 # current layout. The argument name (ie: inline) must to be unquoted
 # and can be set using the equal sign operator (=).
  provisioner "shell" {
    inline = ["sleep 5"]
  }
  provisioner "shell-local" {
    inline = ["echo the address is: $PACKER_HTTP_ADDR and build name is: $PACKER_BUILD_NAME"]
  }
}

Simply save the above files and run packer build folder as normal. Here's another example, using the file builder which won't use cloud resources:

source "file" "example" {
    content = "Happy 2020"
    target = "./test_artifact.txt"
}
build {
  sources = [
    "source.file.example"
  ]
  post-processor "shell-local" {
    inline = ["echo We hope you like your gift!"]
  }
}

You'll notice that the options are identical to what they were in the json templates; currently transferring templates from json to HCL2 will be a straightforward, though manual, process for you. If you choose to wait until later on this year, we intend to create a template converter in an upcoming release.

»HCL2 is Considered a Beta Feature

There are currently 33 builders, 21 provisioners, and 20 post-processors supported by Packer. This new feature makes all the changes necessary to get these components to be configured with HCL2; however, there are some parts of Packer HCL2 that still require polishing. As a result, we have released HCL2 as a beta feature. If you find problems, please open an issue on GitHub so that we can fix any missing tools or strange behavior.

»Variables & Variable Interpolation

Currently, build-specific and “user” variables (see below) are not working for Packer with HCL2. We know that HCL2 will be of limited use until variables and interpolation work, so expect this support to follow relatively soon. Here is a list of template helpers that do and do not yet work in HCL configs:

Working:

• {{timestamp}}
• {{isotime}}
• {{packer_version}}
• {{uuid}}
• {{lower}}
• {{replace}} - {{replace_all}}
• {{split}}
• {{upper}}

Not currently working:

• template variables: {{ .Name }}
• {{user ...}} and as a result {{env}}, {{consul_key ...}}, and {{vault ...}} cannot be used
• {{build ...}}
• {{build_name}}
• {{build_type}}
• {{clean_resource_name}}
• {{template_dir}}
• {{pwd}}

We plan on supporting these features soon with HCL2 and we expect these features to look something like HashiCorp Terraform's input variables and local variables.

»Sharing Variables Between Builders and Provisioners

As Packer has grown, highly custom use cases have appeared. Many of these use cases require access to the instance that cannot be covered by our array of provisioners and communicators. In these cases users need to generate specialized solutions. Historically this has been difficult because we only keep certain connection info in memory. That has now changed: we have added tooling for builders to inject specialized data into provisioners.

Currently, available variables are: ID, Host, Port, User, Password, ConnType, PackerRunUUID, SSHPublicKey, and SSHPrivateKey.

In order to validate that users don't try to access variables that are not set, we require a new template engine to access them.

{{build `ID`}}

These variables can be used in any provisioner argument that is documented as being a template engine. If you find a use case for having this interpolation in a location that is not currently template engine enabled, please open a feature request on GitHub.

For example:

{
	"builders": [
		{
			"type": "null",
			"communicator": "none"
		}
	],
	"provisioners": [
		{
			"type": "shell-local",
			"inline": ["echo {{build `PackerRunUUID`}}"]
		},
		{
			"type": "shell-local",
			"environment_vars": ["TESTVAR={{ build `PackerRunUUID`}}"],
			"inline": ["echo $TESTVAR"]
		}
	]
}

You cannot use this engine inside the variables{} top level template section, as those variables are global whereas the build variables are build-specific; instead, use them directly within the provisioners, as demonstrated in the above example.

»Backwards Incompatibilities

»Everyday Users

Current Packer buildfiles will work just the same. If you do not use any third party plugins, you should not see any major backwards incompatibilities, unless:

• You are still using the now-fully-deprecated clean_ami_name template engine. Use clean_resource_name instead.
• The ansible-local galaxycommand option has been renamed to galaxy_command.

For both of these incompatibilities, calling packer fix yourtemplate.json will output a template with the necessary changes made so that you do not have to search for the fields to correct.

»Plugin Maintainers

Plugin maintainers will see that the interfaces of builders, provisioners, and post-processors have changed.

These changes are summarized by the following diff:

type Builder interface {
+    ConfigSpec() hcldec.ObjectSpec
-    Prepare(...interface{}) ([]string, error)
+    Prepare(...interface{}) ([]string, []string, error)
...
}

type Provisioner interface {
+    ConfigSpec() hcldec.ObjectSpec
-    Provision(context.Context, Ui, Communicator) error
+    Provision(context.Context, Ui, Communicator, map[string]interface{}) error
...
}

type PostProcessor interface {
+    ConfigSpec() hcldec.ObjectSpec
...
}

The new ConfigSpec function returns the HCL2 spec of a component. This function will be called every time a component is used. For more info on the subject, read our guide on how to compute the ConfigSpec of a component.

The changes on Builder.Prepare and Provisioner.Provision are linked: every builder can now define information that can be later used in a provisioner. Builder.Prepare() now returns a new []string which represents the names of variable fields a builder will provide for use by provisioners. These fields can later on be used in a Provision step using the {{ build `field` }} template engine described above

These fields are provided to the provisioner Prepare() call as the first argument. While the function signature of Provisioner.Prepare() has not changed from a ...interface{}, this new map needs to be handled in order to make sure that interpolation succeeds. If you use config.Decode() from github.com/hashicorp/packer/helper/config to read your provisioner template, this is already handled for you (for an example, see how the Prepare() method is being implemented in the shell provisioner).

»Thanks in Advance for Feedback

The Packer team has worked hard on these changes and is excited to be able to bring you these new features. Please try them out and share any bugs with us. As we roll deeper into 2020, we want to thank all of you for being such a great community.