Toward Vagrant 3.0

HashiCorp Vagrant started as a small project to make interacting with virtual machines for local development easy. A growing community of developers, operators, designers, and products rely on Vagrant. In pursuing the goal to continue improving Vagrant, we discovered a path that would allow Vagrant to be ported to Go and still support its Ruby-based features. In this post, we’ll discuss how migration to a Go code base will not only support developer environments of the past decade but also new development workflows, environments, and ecosystems.

The changes we are working on now are designed to allow you to run Vagrant with a client-server architecture, secure it better on different operating systems, and manage its configuration globally. They’ll also offer a more seamless plugin development experience by improving the portability of plugins and reducing the complexity of package management.

Over the next year, Vagrant 2.3 and 2.4 will not break compatibility promises of Vagrantfiles or plugin interfaces. Vagrant 3.0 will introduce new methods for configuration but retain tooling for continued compatibility of Vagrantfiles. This includes detection for Ruby Vagrantfiles and installation of compatibility helpers to minimize interruptions to user workflow. While we don't yet have specific timeframes, we will announce our plans for these new versions of Vagrant as they solidify over the next few months.

»Improvements and New Features

You can look forward to many new features, capabilities, and improvements made possible by these updates to Vagrant’s architecture:

»Client-Server Architecture

The new architecture will allow you to run Vagrant on a remote host and secure actions on the machine. You will be able to install Vagrant on a resource-intensive machine and interact with it on a thin client, which will allow sharing of one Vagrant environment among multiple team members.

A new client-server model will facilitate easy sharing of configuration and resources while also giving administrators power over Vagrant’s privileged actions. This streamlines the collaboration experience offered by the existing Vagrant Share plugin. You will still be able to run Vagrant locally on a desktop machine and use the new security capabilities.

»Security

Vagrant runs as a user process that requires acceptance from a user to run privileged actions. You can work around request and acceptance on POSIX platforms using a sudoers file to define each Vagrant command with escalated privileges. Unfortunately, you need to update this file any time you change the privileged Vagrant commands. On Windows systems, you cannot use a sudoers file. Instead, you must confirm a User Account Control (UAC) prompt, which makes it difficult to run Vagrant in a headless context. Vagrant 3.0 will enable a privileged service to execute known and trusted commands from Vagrant and its plugins without the need for direct user interaction.

In addition to finer control of trusted commands, you will be able to use controls to prevent automatic evaluation of Ruby-based Vagrantfiles. You will also be able to write configuration in HashiCorp Configuration Language (HCL) to limit the potential for mistakes or risk arising from dynamic configuration. This prevents arbitrary code execution inline in Vagrantfiles.

»Global Configuration Management

Currently, you can reference only your own local configuration of Vagrant. Different users on the same system do not have access to the same configuration. A new, server-based approach will allow for management of a global Vagrant configuration, which you can also store in a database. This will fix long-running issues where provider-specific information results in lost configuration settings. Lost settings can result in unexpected and unintended system modifications.

Global configuration management helps organize Vagrant’s state and lays the foundation for adding a web-based UI, error checking, and transaction-based interactions. This improves Vagrant’s resilience to bad changes. The update will also resolve issues around Vagrant’s index file growing too big or being too fragile.

»Plugin API and Portability

While the existing Ruby plugin API will continue to be supported, Vagrant will support a new plugin API with the help of the HashiCorp go-plugin library. Plugin authors will be able to create Vagrant plugins in Go or any other language with GRPC support. A plugin SDK will provide common utilities and helpers for plugin development.

You will be able to port plugins written with the new API across platforms and release them independently of Vagrant’s releases. A compiled Go plugin's cross-platform capability will ease distribution and usage by simplifying current requirements for complex dependency management.

»Bring Your Own Ruby

A long-standing issue with some plugins and third-party distributions packaging Vagrant revolves around Vagrant shipping its own Ruby. This causes friction when you want to run Vagrant on the latest versions of Ruby or when Ruby plugins depend on packages that require specific libraries. Moving away from bundling Ruby with Vagrant will give users more control over the Ruby runtime. For example, systems such as Arch or Fedora that distribute their own Vagrant package can safely depend on system Ruby.

»Using Ecosystem Libraries

We have developed a number of Go libraries for other HashiCorp products and other infrastructure tooling by various vendors and communities. The move to Go will allow Vagrant to more fully join this community, gaining the ability to use (and contribute to) tools that already exist, as well as lean on the Go expertise that exists at Hashicorp.

HashiCorp tools such as go-plugin and glint helped us explore the possibilities around writing Vagrant in Go while maintaining backward compatibility. Moving forward, other Go libraries will allow Vagrant to accommodate a wide variety of new plugins. For example, you can use Go bindings for different virtualization software to build new plugins for many more backends.

»Plugin Externalization

Vagrant has more than 70 built-in plugins. This proliferation has introduced new development constraints. Currently, any fix that may be required on a built-in plugin is dependent on the entirety of Vagrant before it can be released. With Vagrant 3.0, as we continue to improve the plugin API and portability, we can extract plugins from Vagrant so you can update them faster and easier.

The focus will mainly be on provider and provisioner plugins. Once these plugins are externalized, you will be able to upgrade provider and provisioner plugins independent of Vagrant. Command plugins will remain internal and still depend on Vagrant’s release cycle.

»Planned Milestones

We recognize that porting a project to another language can be disruptive, so we aim to make it as seamless as possible. We will not deprecate any interfaces or features you require as a Vagrant end user or plugin developer during this process. First, we will port the core of Vagrant to Go. Once that is stable, we will work on built-in plugins. This iterative process ensures full compatibility between the Go and Ruby Vagrant runtimes, a requirement for our own built-in plugins to remain functional.

We will spread these changes over two minor releases and one major release:

»Vagrant 2.3

The Vagrant 2.3 release will include the initial alpha version of the Go-based Vagrant implementation. It will be shipped in the Vagrant installer packages and made available as an optional executable. You will be able to choose to use the new implementation, but you may experience some instability. Throughout the Vagrant 2.3 release cycle, we will work to stabilize the Go-based Vagrant implementation and begin to port built-in plugins. The vagrant command will continue to run the traditional Ruby-based Vagrant implementation.

»Vagrant 2.4

The Vagrant 2.4 release will feature the new Go implementation of Vagrant as the primary executable. It will include a migration command, which will import existing data from the Vagrant home and project directories. Vagrantfiles will also support HCL and data structures.

We will focus on identifying and resolving compatibility issues in the Vagrant 2.4 release cycle. You will still be able to use the Ruby-based implementation by using the alternate executable. Based on feedback and the consistency of user experience in the 2.4 series, we may have additional 2.x series releases.

»Vagrant 3.0

The Vagrant 3.0 release will include only the Go-based Vagrant implementation. By default, it will support only plugins that use the new API and HCL-based Vagrantfiles. The installer package will no longer include Ruby nor the extra dependencies required for running the Ruby-based Vagrant implementation.

You will still be able to use Ruby-based Vagrantfiles and Vagrant plugins. However, you will need to install Ruby on the host system. Vagrant will detect Ruby-based Vagrantfile or plugin usage and prompt you with instructions for installing Ruby on your platform. Once installed, Ruby-based Vagrantfiles and plugins will work normally.

»Conclusion

From a new Go-based implementation to the externalization of plugins, we are designing Vagrant 3.0 to provide a foundation to integrate with new developer environments while maintaining the environments of the last decade. We look forward to making these changes available to help you scale your use and development of Vagrant. If you have feedback on our plans for Vagrant, or would like to ask questions about it, please reply with a comment in our topic on the Vagrant forum.