Vault AIDE: A ChatOps Bot for HashiCorp Vault

The following is a guest blog from DigitalOnUs, a provider of multi-cloud development, DevOps, SRE Services, and HashiCorp SI partner of the year.  

Vault AIDE is a chatbot solution that connects to Slack and provides operational insights on HashiCorp Vault in real time. A secrets management tool is for securely accessing secret data. A secret is anything that you want to tightly control access to, such as API keys, passwords, or certificates. HashiCorp Vault provides a unified interface to any secret while providing tight access control and recording a detailed audit log.

However, like any software, it requires administrative effort and operationally, Vault needs to be treated as a Tier zero application because many highly available applications will start to depend on it as a central secret store. Providing insights into operational aspects early on via real-time monitoring will help the Service Reliability Engineer (SRE) engineers and operators better manage the Vault platform. This includes receiving an early warning when high privilege actions are performed or wanting to ensure that the Vault clusters were configured according to the best practices.

After dozens-plus enterprise-scale Vault implementations over the last three years, DigitalOnUs has created a solution called ‘Vault AIDE’ to solve the above problem. 

What is Vault AIDE?

As we mentioned before Vault AIDE is a chatbot and consists of continuous analysis of actions and logs that allow us to send Slack notifications when relevant, in such a way that when you receive notifications from your instance, they contain reliable data in real time so that you can take action and your security is not affected.

We cover the cases that we consider are key to ensuring you are as secure as possible in your implementation. There are currently 9 use cases:

• Version Updates: As a Vault SRE, you want to know when you have an update available that's important to you, so that you can pay attention to deciding whether to update Vault without needing to constantly check Vault's CHANGELOG.md

• Adoption Stats: As a Vault operator, you want to see the usage of your Vault deployment, with the goal of justifying its continued use and budget. 

• Extant Leases: As a Vault SRE, you want to be sure that your leases have an adequate time-to-live and you are not leaving infinite leases that could be used for purposes other than those established. 

• Admin Creation Alert: As a Vault SRE, you need to know immediately when extremely high privileged roles are being added to Vault.

• High privilege action: As a Vault SRE, you need to know immediately when extremely high privileged roles are being used in Vault.

• Vault Posture Score: As an InfoSec leader, you want to know how your Vault's internal configuration compares with recommended practices, and how close you are to being "as secure as possible" with it, in an "at a glance" score. This score provides a single and accurate view into your Vault's Application Level security posture.

• Auth Method Suggestions: As a Vault SRE or application developer, you want to know the opportunities to reduce manual work or enhance security.

• Statusserer: As a Vault SRE you need to check Vault status beyond the basics. For example the seal state, if you have audit log enabled, or number of endpoints enabled

Vault AIDE Benefits

Chatbot applications help to streamline interactions between people and services, and can also enhance customer experience. At the same time, chatbots offer new opportunities to improve the customer’s engagement process, and operational efficiency with HashiCorp Vault by reducing the typical cost of customer service. For example in the traditional way if you want to be aware of the most recent changes and improvements of Hashicorp Vault you need to invest time or resources in reviewing the Changelog.md with Vault AIDE you are a notification away so you can invest that time in other actions

From a management point of view, Vault AIDE takes the data directly from Vault, and pretty up the information to present it in a human-readable format as we can see in the image below.

From a technical point of view, the company has a smoother workflow since they can use the same tool with which they communicate internally and check the health or status of the Vault implementation to be sure that it’s working correctly.

Vault AIDE may be of particular interest to those who, without having the technical capacity, want to be informed of the impact that HashiCorp Vault is having within their organization.

For example, if a new version of Vault is released, then you will be prompted with relevant information on how you can upgrade.

The information that Vault exposes is also relevant to those who have certain technical capabilities, through the Vault UI, CLI, or API you can get valuable information but with Vault AIDE you can get a quick heads up if something weird is happening. Like someone accessing or creating root accounts.

Therefore, being in a certain Slack channel can help you take quick action to prevent someone from performing operations for which they are not authorized.

How Does it Work?

Vault AIDE is a python process that runs independent of the Vault implementation. Logs are exposed from HashiCorp Vault through a socket into Vault AIDE, which takes the relevant information and communication that to Slack.

The information that Vault AIDE consumes and displays is about the actions that the users of the Vault instance perform. Vault AIDE do not use or read secrets under any circumstances. 

When installing Vault AIDE we provide you with a list of permissions that must be granted to a token that will be used to read the vault log, the token with which Vault AIDE will communicate with your instance so you can control specifically what you want to allow and what not.

In addition to what you control, Vault itself takes care of your information since it encrypts the sensitive data that reaches the log in such a way that what can be used is simply information for general use. 

For example, this is an entry for token creation in the log that Vault AIDE will analyze and determine if it is necessary to let you know about any action that needs to be taken.

As you can see, all the sensitive information (highlighted with yellow) is encrypted, and other information for general use (highlighted with pink) is plain text.

Finally to put your security concerns aside, here is an example list of what we get from each log query

Lease Optimization:

• Audit Log entries to get the time-to-live and time-of-use for every lease

Admin Access Alert:

• Audit Log entries to know when a root token is used

Admin Creation Alert:

• Audit Log entries to know when a root token is created

Version Updates:

• HTTP request to get the version value

What to Expect From Vault AIDE in the Future

We want to cover 5 phases which are: give data, inform, analyze, recommend and automate for now Vault AIDE is in phase 4 and working to get to the automation phase soon.

We are planning to incorporate machine learning in the last phase so that Vault AIDE can predict and recommend or detect more possible failures according to the data collected from the operation of each instance, with this we expect to remove more work from your shoulders.

Next Steps

We have a version of Vault AIDE that is open source so anyone can contribute, so if you are interested follow this link to take a look and help us to continue growing this tool. If you are interested in contributing to our project on GitHub, read our Installation Guide.

Learn more about Vault AIDE by watching this demo screencast or by emailing us at hashicorp@digitalonus.com: