Transparent Data Encryption in the Modern Datacenter

Transparent Data Encryption (TDE) is a technology coined by Microsoft, IBM, and Oracle to encrypt databases on the filesystem. TDE attempts to solve the problem of protecting data at rest, encrypting databases both on the hard drive and consequently on backup media. It does not protect data in transit, nor data in use. Organizations often employ TDE to solve compliance issues which require the protection of data at rest.

While TDE is a step in the right direction for better security, it was founded in an era where one of the biggest threats to security was physical infiltration of the datacenter. There were genuine concerns that an attacker would steal physical tape media and walk out. Additionally, to enable automatic booting, many databases that implement TDE store the lowest level encryption keys in persistent storage, a security issue vendors acknowledge because the stored keys can be directly recovered from an online system and used to decrypt the databases. Even though TDE solves a compliance problem, it does not solve emerging challenges in the modern security space.

The shift to cloud computing, specifically public cloud, forces us to rethink the traditional approaches to security and to adapt existing threat models in this hybrid world.

»A Hybrid Approach

Just as organizations are quickly adopting multiple cloud technologies, security teams should be adopting hybrid threat models. With the move to public and hybrid cloud, physical security threats are being superseded by a new type of digital threat. Public cloud vendors employ industry-leading research and threat assessment experts to protect against physical access. Many of these vendors have published numerous white papers and have dedicated resource pages that describe their processes for physical security compliance. Simply put - physical security is a solved problem in the public cloud.

Researchers often describe security as Defense in Depth, a concept in which multiple layers of security controls are placed throughout a system. IT Security should be treated as an onion or castle. Implementing only TDE is like putting only a moat around a castle. Once an attacker circumvents the moat, there are no additional protections in place to mitigate the attack. Just like a castle, we should implement a multi-faceted, onion-like approach to security. TDE is just the outermost layer of the encryption castle.

HashiCorp's Vault provides Encryption as a Service (EaaS) and assists security teams in fortifying their security architectures. Vault encrypts data in transit with TLS and at rest with AES 256-bit GCM encryption. With Vault's transit service, databases never store plaintext data; they only persist encrypted data. Even if an attacker were able to access the raw data, they would only have encrypted bits. Furthermore, Vault manages the encryption keys out-of-band, on a separate system. This means an attacker would need to compromise multiple systems before exfiltrating data.

With Vault's EaaS, applications send plaintext data to Vault (encrypted in transit with TLS). Vault encrypts this plaintext data using a high-entropy encryption key and returns the data to the application. The application then persists this ciphertext in the database. Visually:

It is important to note that Vault's EaaS does not require any plugins or special database integrations. The ciphertext is simply stored in the database as text (VARCHAR, text, string, etc.). This makes this solution incredibly portable, even with legacy databases.

When an application needs the plaintext data, the process is simply reversed. The application requests the ciphertext data from the database. It then sends the ciphertext along with Vault authentication information to Vault. Vault authenticates and authorizes the decryption operation and, if permitted, returns the plaintext data to the application. Note that the database never has the plaintext data, and the application only has the plaintext data for the duration which it is needed.

For maximum security, we can combine Vault's EaaS and TDE. The following graphic shows the differences between TDE, Vault's EaaS, and TDE combined with EaaS.

By combining TDE and Vault EaaS, organizations can adopt modern security best practices while adhering to legacy compliance requirements.

»Per-Row Encryption Keys

As an added benefit, Vault also supports derived encryption. In this mode, Vault can encrypt each row in each table of the database with a unique encryption key. Even if an attacker were able to access the raw data, each row of the database is encrypted with a unique, high-entropy key. The database never sees the plain-text value. Even as modern processing power increases, an attacker would still need to brute-force an NP-hard problem for each row of the database to retrieve the plaintext values. You can read more about derived encryption in the transit secrets backend in the Vault documentation or learn more how Vault encrypts application data in transit and at rest.

»Programmatic, Unique Access

In addition to encrypting data in transit and at rest, Vault also provides programmatic access to database credentials. Traditionally, application developers request credentials from the database or security teams and then inject those credentials into the application at runtime. There are a number of shortcomings in this approach, including lack of provenance and secret sprawl. Credential rotation and revocation are often an afterthought, and since each instance of the application uses the same database credentials, these operations typically involved scheduled downtime.

The security and database teams collaborate to configure Vault with the stored procedures and functions required to create a new user. This information is codified and templatized. Applications authenticate to Vault and request new credentials via Vault's API. Vault programmatically connects to the database, runs the configured SQL, and generates a new user. With every service accessing the database with unique credentials, it makes auditing much easier when questionable data access is discovered: you can track it down to the specific instance of a service based on the SQL username.

Lastly, Vault makes use of its own internal revocation system to ensure that users become invalid within a reasonable time of the lease expiring. Unlike a traditional authentication credential which is valid for an infinite amount of time, Vault enforces time-based access control through its leasing model. This prevents long-lived credentials, reducing the surface area for an attack. Even if an attacker could gain access to valid credentials, they would only be able to retrieve encrypted data from the database with Vault's EaaS. The attacker would additionally need to compromise Vault to decrypt the encrypted data.

»Conclusion

Transparent Data Encryption is just one layer of the security onion. While it assists in compliance requirements, it does not protect against more modern security threats, nor does it provide many benefits when running on a public cloud provider. By combining TDE with Vault's Encryption as a Service, organizations can easily adopt modern security best practices while adhering to legacy compliance requirements.

Interested in more Vault’s features like a managed UI, cluster management, HSM integration, or multi-datacenter replication and disaster recovery? Be sure to check out Vault Enterprise.