Skip to main content
Webinar Learn how to move from vulnerability insights to governed remediation across complex infrastructure environments Join the webinar

Building a secure foundation for AI on Kubernetes with HashiCorp Vault

As AI goes cloud native, learn how HashiCorp Vault helps secure Kubernetes data, workload trust, certificates, and application credentials.
HashiCorp Vault and Kubernetes logos side by side

AI is moving beyond experimentation and into production, but security controls aren’t necessarily keeping pace. Recent IBM research shows 48% of surveyed executives say their organizations lack adequate protection for model inference and live AI usage, while 56% say the same about AI development and training. As organizations move AI applications, models, and autonomous agents into production environments orchestrated by platforms such as Kubernetes, closing that security gap requires looking beyond the AI model itself.

This shift is reflected in team priorities and thought leadership, including the agenda at this year’s KubeCon. Moving a model from demo to production introduces challenges across compute, GPU utilization, model serving, networking, storage, observability, cost and most importantly, security. Autonomous AI agents add the consideration: how can workloads safely and securely interact with infrastructure, APIs, tools, and data. 

For platform and security teams, that changes the conversation. Securing AI on Kubernetes isn't only about protecting a model or application. It requires securing the infrastructure underneath it, including the data stored by Kubernetes, the identities and certificates workloads use to establish trust, and the credentials applications need to access other systems. 

»Protect the Kubernetes foundation beneath AI workloads

Kubernetes uses etcd as its backing store for cluster data, which can include Kubernetes Secrets and other sensitive configuration associated with workloads running in the cluster. By default, the Kubernetes API server stores unencrypted representations of resources in etcd unless encryption at rest is configured. For organizations running sensitive workloads, including AI applications, protecting this data adds an important layer of defense if unauthorized access to the underlying storage occurs. 

Vault Kubernetes Key Management reduces this risk by integrating Kubernetes encryption with the Vault Transit secrets engine. The vault-kube-kms process exposes the Kubernetes KMS v2 interface and connects the Kubernetes API server to Vault, where the Transit engine is used for cryptographic key wrapping operations.

With Vault 2.2, Vault Kubernetes Key Management advances to full support for general Kubernetes deployment. This approach helps organizations centralize control over the keys protecting Kubernetes data rather than managing encryption independently for every cluster.

For production AI environments, Kubernetes hosts far more than just the model itself. Model-serving infrastructure, AI gateways, MCP servers, agents, application services, and the configuration required to operate them may all depend on the Kubernetes control plane. Securing AI starts with securing the platform it runs on.

»Automate trust with Vault and cert-manager

As cloud native architectures become more distributed, applications increasingly rely on certificates to establish trusted, encrypted connections between workloads. Consider an agentic application running on Kubernetes. An AI agent might communicate with an MCP server, which connects to an internal API, which in turn accesses a database or another service. Each connection creates another trust relationship that platform teams need to secure.This web of interdependent services makes automated certificate lifecycle management a critical part of the Kubernetes security model.

A new external Vault issuer for cert-manager means application developers can continue requesting certificates using standard cert-manager Certificate resources, while VSO 2.0 sends approved certificate requests to Vault PKI for signing. The approach also reuses VSO's existing authentication and client infrastructure rather than requiring teams to establish a separate Vault integration for certificate management. And with shrinking certificate lifecycles, integrations with Vault and automations in it are even more relevant for teams when it comes to managing certificates. 

The benefit is not a new certificate workflow. It leverages familiar Kubernetes-native workflows, so that teams that have standardized on cert-manager can continue using these familiar workflows, while bringing certificate signing under centralized Vault PKI controls. This centralized control helps platform teams automate workload trust without asking every application team to learn a separate security process, ensuring identity-based access and secure execution supporting autonomous workload access. 

»Keep secrets synchronized as workloads change

Secret delivery is only part of the problem. Credentials also rotate, applications scale, and configurations change.  Common polling approaches can leave a window for downtime between when a credential changes and when a workload receives that change.

Vault 2.2 expands Vault Secrets Operator instant updates. VSO 1.6 can immediately deliver changes for dynamic, auto-rotating, and manually rotated database, KV, Azure, and LDAP secrets. VSO reconciliation through Vault event notifications is triggered immediately on secrets changes. Reducing the gap between credential rotation and availability helps security controls keep pace with the applications they protect, which is necessary for dynamic applications, including AI and agentic workloads that interact with multiple services and must move securely at machine speed. 

»Give workloads secure access without the full Vault server footprint

Secrets present another challenge. Applications, AI services, and autonomous agents all need credentials to access databases, APIs, cloud services, model endpoints, and other tools at runtime. But the component consuming those secrets doesn't need all the functionality required to operate a Vault server.

Vault 2.2 introduces Vault Client. Vault Client packages Vault Agent, Vault Proxy, and a limited set of Vault CLI commands without requiring the complete Vault server binary. Not all organizations have embraced Kubernetes secrets. For deployments and secrets that require additional isolation or more-limited secrets visibility, Vault Agent and Agent Sidecar Injector are now available in a smaller footprint for workloads that need client-side Vault functionality while establishing a clear separation between the capabilities required to operate Vault and those required to consume services from it. For increasingly dynamic cloud native environments, that distinction can help simplify how Vault functionality is deployed alongside applications.

»Build a security foundation for cloud native AI

But the bigger story is not any individual Kubernetes integration. It's how these capabilities work across different layers of the cloud native stack:

  • Protect cluster data: Vault Kubernetes Key Management connects Kubernetes KMS with Vault Transit to protect Kubernetes data stored in etcd. 

  • Establish workload trust: The Vault issuer for cert-manager brings Vault PKI into familiar Kubernetes certificate workflows.

  • Keep credentials current: VSO instant updates help Kubernetes applications receive secret changes as they happen.

  • Secure runtime access: Vault Client provides a lightweight way for workloads to use common Vault client functionality. 

Together, these capabilities point toward a consistent security model: Kubernetes-native workflows for developers and platform teams, backed by centralized controls for secrets, certificates, and encryption.

»Secure what comes after the AI demo

Organizations focused on securing AI at scale are challenged beyond just getting a model to run securely in production. Teams need the infrastructure and operational practices to serve models, operate agents, secure workloads, control access, and connect the growing number of systems surrounding AI applications. Security must evolve alongside that infrastructure.

As organizations bring AI applications and autonomous workloads onto Kubernetes, Vault can help provide the security foundation beneath them, protecting Kubernetes data, automating workload trust, and giving applications secure access to the credentials they need. Ready to secure your Kubernetes workloads? Explore the HashiCorp Vault Kubernetes developer documentation to get started. 

More posts like this