Nomad Tech Deep Dive: Autoscaling, CSI Plugins, and More

Nomad 0.11 recently became GA, bringing a significant number of exciting new features that HashiCorp developer advocates Erik Veld and Jacquie Grindrod want to share with you.

This deep dive session will take a close look at the following new features in Nomad 0.11:

Task Dependencies: Define and run interdependent workloads in their sequential orders. Allow users easily and efficiently run startup, sidecar, and other dependent applications at scale. Intro article

Container Storage Interface (CSI) plugins: Extend stateful workloads support through the Container Storage Interface. Users can natively connect and run stateful services with storage volumes from their third-party provider of choice via the CSI plugin system. Intro article

Autoscaling: Dynamically scale up and down application instances based on real-time load or business SLAs without manual intervention. Intro article

Remote Exec (UI): Directly execute commands in a running allocation through the Nomad UI. Users can now easily inspect container states and debug deployment failures through the web interface without the need for CLI or SSH machine access. Intro article

Audit Logging (ENTERPRISE): This Enterprise feature provides administrators with a complete set of records for all user-issued actions in Nomad. It improves operational traceability to fulfill compliance requirements in regulated sectors such as finance, healthcare, and online commerce. Intro article

Outline

0:00 — Introduction to Nomad 0.11

1:20 — Audit Logging

2:45 — Demo: Autoscaling (Demo code)

14:07 — Demo: Task Dependencies (Demo code)

26:44 — Demo: Container Storage Interface

38:22 — Q&A

Q&A

• What are you running under the covers? What are you running to support the “pods”

• What is the difference between group.count and group.scaling.min (or max) in a job specification?

• The localbindport on the sidecar upstream completely kills bin-packing, but I don't believe there are any logical blockers to why this has to be statically defined. Is it on the roadmap for localbindport to be generated dynamically?

• New versions of Nomad give the possibility to deploy Nomad jobs in the UI Dashboard. Is there an easy way to disable that functionality?

• If I enabled TLS encryption with Nomad, how would I deal with the Consul HTTP health checks I have in my Nomad jobs? By the way, my apps are in HTTP behind a load balancer that provides HTTPS.