Keynote - Aligning Principles to Nomad 1.0

Transcript

Yishan Lin: Great, thank you, Mitchell and Armon. While we strive to embody these nine HashiCorp principles in our day-to-day work, there are a few of these principles that stand out in the context of Nomad.

One, as we progress towards a Nomad 1.0 release, and two is how we think about Nomad’s future in a rapidly maturing space with Kubernetes. To build for the future, we have to start from the today.

In the past year, the first HashiCorp principle we've focused on is reflection. We've reflected on how Nomad is used, talking with hundreds of different Nomad users around the world to learn and understand their core drivers for choosing and adopting Nomad. As well as exploring their production deployments and usage of Nomad in all sorts of industries — from fast food, to solar, to IoT, and to gaming.

We employ our pragmatism over the course of this reflection. We look to curate the core use cases for what people are trying to achieve when they choose Nomad. And we look to build, strengthen, and refine those core use cases for our practitioners.

Every product has a vision, but in Nomad's approach, we build that vision from a place of humility. With the growth of Kubernetes in the past two years, we need to be honest on where Nomad fits and uniquely excels for organizations. And last — but certainly not least — we need to be able to execute to get to Nomad 1.0.

Reflection and Pragmatism

Let's start with reflection and pragmatism as our first two principles and talk about how those have applied over the course of our development towards Nomad 1.0.

In 2020, we've seen a huge uptake in the number of enterprises that are running Nomad in production. One of the most fascinating observations that we've seen is how practitioners are able to translate Nomad's product simplicity into real, tangible, and quantifiable business results for their organizations.

That means whether we've been talking to large Fortune 500 companies like Citadel, or small boutique technical firms like BetterHelp, we find that it takes teams on average 2-4 weeks to get Nomad from a technical proof of concept and running into production.

The product simplicity of Nomad also carries over into its maintainability. We hear remarkable stories from companies all over the world who leveraged Nomad that are able to take very small, lean DevOps teams of just 1-4 staff members, and in turn, learn Nomad, pick it up quickly with very little familiarity and background. Then host it, operate it themselves and service tens to hundreds of developers, deploy and manage up to thousands of applications on a regular basis, achieve higher resource utilization, and cost savings — and hit all the business SLAs using Nomad.

That's the core story and tale of Nomad that we hear time and time again. That resonates with me and the Nomad team every time we get the chance to hear it; the tale of small teams of very few people who are able to come together and use Nomad to accomplish large, and exciting, and awesome things in their space.

But the story for Nomad this year hasn't just stopped in simplicity and maintainability. It has also extended into scalability. Some of the most exciting and largest stories about Nomad's usage this year have come from our friends at Cloudflare and Roblox.

Cloudflare is the world's leading CDN who, as of today, routes 10% of the world's global internet traffic. Their company has built all their infrastructure today on Nomad. Our friends at Roblox are also a great example of a company who recently surpassed 150 million monthly active players — and the number two largest video gaming platform in the world — as a company that has also built all their infrastructure on Nomad to date.

Since its inception, the guiding product principle for Nomad has been to orchestrate any application — containerized or non-containerized. In the past few months, we've explored deployments of Nomad in all sorts of environments. From your run-of-the-mill traditional bare metal on-prem datacenters to your standard public cloud instances like AWS, GCP, and Azure, agricultural smart farms — like we'll hear about from Bowery later in this conference — industrial factories, solar telescopes, and even biology laboratories.

The product simplicity of Nomad is a single lightweight binary. Clocking at just over 50 megabytes is exactly what enables this diversity of deployment environments. DevOps practitioners are able to look at spaces and industries that have traditionally worked with very physically constrained areas and find ways to deploy Nomad in Edge, IoT, and other very innovative ways.

Yet no matter the environment that Nomad is deployed in, Nomad’s usage fundamentally falls into two core use cases. Time and time again, practitioners call out Nomad’s unique simplicity as a container orchestrator and Nomad’s unique flexibility in deploying and managing non-containerized applications like Windows applications and Java applications. We’ve built Nomad around these two pillars — simplicity and flexibility — and they are the reasons and the core drivers for whenever practitioners call out why they use Nomad.

Vision with Humility

By looking back at the roots of what’s brought Nomad to where it is today, we can then form a meaningful vision for where the product can go in the future. If we take a quick step through history, we see that Nomad and Kubernetes both launched at the same time in 2015.

Now that we're five years later along the journey of these two tools and the growth of Kubernetes, we have to be upfront with ourselves and objective in what Nomad’s strengths are, as well as our own limitations. Our vision has to be humble and grounded in real-world usage patterns — on how people use Nomad and Kubernetes today — rather than how we think they should use these two tools.

In prior years, the conversation around orchestration has been that everyone is standardizing on Kubernetes — and that when it's all said and done there would only be one orchestrator left to adopt in Kubernetes.

Yet, as we've covered earlier with Nomad's adoption and other data that we've seen, orchestration has not been that winner-take-all market. And when we look at other products, and other markets, and other spaces that we interact with on a daily basis — whether it's software, hardware, technical, or non-technical — we see that most markets are run in pairs. Most markets and most spaces are all led by two leading products.

As some quick examples, whether I choose between Dunkin' and Starbucks for a morning latte when I'm on the road, and I need some coffee and caffeine to get me through the rest of my long-distance trip. Or if I'm building a workstation, do I go with AMD or Intel for my CPUs? If I'm stuck somewhere and have to get home, do I hail a ride using an Uber or Lyft? Or if I'm trying to add to my already big enough sneaker collection, do I go with Nike or Adidas shoes?

At the end of the day, each space is defined by two leading products, and each of these products excels at different things. And in many of these cases and situations, it ultimately comes down to leveraging the right product for your personal needs.

If we play out the same example, it's interesting that we see cases where markets and spaces hit a peak level of maturity — where consumers end up inevitably using both interchangeably.

That means when I pull out my wallet these days, and I look at my credit cards, I'm looking at both MasterCard and Visa cards. Or if I'm trying to physically ship something somewhere — a box to a different state or a box overseas to my grandparents — I use UPS and FedEx interchangeably. Or when my television remote runs out of batteries, and I need to restock, I go to the fridge, and I pull out both Duracell and Energizer batteries interchangeably. Markets always have room for a second, and we see Nomad as the alternative and supplement to Kubernetes.

It's been 10 years since Twitter first ran Apache Mesos in production as the first major and well-known use of an orchestrator in production. As a problem space and as a market, orchestration has shown that it's big enough — and enterprise needs are also diverse enough — that no one tool and no one solution can solve it all. But all product visions need to be grounded. We have to be upfront about where the gaps are for Nomad today so we can understand where we can take the product in the future.

Humility

Nomad's strength is not in its ecosystem. When we look at the CNCF diagram that showcases the span of the Kubernetes ecosystem today, we see it's this incredible diagram that's ever-changing and ever-growing of all these different vendors, integrations, and logos that we in Nomad can't easily replicate.

When you throw in things like Helm charts that make it even easier for Kubernetes practitioners to spin up these different integrations, we see that Kubernetes' strength is objectively in its ecosystem. We understand how ecosystem can be a deciding factor for practitioners when it comes to a situation of choosing an orchestrator.

Our ambition for Nomad, though, is to build an ecosystem in a very fundamentally different way. While the breadth and ever-growing scope of the Kubernetes landscape can be exciting, we think that there's a path to simplicity that we can uniquely carve out with Nomad. Nomad's ecosystem won't ever reach as many logos as Kubernetes. But we do believe we can come up with a simpler, leaner, and more prescriptive path to ecosystem.

In the past six months, we've factored this approach into our development. We've shipped integrations with Prometheus for observability and analysis. We've held community webinars on how to hook up Nomad with GitLab CI, and we've partnered with our friends at Datadog to come up with out-of-the-box consumable, ready-to-go Datadog dashboards for Nomad practitioners who are looking to get more analysis and observability into their clusters without digging through metrics. On top of all that, we've built, shipped, and maintained our own autoscaler in cluster and application auto-scaling for AWS.

By building for depth and on breadth, we believe that Nomad's ecosystem can be built through strategic partnerships and integrations with 1-2 of the leading partners from each of these wider orchestrator ecosystem categories.

When we factor in that approach with our existing integrations; with JFrog's Artifactory for artifact storage, with CNI and Portworx for stateful workloads, and with Spinnaker in the CI/CD space — that's coming soon — we believe that Nomad can provide a leaner, simpler, and prescriptive ecosystem for those practitioners that prefer simplicity and definition over comprehensiveness and complexity.

Yet we understand how our more deliberate, precise, calculated, and measured approach to building our ecosystem may not be a fit for those practitioners that want to move fast and be on the cutting edge of technology adoption when it comes to choosing an orchestrator.

 How Organizations Leverage Nomad and Kubernetes

As a result of this split — this contrast in strengths — one of the most exciting things that we've seen this year has been the organic emergence of a multi-orchestrator pattern. Where large enterprises like Intel — who will have this in their talk later on in our conference — are organically running both Nomad and Kubernetes at their companies. They're leveraging each of these tools towards its strengths.

As we've covered, Nomad's strengths objectively lie in its simplicity and flexibility. Its simplicity and easy to operate for small, lean teams with very little competence and background is one of its strengths — as well as its flexibility to deploy and manage containerized and non-containerized applications. Kubernetes' strength is a stark contrast in that there’s a lot of complexity. But it comes with a lot of powerful features and access to a cutting-edge ecosystem that is thriving and growing every day.

We see these companies utilize Kubernetes for the greenfield teams and projects that are ecosystem-dependent. They need the cutting-edge technology to power the revolutionary use cases like machine learning, artificial intelligence, big data, and serverless.

At the same time, we see this business value is there for Kubernetes for these business teams at these organizations. These high-value teams and greenfield projects naturally have the high staffing, budget, and the long-term timelines that are needed — and often required — to support running Kubernetes in the first place.

On the flip side, these same organizations take Nomad and insert Nomad for different business teams and projects. They leverage Nomad in the situations and environments that Nomad excels in with its simplicity. We're often talking about business teams or projects that are much smaller and leaner in staffing and resourcing.

These team profiles can range from anything like a brownfield team that has to build and maintain legacy Windows or Java applications to even greenfield product teams that are trying to build new products for their company that have to work on bare metal. We see in these situations that Nomad objectively excels in — like on-prem environmental constraints — that there are a lot less dependencies and requirements for that sprawling orchestrator ecosystem.

This is the multi-orchestrator future that we're beginning to see at enterprises where practitioners are looking at Nomad and Kubernetes like tools out of a toolbox. It's no longer a conversation of a one-size-fits-all of why you should use Nomad exclusively or why you should use Kubernetes exclusively. But rather when you should use Nomad and when you should use Kubernetes. How would you leverage each tool towards the strength and context of your overhead, your resourcing, the people that you have, the workload type you're dealing with, and the environmental constraints that you may have on your team?

Announcing Nomad 1.0

Ultimately, talk is great, but execution is what will matter most and will get us to a Nomad 1.0. I'm super proud and excited to announce on behalf of the team that Nomad 1.0 will be available two weeks from today on October 27.

As our first major release, Nomad 1.0 embodies a lot of the principles, direction, vision that we've covered today in our presentation. We're excited to share more and hope you can join us on the launch event on October 27, where Armon will be hosting us all again and walking us through the latest and greatest that's coming in Nomad 1.0.

There are a whole slew of new features and major improvements coming in Nomad 1.0, but also one cool and innovative one that we're working hard on behind the scenes — that we can't wait to reveal at that stage.

Open Source Namespaces

To close, we do have one more thing to share with everyone. Namespaces — which has historically been part of our closed source feature set — will be available for all open source users in Nomad 1.0.

Open-sourcing Namespaces means that as a Nomad practitioner, you can natively convert your clusters overnight into multitenant clusters. This means you can take your compute resources and your existing application deployments and securely and safely segment them across different development teams or projects within your organization.

We know that open-sourcing namespaces has been a long-awaited and much-asked request from the Nomad community. And on behalf of the team, we're happy and privileged to do this as a token of our gratitude and appreciation to those thousands of Nomad users around the world today — along with those to come in the future.

Thank you all for tuning in today. We hope to see you all again when we launch Nomad 1.0 on October 27.