The multi-million player platform

In 2017, Roblox was growing rapidly, as its monthly active users (MAUs) reached new heights. Internal engineering teams were scaling as well — driving significantly higher levels of resource consumption, capacity demands, and frequency of changes.

Rob Cameron, technical director of cloud services, realized that their infrastructure could not keep up. Dedicated servers were leading to increased resource waste. The company’s reliance on manual workflows and homegrown tooling resulted in significant productivity bottlenecks. Prior to deploying Nomad, Roblox was challenged with:

Cameron knew that Roblox needed an orchestrator to modernize its infrastructure — a tool that could enable resource management, efficient scheduling, container adoption, and developer velocity at scale.

"Almost every weekend is the biggest weekend we've ever had in Roblox, our infrastructure had just become unmanageable to deal with application deployment in the old way," said Cameron.

Today, Roblox has deployed Nomad on thousands of nodes in clusters across bare metal and cloud.

Improved Productivity

Roblox wants to maintain performance of a large scale platform without an overreliance on additional headcount. The operational simplicity of Nomad is the key to fast adoption and high productivity. Nomad allows Roblox to reduce time on learning, debugging, and fixing infrastructure so that engineers can spend more time on adding value to its core gaming business. Other teams within the infrastructure group are able to assist in managing the Nomad deployments because the technology is easy to learn.

• Minutes to deploy an application globally
• Minutes to onboard a new developer into deploying applications onto Nomad
• Created flexible workload support that enables Roblox to reliably serve users in 180+ countries

"We have people who are first-time system administrators deploying applications, building containers, maintaining Nomad. There is a guy on our team who worked in the IT help desk for eight years — just today he upgraded an entire cluster himself."

"That’s the value proposition that I hope people understand. People seem to get stuck on ‘I need to run Kubernetes because my friend runs it’ — but do you really use it? Can you operate it at the level that’s needed?"

Cost Savings

With the right technologies and focus, Roblox successfully implemented its containerization strategy, which helps the company to scale efficiently in dollars and personnel. By containerizing their legacy game engine, upgrading to 64-bit Linux CPU, and adopting Nomad as the single orchestration platform, Roblox achieved:

• Resource utilization — run double the workload on the same hardware
• Save millions in Windows licensing
• Zero downtime to migrate application deployments from on-prem to AWS

"It takes no effort. We take a data center in Nomad and split it across AWS and on-prem and just increase compute in AWS and decrease compute in our local data center. We drain nodes in Nomad and the applications/jobs migrate silently from on-prem to AWS with no understanding needed by the end developers."