HashiCorp Products Used
- 100s of billions in loans for development programs
- 2,000+ configurations items in datacenter
- Reduced VM provisioning time from 3 days to 2 minutes
- 40 locations within Asia-Pacific region
- Automated processes available 24/7 instead of 8/5
- Improved regulatory compliance around provisioning governance
Asian Development Bank (ADB) is a mission-based bank providing grants, loans, and technical assistance to developing countries to help them tackle complex problems, such as water, transport, climate change, and even gender equality. Established in 1966 and based in Manila, Philippines, ADB is owned by 68 member countries, including 48 from the region, and employs more than 6,000 workers worldwide to help carry out its mission of eradicating extreme poverty and creating a prosperous, inclusive, resilient, and sustainable Asia-Pacific region.
By writing every single change as code, we have the latest source of truth — or the latest configuration in place — as a blueprint in our code.
Krista Camille Lozada, Sr. IT Specialist, Innovation & Engineering
A bank on a mission
When most people think of banks, they conjure images of checking and savings accounts or meeting with a mortgage officer for help buying a new house. But Asian Development Bank (ADB) isn’t your average bank.
Similar to well-known, mission-based organizations like the World Bank, ADB exists to provide technical assistance and financial support through grants and government-backed loans to programs aimed at tackling complex problems.
Reliably delivering much-needed support for more than a quarter of a billion people demands efficient, stable, and secure technology infrastructure. The organization had already begun its digital transformation in 2018 as part of an initiative to gain greater flexibility and reliability across its infrastructure — particularly in a region prone to devastating earthquakes that could take essential services offline.
But like so many organizations around the world, the onset of the COVID-19 pandemic dramatically accelerated ADB’s timeline for moving more of its core infrastructure to the cloud.
“Before COVID, we had already felt the importance of strengthening our resilience via automation.,” says Krista Lozada, who is in charge of infrastructure innovation, and engineering at ADB, handling everything from integration to mobility. “So when there was a massive surge in demand and network traffic as ADB began operating completely remotely, coupled with the bank tripling its COVID-response to more than $20 billion worth of initiatives, we knew it was the best time to move away from conventional cloud migration strategies. We needed an intuitive, automated way to stand up cloud infrastructure quickly but without sacrificing security or resilience.”
Extraordinary times demand an extraordinary approach to infrastructure
With more than 40 offices in the Asia-Pacific region, ADB works hard to be as close to the people and organizations it serves as possible. However, the same couldn’t be said for the bank’s IT infrastructure, which had traditionally lived at the bank’s Philippines headquarters.
In addition, unlike commercial banks that typically migrate core systems and workloads to the cloud in search of a competitive edge through cost management and business efficiency, ADB sought digital transformation to protect its business — and constituents – from the unknown.
“At one point, we had a disaster recovery (DR) location in Geneva, Switzerland, that was quite far from our home datacenter and wasn’t strategically advantageous for us because we couldn’t easily replicate data from Manila and transfer it over,” explains Lozada. “Knowing that any unforeseen interruption to our operations could adversely affect millions of people and living under the constant threat of a once-a-century earthquake poised to hit Manila at any time, we needed to move as much as we could to the cloud quickly, safely, and intelligently.”
The first step meant swapping ADB’s legacy Lotus Notes secrets management solution for HashiCorp Vault, a more elegant and adaptive secrets engine used to manage the company’s vaccine management system which contains confidential health records. Storing individual user keys and encrypting database information, ADB built an entire vaccine management system for the organization within days and further leveraged Vault’s PKI rotate ephemeral boardroom device certificates for effortless, automated authentication to protect highly sensitive information.
From there, the plan was to begin setting up a new datacenter using Azure located in the Singapore region, closer to its Philippines headquarters, implementing modern practices and technologies to do so. But the rapid and unprecedented spread of COVID-19 threw a wrench in those plans, sending ABD’s infrastructure team scrambling to change course.
“Ordinarily, we’d take a methodical approach to building out a new disaster recovery facility, choosing each tool and platform, testing them, and then strategically deploying them when we’re ready,” Lozada says. “But, like everyone else, we were forced overnight into working fully remotely, which we hadn’t done before. And because the datacenter build was central to our digital transformation agenda, we chose to prioritize building out the foundational infrastructure over other important items on the to-do list, which meant finding a simple, effective, and efficient way to stand it up on a short timeline.”
Challenges
Migrating operations to the cloud while boosting security, speed, and resiliency
Building a new infrastructure leveraging the bank’s existing framework
Rapidly spinning up datacenters in multiple locations around the world
Making sure the transition to the cloud was as seamless as possible
Doing it all just as the COVID-19 pandemic took hold
Why Terraform
With Terraform, we can make changes or spin up new infrastructure with just a few clicks and a couple of minutes.
Krista Camille Lozada, Sr. IT Specialist, Innovation & Engineering
Fast, flexible, and scalable infrastructure deployment Working fully remotely for the first time and under increasing pressure to complete a core component of the bank’s digital transformation efforts, Lozada’s ADB team focused on ways to accelerate provisioning its Day Zero datacenter infrastructure.
Given the team’s positive experience with HashiCorp Vault and Consul for its secrets engine, Lozada says that staying within the HashiCorp ecosystem with HashiCorp Terraform made the most sense because its common user experience and simple, English-like HashiCorp Configuration Language (HCL) didn’t require special skills or a long learning curve.
“Our network team doesn’t write scripts and our Linux and Windows teams usually just work in their own languages on their own projects,” she explains.“Terraform’s HCL is the simplest way of getting everyone speaking the same language and pulling in the same direction toward our common business goal.”
That common business goal — building out the cloud infrastructure for a new datacenter as quickly as possible— featured some uncommon strategies. Specifically, ADB uses Terraform alongside its existing Infrastructure-as-a-Service (IaaS) solution, using the HashiCorp software to build out the initial infrastructure and ADB’s other tools to handle everything that comes afterward.
“People generally just want to use one platform instead of multiple ones, but we saw it a bit differently,” Lozada explains. “If we were creating a new restaurant, Terraform would be the one actually building the facility. Our other tools would be the chef who creates the menu and serves the customers.”
Lozada’s team built out its entire datacenter’s cloud infrastructure using Terraform open source and Microsoft Azure DevOps Server, which provides developer services for teams to plan work, collaborate on code development, and build and deploy applications. The team subsequently used Terraform OSS and Azure DevOps to put in place its Terraform Enterprise instance. They then used that instance of Terraform Enterprise as the automation engine to complete the whole virtual datacenter build.
With Terraform, ADB can automate the provisioning of all its cloud infrastructure in Microsoft Azure with no manual intervention or the need for user access. The solution features infrastructure automation workflows for building composition, collaboration, and reusing infrastructure as code while providing standardization for security, compliance, and management.
While it might have been faster to do the first iterations manually, infrastructure as code gave the bank the flexibility to change on demand. “By writing every single change as code, we have the latest source of truth — or the latest configuration in place — as a blueprint in our code,” Lozada says. “The team loves putting everything in as code so much that users have only read access to the bank’s cloud infrastructure. If you need to change any infrastructure, you change it at the code level. “ Parallel processes for rapid build-out Lozada says that using Terraform — along with complementary HashiCorp solutions like Packer and Consul — helped the ADB team accomplish what, at one time, seemed impossible. “One of the biggest challenges we faced was building everything in parallel because the statuses of other aspects of the datacenter build-out changed every week,” she says. “With Terraform, we can make changes or spin up new infrastructure with just a few clicks and a couple of minutes. If we’d done it manually and sequentially like most processes, it would have taken at least 45 days for our part of the build to be ready.”
Lozada says that deploying every component of their cloud infrastructure — from spinning up new virtual machines and configuring more than 2,000 items in the datacenter to completing a whole new data warehouse — takes a fraction of the time and effort as standard methods.
“Spinning up a new virtual machine used to take two or three days each, but now takes just a few minutes,” she says. “More importantly, with Terraform we can virtually guarantee compliance with data governance standards and meet the demands for systems availability simply by standardizing our processes across the board and reusing the code we’d built in Azure.”
With its DR site fully built out, and the code base available should the bank decide to add more failover options to its footprint in the future, Asian Development Bank is well positioned to continue lending its support to underserved populations across the region. “We play such an important role in the lives of people throughout Asia, and modernizing our IT infrastructure is vital to continuing our mission. Terraform and the support from the HashiCorp team have been instrumental in our transformation.”
Outcomes
Reduced VM provisioning time from 3 days to 2 minutes
Continuously updated build process with Packer and no hardcoded password in the image
Gained the ability to deploy a complete data warehouse in less than 5 minutes
Built out the virtual datacenter in less than 5 days, configured the 2,000+ assets in less than 45 days
Automated and standardized network configuration to enhance resiliency, flexibility, and security
Solution
Asian Development Bank adopted Hashicorp Terraform to reduce its VM provisioning time, automate its infrastructure, and dramatically accelerate the build out of its disaster recovery datacenter without sacrificing improving security or network resiliency.
Asian Development Bank Partner
Krista Camille Lozada Sr. IT Specialist, Innovation & Engineering Asian Development Bank
Krista Camille Lozada is a senior IT specialist at Asian Development Bank where she leads the middleware integration mobility and desktop engineering team. Krista is a passionate advocate for automation, DevOps, and Cloud Native architecture and aims to automate her work so she can kick back, relax, and play as a psionic elf in D&D.
Technology Stack
- Infrastructure :
- Microsoft Azure, on-premises (1% bare metal, 99% virtualized)
- Workload type:
- Windows (50%), Linux (50%)
- Container Runtime :
- Docker, Podman
- Orchestrator :
- Azure Kubernetes Service, HashiCorp Nomad
- CI/CD :
- Azure Devops
- Version Control :
- Git, SVN
- Provisioning:
- Hashicorp Terraform Enterprise
- Configuration management:
- Ansible Automation Platform
- Security management:
- Hashicorp Vault, Azure Key Vault