High performance computing (HPC) in the financial services industry is an ongoing challenge because of the pressures from ever-increasing demand across retail, commercial, and investment groups, combined with growing cost and capital constraints. The approaches to solving these problems have evolved over generations from centralized, monolithic solutions, to business-aligned clusters of commodity hardware, to modern grid architectures with centralized schedulers that manage disparate compute capacity. Regulators and large financial institutions are increasingly accepting hyperscale cloud providers, which has resulted in significant interest in how to best leverage new capabilities while ensuring good governance and cost controls. Cloud concepts such as capacity on demand and pay as you go pricing models offer new opportunities to teams who run HPC platforms. Historically, the challenge has been to manage a fixed set of on-premises resources, while maximizing utilization and minimize queuing. In a model with capacity that is effectively unconstrained, the focus shifts away from managing and throttling demand towards optimizing supply. With this model, decisions become more granular and tailored to each customer, and focus on how fast and at what cost, with the ability to make adjustments as required by the business. With this basically limitless capacity, concepts such as queuing and prioritization become irrelevant as clients are able to submit calculation requests and have them serviced immediately. This also results in upstream consumers increasingly expecting and demanding near instantaneous execution of their workloads at any scale. Initial cloud migrations of HPC platforms are often seen as extensions or evolutions of on-premises grid implementations. However, forward-looking institutions see much in common with the patterns of HPC and serverless execution models, such as AWS Lambda. Both solutions focus on executing code on demand, and customers want the lowest cost allocation of capacity with no provisioning or management of servers. As HPC environments move to the cloud, the applications that are associated with them start to migrate too. Risk management systems which drive compute grids quickly become a bottleneck when the downstream HPC platform is unconstrained. By migrating applications with the compute grid, they also benefit from the elasticity that the cloud provides. In turn, data sources such as market and static data are sourced natively from within the cloud, from the same providers that customers work with today. Many of the building blocks required for fully serverless solutions for risk management and reporting already exist today within AWS services. As financial institutions become increasingly familiar and comfortable with these services, it’s likely that serverless patterns will become the predominant HPC architectures of the future.

Read the entire whitepaper here.