As of the 63^rd edition of the TOP500, the Aurora supercomputer out of Argonne National Laboratory has officially broken the exascale barrier with an HPL score of 1.012 exaflops. This is an enormous improvement over the machine’s score of 585.34 petaflops from the previous list and a major achievement for the Argonne Leadership Computing Facility (ALCF) team.

Aurora still isn’t finished, and this current benchmark was achieved with only 87% of the system online. The machine has 9,264,128 total cores, it is based on the HPE Cray EX – Intel Exascale Computer Blade, and uses Intel Xeon CPU Max series processors, Intel Data Center GPU Max Series accelerators, and a Slingshot-11 interconnect.

Despite taking second place in the overall list, Aurora snagged the top spot on the HPL-MxP mixed-precision benchmark with 10.6 exaflops of AI performance – beating out Frontier’s score of 10.2 exaflops.

As AI continues to dominate every discussion about HPC and computing, the HPL-MxP score will continue to become more relevant. However, distinguishing this benchmark from the traditional HPL requires some understanding of how supercomputer performance is measured.

Here, we’ll differentiate between these two benchmarks and discuss how Aurora’s team plans to use the machine’s AI capabilities.

How Do HPL and HPL-MxP Differ?

 

At its core, all computing is just math. From viewing cute cat photos on Reddit to discovering new medicines, computers work by solving math problems in specific and useful ways. Therefore, to measure these machines, it makes sense to give them math problems and see how long it takes them to solve those problems.

This is the basis for the HPL benchmark, which stands for High-Performance Linpack. HPL is one of a group of measurements that make up the LINPACK Benchmarks. Introduced by Jack Dongarra, all LINPACK Benchmarks measure how fast a computer solves a dense n by n system of linear equations.

HPL is a software package that solves a random dense linear system, asking machines to solve a big math problem using 64-bit numbers. These 64-bit numbers give highly precise answers to the problem.

Conversely, HPL-MxP asks the machine to solve the same big math problem as HPL with one big difference. Instead of doing all the calculations with super precise 64-bit numbers, systems measured by HPL-MxP first do most of the mathematical work using smaller 16-bit or 32-bit numbers. These numbers allow the machine to do a lot of the work faster but with less precision. At the end of this process, HPL-MxP uses a special method to refine the answer given by the machine and get it back to full 64-bit precision.

The reason this method is used to test AI capabilities is that AI and machine learning generally don’t demand a high level of precision – they just need a lot of speed. Additionally, the GPUs that are driving the AI revolution are also really good at quickly doing calculations with smaller, less precise numbers.

As AI projects continue to grow, and as full 64-bit precision becomes less necessary for many real-world applications, HPL-MxP will become more important to model performance.

What Will Aurora Do With All This AI Power?

 

Aurora was always meant to be an AI-centric system, and its win on the HPL-MxP is proof that the machine is an AI powerhouse. In fact, the ALCF has claimed that the supercomputer’s 63,744 GPUs make it the world’s largest GPU-powered system.

But even with all the incredible hardware contained within the machine, Aurora is just an expensive paperweight without any problems to solve. Thankfully, the ALCF has some exciting projects for this system.

“Aurora’s hardware excels at tackling both traditional scientific computing problems and AI-powered research,” said Rick Stevens, Argonne’s associate lab director for Computing, Environment and Life Sciences, in an article from the ALCF. “As AI continues to reshape the scientific landscape, Aurora gives us a platform to develop new tools and approaches that will significantly accelerate the pace of research.”

In the wake of the COVID-19 pandemic, computational drug discovery has become an even hotter topic within the HPC community. Aurora’s AI capabilities make it a perfect candidate for drug discovery, and the ALCF team is already working to put the machine through its paces. Researchers are creating AI workflows to use Aurora to sift through huge databases of chemical compounds in search of medicines that could treat some of the most deadly diseases.

The team was able to screen 11 billion drug molecules per house using 128 nodes of Aurora and then double the number of nodes to 256 to demonstrate linear scaling and screen 22 billion molecules per hour. Work is still progressing here, but ALCF hopes to someday soon screen 1 trillion candidates per hour once Aurora is fully completed.

In a similar vein of computational biology, ALCF scientists are also using Aurora to develop deep learning models to advance research aimed at mapping neurons in the brain along with their tens of thousands of connections.

Early runs of this project have been promising, and the team anticipates being able to reconstruct segments of the brain using datasets that are 1,000 times larger than their initial computations. The computational methods employed by the researchers are facilitating the progression from the current mapping of cubic millimeters of brain tissue to the full mapping of a mouse brain’s cubic centimeter on Aurora and other supercomputers in the future.

On top of working on these microscopic problems, researchers are also using Aurora to model some of the biggest cosmological systems known to humankind. With Aurora, scientists may add more detail and complexity to their cosmological models, leading to new understandings of the universe’s dynamics and structure.

An early science team has employed about 2,000 Aurora nodes in initial runs to generate simulations and images of the universe’s large-scale structure. These efforts have shown excellent single-GPU performance and demonstrated close to perfect scaling of performance for the full machine. Exascale simulations produced by the researchers are anticipated to be crucial in confirming and improving our knowledge of cosmic evolution.

Aurora is already an incredible machine, and the HPC community is anxiously waiting to see what the machine will be capable of when it is fully completed. But even with what we’ve seen so far, Aurora has some incredible potential within AI applications.