Preparing for Exascale: ALCF’s Aurora Early Science Program and Visualizing the Spread of Cancer
Scientists are preparing a cancer modeling study to run on Argonne’s new Aurora supercomputer before it goes online in 2022.
The US Department of Energy’s Argonne National Laboratory (DOE) will be home to one of the country’s first exascale supercomputers – Aurora is slated to arrive in 2022. To prepare codes for the architecture and scale of the system, 15 research teams are participating in the Aurora Early Science Program through the Argonne Leadership Computing Facility (ALCF), a user facility of the DOE Office of Science. With access to preproduction time on the supercomputer, these researchers will be among the first in the world to use an exascale machine for science.
Over 60,000 miles of blood vessels carry nutrients around our body. However, sometimes they can carry something much more dangerous – cancer.
Cancer cells can spread from one part of the body to another by traveling through our blood vessels. Understanding how metastasis occurs can guide the development of new drugs and treatments. Randles’ team will use Aurora to create high-resolution representations of the geometry of the circulatory system, simulating fluid flow, as well as collections of red blood cells and tumor cells moving in the fluid flow.
Amanda Randles, an assistant professor at Duke University, uses modeling and simulation to find answers. She is one of the few researchers selected to participate in the ALCF’s Aurora Early Science Program (ESP). Randles and her team developed HARVEY, a model that simulates blood flow throughout the body. Now they want to use it to predict the movement of cancer cells on a microscopic level.
“As we tackle our new research on the metastasis process and do the intricate simulations we need, we need even more computing power to process the huge amounts of data in real time,” she said. “The Aurora system will help us meet this need.”
The Randles team is working with Argonne scientists to prepare their application to run on the new system. A major challenge: the discrepancy between computing speed and the speed with which data can be saved. Aurora allows the computation to be done at lightning speed, but the speed at which data can be stored for longer periods of time (by writing it to disk) cannot match this speed, a known loophole that has only grown over the years.
Without an address, the researchers would have to slow down their calculations by orders of magnitude in order to write all of their simulation data to the hard drive. However, this approach would be time consuming and require massive storage space. For the Randles team and many others, this is impractical given the volume of data and the workload.
Alternatively, the HPC community endeavors to enable data analysis on site (ie, in computer memory) while simulations are running. By visualizing and analyzing data in this way, the computation can still be done quickly, and researchers can decide which data to store or discard over the long term.
“For example, if you identify a specific area of interest throughout the system, you can store data in that small area with a higher frequency or a higher level of detail. This would allow you to continue to get more science out of the data while reducing the amount of data you actually need to write to disk, ”said Joseph Insley, head of ALCF’s visualization and data analysis team.
As part of a DOE project called SENSEI, Insley and ALCF computer scientist Silvio Rizzi are working with other researchers across the DOE laboratory complex and across the industry to create a unified interface that allows scientists to access a library of frameworks that support the analysis and visualization of in-situ data. With access to early hardware and the Aurora software development kit, the ESP team tested and refined these features before the exascale machine arrived.
“We used the ALCF theta supercomputer and the early Aurora hardware to incorporate this library into the HARVEY code. By enabling visualization and analysis of the data while it is still in memory, the science team can gain more insight from the data than they would otherwise, ”said Insley.
Ultimately, the team’s research on Aurora aims to advance the development of new cancer drugs and treatments that have the potential to save lives in the years to come.
“By understanding the biological mechanisms behind metastatic cancer cells, we hope that our work with HARVEY will ultimately help doctors and their patients fight cancer,” concluded Randles.
Randles’ ESP project “In-Situ Visualization and Analysis of Fluid-Structure-Interaction Simulations on an Extreme Scale” is supported by the DOE’s Advanced Scientific Computing Research program.
Source: US Department of Energy news source