The 2021 International Conference for High Performance Computing, Networking, Storage and Analysis (SC21) on Nov. 18 presented the inaugural Best Reproducibility Advancement Award to a Lawrence Livermore National Laboratory team for a benchmark suite aimed at simplifying the evaluation process of approximation techniques for scientific applications. Photo by Jeremy Thomas/LLNL.
A suite developed by a Lawrence Livermore National Laboratory (LLNL) team to simplify evaluation of approximation techniques for scientific applications has won the first-ever Best Reproducibility Advancement Award at the 2021 International Conference for High Performance Computing, Networking, Storage and Analysis (SC21).
Newly instituted by the conference, the award recognizes outstanding efforts in advancing transparency and reproducibility of methods for high performance computing (HPC). Conference organizers presented the inaugural award to the LLNL researchers for a paper describing their High Performance Approximate Computing (HPAC) framework, which allows users to easily explore the accuracy and performance trade-off of various approximation techniques on OpenMP HPC applications.
"We're quite honored to be receiving the very first SC Best Reproducibility Advancement Award," said principal investigator Harshitha Menon, a computer scientist in LLNL's Center for Applied Scientific Computing. "SC has been a pioneer in the reproducibility effort, so getting an award in this field at SC is very exciting. It speaks to the importance that SC gives to reproducibility and transparency in reproducing results."
As computing reaches the limits of Moore's Law - the concept that processing speed doubles about every two years - researchers are exploring new paradigms for increasing performance in HPC. While approximate computing - techniques that produce results that are "almost correct" - can substantially improve performance, adoption has been limited for scientific applications due to strict accuracy requirements, Menon explained.
The uncertainty and hesitancy of scientists to rely on the end results necessitates a better understanding of the performance and accuracy tradeoff in approximation and ensure methods used for evaluation are reproducible, researchers said.
"Approximate computing has been studied for many years and it has gotten some traction over the past decade, but most of the current solutions are more of an ad hoc solution, and this has resulted in implementations that are lacking," said lead author and LLNL postdoctoral researcher Konstantinos (Dinos) Parasyris. "In our case, we tried to use state-of-the-art tools, and we delivered software that can be used by somebody else."
In designing HPAC, the LLNL team integrated cutting-edge approximation techniques (loop perforation, input/output memorization), with common LLVM/Clang compiler and OpenMP runtime support. By staying close to the OpenMP interface, HPAC makes it simple for users to specify which technique they want to use, annotate an area of the code to try it on and get an initial idea of performance gains, researchers said. Users can compare different approximation techniques and, depending on their application and error tolerance, can make an informed choice about whether techniques are useful to them or not, Menon said.
"The framework provides a way to generate the plots and say, 'this is the range you get for these particular approximate techniques, and now it's up to you to pick the suitable error range for your application and maximum speed-up,'" Menon said. "There's a bit of an accuracy-versus-performance-tradeoff game we are playing here, so depending on what the user needs, they can go one way or another. Usually, there's a sweet spot."
In the paper, the team applied HPAC to eight common HPC benchmarks, finding that the suite provided considerable performance gains for given error thresholds. For example, in the LULESH benchmark - which approximates hydrodynamics equations - they found approximation provides substantial performance gains of up to 1.7 times, due to the reduction of required memory. In an application called leukocyte - which detects and tracks white blood cells in video microscopy of blood vessels - the team discovered that, beyond increasing speed by increasing the number of threads, HPAC provided an additional speed-up of up to 25x at an error threshold of less than 5 percent.
Funded by the Laboratory Directed Research and Development program, HPAC is part of an LLNL "ApproxHPC" project aimed at evaluating approximation techniques and creating tools to improve confidence in approximate computing for scientific applications in the post-Moore's Law era. The team is continuing the work by further exploring techniques and improving understanding of error sensitivity.
The HPAC software is available on github. Co-authors include LLNL scientists Giorgis Georgakoudis, James Diffenderfer, Ignacio Laguna, Daniel Osei-Kuffuor and Markus Schordan.
