Intel oneAPI Academic Center of Excellence in the UK 

Durham Computer Science Department to Launch First oneAPI Academic Center of Excellence in the UK

Durham University’s Department of Computer Science is excited to host Intel’s first United Kingdom (UK) oneAPI Academic Center of Excellence (CoE). The centre will conduct research around task-based programming using oneAPI development for heterogeneous architectures and will champion oneAPI training. Our goal is to extend the hyperbolic PDE engine ExaHyPE into a OneExaHyPE code that scales across a wide variety of GPU-accelerated machines. The algorithmic and methodological insights will be beneficial for many other simulation codes. Furthermore, the centre will organise workshops and tutorials around oneAPI cross-architecture programming. These will be open to Durham students and faculty, and available to colleagues from all over the UK and beyond.

oneAPI Training

ExaHyPE is an engine, i.e. a generic collection of state-of-the-art numerical ingredients (high-order time integration, high-order DG representations, block-structured Finite Volume methods, dynamically adaptive Cartesian meshes, task-based load-balancing, and so forth) to solve hyperbolic equation systems given in first-order formulation. The engine allows users to define what (equations) they want to solve, and then it decides how to solve them, where and in which order. It also commits to a particular set of numerical techniques. This degree of freedom on the application domain side in combination with the methodological focus opens the door for various algorithmic optimisations, as all ingredients can be tightly integrated and are aggressively optimised towards each other. Codes using the engine are currently used to implement solvers simulating various phenomena ranging from gravitational waves to tsunamis and earthquakes. Written originally under the umbrella of a FET HPC H2020 grant, the second generation of the code is a complete rewrite and a UK ExCALIBUR showcase code and backbone of an Excalibur Design and Development Working Group. The Exascale Computing ALgorithms and Infrastructures Benefiting UK Research (ExCALIBUR) programme’s strategic vision is to prepare the UK for the exascale era. It also serves as one of three showcase codes for ExCALIBUR’s cross-cutting task-parallelism theme.

ExaHyPE’s compute core is entirely written in C++, and as of summer 2021, has a backend for OpenMP 5 to realise both its task-based parallelism and generic compute kernels in classic BSP style. It also offers OpenMP offloading to GPUs. This homogeneous software view of the hardware behind the code yields a maintainable, C++-only, vendor-independent code base. Through collaborations between scientists and research software engineers from Intel and Durham’s Advanced Research Computing, the OneExaHyPE team at Durham’s Department of Computer Science will develop an alternative oneAPI version. A oneAPI implementation will allow us to intermix the scheduling of data-parallel and task-parallel regions. In state-of-the-art codes, these two paradigms often compete with each other, while data-parallel regions are either completely deployed to a GPU, an FPGA, or not at all. In OneExaHyPE, we plan to study modes to balance between the parallelisation strategies, and to identify well-suited problem cardinalities for an accelerator on-the-fly.

“Current HPC codes often run efficiently either on multicore nodes or accelerators, but typically struggle to balance between the two paradigms and to get the best performance out of both architectures working together” says Tobias Weinzierl, Durham’s Principal Investigator. “The added value and big promise behind oneAPI is that we get one programming model for all parts of the machine and then can let algorithms decide dynamically which steps of the code to run where.”

ExaHyPE’s new OneExaHyPE generation will provide opportunities to extend new research in astrophysics. The team around Weinzierl joined forces with colleagues from Durham’s Institute for Computational Cosmology (ICC) to study new gravitational models and theories and to better understand how systems of binary black holes or neutron stars merge. Such mergers emit gravitational waves that we can measure, and serves as cosmological test benches for new theories. ExaHyPE’s other traditional application is seismic waves, where achieving better performance and scalability will help scientists improve our understanding of these waves and their interplay with the actual fault geometries and material ruptures. The biggest promise behind OneExaHyPE’s research, however, is the identification, validation and documentation of novel parallelisation patterns and concepts that will help many other codes to exploit the added value behind oneAPI: multi-architecture performance portable, scaling, maintainable code.

“Durham has always been a stronghold of theoretical computer science. It hosts three regional and national supercomputers for the University, DiRAC and the N8 (the N8 Research Partnership is a collaboration of the eight most research-intensive Universities in the North of England). This makes Durham a unique place to be for the enablement of next-generation research through [the embedding of] Exascale methods and technologies” clarifies Alan Real, head of Durham’s directorate of Advanced Research Computing (ARC). Durham also is home to international research institute flagships such as the ICC, whose research is relying on top-notch, large-scale computational infrastructure. Finally, the University recently started to invest into the provision and promotion of Research Software Engineers (RSE) – a role that offers an alternative to the classical academic career path for colleagues who want to focus on software as an enabling technology. It is the combination of theory- and algorithm-driven computer scientists, RSEs, compute centre specialists and team work with Intel engineers which will deliver the research programme.

The oneAPI endeavour also complements Durham’s recent research projects funded under the UK’s exascale programme ExCALIBUR. These include a project around task-based parallel programming in collaboration with Hartree and the MetOffice, two projects around ExaHyPE and the SPH code SWIFT (a former Intel Parallel Computing Centre code), as part of a Design and Development Working Group, along with novel hardware testbeds as part of ExCALIBUR’s Hardware and Enabling Software activities.