HEP has also already reached the level of “exascale data”, however, the present community codebase is completely unmatched to the task of fully exploiting that data. Codes have evolved slowly over time, and are often decades old, dating from an entirely different era of hardware and software technology. What is now needed is a serious programme of software engineering effort, for the full and proper task of refactoring and re-writing HEP code to make it fit for the exascale era – for the data volumes, velocity and veracity that will underpin the exascale era. During the first phase of ExCALIBUR, the HEP working group will provide demonstrators of exascale algorithms and next generation data management infrastructure.
The four focus areas that are at the core of the HEP computing algorithms and infrastructure development are described below.
Exascale Data Organisation and Management – It would be prohibitively expensive, at exascale and beyond, for all data to be managed on high throughput, low latency storage. In practice, data are likely to be distributed across many sites, with different storage technologies providing different qualities of service. Object stores provide extremely scalable, high throughput, cost effective storage, achieved by only providing a basic set of access methods. To reach exascale performance metrics, services need to be able to automatically adapt to rapidly evolving use cases.
Objective – To develop the means to access and transport distributed exabyte-scale data sets and deliver them to matched computational nodes; this is a critical task for all science areas that will eventually generate such distributed big-data sets.
Real-Time Tracking – Data from the LHC (Large Hadron Collider) experiments are produced at rates far in excess of those that can be analysed offline. This means that data need to be processed online through a trigger system that selects interesting collisions and discards less-interesting events.
Objective – Inspecting, classifying and selecting data from exabyte scale data flows, comparing FPGA (Field Programmable Gate Array) and GPU technology. This is relevant to many other areas where data output rates are rising exponentially, and it will be infeasible to store all data (i.e. in-situ analytics).
Detector Simulation – Simulations of particle transport through matter are a foundation of designing and understanding HEP experimental detectors. This represents a complex and highly CPU-intensive computational challenge as the modelled physical processes involve particle interactions with the production of cascades of secondary particles, all of which must be tracked through large, complex geometries to very high precision.
Objective – To advance the means to run detector simulations on compute accelerators, developing a toolkit that will have strategic implications for numerous other disciplines that utilise these codes.
Portable Parallel Strategies
Objective – To evaluate high level APIs that allow that automatically compile code for different architectures (CPU, different GPU brands, FPGA, …) with the goal of decoupling the codebase from hardware technology evolution (creating a so-called “Separation of concerns” and building in performance portability to the HEP code-base.
