Document Repository

 RESTRICTED (TBC)

[…] This chapter discusses the benefits and application of Open Science within the community, and explores the current and future perspectives for the community. This is divided into the several “pillars” of Open Science, namely: Open Science developments, Open Access publications, Open Data and lifecycle, Open Software and workflows, Infrastructures for Open Science, and Nuclear data evaluation.

PUBLIC 

The present data management policy pertains to the ownership of, the curation of and access to experimental data and metadata collected and/or stored at GANIL. […]

PUBLIC 

This document details how to publish data and software to an external repository, make records in the JOIN2 GSI publications repository (https://repository.gsi.de/), and subsequently link them together with the publication record.

 RESTRICTED (TBC)

With the advancing digitization of research and teaching, the number of scientific software solutions is constantly increasing. In general, scientific software should be released as freely as possible on a trusted infrastructure, if there is no exploitation option. Adequate embargo periods may apply under certain conditions, such as termination of theses and publications, as well as maintaining competitive advantage. These guidelines are based on the recommendations formulated by the task force on Open Source licensing at CERN.

PUBLIC 

This document describes the conceptual design for the Offline Software and Computing for the Deep Underground Neutrino Experiment (DUNE). The goals of the experiment include 1) studying neutrino oscillations using a beam of neutrinos sent from Fermilab in Illinois to the Sanford Underground Research Facility (SURF) in Lead, South Dakota, 2) studying astrophysical neutrino sources and rare processes and 3) understanding the physics of neutrino interactions in matter. We describe the development of the computing infrastructure needed to achieve the physics goals of the experiment by storing, cataloging, reconstructing, simulating, and analyzing 

PUBLIC 

This white paper briefly summarized key conclusions of the recent US Community Study on the Future of Particle Physics (Snowmass 2021) workshop on Software and Computing for Small High Energy Physics Experiments.

PUBLIC 

[…] The talk will present the experience from running the O2 framework in production during the 2022 ALICE data taking, with particular regard to the GPU usage, an overview of the current state and the plans for the asynchronous reconstruction, and the current performance of synchronous and asynchronous reconstruction with GPUs for pp and Pb-Pb data.

PUBLIC 

PUBLIC 

Common and community software packages, such as ROOT, Geant4 and event generators have been a key part of the LHC’s success so far and continued development and optimisation will be critical in the future. The challenges are driven by an ambitious physics programme, notably the LHC accelerator upgrade to high-luminosity, HL-LHC, and the corresponding detector upgrades of ATLAS and CMS. The General Purpose Detectors describe their specific challenges elsewhere; in this document we address the issues for software that is used in multiple experiments (usually even more widely than ATLAS and CMS) and maintained by teams of developers who are either not linked to a particular experiment or who contribute to common software within the context of their experiment activity. We also give space to general considerations for future software and projects that tackle upcoming challenges, no matter who writes it, which is an area where community convergence on best practice is extremely useful. […]

PUBLIC 

Software and Computing (S&C) are essential to all High Energy Physics (HEP) experiments and many theoretical studies. The size and complexity of S&C are now commensurate with that of experimental instruments, playing a critical role in experimental design, data acquisition/instrumental control, reconstruction, and analysis. Furthermore, S&C often plays a leading role in driving the precision of theoretical calculations and simulations. Within this central role in HEP, S&C has been immensely successful over the last decade. This report looks forward to the next decade and beyond, in the context of the 2021 Particle Physics Community Planning Exercise ("Snowmass") organized by the Division of Particles and Fields (DPF) of the American Physical Society.

PUBLIC 

The Phase-2 upgrade of CMS, coupled with the projected performance of the HL-LHC, shows great promise in terms of discovery potential. However, the increased granularity of the CMS detector and the higher complexity of the collision events generated by the accelerator pose challenges in the areas of data acquisition, processing, simulation, and analysis. These challenges cannot be solved solely by increments in the computing resources available to CMS, but must be accompanied by major improvements of the computing model and computing software tools, as well as data processing software and common software tools. In this document we present aspects of our roadmap for those improvements, focusing on the plans to reduce storage and CPU needs as well as take advantage of heterogeneous platforms, such as the ones equipped with GPUs, and High Performance Computing Centers. We describe the most prominent research and development activities being carried out in the experiment, demonstrating their potential effectiveness in either mitigating risks or quantitatively reducing computing resource needs on the road to the HL-LHC.

PUBLIC 

[…] ATLAS produced a Conceptual Design Report (CDR) for HL-LHC Computing during the spring of 2020 for an initial review by the LHCC. The CDR laid out the issues discussed above, and the general approaches that will be taken to address them. This new document serves as a software-focused update to the first CDR, providing more concrete information on development work that will be undertaken in the coming years, listing specific milestones and target dates. Additionally, the document describes how ATLAS collaborates with external activities and projects, and how such collaboration will impact the overall development for HL-LHC.

PUBLIC 

[…] The Portable Parallelization Strategies team of the HEP Center for Computational Excellence is investigating the use of Kokkos, SYCL, OpenMP, std::execution::parallel and alpaka as potential portability solutions that promise to execute on multiple architectures from the same source code, using representative use cases from major HEP experiments, including the DUNE experiment of the Long Baseline Neutrino Facility, and the ATLAS and CMS experiments of the Large Hadron Collider. This cross-cutting evaluation of portability solutions using real applications will help inform and guide the HEP community when choosing their software and hardware suites for the next generation of experimental frameworks. We present the outcomes of our studies, including performance metrics, porting challenges, API evaluations, and build system integration.

PUBLIC 

This paper summarizes the needs of the dark matter and neutrino communities as it relates to computation. The scope includes data acquisition, triggers, data management and processing, data preservation, simulation, machine learning, data analysis, software engineering, career development, and equity and inclusion. Beyond identifying our community needs, we propose actions that can be taken to strengthen this community and to work together to overcome common challenges.

PUBLIC 

The fundamental nature of Dark Matter is a central theme of the Snowmass 2021 process, extending across all frontiers. In the last decade, advances in detector technology, analysis techniques and theoretical modeling have enabled a new generation of experiments and searches while broadening the types of candidates we can pursue. Over the next decade, there is great potential for discoveries that would transform our understanding of dark matter. In the following, we outline a road map for discovery developed in collaboration among the frontiers. A strong portfolio of experiments that delves deep, searches wide, and harnesses the complementarity between techniques is key to tackling this complicated problem, requiring expertise, results, and planning from all Frontiers of the Snowmass 2021 process.

PUBLIC 

This paper summarizes the modeling, statistics, simulation, and computing needs of direct dark matter detection experiments in the next decade.

PUBLIC 

IceCube is a one-gigaton instrument located at the geographic South Pole, designed to detect cosmic neutrinos, identify the particle nature of dark matter, and study high-energy neutrinos themselves. Simulation of the IceCube detector and processing of data require a significant amount of computational resources. This paper presents the first detailed description of IceProd, a lightweight distributed management system designed to meet these requirements. It is driven by a central database in order to manage mass production of simulations and analysis of data produced by the IceCube detector. IceProd runs as a separate layer on top of other middleware and can take advantage of a variety of computing resources, including grids and batch systems such as CREAM, HTCondor, and PBS. This is accomplished by a set of dedicated daemons that process job submission in a coordinated fashion through the use of middleware plugins that serve to abstract the details of job submission and job management from the framework

PUBLIC 

The KM3NeT Research Infrastructure will, over a period of at least a decade, produce a large amount of unique scientific data that are to be made available to the scientific communities concerned and to the broader general public. This requires the set-up of tools, procedures, documentation and rules to provide this service. For all aspects of the open data access system, including data processing methods, data structure, access and usage examples, sufficient documentation for the effective use of the open data must be provided. In this document, the documentation strategy for the different components is described.

PUBLIC 

The discovery of gravitational waves, first observed in September 2015 following the merger of a binary black hole system, has already revolutionised our understanding of the Universe. This was further enhanced in August 2017, when the coalescence of a binary neutron star system was observed both with gravitational waves and a variety of electromagnetic counterparts; this joint observation marked the beginning of gravitational multimessenger astronomy. The Einstein Telescope, a proposed next-generation ground-based gravitational-wave observatory, will dramatically increase the sensitivity to sources: the number of observations of gravitational waves is expected to increase from roughly 100 per year to roughly 100’000 per year, and signals may be visible for hours at a time, given the low frequency cutoff of the planned instrument. This increase in the number of observed events, and the duration with which they are observed, is hugely beneficial to the scientific goals of the community but poses a number of significant computing challenges. Moreover, the currently used computing algorithms do not scale to this new environment, both in terms of the amount of resources required and the speed with which each signal must be characterised. This contribution will discuss the Einstein Telescope's computing challenges, and the activities that are underway to prepare for them. Available computing resources and technologies will greatly evolve in the years ahead, and those working to develop the Einstein Telescope data analysis algorithms will need to take this into account. It will also be important to factor into the initial development of the experiment's computing model the availability of huge parallel HPC systems and ubiquitous Cloud computing; the design of the model will also, for the first time, include the environmental impact as one of the optimisation metrics.

PUBLIC 

Cyber infrastructure will be a critical consideration in the development of next generation gravitational-wave detectors. The demand for data analysis computing in the 3G era will be driven by the high number of detections as well as the expanded search parameter space for compact astrophysical objects and the subsequent parameter estimation follow-up required to extract the nature of the sources. Additionally, there will be an increased need to develop appropriate and scalable computing cyberinfrastructure, including data access and transfer protocols, and storage and management of software tools, that have sustainable development, support, and management processes. This report identifies the major challenges and opportunities facing 3G gravitational-wave observatories and presents recommendations for addressing them.
This report is the fourth in a six part series of reports by the GWIC 3G Subcommittee: i) Expanding the Reach of Gravitational Wave Observatories to the Edge of the Universe, ii) The Next Generation Global Gravitational Wave Observatory: The Science Book, iii) 3G R&D: R&D for the Next Generation of Ground-based Gravitational Wave Detectors, iv) Gravitational Wave Data Analysis: Computing Challenges in the 3G Era (this report), v) Future Ground-based Gravitational-wave Observatories: Synergies with Other Scientific Communities, and vi) An Exploration of Possible Governance Models for the Future Global Gravitational-Wave Observatory Network.

PUBLIC 

Given the urgency to reduce fossil fuel energy production to make climate tipping points less likely, we call for resource-aware knowledge gain in the research areas on Universe and Matter with emphasis on the digital transformation. A portfolio of measures is described in detail and then summarized according to the timescales required for their implementation. The measures will both contribute to sustainable research and accelerate scientific progress through increased awareness of resource usage. This work is based on a three-days workshop on sustainability in digital transformation held in May 2023.

PUBLIC 

The climate crisis and the degradation of the world's ecosystems require humanity to take immediate action. The international scientific community has a responsibility to limit the negative environmental impacts of basic research. The HECAP+ communities (High Energy Physics, Cosmology, Astroparticle Physics, and Hadron and Nuclear Physics) make use of common and similar experimental infrastructure, such as accelerators and observatories, and rely similarly on the processing of big data. Our communities therefore face similar challenges to improving the sustainability of our research. This document aims to reflect on the environmental impacts of our work practices and research infrastructure, to highlight best practice, to make recommendations for positive changes, and to identify the opportunities and challenges that such changes present for wider aspects of social responsibility.

PUBLIC 

Batch scheduling systems are usually designed to maximise fair resource utilisation and efficiency, but are less well designed for demanding interactive processing, which requires fast access to resources while low upstart latency is only of secondary significance for high throughput of high performance computing scheduling systems. The computing clusters at DESY are intended as batch systems for end users to run massive analysis and simulation jobs enabling fast turnaround systems, in particular when processing is expected to feed back to operation of instruments in near real-time. The continuously increasing popularity of Jupyter Notebooks for interactive and online processing made an integration of this technology into the DESY batch systems indispensable. We present here our approach to utilise the HTCondor and SLURM backends to integrate Jupyter Notebook servers and the techniques involved to provide fast access. The chosen approach offers a smooth user experience allowing users to customize resource allocation tailored to their computational requirements. In addition, we outline the differences between the HPC and the HTC implementations and give an overview of the experience of running Jupyter Notebook services.

PUBLIC 

[...] We will present DESY compute cloud and container orchestration plans as a basis for infrastructure and platform services. We will show examples of Jupyter notebooks for small scale interactive analysis, as well as its integration into large scale resources such as batch systems or Spark clusters. To overcome the fragmentation of the various resources for all scientific communities at DESY, we explore how to integrate them into a seamless user experience in an Interdisciplinary Data Analysis Facility.

PUBLIC 

Increased operational effectiveness and the dynamic integration of only temporarily available compute resources (opportunistic resources) becomes more and more important in the next decade, due to the scarcity of resources for future high energy physics experiments as well as the desired integration of cloud and high performance computing resources. This results in a more heterogenous compute environment, which gives rise to huge challenges for the computing operation teams of the experiments.

At the Karlsruhe Institute of Technology (KIT) we design solutions to tackle these challenges. In order to ensure an efficient utilization of opportunistic resources and unified access to the entire infrastructure, we developed the Transparent Adaptive Resource Dynamic Integration System (TARDIS). A scalable multi-agent resource manager providing interfaces to provision as well as dynamically and transparently integrate resources of various providers into one common overlay batch system. Operational effectiveness is guaranteed by relying on COBalD – the Opportunistic Balancing Daemon and its simple approach of taking into account the utilization and allocation of the different resource types, in order to run the individual workflows on the best-suited resource respectively.

In this contribution we will present the current status of integrating various HPC centers and cloud providers into the compute infrastructure at the Karlsruhe Institute of Technology as well as our experiences gained in a production environment.

PUBLIC 

To satisfy future computing demands of the Worldwide LHC Computing Grid (WLCG), opportunistic usage of third-party resources is a promising approach. While the means to make such resources compatible with WLCG requirements are largely satisfied by virtual machines and containers technologies, strategies to acquire and disband many resources from many providers are still a focus of current research. Existing meta-schedulers that manage resources in the WLCG are hitting the limits of their design when tasked to manage heterogeneous resources from many diverse resource providers.

To provide opportunistic resources to the WLCG as part of a regular WLCG site, we propose a new meta-scheduling approach suitable for opportunistic, heterogeneous resource provisioning. Instead of anticipating future resource requirements, our approach observes resource usage and promotes well-used resources. Following this approach, we have developed an inherently robust meta-scheduler, COBalD, for managing diverse, heterogeneous resources given unpredictable resource requirements. This paper explains the key concepts of our approach, and discusses the benefits and limitations of our new approach to dynamic resource provisioning compared to previous approaches.

PUBLIC 

The inclusion of opportunistic resources, for example from High Performance Computing (HPC) centers or cloud providers, is an important contribution to bridging the gap between existing resources and future needs by the LHC collaborations, especially for the HL-LHC era. However, the integration of these resources poses new challenges and often needs to happen in a highly dynamic manner. To enable an effective and lightweight integration of these resources, the tools COBalD and TARDIS are developed at KIT.

In this contribution we report on the infrastructure we use to dynamically offer opportunistic resources to collaborations in the World Wide LHC Computing Grid (WLCG). The core components are COBalD/TARDIS, HTCondor, CVMFS and modern virtualization technology. The challenging task of managing the opportunistic resources is performed by COBalD/TARDIS. We showcase the challenges, employed solutions and experiences gained with the provisioning of opportunistic resources from several resource providers like university clusters, HPC centers and cloud setups in a multi VO environment. This work can serve as a blueprint for approaching the provisioning of resources from other resource providers.

PUBLIC 

Particle accelerators are an important tool to study the fundamental properties of elementary particles. Currently the highest energy accelerator is the LHC at CERN, in Geneva, Switzerland. Each of its four major detectors, such as the CMS detector, produces dozens of Petabytes of data per year to be analyzed by a large international collaboration. The processing is carried out on the Worldwide LHC Computing Grid, that spans over more than 170 compute centers around the world and is used by a number of particle physics experiments. Recently the LHC experiments were encouraged to make increasing use of HPC resources. While Grid resources are homogeneous with respect to the used Grid middleware, HPC installations can be very different in their setup. In order to integrate HPC resources into the highly automatized processing setups of the CMS experiment a number of challenges need to be addressed. For processing, access to primary data and metadata as well as access to the software is required. At Grid sites all this is achieved via a number of services that are provided by each center. However at HPC sites many of these capabilities cannot be easily provided and have to be enabled in the user space or enabled by other means. At HPC centers there are often restrictions regarding network access to remote services, which is again a severe limitation. The paper discusses a number of solutions and recent experiences by the CMS experiment to include HPC resources in processing campaigns.

PUBLIC 

This document describes the evolution of the ALICE Online-Offline computing system since the TDR, published in June 2015, and gives an account of its implementation schedule in the next two years. After the LS2 upgrade, ALICE will operate at a peak Pb–Pb collision rate of 50 kHz. All events will be read out, reconstructed, compressed and written to permanent storage without any selective trigger. […]

PUBLIC 

Here, we present a series of sample use cases that highlight some of the scientific objectives that could be enabled by phase 1 of the SKA (SKA1). This set consists of examples that include a broad range of scientific applications requiring the frequency coverage of SKA1-LOW and SKA1-MID ([AD3]), as well as the extended high frequency coverage that could be enabled by the advanced instrumentation programme band 6 receivers (beyond 15 GHz; [RD6]). These are intended to serve as examples, and should not be regarded as a substitute for the system level 1 requirements document. […]