Collaborative Computing Support for Analysis Facilities Exploiting Software as Infrastructure Techniques

Authors

Maria Acosta Flechas, Fermi National Accelerator Laboratory
Garhan Attebury, University of Nebraska-LincolnFollow
Kenneth Bloom, University of Nebraska-LincolnFollow
Brian Bockelman, Morgridge Institute for Research
Lindsey Gray, Fermi National Accelerator Laboratory
Burt Holzman, Fermi National Accelerator Laboratory
Carl Lundstedt, Morgridge Institute for Research
Oksana Shadura, Morgridge Institute for Research
Nicholas Smith, Fermi National Accelerator Laboratory
John Thiltges, Morgridge Institute for Research

Date of this Version

3-22-2022

Citation

Prepared for submission to Snowmass, CompF4. arXiv:2203.10161v2

Comments

Used by permission.

Abstract

Prior to the public release of Kubernetes it was difficult to conduct joint development of elaborate analysis facilities due to the highly non-homogeneous nature of hardware and network topology across compute facilities. However, since the advent of systems like Kubernetes and OpenShift, which provide declarative interfaces for building fault-tolerant and self-healing deployments of networked software, it is possible for multiple institutes to collaborate more effectively since resource details are abstracted away through various forms of hardware and software virtualization. In this whitepaper we will outline the development of two analysis facilities: “Coffea-casa” at University of Nebraska Lincoln and the “Elastic Analysis Facility” at Fermilab, and how utilizing platform abstraction has improved the development of common software for each of these facilities, and future development plans made possible by this methodology.