Scalable Resilience Against Node Failures for Communication-Hiding Preconditioned Conjugate Gradient and Conjugate Residual Methods

Scalable Resilience Against Node Failures for Communication-Hiding Preconditioned Conjugate Gradient and Conjugate Residual Methods

Abstract

The observed and expected continued growth in the number of nodes in large-scale parallel computers gives rise to two major challenges: global communication operations are becoming major bottlenecks due to their limited scalability, and the likelihood of node failures is increasing. We study an approach for addressing these challenges in the context of solving large sparse linear systems. In particular, we focus on the pipelined preconditioned conjugate gradient (PPCG) method, which has been shown to successfully deal with the first of these challenges. In this paper, we address the second challenge. We present extensions to the PPCG solver and two of its variants which make them resilient against the failure of a compute node while fully preserving their communication-hiding properties and thus their scalability. The basic idea is to efficiently communicate a few redundant copies of local vector elements to neighboring nodes with very little overhead. In case a node fails, these redundant copies are gathered at a replacement node, which can then accurately reconstruct the lost parts of the solver's state. After that, the parallel solver can continue as in the failure-free scenario. Experimental evaluations of our approach illustrate on average very low runtime overheads compared to the standard non-resilient algorithms. This shows that scalable algorithmic resilience can be achieved at low extra cost.

Grafik Top
Authors
  • Levonyak, Markus
  • Pacher, Christina
  • Gansterer, Wilfried
Grafik Top
Shortfacts
Category
Paper in Conference Proceedings or in Workshop Proceedings (Paper)
Event Title
SIAM Conference on Parallel Processing for Scientific Computing (PP20)
Divisions
Theory and Applications of Algorithms
Subjects
Parallele Datenverarbeitung
Event Location
Seattle, Washington, USA
Event Type
Conference
Event Dates
12-15 Feb 2020
Series Name
Proceedings of the 2020 SIAM Conference on Parallel Processing for Scientific Computing (PP20)
ISSN/ISBN
978-1-61197-613-7
Page Range
pp. 81-92
Date
January 2020
Export
Grafik Top