Mixed Precision Algorithms in Numerical Linear Algebra

Higham, Nicholas J. and Mary, Theo (2021) Mixed Precision Algorithms in Numerical Linear Algebra. [MIMS Preprint] (In Press)

This is the latest version of this item.

[thumbnail of paper_eprint.pdf] Text
paper_eprint.pdf
Download (530kB)

Abstract

Today's floating-point arithmetic landscape is broader than ever. While scientific computing has traditionally used single precision and double precision floating-point arithmetics, half precision is increasingly available in hardware and quadruple precision is supported in software. Lower precision arithmetic brings increased speed and reduced communication and energy costs, but it produces results of correspondingly low accuracy. Higher precisions are more expensive but can potentially provide great benefits, even if used sparingly. A variety of mixed precision algorithms have been developed that combine the superior performance of lower precisions with the better accuracy of higher precisions. Some of these algorithms aim to provide results of the same quality as algorithms running in a fixed precision but at a much lower cost; others use a little higher precision to improve the accuracy of an algorithm. This survey treats a broad range of mixed precision algorithms in numerical linear algebra, both direct and iterative, for problems including matrix multiplication, matrix factorization, linear systems, least squares, eigenvalue decomposition, and singular value decomposition. We identify key algorithmic ideas, such as iterative refinement, adapting the precision to the data, and exploiting mixed precision block fused multiply--add operations. We also describe the possible performance benefits and explain what is known about the numerical stability of the algorithms. This survey should be useful to a wide community of researchers and practitioners who wish to develop or benefit from mixed precision numerical linear algebra algorithms.

Item Type: MIMS Preprint
Additional Information: To appear in Acta Numerica.
Subjects: MSC 2010, the AMS's Mathematics Subject Classification > 15 Linear and multilinear algebra; matrix theory
MSC 2010, the AMS's Mathematics Subject Classification > 65 Numerical analysis
Depositing User: Nick Higham
Date Deposited: 18 Feb 2022 18:33
Last Modified: 18 Feb 2022 18:33
URI: https://eprints.maths.manchester.ac.uk/id/eprint/2849

Available Versions of this Item

Actions (login required)

View Item View Item