Mixed Precision Algorithms in Numerical Linear Algebra

Higham, Nicholas J. and Mary, Theo (2021) Mixed Precision Algorithms in Numerical Linear Algebra. [MIMS Preprint]

Text paper_eprint.pdf Download (567kB)

Abstract

Today's floating-point arithmetic landscape is broader than ever. While scientific computing has traditionally used single precision and double precision floating-point arithmetics, half precision is increasingly available in hardware and quadruple precision is supported in software. Lower precision arithmetic brings increased speed and reduced communication and energy costs, but it produces results of correspondingly low accuracy. Higher precisions are more expensive but can potentially provide great benefits, even if used sparingly. A variety of mixed precision algorithms have been developed that combine the superior performance of lower precisions with the better accuracy of higher precisions. Some of these algorithms aim to provide results of the same quality as algorithms running in a fixed precision but at a much lower cost; others use a little higher precision to improve the accuracy of an algorithm. This survey treats a broad range of mixed precision algorithms in numerical linear algebra, both direct and iterative, for problems including matrix multiplication, matrix factorization, linear systems, least squares, eigenvalue decomposition, and singular value decomposition. We identify key algorithmic ideas, such as iterative refinement, adapting the precision to the data, and exploiting mixed precision block fused multiply--add operations. We also describe the possible performance benefits and explain what is known about the numerical stability of the algorithms. This survey should be useful to a wide community of researchers and practitioners who wish to develop or benefit from mixed precision numerical linear algebra algorithms.

Item Type:	MIMS Preprint
Additional Information:	To appear in Acta Numerica.
Subjects:	MSC 2010, the AMS's Mathematics Subject Classification > 15 Linear and multilinear algebra; matrix theory MSC 2010, the AMS's Mathematics Subject Classification > 65 Numerical analysis
Depositing User:	Nick Higham
Date Deposited:	29 Dec 2021 14:46
Last Modified:	29 Dec 2021 14:46
URI:	https://eprints.maths.manchester.ac.uk/id/eprint/2841

Available Versions of this Item

• Mixed Precision Algorithms in Numerical Linear Algebra. (deposited 29 Dec 2021 14:46) [Currently Displayed]
  • Mixed Precision Algorithms in Numerical Linear Algebra. (deposited 18 Feb 2022 18:33)

Actions (login required)

View Item