Exploiting Lower Precision Arithmetic in Solving Symmetric Positive Definite Linear Systems and Least Squares Problems

Higham, Nicholas and Pranesh, Srikara (2019) Exploiting Lower Precision Arithmetic in Solving Symmetric Positive Definite Linear Systems and Least Squares Problems. [MIMS Preprint]

Text paper.pdf Download (534kB)

Abstract

What is the fastest way to solve a linear system $Ax= b$ in arithmetic of a given precision when $A$ is symmetric positive definite and otherwise unstructured? The usual answer is by Cholesky factorization, assuming that $A$ can be factorized. We develop an algorithm that can be faster, given an arithmetic of precision lower than the working precision as well as (optionally) one of higher precision. The arithmetics might, for example, be of precisions half, single, and double; half and double, possibly with quadruple; or single and double, possibly with quadruple. We compute a Cholesky factorization at the lower precision and use the factors as preconditioners in GMRES-based iterative refinement. To avoid breakdown of the factorization we shift the matrix by a small multiple of its diagonal. We explain why this is preferable to the common approach of shifting by a multiple of the identity matrix, We also incorporate scaling in order to avoid overflow and reduce the chance of underflow when working in IEEE half precision arithmetic. We extend the algorithm to solve a linear least squares problem with a well conditioned coefficient matrix by forming and solving the normal equations. In both algorithms most of the work is done at low precision provided that iterative refinement and the inner iterative solver converge quickly. We explain why replacing GMRES by the conjugate gradient method causes convergence guarantees to be lost, but show that this change has little effect on convergence in practice. Our numerical experiments confirm the potential of the new algorithms to provide faster solutions in environments that support multiple precisions of arithmetic.

Item Type:	MIMS Preprint
Subjects:	MSC 2010, the AMS's Mathematics Subject Classification > 65 Numerical analysis
Depositing User:	Dr Srikara Pranesh
Date Deposited:	06 Nov 2019 17:42
Last Modified:	06 Nov 2019 17:52
URI:	https://eprints.maths.manchester.ac.uk/id/eprint/2736

Available Versions of this Item

• Exploiting Lower Precision Arithmetic in Solving Symmetric Positive Definite Linear Systems and Least Squares Problems. (deposited 06 Nov 2019 17:42) [Currently Displayed]
  • Exploiting Lower Precision Arithmetic in Solving Symmetric Positive Definite Linear Systems and Least Squares Problems. (deposited 09 Jul 2020 09:25)

Actions (login required)

View Item