Higham, Nicholas and Pranesh, Srikara (2019) Exploiting Lower Precision Arithmetic in Solving Symmetric Positive Definite Linear Systems and Least Squares Problems. [MIMS Preprint]
This is the latest version of this item.
Text
paper.pdf Download (554kB) 
Abstract
What is the fastest way to solve a linear system $Ax= b$ in arithmetic of a given precision when $A$ is symmetric positive definite and otherwise unstructured? The usual answer is by Cholesky factorization, assuming that $A$ can be factorized. We develop an algorithm that can be faster, given an arithmetic of precision lower than the working precision as well as (optionally) one of higher precision. The arithmetics might, for example, be of precisions half, single, and double; half and double, possibly with quadruple; or single and double, possibly with quadruple. We compute a Cholesky factorization at the lower precision and use the factors as preconditioners in GMRESbased iterative refinement. To avoid breakdown of the factorization we shift the matrix by a small multiple of its diagonal. We explain why this is preferable to the common approach of shifting by a multiple of the identity matrix, We also incorporate scaling in order to avoid overflow and reduce the chance of underflow when working in IEEE half precision arithmetic. We extend the algorithm to solve a linear least squares problem with a well conditioned coefficient matrix by forming and solving the normal equations. In both algorithms most of the work is done at low precision provided that iterative refinement and the inner iterative solver converge quickly. We explain why replacing GMRES by the conjugate gradient method causes convergence guarantees to be lost, but show that this change has little effect on convergence in practice. Our numerical experiments confirm the potential of the new algorithms to provide faster solutions in environments that support multiple precisions of arithmetic.
Item Type:  MIMS Preprint 

Subjects:  MSC 2010, the AMS's Mathematics Subject Classification > 65 Numerical analysis 
Depositing User:  Dr Srikara Pranesh 
Date Deposited:  09 Jul 2020 09:25 
Last Modified:  09 Jul 2020 09:25 
URI:  http://eprints.maths.manchester.ac.uk/id/eprint/2771 
Available Versions of this Item

Exploiting Lower Precision Arithmetic in Solving Symmetric Positive Definite Linear Systems and Least Squares Problems. (deposited 06 Nov 2019 17:42)
 Exploiting Lower Precision Arithmetic in Solving Symmetric Positive Definite Linear Systems and Least Squares Problems. (deposited 09 Jul 2020 09:25) [Currently Displayed]
Actions (login required)
View Item 