Higham, Nicholas J. (2021) Numerical Stability of Algorithms at Extreme Scale and Low Precisions. [MIMS Preprint]
![[thumbnail of paper.pdf]](https://eprints.maths.manchester.ac.uk/style/images/fileicons/text.png) |
Text
paper.pdf Download (331kB) |
Abstract
The largest dense linear systems that are being solved today are of order $n = 10^7$. Single precision arithmetic, which has a unit roundoff $u \approx 10^{-8}$, is widely used in scientific computing, and half precision arithmetic, with $u \approx 10^{-4}$, is increasingly being exploited as it becomes more readily available in hardware. Standard rounding error bounds for numerical linear algebra algorithms are proportional to $p(n)u$, with $p$ growing at least linearly with $n$. Therefore we are at the stage where these rounding error bounds are not able to guarantee any accuracy or stability in the computed results for some extreme-scale or low-accuracy computations. We explain how rounding error bounds with much smaller constants can be obtained. Blocked algorithms, which break the data into blocks of size $b$, lead to a reduction in the error constants by a factor $b$ or more. Two architectural features also reduce the error constants: extended precision registers and fused multiply--add operations, either at the scalar level or in mixed precision block form. We also discuss a new probabilistic approach to rounding error analysis that provides error constants that are the square roots of those of the worst-case bounds. Combining these different considerations provides new understanding of the numerical stability of extreme scale and low precision computations in numerical linear algebra.
| Item Type: | MIMS Preprint |
|---|---|
| Uncontrolled Keywords: | numerical stability, low precision, exascale computing, floating-point arithmetic, rounding error analysis, numerical linear algebra probabilistic error bounds, summation, inner product, matrix--vector product, matrix multiplication, blocked algorithm, fused multiply-add |
| Subjects: | MSC 2010, the AMS's Mathematics Subject Classification > 65 Numerical analysis |
| Depositing User: | Nick Higham |
| Date Deposited: | 14 Sep 2021 13:21 |
| Last Modified: | 14 Sep 2021 13:21 |
| URI: | https://eprints.maths.manchester.ac.uk/id/eprint/2833 |
Actions (login required)
 |
View Item |