Connolly, Michael P. and Higham, Nicholas J. and Mary, Theo (2020) Stochastic Rounding and its Probabilistic Backward Error Analysis. [MIMS Preprint]
This is the latest version of this item.
Text
paper.pdf Download (509kB) 
Abstract
Stochastic rounding rounds a real number to the next larger or smaller floatingpoint number with probabilities $1$ minus the relative distances to those numbers. It is gaining attention in deep learning because it can increase the success of low precision computations. We compare basic properties of stochastic rounding with those for round to nearest, finding properties in common as well as significant differences. We prove that for stochastic rounding the rounding errors are mean independent random variables with zero mean. We derive a new version of our probabilistic error analysis theorem from [{\em SIAM J. Sci. Comput.}, 41 (2019), pp.\ A2815A2835], weakening the assumption of independence of the random variables to mean independence. These results imply that for a wide range of linear algebra computations the backward error for stochastic rounding is unconditionally bounded by a multiple of $\sqrt{n}\mkern1muu$ to first order, with a certain probability, where $n$ is the problem size and $u$ is the unit roundoff. This is the first scenario where the rule of thumb that one can replace $nu$ by $\sqrt{n}\mkern1muu$ in a rounding error bound has been shown to hold without any additional assumptions on the rounding errors. We also explain how stochastic rounding avoids the phenomenon of stagnation in sums, whereby small addends are obliterated by round to nearest when they are too small relative to the sum.
Item Type:  MIMS Preprint 

Uncontrolled Keywords:  floatingpoint arithmetic, rounding error analysis, numerical linear algebra, stochastic rounding, round to nearest, probabilistic backward error analysis, stagnation 
Subjects:  MSC 2010, the AMS's Mathematics Subject Classification > 65 Numerical analysis 
Depositing User:  Nick Higham 
Date Deposited:  10 Aug 2020 17:15 
Last Modified:  10 Aug 2020 17:15 
URI:  http://eprints.maths.manchester.ac.uk/id/eprint/2778 
Available Versions of this Item

Stochastic Rounding and its Probabilistic Backward Error Analysis. (deposited 28 Apr 2020 10:01)
 Stochastic Rounding and its Probabilistic Backward Error Analysis. (deposited 10 Aug 2020 17:15) [Currently Displayed]
Actions (login required)
View Item 