4 ms·

Here's an example where higher precision could be useful. You need to do an expectation of some function over a distribution. You only have the moment generatin

Here's an example where higher precision could be useful. You need to do an expectation of some function over a distribution. You only have the moment generating function for the distribution. The moments are the derivatives of this MGF, but a rough rule of thumb is that if the precision of a function is epsilon, then the precision of it's n'th derivative is epsilon^(1/n). If you have 12 significant digits for the MGF, then you can count on only 1 significant digit for the 12'th derivative. Let's say that this is simply unacceptable for you, you need 4 digits, so you can use the first 3 derivatives only. If you double the precision, you can use 6 derivatives, so you can match 6 moments. This might be enough for your problem. Of course, you can find better algorithms, like you use cumulants instead of moments, find a way to devise a Gaussian quadrature method (a Golub-Welsch algorithm) that uses these cumulants, etc. But that requires doing a lot of hard math (with the inevitable initial bugs), while plugging in a double precision library gives you instantaneous results.

nestorD 2y ago
Yes! And no! You will indeed need higher precision if you need a very large number of digits (more than what double precision gives you). However, even with high precision, the algorithms you use to compute your moments matter: you might not have as many digits as you hope for. Variance, for example, is famously numerically unstable (sometimes giving a negative variance) when computed naively. A stable summation algorithm plugged into a variance computation (even a carefully written one) can sometimes give you six additional digits of precision in the variance that you might not get when doubling the precision (it depends on how big of a cancellation you had in your operations; too big, and doubling the precision will not be enough to recover, while a compensated summation would do fine). I have seen this play out once in a clustering algorithm that would only work on some data when using a compensated summation to compute the variance.
- credit_guy 2y ago
  Well, my point is that it's not as easy as saying "just pick a smarter algorithm". Not all problems needing high precision can resort to a smart summation. Sometimes that smart algorithm just doesn't exist yet. You are also right, that if you can solve your problem with high precision, it's unlikely that just quad precision will be enough. For example the inverse Laplace transform. In some cases you need thousands of digits. The cases where double precision is not enough, but quad precision is are probably rare. But it's worth a try.

nestorD 2y ago

Yes! And no! You will indeed need higher precision if you need a very large number of digits (more than what double precision gives you). However, even with high precision, the algorithms you use to compute your moments matter: you might not have as many digits as you hope for. Variance, for example, is famously numerically unstable (sometimes giving a negative variance) when computed naively. A stable summation algorithm plugged into a variance computation (even a carefully written one) can sometimes give you six additional digits of precision in the variance that you might not get when doubling the precision (it depends on how big of a cancellation you had in your operations; too big, and doubling the precision will not be enough to recover, while a compensated summation would do fine). I have seen this play out once in a clustering algorithm that would only work on some data when using a compensated summation to compute the variance.

credit_guy 2y ago
Well, my point is that it's not as easy as saying "just pick a smarter algorithm". Not all problems needing high precision can resort to a smart summation. Sometimes that smart algorithm just doesn't exist yet. You are also right, that if you can solve your problem with high precision, it's unlikely that just quad precision will be enough. For example the inverse Laplace transform. In some cases you need thousands of digits. The cases where double precision is not enough, but quad precision is are probably rare. But it's worth a try.