4 ms·

Why bother? The precision of double is sufficient for the vast majority of engineering problems. Seldom do you have measurements that are more precice than the

by ambrop7 10y ago

Why bother? The precision of double is sufficient for the vast majority of engineering problems. Seldom do you have measurements that are more precice than the calculations you do with them.

dnautics 10y ago
A big problem is that chained floating point operations are a O(sqrt n) martingale. Also, some floating point ops that use pade approximants (most log algorithms, sin, cos) lose onoe or two precision points (versus half for arithmetic ops) of floating point accuracy.
- ambrop7 10y ago
  Yeah, I know, there are cases where it matters. But I suppose many of these cases are also the ones where performance (and a hardware implementation) matters, and you're better off designing your algorithm to deal with roundoff error than use a magically precise FP implementation.
  - dnautics 10y ago
    sure. You could go the KSP route and write your own FP. There are still problems like the various krakens. KSP is obviously not so critical that krakens are life- or project- jeopardizing... But imagine being NASA for example, designing real world satellite guidance systems that must operate over many orders of magnitude. If you're using IEEE floating points, your systems will silently fail and (rarely) give an error like NaN. IEEE intervals will work, but often give a diagnostically useless answer. Unums have characteristic signatures that hint where to look to find the problematic calculation. You can then backtrack and redo the calculation at a higher precision and drop back to the standard precision as necessary. This is, of course, automatable.
- acidflask 10y ago
  > A big problem is that chained floating point operations are a O(sqrt n) martingale. Mostly false; see https://www.cs.berkeley.edu/~wkahan/Mindless.pdf https://www.cs.berkeley.edu/~wkahan/Mindless.pdf It is true that an arbitrary, independent sequence of n floating point operations can potentially lose up to 2n bits of precision. However, it is false that most scientific computing code has such structure. On the contrary, the loss of precision in numerical algorithms can be estimated quite well using standard techniques of error analysis, especially when there is underlying matrix structure to exploit. The growth of error in properly written numerical code should NOT grow exponentially with the number of floating point operations. It should be bounded by intrinsic properties of the underlying matrix, such as its condition number or spectral norm.
bsder 10y ago
"vast majority"? Maybe. But anything which uses matrices (aka. systems of linear differential equations), always runs into some pathological cases. Quite often, they aren't even uncommon. Systems which have vastly different time constants in the same system are ALWAYS a problem. Simulating RF circuits or phase locked loops is always an issue. Computational fluid dynamics always has boundary conditions that challenge infinity/zero and drive the solver crazy. Something as pedestrian as simulating basic cotton cloth accurately is problematic (the warp/weft redistributes energy on a much faster time scale than gravity pulls the cloth down). A lot of people did a lot of research and put in a lot of programming hours to make solving these problems relatively easy without understanding the gory details.
- deleted 10y ago
  [deleted]
- zevets 10y ago
  But will unums help? No matter how an ill conditioned matrix is represented, it will still be ill conditioned, and we'll still be iterating, unless I've missed something in this article. I'm skeptical that changing how we represent the numbers will help - when the physics are icky, they're icky.
  - tjl 10y ago
    It won't really help. You'll get the same issues with stiff DEs or ill-conditioned matrices for perfect precision or floating-point. There will be some cases where it will help, but not a lot. I've implemented a few DE solvers and sparse matrix implementations before. I think sparse matrices is one case where it would help.
  - tanderson92 10y ago
    Yes, this won't change the challenging nature of solving the underlying problem. More problematic is that when a matrix is ill-conditioned it may not be all that helpful to have "an answer" since the ill-conditioning tells you there are many solutions closely satisfying the system of equations.
dbcurtis 10y ago
Naive geometry calculations fail very hard, very fast. There is a reason why Solidworks has you draw on the surface of something that already exists, and extrude a feature up from that, instead of computing the union of two 3D primitives. Computing the coincidence of two lines or planes is a quick path to woe. The geometry package CGAL uses infinite precision rationals instead of floating point, and also bans curves. A circle is a many-sided polygon, and pi is not acknowledged to exist. Floating point and computational geometry are uneasy aquaintances.
jhayward 10y ago
He gives an example of solving complex nonlinear systems [1] in a very few operations by using the inherent parallelism available via subdividing intervals. He claims both accuracy and huge speed gains are possible. [1] http://www.johngustafson.net/presentations/Multicore2016-JLG.pdf http://www.johngustafson.net/presentations/Multicore2016-JLG... (see example starting p.10)
effie 10y ago
Calculations with fractional numbers are done for many mathematical and scientific purposes as well, where arithmetic system obeying clear set of mathematical rules, designed with correctness and reliability of results in mind, would be useful and liberating. Even engineering calculations would benefit from faster and more economical operation that unum hardware can possibly provide.
atemerev 10y ago
Because, you know, computing is not restricted to engineering problems. People occasionally use computers for all other weird things, like counting _money_. Or formatting text.
Houshalter 10y ago
Not just precision, but it also allows more extreme values. Very large or very small numbers. In statistics numbers often get very small because they represent probabilities. Sometimes they transform into the log domain, but that has a lot of disadvantages. You lose zero, negative numbers, addition is difficult, and you get vastly more precision near zero than near one.
devishard 10y ago
You'd be surprised how many people don't build rails apps for advertising companies.
devishard 10y ago
You'd be surprised how many people don't build rails apps for advertising companies.

by ambrop7 10y ago

Why bother? The precision of double is sufficient for the vast majority of engineering problems. Seldom do you have measurements that are more precice than the calculations you do with them.

dnautics 10y ago

A big problem is that chained floating point operations are a O(sqrt n) martingale. Also, some floating point ops that use pade approximants (most log algorithms, sin, cos) lose onoe or two precision points (versus half for arithmetic ops) of floating point accuracy.

ambrop7 10y ago
Yeah, I know, there are cases where it matters. But I suppose many of these cases are also the ones where performance (and a hardware implementation) matters, and you're better off designing your algorithm to deal with roundoff error than use a magically precise FP implementation.
- dnautics 10y ago
  sure. You could go the KSP route and write your own FP. There are still problems like the various krakens. KSP is obviously not so critical that krakens are life- or project- jeopardizing... But imagine being NASA for example, designing real world satellite guidance systems that must operate over many orders of magnitude. If you're using IEEE floating points, your systems will silently fail and (rarely) give an error like NaN. IEEE intervals will work, but often give a diagnostically useless answer. Unums have characteristic signatures that hint where to look to find the problematic calculation. You can then backtrack and redo the calculation at a higher precision and drop back to the standard precision as necessary. This is, of course, automatable.
acidflask 10y ago
> A big problem is that chained floating point operations are a O(sqrt n) martingale. Mostly false; see https://www.cs.berkeley.edu/~wkahan/Mindless.pdf https://www.cs.berkeley.edu/~wkahan/Mindless.pdf It is true that an arbitrary, independent sequence of n floating point operations can potentially lose up to 2n bits of precision. However, it is false that most scientific computing code has such structure. On the contrary, the loss of precision in numerical algorithms can be estimated quite well using standard techniques of error analysis, especially when there is underlying matrix structure to exploit. The growth of error in properly written numerical code should NOT grow exponentially with the number of floating point operations. It should be bounded by intrinsic properties of the underlying matrix, such as its condition number or spectral norm.

bsder 10y ago

"vast majority"? Maybe. But anything which uses matrices (aka. systems of linear differential equations), always runs into some pathological cases. Quite often, they aren't even uncommon. Systems which have vastly different time constants in the same system are ALWAYS a problem. Simulating RF circuits or phase locked loops is always an issue. Computational fluid dynamics always has boundary conditions that challenge infinity/zero and drive the solver crazy. Something as pedestrian as simulating basic cotton cloth accurately is problematic (the warp/weft redistributes energy on a much faster time scale than gravity pulls the cloth down). A lot of people did a lot of research and put in a lot of programming hours to make solving these problems relatively easy without understanding the gory details.

deleted 10y ago
[deleted]
zevets 10y ago
But will unums help? No matter how an ill conditioned matrix is represented, it will still be ill conditioned, and we'll still be iterating, unless I've missed something in this article. I'm skeptical that changing how we represent the numbers will help - when the physics are icky, they're icky.
- tjl 10y ago
  It won't really help. You'll get the same issues with stiff DEs or ill-conditioned matrices for perfect precision or floating-point. There will be some cases where it will help, but not a lot. I've implemented a few DE solvers and sparse matrix implementations before. I think sparse matrices is one case where it would help.
- tanderson92 10y ago
  Yes, this won't change the challenging nature of solving the underlying problem. More problematic is that when a matrix is ill-conditioned it may not be all that helpful to have "an answer" since the ill-conditioning tells you there are many solutions closely satisfying the system of equations.

dbcurtis 10y ago

Naive geometry calculations fail very hard, very fast. There is a reason why Solidworks has you draw on the surface of something that already exists, and extrude a feature up from that, instead of computing the union of two 3D primitives. Computing the coincidence of two lines or planes is a quick path to woe. The geometry package CGAL uses infinite precision rationals instead of floating point, and also bans curves. A circle is a many-sided polygon, and pi is not acknowledged to exist. Floating point and computational geometry are uneasy aquaintances.

jhayward 10y ago

He gives an example of solving complex nonlinear systems [1] in a very few operations by using the inherent parallelism available via subdividing intervals. He claims both accuracy and huge speed gains are possible. [1] http://www.johngustafson.net/presentations/Multicore2016-JLG.pdf http://www.johngustafson.net/presentations/Multicore2016-JLG... (see example starting p.10)

effie 10y ago

Calculations with fractional numbers are done for many mathematical and scientific purposes as well, where arithmetic system obeying clear set of mathematical rules, designed with correctness and reliability of results in mind, would be useful and liberating. Even engineering calculations would benefit from faster and more economical operation that unum hardware can possibly provide.

atemerev 10y ago

Because, you know, computing is not restricted to engineering problems. People occasionally use computers for all other weird things, like counting _money_. Or formatting text.

Houshalter 10y ago

Not just precision, but it also allows more extreme values. Very large or very small numbers. In statistics numbers often get very small because they represent probabilities. Sometimes they transform into the log domain, but that has a lot of disadvantages. You lose zero, negative numbers, addition is difficult, and you get vastly more precision near zero than near one.

devishard 10y ago

You'd be surprised how many people don't build rails apps for advertising companies.

devishard 10y ago

You'd be surprised how many people don't build rails apps for advertising companies.