3 ms·
Rounding errors might be small, but catastrophic cancellation can make errors much larger than parts per billion. Maybe for your application it doesn't matter,
by SuchAnonMuchWow 5y ago
Rounding errors might be small, but catastrophic cancellation can make errors much larger than parts per billion. Maybe for your application it doesn't matter, but sometimes it does, and you need to be aware of these cases.
You wouldn't want those kind of rounding errors in financial applications, in rocket science applications, etc ...
- andi999 5y agoAnd how does the mentioned averages procedure help?