2 ms·
Right, the number of digits you need for any given calculation depends on the magnitudes of the numbers involved, and the acceptable precision of the result. BT
by dahart 1y ago
Right, the number of digits you need for any given calculation depends on the magnitudes of the numbers involved, and the acceptable precision of the result. BTW round to nearest might be unsafe, I’m certain that there are many situations where people will specifically avoid rounding to nearest, I would not assume that companies use that scheme.
It seems like a decent assumption that financial companies are not spending engineering time (money) on a floating point analysis of every single computation. They must generally have a desired accuracy, some default numeric types, and workflows that use more bits than necessary for most calculations in exchange for not having to spend time thinking hard about every math op, right? That’s how it works everywhere else.
The accuracy used for reporting doesn’t seem relevant to the accuracy used for internal calculations. It’s fine for large companies to report incomes to rounded millions, while it’s absolutely unacceptable to round compound interest calculations in a bank account to millions, regardless of the balance.