3 ms·
> You run out of integer precision at 2^24, which is only 16 million To be fair, the author seems to be suggesting using double-precision floating point. If yo
by pcwalton 7y ago
> You run out of integer precision at 2^24, which is only 16 million
To be fair, the author seems to be suggesting using double-precision floating point. If you use integer numbers of pennies, signed 32-bit integers cap out at a similar value of $21M. 32 bits is just too small for financial calculations.
- dahart 7y agoYou’re absolutely right; he did say 64 bit. I just wouldn’t do that blindly either, and the author admitted to not being fluent in error analysis. The issue with even doubles is that the magnitude of your error in a running total calculation is a sum of all the errors of your largest intermediate results (the results of multiplies you don’t see or store explicitly). That means with a bank account, the error of your calculations continues to grow forever unless you are explicitly correcting the errors. Rounding does not solve that, so even using doubles for money is a sketchy proposition unless you really know what you’re doing.
- jdashg 7y agoIsn't the proposal in the article is "round after every operation"? Since rounding to nearest cent values corrects each subtotal's error to zero, this should work up to 2^53 cents. ($90T)
- dahart 7y agoYes, that is the proposal. Rounding doesn’t correct errors though, it can make the error grow faster. Rounding just keeps you from dealing with sub-pennies.