3 ms·
Had to deal with accumulating noise and unpredictable rounding in a double precision supported application. Ended up scaling the amounts to integers, then conv
by EEGuy 14y ago
Had to deal with accumulating noise and unpredictable rounding in a double precision supported application.
Ended up scaling the amounts to integers, then converting those known and stable amounts to _character strings_ then doing the multiplication and division by character manipulation!
Don't laugh too hard; performance wasn't a consideration but _decimal_ accuracy was. It did the job.