5 ms·
I remember, once upon a time... a bug where we were trying to find the average credit rating (AAA, AA+, AA, etc) of a portfolio of companies. And we had this i
by whitej125 11y ago
I remember, once upon a time... a bug where we were trying to find the average credit rating (AAA, AA+, AA, etc) of a portfolio of companies. And we had this issue where, in a simple degenerate case, the average of AA+ and AA+ became AA (AA+ is expected). In the end, it was an issue of floating point error accounting! (This was in C++ btw on a probably not-so-familiar platform).
TLDR;
floor(13.0) != 13
floor(13.0) == 12
Each credit rating was mapped to a integer. We'd take the floating point average of all the integers and then floor it. The integral answer would be used to map back to a credit rating.
Let's say AA+ was mapped to the number 13. Well floor( (float(13) + float(13)) / 2.0) == 12. Not 13.
That's because floating point 13.0 is really 12.99999999999.
The fix was to add accounting for floating point error by adding in ULP (https://en.wikipedia.org/wiki/Unit_in_the_last_place https://en.wikipedia.org/wiki/Unit_in_the_last_place) which in essence tips the representation of 13.0 to 13.00000000001 (or whatever the real number is).
- nwhitehead 11y agoThis might be true on really weird platforms, but with IEEE-754 floating point this isn't true. Integers within a big range are exactly representable without any rounding error. Doing floor((float(13) + float(13))/2.0) should be exactly 13. It is entirely possible that the weird platform was supposed to be using IEEE-754 arithmetic but had a buggy math library (depressingly common).