3 ms·
> but you'll know it's approximately 0.333, definitely more than 6/20 but definitely less than 7/20, for instance What strikes me as odd is that in my intro co
by geezerjay 9y ago
> but you'll know it's approximately 0.333, definitely more than 6/20 but definitely less than 7/20, for instance
What strikes me as odd is that in my intro course to numerical methods taught how to calculate floating point error bounds when introducing the concept of floating point numbers, including how errors propagated with each flop.
- TallGuyShort 9y agoYeah I doubt this is anything novel in the purely mathematical realm. It sounds like what's patented is a practical design for doing this on a chip. Not sure if you were taught a different method, but I envision this being similar to counting "significant digits" in scientific notation, and it sounds like that's very similar to the approach he took. I wonder if that explains there statements about tracking to the last "digit". They obviously can't do that for irrational number, so maybe they mean the last "significant" digit as far as the underlying floating point implementation is concerned.