8 ms·
I feel like the implementation of floating point is a bit weird, but the concept really isn’t. You have a big string of numbers of more or less a fixed length,
by zoky 3y ago
I feel like the implementation of floating point is a bit weird, but the concept really isn’t. You have a big string of numbers of more or less a fixed length, with a decimal point somewhere therein. Floating point lets you specify what the numbers are, and where the decimal point lies. Smaller numbers have the decimal point to the left, so more room for precision to the right. Larger numbers need to move the decimal point to the right, so there’s less room for precision. Oh, and everything is secretly in base two, so sometimes numbers don’t work out quite the way you think they should, which pretty much means don’t use it for money or anything else where precision really makes a difference.
Yeah, the actual implementation is fairly different, but that’s just computer gibberish. Most of us no longer need to think of strings as an offset into memory to an array of characters that happen to be interpretable as a group, but that’s really what it is despite any abstractions.
The real mystery is why most modern languages continue to use floats and integers as the default numeric types rather than some sort of BigDecimal representation, given that precision and programmer intuitiveness trumps raw performance in the vast majority of cases.
- di4na 3y agoBecause they do not for integer. The vast majority of languages you use out there that do not care about speed use Bigint. Python. Haskell. Ruby. Perl. PHP. Erlang. Etc etc. Floating point are used because quite simply they are the only way to do it that has wide support that can be reused. A good fixed width or Decimal implementation that could be reused and implement all the maths you want for Reals are... Rare. The exception is Java and JavaScript mostly. But JS has Bigint now. Just not as a default.
- zoky 3y agoI guess what I’m saying is that they should. We all grew up learning math before programming, and mathematically speaking, there is no qualitative difference between 15 and 14.7, and no difference whatsoever between 15 and 15.0. So why do programming languages, particularly ones like the ones you mention that purport to abstract such distinctions away, treat them differently by default? There is no logical reason why 5/2 should be 2 rather than 2.5, yet by default it is, for apparently no other reason than that is faster for the CPU to calculate. I’m not saying get rid of these distinctions entirely. But where they matter, for performance reasons or whatever, they should be opt-in rather than the default.
- defrost 3y agoIntegers are exact and give repeatable results. Many floats (such as 14.7) are not even represented exactly, float operations have a large number of issues. If you're wondering about all this, see: What Every Computer Scientist Should Know About Floating-Point Arithmetic ~ David Goldberg, March, 1991 issue of Computing Surveys https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.h... (and elsewhere)
- HPsquared 3y agoIt's a bit like musical scales. It all seems nice and neat, until you start looking for exactness.
- sfpotter 3y agoPlease don’t spread FUD about floating point arithmetic. IEEE754 is an extremely carefully worked out standard which provides a good trade off between speed and accuracy. The problem of representing a continuum of real numbers in a fixed amount of memory is highly nontrivial. The error incurred by each operation is bounded and predictable—not exact, but as good as you’re going to get given the constraints. Also, results are replicable. Speaking as someone working in numerical analysis and scientific computing, so I work with floats all day long.
- defrost 3y ago> Please don’t spread FUD about floating point arithmetic. I'm sorry, my apologies. Are you saying that IEEE754 does exactly represent 14.7 ? > Speaking as someone working in numerical analysis and scientific computing You're hardly the only Scot in Glasgow. I feel certain a few here abouts have arc'd through PDP-11's, Cyber 205's, and landed in modern centres such as Pawsey.
- sfpotter 3y agoNo, I’m not. In the applications IEEE754 was developed to support through standardization, being able to represent 14.7 exactly is simply unimportant.
- sigsev_251 3y agoC also offers some kind of Bigint, though C's _BitInt was standardized because it's useful for high level synthesis (otherwise you need a custom preprocessor or C++ templates).
- nsajko 3y ago_BitInt may offer longer bit widths than usual, however the width is still fixed (compile time constant).
- mungoman2 3y agoYes, completely agree about the concept. I've had success discussing floats with compute engineers by framing it like a really large fixed point number with all zeroes, except a small bit range. The location and value of the bit range are set by the exponent and mantissa, respectively. This makes precision issues and things like denormals straight forward to discuss.
- HPsquared 3y agoThat's a really intuitive way to visualise it, thanks!
- wongarsu 3y agoBecause floats and integers are multiple orders of magnitude faster than BigDecimal. And even with BigDecimal you have to set some desired precision, because you can't represent all of PI or e in finite memory (and as it turns out, you rarely care for more than 8 digits of Pi). If there was hardware support for 128-bit floating point, we could just use that: 113 bits of precision and 15 bits of exponent is enough for nearly any number you would ever want to store or process. We'd just need to agree on a sensible rounding scheme for turning those into array indexes.
- KeplerBoy 3y agoPeople might not care about more than 7 digits of Pi, but they sure care about the first significant digit of 0.1+0.2 not being "2". Also I don't think it would matter if a language like Python would switch to BigDecimals as the default option. Python does such a ridiculous large amount of stuff unrelated to the actual math, that bit of lost performance won't break the camel's back.
- bla3 3y agoThe first significant digit of "0.1" and "0.2" _is_ 3! The first significant digit of "0.3" just isn't 3. In case anyone hasn't seen it yet, if you print a double with enough fractional values, you'll eventually get its "true" value (because 2 divides 10). >>> print('%.60f' % 0.1) 0.100000000000000005551115123125782702118158340454101562500000 It's only zeros after these zeros. Here are the "true" values of 0.2, 0.3, and their sum: >>> print('%.60f' % 0.2) 0.200000000000000011102230246251565404236316680908203125000000 >>> print('%.60f' % 0.3) 0.299999999999999988897769753748434595763683319091796875000000 >>> print('%.60f' % (0.1 + 0.2)) 0.300000000000000044408920985006261616945266723632812500000000 Turns out 0.1 + 0.2 is actually the number right after "0.3": >>> print('%.60f' % math.nextafter(0.3, 100)) 0.300000000000000044408920985006261616945266723632812500000000 If you don't use a ton of decimal digits, by default the runtime prints the shortest decimal number that has the same binary representation as the true value when converting that decimal number to binary. Much of the confusion around floats is due to that convenience feature that aims to make floats less confusing.
- amelius 3y ago> The real mystery is why most modern languages continue to use floats and integers as the default numeric types rather than some sort of BigDecimal representation, given that precision and programmer intuitiveness trumps raw performance in the vast majority of cases. It's also strange that people use floats to represent 3D coordinates. Why should coordinates closer to the origin of the scene have more precision than those further away?
- Joker_vD 3y agoWell, why should coordinates (and everything else, really) closer to the camera/player have more precision than those further away? And then you accidentally mix these two things together and here we are.
- amelius 3y agoI don't think many programs place the origin at the camera or player, and besides, what if you have more than one player?
- atq2119 3y agoAll 3D rasterization techniques ultimately put the camera at the origin. That's why you have different coordinate systems (model vs. world vs. view/camera) and transforms between them.
- amelius 3y agoYes, of course, but the coordinates in the original scene (before transformation) are typically done in floats. I also get that this is convenient, but the point is that it is not how a purist would do it.
- sk0g 3y agoBecause a lot of physics simulations and actor ticking is either reduced, or turned off entirely, when not within close proximity of the player. There are only so many CPU cycles in 1/60 seconds, so games have to be selective about what they update at full fidelity. (Single precision) float-induced rounding at large offsets would introduce a lot of jitter to these calculations, which looks and feels very janky.
- 6510 3y agoIt doesn't even need to go all the way, it is supper easy to shift the precision depending on the input. For example 0.01 + 2.01 can already be written like ((100*0.01) + (100*2.01)) / 100 which seems trivial to implement in the language. with something like 1000000000000000.0000001 + 1000000000000002.0000001 it seems to me that one could split the operation into 2? One could even discard 2 numbers worth of accuracy and have a representation of 1/3 and 2/3. I'm not that attached to 4294967295 Having 1/3 would be more useful. Why would we call an implementation correct if it produces the wrong result?
- taeric 3y agoYou might be interested in https://en.wikipedia.org/wiki/Numerical_tower https://en.wikipedia.org/wiki/Numerical_tower.
- 6510 3y agoI'm really bad at math and completely unfamiliar with pretty much every relevant implementations. That makes it even worse that the implementation looks like.... well.... dumb shit? How does one do anything without considering the input?
- taeric 3y agoNot sure why you would be insulting on it? My understanding was that it works really well; as does IEEE floating numbers, all told. Understanding the constraints and reasoning behind things goes a long way.
- eschneider 3y agoFor a lot of applications, the performance and accuracy limitations for various numeric representations are very real. Part one f the engineering work is figuring out what your application needs. It’ll vary depending on what you’re doing.
- taeric 3y agoFloating point makes a ton of sense if you ever learned "scientific notation" by hand. As, it is basically that, but in binary. It isn't even gibberish. And we use floats for much the same reason as you would use scientific notation. It is faster and, at large, "significant digits" don't go away just because you can write more digits. Appeals to intuition are tough. Intuition is not some intrinsic property of the number system. It is a learned property. There is really no reason we couldn't be better at teaching it.
- jancsika 3y ago> As, it is basically that, but in binary. And with an exponent bias which isn't part of what anyone learns as basic scientific notation. Used for reasons of efficiency that don't have anything to do with the efficiency humans have in reading the exponent of scientific notation. It's like how a theremin is basically just a monophonic piano that ramps smoothly between notes, floating free in space, scaled and rotated vertically 90-degrees, with one's entire hand being used as a single finger.
- taeric 3y agoMaybe I'm wrong, but it doesn't seem hard to argue that 10 is the base in scientific notation purely for the efficiency of reading that humans have with our common numbers. No reason we couldn't have learned hex with 16 as the base exponent, after all? Indeed, for imperial units, this would have made a ton of sense? 1/16th of an inch would directly be .1 (1*16^-1), after all. Would be better if we used 16 symbols for counting, of course, such that we didn't mix decimal in that notation. This, sadly, quickly devolves into debates about metric versus imperial. :(
- masklinn 3y ago> Indeed, for imperial units, this would have made a ton of sense? No. The number of ratios to 16 units is 2 to a few dozens. You have 12in to a foot, 3 ft to a yard, 1760 yd to a mile (because 80 chains or 22 yards). A nautical mile is 10 cables, of 100 fathoms each. 640 acres to a square mile. 20 fl oz to a pint, 2 pt to a quart, 4 qt to a gallon (8 pt to a gal, 160 oz to a gal). There's 16 drachm to an ounce and 16 oz to a pound. Sadly there's then 14 pounds to a stone, 8 st to a hundredweight unless you have abandoned stones and have 100-lb hundredweight because customary units are dumb, which then gives you a different ton of 20 cwt depending on, well, your cwt. > 1/16th of an inch would directly be .1 (1*16^-1), after all. Shame that would be useful for just about nothing else. > This, sadly, quickly devolves into debates about metric versus imperial. :( I wonder why...
- analog31 3y agoLearning scientific notation before starting computer programming made it easy for me to understand floats. The limitations of floating point were covered in every class that used computation within the science and math curriculum.
- jcranmer 3y ago> The real mystery is why most modern languages continue to use floats and integers as the default numeric types rather than some sort of BigDecimal representation, given that precision and programmer intuitiveness trumps raw performance in the vast majority of cases. BigDecimal actually doesn't fix anything. The underlying problem is that decimal representations are rarely finite--in base 10, 1 / 3 has to get rounded to 0.333, so the same problem comes up as 1 / 3 * 3 = 0.999 ≠ 1. If you want to fix that, you have to move to rational representations, but then you get flummoxed by things like sqrt(2) or atan(1/10) or pi. No, the thing is that 0.1 + 0.2 is 0.3 in double arithmetic. The result differs from the closest representation to 0.3 only a tiny bit--like measuring the distance between LA and NY and being off by a few atoms. The problem is that == is checking for equality of all bits rather than a tolerance, and that string printing is using rules that it prints the shortest base-10 representation that rounds to that extra floating-point value, both of which exaggerate the importance of the last few bits. So the real mystery is why no one's designed a language that makes floating-point comparisons default to relative tolerance of 10^-10 or so.
- diydsp 3y ago> real mystery is why no one's designed a language that makes floating-point comparisons default to relative tolerance of 10^-10 or so. Bc the very next step is a footgun: "the function said these two values are equal but they're not."
- taeric 3y agoI think it is fair to wonder why no language made it so that you can only compare floats with a defined tolerance?
- bee_rider 3y agoI suspect most languages use floats because they come built in and have reasonably sensible behavior. Implementing a big decimal with feature-parity with floats is going to be a big pain. Anyway, what about irrational numbers? Python does big ints by default. Giving up at E-1 seems like a reasonably pragmatic but arbitrary choice.