10 ms·

New zine: How integers and floats work

153 points by signa11 3y ago

zoky 3y ago
I feel like the implementation of floating point is a bit weird, but the concept really isn’t. You have a big string of numbers of more or less a fixed length, with a decimal point somewhere therein. Floating point lets you specify what the numbers are, and where the decimal point lies. Smaller numbers have the decimal point to the left, so more room for precision to the right. Larger numbers need to move the decimal point to the right, so there’s less room for precision. Oh, and everything is secretly in base two, so sometimes numbers don’t work out quite the way you think they should, which pretty much means don’t use it for money or anything else where precision really makes a difference. Yeah, the actual implementation is fairly different, but that’s just computer gibberish. Most of us no longer need to think of strings as an offset into memory to an array of characters that happen to be interpretable as a group, but that’s really what it is despite any abstractions. The real mystery is why most modern languages continue to use floats and integers as the default numeric types rather than some sort of BigDecimal representation, given that precision and programmer intuitiveness trumps raw performance in the vast majority of cases.
- di4na 3y ago
  Because they do not for integer. The vast majority of languages you use out there that do not care about speed use Bigint. Python. Haskell. Ruby. Perl. PHP. Erlang. Etc etc. Floating point are used because quite simply they are the only way to do it that has wide support that can be reused. A good fixed width or Decimal implementation that could be reused and implement all the maths you want for Reals are... Rare. The exception is Java and JavaScript mostly. But JS has Bigint now. Just not as a default.
  - zoky 3y ago
    I guess what I’m saying is that they should. We all grew up learning math before programming, and mathematically speaking, there is no qualitative difference between 15 and 14.7, and no difference whatsoever between 15 and 15.0. So why do programming languages, particularly ones like the ones you mention that purport to abstract such distinctions away, treat them differently by default? There is no logical reason why 5/2 should be 2 rather than 2.5, yet by default it is, for apparently no other reason than that is faster for the CPU to calculate. I’m not saying get rid of these distinctions entirely. But where they matter, for performance reasons or whatever, they should be opt-in rather than the default.
    - defrost 3y ago
      Integers are exact and give repeatable results. Many floats (such as 14.7) are not even represented exactly, float operations have a large number of issues. If you're wondering about all this, see: What Every Computer Scientist Should Know About Floating-Point Arithmetic ~ David Goldberg, March, 1991 issue of Computing Surveys https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.h... (and elsewhere)
      - HPsquared 3y ago
        It's a bit like musical scales. It all seems nice and neat, until you start looking for exactness.
      - sfpotter 3y ago
        Please don’t spread FUD about floating point arithmetic. IEEE754 is an extremely carefully worked out standard which provides a good trade off between speed and accuracy. The problem of representing a continuum of real numbers in a fixed amount of memory is highly nontrivial. The error incurred by each operation is bounded and predictable—not exact, but as good as you’re going to get given the constraints. Also, results are replicable. Speaking as someone working in numerical analysis and scientific computing, so I work with floats all day long.
        defrost 3y ago
        > Please don’t spread FUD about floating point arithmetic. I'm sorry, my apologies. Are you saying that IEEE754 does exactly represent 14.7 ? > Speaking as someone working in numerical analysis and scientific computing You're hardly the only Scot in Glasgow. I feel certain a few here abouts have arc'd through PDP-11's, Cyber 205's, and landed in modern centres such as Pawsey.
        sfpotter 3y ago
        No, I’m not. In the applications IEEE754 was developed to support through standardization, being able to represent 14.7 exactly is simply unimportant.
  - sigsev_251 3y ago
    C also offers some kind of Bigint, though C's _BitInt was standardized because it's useful for high level synthesis (otherwise you need a custom preprocessor or C++ templates).
    - nsajko 3y ago
      _BitInt may offer longer bit widths than usual, however the width is still fixed (compile time constant).
- mungoman2 3y ago
  Yes, completely agree about the concept. I've had success discussing floats with compute engineers by framing it like a really large fixed point number with all zeroes, except a small bit range. The location and value of the bit range are set by the exponent and mantissa, respectively. This makes precision issues and things like denormals straight forward to discuss.
  - HPsquared 3y ago
    That's a really intuitive way to visualise it, thanks!
- wongarsu 3y ago
  Because floats and integers are multiple orders of magnitude faster than BigDecimal. And even with BigDecimal you have to set some desired precision, because you can't represent all of PI or e in finite memory (and as it turns out, you rarely care for more than 8 digits of Pi). If there was hardware support for 128-bit floating point, we could just use that: 113 bits of precision and 15 bits of exponent is enough for nearly any number you would ever want to store or process. We'd just need to agree on a sensible rounding scheme for turning those into array indexes.
  - KeplerBoy 3y ago
    People might not care about more than 7 digits of Pi, but they sure care about the first significant digit of 0.1+0.2 not being "2". Also I don't think it would matter if a language like Python would switch to BigDecimals as the default option. Python does such a ridiculous large amount of stuff unrelated to the actual math, that bit of lost performance won't break the camel's back.
    - bla3 3y ago
      The first significant digit of "0.1" and "0.2" _is_ 3! The first significant digit of "0.3" just isn't 3. In case anyone hasn't seen it yet, if you print a double with enough fractional values, you'll eventually get its "true" value (because 2 divides 10). >>> print('%.60f' % 0.1) 0.100000000000000005551115123125782702118158340454101562500000 It's only zeros after these zeros. Here are the "true" values of 0.2, 0.3, and their sum: >>> print('%.60f' % 0.2) 0.200000000000000011102230246251565404236316680908203125000000 >>> print('%.60f' % 0.3) 0.299999999999999988897769753748434595763683319091796875000000 >>> print('%.60f' % (0.1 + 0.2)) 0.300000000000000044408920985006261616945266723632812500000000 Turns out 0.1 + 0.2 is actually the number right after "0.3": >>> print('%.60f' % math.nextafter(0.3, 100)) 0.300000000000000044408920985006261616945266723632812500000000 If you don't use a ton of decimal digits, by default the runtime prints the shortest decimal number that has the same binary representation as the true value when converting that decimal number to binary. Much of the confusion around floats is due to that convenience feature that aims to make floats less confusing.
- amelius 3y ago
  > The real mystery is why most modern languages continue to use floats and integers as the default numeric types rather than some sort of BigDecimal representation, given that precision and programmer intuitiveness trumps raw performance in the vast majority of cases. It's also strange that people use floats to represent 3D coordinates. Why should coordinates closer to the origin of the scene have more precision than those further away?
  - Joker_vD 3y ago
    Well, why should coordinates (and everything else, really) closer to the camera/player have more precision than those further away? And then you accidentally mix these two things together and here we are.
    - amelius 3y ago
      I don't think many programs place the origin at the camera or player, and besides, what if you have more than one player?
      - atq2119 3y ago
        All 3D rasterization techniques ultimately put the camera at the origin. That's why you have different coordinate systems (model vs. world vs. view/camera) and transforms between them.
        amelius 3y ago
        Yes, of course, but the coordinates in the original scene (before transformation) are typically done in floats. I also get that this is convenient, but the point is that it is not how a purist would do it.
    - sk0g 3y ago
      Because a lot of physics simulations and actor ticking is either reduced, or turned off entirely, when not within close proximity of the player. There are only so many CPU cycles in 1/60 seconds, so games have to be selective about what they update at full fidelity. (Single precision) float-induced rounding at large offsets would introduce a lot of jitter to these calculations, which looks and feels very janky.
- 6510 3y ago
  It doesn't even need to go all the way, it is supper easy to shift the precision depending on the input. For example 0.01 + 2.01 can already be written like ((100*0.01) + (100*2.01)) / 100 which seems trivial to implement in the language. with something like 1000000000000000.0000001 + 1000000000000002.0000001 it seems to me that one could split the operation into 2? One could even discard 2 numbers worth of accuracy and have a representation of 1/3 and 2/3. I'm not that attached to 4294967295 Having 1/3 would be more useful. Why would we call an implementation correct if it produces the wrong result?
  - taeric 3y ago
    You might be interested in https://en.wikipedia.org/wiki/Numerical_tower https://en.wikipedia.org/wiki/Numerical_tower.
    - 6510 3y ago
      I'm really bad at math and completely unfamiliar with pretty much every relevant implementations. That makes it even worse that the implementation looks like.... well.... dumb shit? How does one do anything without considering the input?
      - taeric 3y ago
        Not sure why you would be insulting on it? My understanding was that it works really well; as does IEEE floating numbers, all told. Understanding the constraints and reasoning behind things goes a long way.
- eschneider 3y ago
  For a lot of applications, the performance and accuracy limitations for various numeric representations are very real. Part one f the engineering work is figuring out what your application needs. It’ll vary depending on what you’re doing.
- taeric 3y ago
  Floating point makes a ton of sense if you ever learned "scientific notation" by hand. As, it is basically that, but in binary. It isn't even gibberish. And we use floats for much the same reason as you would use scientific notation. It is faster and, at large, "significant digits" don't go away just because you can write more digits. Appeals to intuition are tough. Intuition is not some intrinsic property of the number system. It is a learned property. There is really no reason we couldn't be better at teaching it.
  - jancsika 3y ago
    > As, it is basically that, but in binary. And with an exponent bias which isn't part of what anyone learns as basic scientific notation. Used for reasons of efficiency that don't have anything to do with the efficiency humans have in reading the exponent of scientific notation. It's like how a theremin is basically just a monophonic piano that ramps smoothly between notes, floating free in space, scaled and rotated vertically 90-degrees, with one's entire hand being used as a single finger.
    - taeric 3y ago
      Maybe I'm wrong, but it doesn't seem hard to argue that 10 is the base in scientific notation purely for the efficiency of reading that humans have with our common numbers. No reason we couldn't have learned hex with 16 as the base exponent, after all? Indeed, for imperial units, this would have made a ton of sense? 1/16th of an inch would directly be .1 (1*16^-1), after all. Would be better if we used 16 symbols for counting, of course, such that we didn't mix decimal in that notation. This, sadly, quickly devolves into debates about metric versus imperial. :(
      - masklinn 3y ago
        > Indeed, for imperial units, this would have made a ton of sense? No. The number of ratios to 16 units is 2 to a few dozens. You have 12in to a foot, 3 ft to a yard, 1760 yd to a mile (because 80 chains or 22 yards). A nautical mile is 10 cables, of 100 fathoms each. 640 acres to a square mile. 20 fl oz to a pint, 2 pt to a quart, 4 qt to a gallon (8 pt to a gal, 160 oz to a gal). There's 16 drachm to an ounce and 16 oz to a pound. Sadly there's then 14 pounds to a stone, 8 st to a hundredweight unless you have abandoned stones and have 100-lb hundredweight because customary units are dumb, which then gives you a different ton of 20 cwt depending on, well, your cwt. > 1/16th of an inch would directly be .1 (1*16^-1), after all. Shame that would be useful for just about nothing else. > This, sadly, quickly devolves into debates about metric versus imperial. :( I wonder why...
- analog31 3y ago
  Learning scientific notation before starting computer programming made it easy for me to understand floats. The limitations of floating point were covered in every class that used computation within the science and math curriculum.
- jcranmer 3y ago
  > The real mystery is why most modern languages continue to use floats and integers as the default numeric types rather than some sort of BigDecimal representation, given that precision and programmer intuitiveness trumps raw performance in the vast majority of cases. BigDecimal actually doesn't fix anything. The underlying problem is that decimal representations are rarely finite--in base 10, 1 / 3 has to get rounded to 0.333, so the same problem comes up as 1 / 3 * 3 = 0.999 ≠ 1. If you want to fix that, you have to move to rational representations, but then you get flummoxed by things like sqrt(2) or atan(1/10) or pi. No, the thing is that 0.1 + 0.2 is 0.3 in double arithmetic. The result differs from the closest representation to 0.3 only a tiny bit--like measuring the distance between LA and NY and being off by a few atoms. The problem is that == is checking for equality of all bits rather than a tolerance, and that string printing is using rules that it prints the shortest base-10 representation that rounds to that extra floating-point value, both of which exaggerate the importance of the last few bits. So the real mystery is why no one's designed a language that makes floating-point comparisons default to relative tolerance of 10^-10 or so.
  - diydsp 3y ago
    > real mystery is why no one's designed a language that makes floating-point comparisons default to relative tolerance of 10^-10 or so. Bc the very next step is a footgun: "the function said these two values are equal but they're not."
    - taeric 3y ago
      I think it is fair to wonder why no language made it so that you can only compare floats with a defined tolerance?
- bee_rider 3y ago
  I suspect most languages use floats because they come built in and have reasonably sensible behavior. Implementing a big decimal with feature-parity with floats is going to be a big pain. Anyway, what about irrational numbers? Python does big ints by default. Giving up at E-1 seems like a reasonably pragmatic but arbitrary choice.
Waterluvian 3y ago
Julia’s style is charming and so easy to learn from. In my opinion the float number line is the one thing anyone needs to look at to understand the most important part about floats, and it visualizes the concept more intuitively than most sources I’ve found.
ckdot2 3y ago
Is this an ad? Not cool, HN.
- adiM 3y ago
  Curious as to why you thought that this was an ad? Julia Evans has written a bunch of such zines in the past as well and they are quite informative
  - balls187 3y ago
    Because you have to pay to get the actual content. The title says “How integers and floats work.”But it isn’t a post about how integers and floating point precision numbers are represented in binary. Rather it’s a post announcing an illustrated guide to ints and floats, which you have to pay for.
    - dang 3y ago
      The submitted title was "How integers and floats work" but really it should have included the "New zine" part. I agree that that changes the meaning and that an article with the submitted title ought to be about how integers and floats work.
  - deleted 3y ago
    [deleted]
  - krisoft 3y ago
    > Curious as to why you thought that this was an ad? Because it is clearly an ad? It explains what will you get and at what price. It does not explain "how Integers and Floats work". They want you to buy the zine, which is what they are advertising. > Julia Evans has written a bunch of such zines in the past as well and they are quite informative Good for them. Why do you think this makes it not an ad?
- t-3 3y ago
  It is an ad, but it's also an announcement with information that many hners will be interested in and enjoy discussing. There are links to tools and blog posts on the topic as well as links to the advertised product. I don't think it's out of place here.
voxadam 3y ago
Julia Evans,[0] the author of this zine, was recently interviewed[1] about this specific topic and publication on the Embedded.fm podcast.[2] [0] https://jvns.ca/ https://jvns.ca/ [1] https://embedded.fm/episodes/452 https://embedded.fm/episodes/452 [2] https://embedded.fm https://embedded.fm
benreesman 3y ago
I’ve been reading Julia Evans’s blog for years and years now, and it’s just so cool to watch someone go from blogging about technology mostly as a way to learn technology but just really stick with it long enough that at some point it’s mostly about teaching technology. Julia has become a really, really serious pro over the years, but never lost the passion for blogging the journey. In terms of mostly self-taught hackers, an iconic legends like jwz or something at this point. Keep it up another decade!
javier_e06 3y ago
I was computer science major and a DELL Phone Tech support evening shift guy in Austin back in the day when Intel's Pentium V arrived in the market with a with Floating Point Error: https://en.wikipedia.org/wiki/Pentium_FDIV_bug https://en.wikipedia.org/wiki/Pentium_FDIV_bug The lines got busy with people asking to return the brand new DELLs. Old ladies pulled the excel spreadsheets and start copying paste values to test for the floating point error. Then they will call me to return their system. After a few days explaining that unless they are planning to plot a orbit into Jupiter the PC was fine they all insisted to return the PCs. We ended with a lot of refurbished PCs. 6 nines anyone? https://en.wikipedia.org/wiki/Six_nines_in_pi https://en.wikipedia.org/wiki/Six_nines_in_pi
- balls187 3y ago
  It’s likely Intel compensated Dell for the problem. I don’t like arguing by analogy, but I don’t have a better way to make my point—-what if instead, old lady paid for a cpu that was clocked at 3.2ghz, and instead the cpu was hardware throttled down to 2.8ghz. I expect when I purchase an item, all the features I paid for operate correctly—-even if I never expect to use all of them or use them to their full potential.
klelatti 3y ago
Obligatory link to Bartosz Ciechanowski's (fantastic) post on Floating Point: https://ciechanow.ski/exposing-floating-point/ https://ciechanow.ski/exposing-floating-point/ and his companion website https://float.exposed/0x44bf9400 https://float.exposed/0x44bf9400
victor106 3y ago
From the post: And the way integers and floats are represented isn’t going to change (floating point was standardized in 1985!), so you only have to learn it once. This got me thinking, what are some things that DON’T change that are worth learning? HN I think is mostly focused on the latest things that change but there’s lot of value in focusing on things that don’t change.
- wongarsu 3y ago
  SQL was invented in the 70s, and by 2003 had most of the features you will ever use. New stuff gets added over time, and some of it is useful (JSON support), but 99% of use cases are covered by extremely simple stuff that has always been there. Database normalization (1NF, 3NF, BCNF, etc) is worth knowing about when designing any kind of datastore. At least consider the tradeoffs of breaking BCNF. Almost all work on this was done in the 70s. Linear algebra always turns up in unexpected places. Traditionally you would see it mostly in anything even vaguely related to computer graphics, but now it turns out to be quite useful for ML as well. Mostly stuff from the late 1800s, some a couple millennia older. TCP/IP is mostly unchanged since the 70s, ARP is from the 80s, and HTTP1.0 is from the 90s. Combine those 4 and you have a good mental model of how the network stack works, even if HTTP3.0 is a bit more fancy. Linux and Windows are quite different from each other, but the broad strokes of each hasn't changed much in the last two decades. For example knowing how they manage memory or how their respective thread schedulers work is potentially useful (though Linux thread schedulers do change from time to time. Windows seems to be happy with the architecture of theirs though). Backus–Naur Form is an incredibly useful notation for describing context-free grammars. Stuff like valid syntax of a programming language, or how an address can look like. It's from the late 50s, though predecessors reach back much further. Extended Backus–Naur Form from the 90s is probably what you want to use today, it has an ISO standard and a great wikipedia page.

153 points by signa11 3y ago

zoky 3y ago

I feel like the implementation of floating point is a bit weird, but the concept really isn’t. You have a big string of numbers of more or less a fixed length, with a decimal point somewhere therein. Floating point lets you specify what the numbers are, and where the decimal point lies. Smaller numbers have the decimal point to the left, so more room for precision to the right. Larger numbers need to move the decimal point to the right, so there’s less room for precision. Oh, and everything is secretly in base two, so sometimes numbers don’t work out quite the way you think they should, which pretty much means don’t use it for money or anything else where precision really makes a difference. Yeah, the actual implementation is fairly different, but that’s just computer gibberish. Most of us no longer need to think of strings as an offset into memory to an array of characters that happen to be interpretable as a group, but that’s really what it is despite any abstractions. The real mystery is why most modern languages continue to use floats and integers as the default numeric types rather than some sort of BigDecimal representation, given that precision and programmer intuitiveness trumps raw performance in the vast majority of cases.

di4na 3y ago
Because they do not for integer. The vast majority of languages you use out there that do not care about speed use Bigint. Python. Haskell. Ruby. Perl. PHP. Erlang. Etc etc. Floating point are used because quite simply they are the only way to do it that has wide support that can be reused. A good fixed width or Decimal implementation that could be reused and implement all the maths you want for Reals are... Rare. The exception is Java and JavaScript mostly. But JS has Bigint now. Just not as a default.
- zoky 3y ago
  I guess what I’m saying is that they should. We all grew up learning math before programming, and mathematically speaking, there is no qualitative difference between 15 and 14.7, and no difference whatsoever between 15 and 15.0. So why do programming languages, particularly ones like the ones you mention that purport to abstract such distinctions away, treat them differently by default? There is no logical reason why 5/2 should be 2 rather than 2.5, yet by default it is, for apparently no other reason than that is faster for the CPU to calculate. I’m not saying get rid of these distinctions entirely. But where they matter, for performance reasons or whatever, they should be opt-in rather than the default.
  - defrost 3y ago
    Integers are exact and give repeatable results. Many floats (such as 14.7) are not even represented exactly, float operations have a large number of issues. If you're wondering about all this, see: What Every Computer Scientist Should Know About Floating-Point Arithmetic ~ David Goldberg, March, 1991 issue of Computing Surveys https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.h... (and elsewhere)
    - HPsquared 3y ago
      It's a bit like musical scales. It all seems nice and neat, until you start looking for exactness.
    - sfpotter 3y ago
      Please don’t spread FUD about floating point arithmetic. IEEE754 is an extremely carefully worked out standard which provides a good trade off between speed and accuracy. The problem of representing a continuum of real numbers in a fixed amount of memory is highly nontrivial. The error incurred by each operation is bounded and predictable—not exact, but as good as you’re going to get given the constraints. Also, results are replicable. Speaking as someone working in numerical analysis and scientific computing, so I work with floats all day long.
      - defrost 3y ago
        > Please don’t spread FUD about floating point arithmetic. I'm sorry, my apologies. Are you saying that IEEE754 does exactly represent 14.7 ? > Speaking as someone working in numerical analysis and scientific computing You're hardly the only Scot in Glasgow. I feel certain a few here abouts have arc'd through PDP-11's, Cyber 205's, and landed in modern centres such as Pawsey.
        sfpotter 3y ago
        No, I’m not. In the applications IEEE754 was developed to support through standardization, being able to represent 14.7 exactly is simply unimportant.
- sigsev_251 3y ago
  C also offers some kind of Bigint, though C's _BitInt was standardized because it's useful for high level synthesis (otherwise you need a custom preprocessor or C++ templates).
  - nsajko 3y ago
    _BitInt may offer longer bit widths than usual, however the width is still fixed (compile time constant).
mungoman2 3y ago
Yes, completely agree about the concept. I've had success discussing floats with compute engineers by framing it like a really large fixed point number with all zeroes, except a small bit range. The location and value of the bit range are set by the exponent and mantissa, respectively. This makes precision issues and things like denormals straight forward to discuss.
- HPsquared 3y ago
  That's a really intuitive way to visualise it, thanks!
wongarsu 3y ago
Because floats and integers are multiple orders of magnitude faster than BigDecimal. And even with BigDecimal you have to set some desired precision, because you can't represent all of PI or e in finite memory (and as it turns out, you rarely care for more than 8 digits of Pi). If there was hardware support for 128-bit floating point, we could just use that: 113 bits of precision and 15 bits of exponent is enough for nearly any number you would ever want to store or process. We'd just need to agree on a sensible rounding scheme for turning those into array indexes.
- KeplerBoy 3y ago
  People might not care about more than 7 digits of Pi, but they sure care about the first significant digit of 0.1+0.2 not being "2". Also I don't think it would matter if a language like Python would switch to BigDecimals as the default option. Python does such a ridiculous large amount of stuff unrelated to the actual math, that bit of lost performance won't break the camel's back.
  - bla3 3y ago
    The first significant digit of "0.1" and "0.2" _is_ 3! The first significant digit of "0.3" just isn't 3. In case anyone hasn't seen it yet, if you print a double with enough fractional values, you'll eventually get its "true" value (because 2 divides 10). >>> print('%.60f' % 0.1) 0.100000000000000005551115123125782702118158340454101562500000 It's only zeros after these zeros. Here are the "true" values of 0.2, 0.3, and their sum: >>> print('%.60f' % 0.2) 0.200000000000000011102230246251565404236316680908203125000000 >>> print('%.60f' % 0.3) 0.299999999999999988897769753748434595763683319091796875000000 >>> print('%.60f' % (0.1 + 0.2)) 0.300000000000000044408920985006261616945266723632812500000000 Turns out 0.1 + 0.2 is actually the number right after "0.3": >>> print('%.60f' % math.nextafter(0.3, 100)) 0.300000000000000044408920985006261616945266723632812500000000 If you don't use a ton of decimal digits, by default the runtime prints the shortest decimal number that has the same binary representation as the true value when converting that decimal number to binary. Much of the confusion around floats is due to that convenience feature that aims to make floats less confusing.
amelius 3y ago
> The real mystery is why most modern languages continue to use floats and integers as the default numeric types rather than some sort of BigDecimal representation, given that precision and programmer intuitiveness trumps raw performance in the vast majority of cases. It's also strange that people use floats to represent 3D coordinates. Why should coordinates closer to the origin of the scene have more precision than those further away?
- Joker_vD 3y ago
  Well, why should coordinates (and everything else, really) closer to the camera/player have more precision than those further away? And then you accidentally mix these two things together and here we are.
  - amelius 3y ago
    I don't think many programs place the origin at the camera or player, and besides, what if you have more than one player?
    - atq2119 3y ago
      All 3D rasterization techniques ultimately put the camera at the origin. That's why you have different coordinate systems (model vs. world vs. view/camera) and transforms between them.
      - amelius 3y ago
        Yes, of course, but the coordinates in the original scene (before transformation) are typically done in floats. I also get that this is convenient, but the point is that it is not how a purist would do it.
  - sk0g 3y ago
    Because a lot of physics simulations and actor ticking is either reduced, or turned off entirely, when not within close proximity of the player. There are only so many CPU cycles in 1/60 seconds, so games have to be selective about what they update at full fidelity. (Single precision) float-induced rounding at large offsets would introduce a lot of jitter to these calculations, which looks and feels very janky.
6510 3y ago
It doesn't even need to go all the way, it is supper easy to shift the precision depending on the input. For example 0.01 + 2.01 can already be written like ((100*0.01) + (100*2.01)) / 100 which seems trivial to implement in the language. with something like 1000000000000000.0000001 + 1000000000000002.0000001 it seems to me that one could split the operation into 2? One could even discard 2 numbers worth of accuracy and have a representation of 1/3 and 2/3. I'm not that attached to 4294967295 Having 1/3 would be more useful. Why would we call an implementation correct if it produces the wrong result?
- taeric 3y ago
  You might be interested in https://en.wikipedia.org/wiki/Numerical_tower https://en.wikipedia.org/wiki/Numerical_tower.
  - 6510 3y ago
    I'm really bad at math and completely unfamiliar with pretty much every relevant implementations. That makes it even worse that the implementation looks like.... well.... dumb shit? How does one do anything without considering the input?
    - taeric 3y ago
      Not sure why you would be insulting on it? My understanding was that it works really well; as does IEEE floating numbers, all told. Understanding the constraints and reasoning behind things goes a long way.
eschneider 3y ago
For a lot of applications, the performance and accuracy limitations for various numeric representations are very real. Part one f the engineering work is figuring out what your application needs. It’ll vary depending on what you’re doing.
taeric 3y ago
Floating point makes a ton of sense if you ever learned "scientific notation" by hand. As, it is basically that, but in binary. It isn't even gibberish. And we use floats for much the same reason as you would use scientific notation. It is faster and, at large, "significant digits" don't go away just because you can write more digits. Appeals to intuition are tough. Intuition is not some intrinsic property of the number system. It is a learned property. There is really no reason we couldn't be better at teaching it.
- jancsika 3y ago
  > As, it is basically that, but in binary. And with an exponent bias which isn't part of what anyone learns as basic scientific notation. Used for reasons of efficiency that don't have anything to do with the efficiency humans have in reading the exponent of scientific notation. It's like how a theremin is basically just a monophonic piano that ramps smoothly between notes, floating free in space, scaled and rotated vertically 90-degrees, with one's entire hand being used as a single finger.
  - taeric 3y ago
    Maybe I'm wrong, but it doesn't seem hard to argue that 10 is the base in scientific notation purely for the efficiency of reading that humans have with our common numbers. No reason we couldn't have learned hex with 16 as the base exponent, after all? Indeed, for imperial units, this would have made a ton of sense? 1/16th of an inch would directly be .1 (1*16^-1), after all. Would be better if we used 16 symbols for counting, of course, such that we didn't mix decimal in that notation. This, sadly, quickly devolves into debates about metric versus imperial. :(
    - masklinn 3y ago
      > Indeed, for imperial units, this would have made a ton of sense? No. The number of ratios to 16 units is 2 to a few dozens. You have 12in to a foot, 3 ft to a yard, 1760 yd to a mile (because 80 chains or 22 yards). A nautical mile is 10 cables, of 100 fathoms each. 640 acres to a square mile. 20 fl oz to a pint, 2 pt to a quart, 4 qt to a gallon (8 pt to a gal, 160 oz to a gal). There's 16 drachm to an ounce and 16 oz to a pound. Sadly there's then 14 pounds to a stone, 8 st to a hundredweight unless you have abandoned stones and have 100-lb hundredweight because customary units are dumb, which then gives you a different ton of 20 cwt depending on, well, your cwt. > 1/16th of an inch would directly be .1 (1*16^-1), after all. Shame that would be useful for just about nothing else. > This, sadly, quickly devolves into debates about metric versus imperial. :( I wonder why...
analog31 3y ago
Learning scientific notation before starting computer programming made it easy for me to understand floats. The limitations of floating point were covered in every class that used computation within the science and math curriculum.
jcranmer 3y ago
> The real mystery is why most modern languages continue to use floats and integers as the default numeric types rather than some sort of BigDecimal representation, given that precision and programmer intuitiveness trumps raw performance in the vast majority of cases. BigDecimal actually doesn't fix anything. The underlying problem is that decimal representations are rarely finite--in base 10, 1 / 3 has to get rounded to 0.333, so the same problem comes up as 1 / 3 * 3 = 0.999 ≠ 1. If you want to fix that, you have to move to rational representations, but then you get flummoxed by things like sqrt(2) or atan(1/10) or pi. No, the thing is that 0.1 + 0.2 is 0.3 in double arithmetic. The result differs from the closest representation to 0.3 only a tiny bit--like measuring the distance between LA and NY and being off by a few atoms. The problem is that == is checking for equality of all bits rather than a tolerance, and that string printing is using rules that it prints the shortest base-10 representation that rounds to that extra floating-point value, both of which exaggerate the importance of the last few bits. So the real mystery is why no one's designed a language that makes floating-point comparisons default to relative tolerance of 10^-10 or so.
- diydsp 3y ago
  > real mystery is why no one's designed a language that makes floating-point comparisons default to relative tolerance of 10^-10 or so. Bc the very next step is a footgun: "the function said these two values are equal but they're not."
  - taeric 3y ago
    I think it is fair to wonder why no language made it so that you can only compare floats with a defined tolerance?
bee_rider 3y ago
I suspect most languages use floats because they come built in and have reasonably sensible behavior. Implementing a big decimal with feature-parity with floats is going to be a big pain. Anyway, what about irrational numbers? Python does big ints by default. Giving up at E-1 seems like a reasonably pragmatic but arbitrary choice.

Waterluvian 3y ago

Julia’s style is charming and so easy to learn from. In my opinion the float number line is the one thing anyone needs to look at to understand the most important part about floats, and it visualizes the concept more intuitively than most sources I’ve found.

ckdot2 3y ago

Is this an ad? Not cool, HN.

adiM 3y ago
Curious as to why you thought that this was an ad? Julia Evans has written a bunch of such zines in the past as well and they are quite informative
- balls187 3y ago
  Because you have to pay to get the actual content. The title says “How integers and floats work.”But it isn’t a post about how integers and floating point precision numbers are represented in binary. Rather it’s a post announcing an illustrated guide to ints and floats, which you have to pay for.
  - dang 3y ago
    The submitted title was "How integers and floats work" but really it should have included the "New zine" part. I agree that that changes the meaning and that an article with the submitted title ought to be about how integers and floats work.
- deleted 3y ago
  [deleted]
- krisoft 3y ago
  > Curious as to why you thought that this was an ad? Because it is clearly an ad? It explains what will you get and at what price. It does not explain "how Integers and Floats work". They want you to buy the zine, which is what they are advertising. > Julia Evans has written a bunch of such zines in the past as well and they are quite informative Good for them. Why do you think this makes it not an ad?
t-3 3y ago
It is an ad, but it's also an announcement with information that many hners will be interested in and enjoy discussing. There are links to tools and blog posts on the topic as well as links to the advertised product. I don't think it's out of place here.

voxadam 3y ago

Julia Evans,[0] the author of this zine, was recently interviewed[1] about this specific topic and publication on the Embedded.fm podcast.[2] [0] https://jvns.ca/ https://jvns.ca/ [1] https://embedded.fm/episodes/452 https://embedded.fm/episodes/452 [2] https://embedded.fm https://embedded.fm

benreesman 3y ago

I’ve been reading Julia Evans’s blog for years and years now, and it’s just so cool to watch someone go from blogging about technology mostly as a way to learn technology but just really stick with it long enough that at some point it’s mostly about teaching technology. Julia has become a really, really serious pro over the years, but never lost the passion for blogging the journey. In terms of mostly self-taught hackers, an iconic legends like jwz or something at this point. Keep it up another decade!

javier_e06 3y ago

I was computer science major and a DELL Phone Tech support evening shift guy in Austin back in the day when Intel's Pentium V arrived in the market with a with Floating Point Error: https://en.wikipedia.org/wiki/Pentium_FDIV_bug https://en.wikipedia.org/wiki/Pentium_FDIV_bug The lines got busy with people asking to return the brand new DELLs. Old ladies pulled the excel spreadsheets and start copying paste values to test for the floating point error. Then they will call me to return their system. After a few days explaining that unless they are planning to plot a orbit into Jupiter the PC was fine they all insisted to return the PCs. We ended with a lot of refurbished PCs. 6 nines anyone? https://en.wikipedia.org/wiki/Six_nines_in_pi https://en.wikipedia.org/wiki/Six_nines_in_pi

balls187 3y ago
It’s likely Intel compensated Dell for the problem. I don’t like arguing by analogy, but I don’t have a better way to make my point—-what if instead, old lady paid for a cpu that was clocked at 3.2ghz, and instead the cpu was hardware throttled down to 2.8ghz. I expect when I purchase an item, all the features I paid for operate correctly—-even if I never expect to use all of them or use them to their full potential.

klelatti 3y ago

Obligatory link to Bartosz Ciechanowski's (fantastic) post on Floating Point: https://ciechanow.ski/exposing-floating-point/ https://ciechanow.ski/exposing-floating-point/ and his companion website https://float.exposed/0x44bf9400 https://float.exposed/0x44bf9400

victor106 3y ago

From the post: And the way integers and floats are represented isn’t going to change (floating point was standardized in 1985!), so you only have to learn it once. This got me thinking, what are some things that DON’T change that are worth learning? HN I think is mostly focused on the latest things that change but there’s lot of value in focusing on things that don’t change.

wongarsu 3y ago
SQL was invented in the 70s, and by 2003 had most of the features you will ever use. New stuff gets added over time, and some of it is useful (JSON support), but 99% of use cases are covered by extremely simple stuff that has always been there. Database normalization (1NF, 3NF, BCNF, etc) is worth knowing about when designing any kind of datastore. At least consider the tradeoffs of breaking BCNF. Almost all work on this was done in the 70s. Linear algebra always turns up in unexpected places. Traditionally you would see it mostly in anything even vaguely related to computer graphics, but now it turns out to be quite useful for ML as well. Mostly stuff from the late 1800s, some a couple millennia older. TCP/IP is mostly unchanged since the 70s, ARP is from the 80s, and HTTP1.0 is from the 90s. Combine those 4 and you have a good mental model of how the network stack works, even if HTTP3.0 is a bit more fancy. Linux and Windows are quite different from each other, but the broad strokes of each hasn't changed much in the last two decades. For example knowing how they manage memory or how their respective thread schedulers work is potentially useful (though Linux thread schedulers do change from time to time. Windows seems to be happy with the architecture of theirs though). Backus–Naur Form is an incredibly useful notation for describing context-free grammars. Stuff like valid syntax of a programming language, or how an address can look like. It's from the late 50s, though predecessors reach back much further. Extended Backus–Naur Form from the 90s is probably what you want to use today, it has an ISO standard and a great wikipedia page.