4 ms·

I vaguely remember learning similar fact years ago, but it was phrased something like "There are as many floats in [0, 1] range as in [1, infty]". Leaving aside

by rednum 6y ago

I vaguely remember learning similar fact years ago, but it was phrased something like "There are as many floats in [0, 1] range as in [1, infty]". Leaving aside where exactly "midpoint" of floats lies (either in Ruby's implementation or other languages'): what would be implications of this for writing code dealing with floats? Can I shoot myself in the foot somehow with low precision if numbers I'm using are "too close" to infinity?

pantulis 6y ago
I guess the fact would be something like: "There are as many floats in [0,1] as integers in [1, infty]". Probably you will have problems mixing operations between numbers too close to infinity with numbers too close to zero. But if you are working with floats you are probably going to shoot yourself in the foot anyway unless you understand IEEE floating point arithmetic, precisions and so on, so better stick to Decimals and BigDecimals if you can.
TeMPOraL 6y ago
Yes, you can. A good way to think about it is that a floating point number has a fixed amount of significant digits. A 32-bit float has between 6 to 9 significant digits[0]; a 64-bit float ("a double") can represent just double that - i.e. ~16 significant figures. This means you can represent 12345678 and 0.12345678 as 32-bit floats just fine, but if you try to add the two, you'll just get 12345678. But there's also a more subtle case, in which you try to add 1234.5678 to 0.12345678, and lose half of the significant digits from the smaller number. These types of errors are common in calculations dealing with large differences in orders of magnitude, and a part of the common wisdom to not use floats for anything involving money. Another common occurrence of precision errors happens when modelling large 2D or 3D environments in video games. The simplest approach starts with a global coordinate system, in which calculations take place. Since the number of significant digits remains fixed, the further you are from origin, the less precision you have available for your unit of measure of position - meaning the same calculations happening far away from (0, 0, 0) will have more errors than those happening close. A somewhat well-known example of that was[1] Kerbal Space Program - a game that models realistic spaceflight within a fictional solar system. As you ventured further out and visited the most distant celestial bodies, you'd notice your ships would spontaneously shake apart and/or explode, as if physics stopped working. Affectionately called a "Kraken attack", it was due to floating point precision errors - that far out, in the global coordinate system, the difference between two closest representable positions became comparable with the size of parts from which the ships were built. With that big of a delta, physical calculations would round some forces down to zero, and massively magnify others, as simulated objects could suddenly only move in increments comparable with their own size. -- [0] - The limitation is in binary, which doesn't map cleanly to base 10. [1] - I think they got around to mitigating this eventually. One way you can approach such mitigation is by dividing your space into cells (and possibly subcells), each with its own coordinate system, and do as many calculations as you can within these cells, so that everything is close to the origin of their local coordinate system. Then, for (hopefully few) remaining calculations that involve multiple distant cells, you can use larger-precision representations, which are much more computationally expensive.
- navaati 6y ago
  > [1] - I think they got around to mitigating this eventually. Yes, as far as I know nowadays KSP works by inversing the point of reference, that is, your spaceship does not move and sits at (0,0,0) at all times, it's the whole solar system that moves around you. Of course, functionnally it does not make a difference, and it solves the problem ;) EDIT: ah, interesting that you have another theory, now I wonder :)
  - TeMPOraL 6y ago
    I don't know how they solved it in KSP, I was just presenting a general approach for when you need to have things happen in multiple places around a large simulated universe. I can imagine KSP going with what you described, because in that game, things you aren't actively controlling and that aren't in "physics range" (2.5 kilometers IIRC) are either treated as stationary or on-rails (i.e. they're in orbit, and their positions are computed with a closed-form equation); anything that's neither stationary nor in orbit will be deleted once it leaves the physics range. This way, there is no physics simulation to be done on things you don't control or aren't withing few kilometers of, so centering the coordinate system at your active vessel would be a very good approach.
simias 6y ago
What you describe is a common problem for video games, because if the engine uses "absolute" floats for everything in the game you effectively lose precision as you move away from the origin. If your maps get very large it gets very noticeable. I remember for instance in No Man's Sky when you traveled very far in a system you'd start to see the various animations become very jerky because the precision would become too low to represent the coordinates of the intermediary steps correctly. See a random video showcasing the issue here: https://youtu.be/D2XX2ZnRk8M?t=197 https://youtu.be/D2XX2ZnRk8M?t=197 You can see all the moving elements jerking around as they "snap" to the next available float value. A common fix for this issue is to use two sets of coordinates: you can for instance represent your world as a grid with fixed-size cells, then you translate all your models into the local cell before computing anything, this way you always have good enough precision since you effectively limit the amplitude of your floats. Of course I handwave many complications here, such as what happens when you're at the edge of a cell for instance, but it's workable.
- kqr 6y ago
  > A common fix for this issue is to use two sets of coordinates: you can for instance represent your world as a grid with fixed-size cells, then you translate all your models into the local cell before computing anything, this way you always have good enough precision since you effectively limit the amplitude of your floats. Isn't this in practise creating a double-precision float by adding a second "significant figure" in a "base float" system?
  - ender341341 6y ago
    Kind of but you get significantly better precision than if you just use a normal double float type because doubles only push the issues further out. Especially in games you're likely already breaking the world into cells for streaming chunks into/out of memory anyways.
    - shawnz 6y ago
      Isn't this also just pushing the problem out? Once you get far enough out, you wouldn't be able to transition between cells because of the rounding error of the cell boundary being bigger than the cell size. You can't represent infinite precision with finite bits so you must run into an issue eventually.
kazinator 6y ago
> Can I shoot myself in the foot somehow with low precision if numbers I'm using are "too close" to infinity? The "number of sig figs" precision will be the same for large values far away from zero, as for values close to zero. E.g. IEEE 64 bit reliably gives you about 15 decimal digits of precision. Any decimal number with 15 digits of precision that is in the range of the IEEE 64 bit double can be converted to that type, and then back to decimal, such that all those digits of precision are recovered. You can shoot yourself in the foot if you rely on floating-point values being able to exactly represent integers, but the values go beyond the range where that is possible. Beyond a certain range, all consecutive integers are no longer representable; some integers get approximated by nearby integers.

by rednum 6y ago

pantulis 6y ago

I guess the fact would be something like: "There are as many floats in [0,1] as integers in [1, infty]". Probably you will have problems mixing operations between numbers too close to infinity with numbers too close to zero. But if you are working with floats you are probably going to shoot yourself in the foot anyway unless you understand IEEE floating point arithmetic, precisions and so on, so better stick to Decimals and BigDecimals if you can.

TeMPOraL 6y ago

Yes, you can. A good way to think about it is that a floating point number has a fixed amount of significant digits. A 32-bit float has between 6 to 9 significant digits[0]; a 64-bit float ("a double") can represent just double that - i.e. ~16 significant figures. This means you can represent 12345678 and 0.12345678 as 32-bit floats just fine, but if you try to add the two, you'll just get 12345678. But there's also a more subtle case, in which you try to add 1234.5678 to 0.12345678, and lose half of the significant digits from the smaller number. These types of errors are common in calculations dealing with large differences in orders of magnitude, and a part of the common wisdom to not use floats for anything involving money. Another common occurrence of precision errors happens when modelling large 2D or 3D environments in video games. The simplest approach starts with a global coordinate system, in which calculations take place. Since the number of significant digits remains fixed, the further you are from origin, the less precision you have available for your unit of measure of position - meaning the same calculations happening far away from (0, 0, 0) will have more errors than those happening close. A somewhat well-known example of that was[1] Kerbal Space Program - a game that models realistic spaceflight within a fictional solar system. As you ventured further out and visited the most distant celestial bodies, you'd notice your ships would spontaneously shake apart and/or explode, as if physics stopped working. Affectionately called a "Kraken attack", it was due to floating point precision errors - that far out, in the global coordinate system, the difference between two closest representable positions became comparable with the size of parts from which the ships were built. With that big of a delta, physical calculations would round some forces down to zero, and massively magnify others, as simulated objects could suddenly only move in increments comparable with their own size. -- [0] - The limitation is in binary, which doesn't map cleanly to base 10. [1] - I think they got around to mitigating this eventually. One way you can approach such mitigation is by dividing your space into cells (and possibly subcells), each with its own coordinate system, and do as many calculations as you can within these cells, so that everything is close to the origin of their local coordinate system. Then, for (hopefully few) remaining calculations that involve multiple distant cells, you can use larger-precision representations, which are much more computationally expensive.

navaati 6y ago
> [1] - I think they got around to mitigating this eventually. Yes, as far as I know nowadays KSP works by inversing the point of reference, that is, your spaceship does not move and sits at (0,0,0) at all times, it's the whole solar system that moves around you. Of course, functionnally it does not make a difference, and it solves the problem ;) EDIT: ah, interesting that you have another theory, now I wonder :)
- TeMPOraL 6y ago
  I don't know how they solved it in KSP, I was just presenting a general approach for when you need to have things happen in multiple places around a large simulated universe. I can imagine KSP going with what you described, because in that game, things you aren't actively controlling and that aren't in "physics range" (2.5 kilometers IIRC) are either treated as stationary or on-rails (i.e. they're in orbit, and their positions are computed with a closed-form equation); anything that's neither stationary nor in orbit will be deleted once it leaves the physics range. This way, there is no physics simulation to be done on things you don't control or aren't withing few kilometers of, so centering the coordinate system at your active vessel would be a very good approach.

simias 6y ago

What you describe is a common problem for video games, because if the engine uses "absolute" floats for everything in the game you effectively lose precision as you move away from the origin. If your maps get very large it gets very noticeable. I remember for instance in No Man's Sky when you traveled very far in a system you'd start to see the various animations become very jerky because the precision would become too low to represent the coordinates of the intermediary steps correctly. See a random video showcasing the issue here: https://youtu.be/D2XX2ZnRk8M?t=197 https://youtu.be/D2XX2ZnRk8M?t=197 You can see all the moving elements jerking around as they "snap" to the next available float value. A common fix for this issue is to use two sets of coordinates: you can for instance represent your world as a grid with fixed-size cells, then you translate all your models into the local cell before computing anything, this way you always have good enough precision since you effectively limit the amplitude of your floats. Of course I handwave many complications here, such as what happens when you're at the edge of a cell for instance, but it's workable.

kqr 6y ago
> A common fix for this issue is to use two sets of coordinates: you can for instance represent your world as a grid with fixed-size cells, then you translate all your models into the local cell before computing anything, this way you always have good enough precision since you effectively limit the amplitude of your floats. Isn't this in practise creating a double-precision float by adding a second "significant figure" in a "base float" system?
- ender341341 6y ago
  Kind of but you get significantly better precision than if you just use a normal double float type because doubles only push the issues further out. Especially in games you're likely already breaking the world into cells for streaming chunks into/out of memory anyways.
  - shawnz 6y ago
    Isn't this also just pushing the problem out? Once you get far enough out, you wouldn't be able to transition between cells because of the rounding error of the cell boundary being bigger than the cell size. You can't represent infinite precision with finite bits so you must run into an issue eventually.

kazinator 6y ago

> Can I shoot myself in the foot somehow with low precision if numbers I'm using are "too close" to infinity? The "number of sig figs" precision will be the same for large values far away from zero, as for values close to zero. E.g. IEEE 64 bit reliably gives you about 15 decimal digits of precision. Any decimal number with 15 digits of precision that is in the range of the IEEE 64 bit double can be converted to that type, and then back to decimal, such that all those digits of precision are recovered. You can shoot yourself in the foot if you rely on floating-point values being able to exactly represent integers, but the values go beyond the range where that is possible. Beyond a certain range, all consecutive integers are no longer representable; some integers get approximated by nearby integers.