6 ms·

If you are doing signal processing, ML, simulations, gaming or the like, you can know going in that you will need to use primitive types from a performance stan

If you are doing signal processing, ML, simulations, gaming or the like, you can know going in that you will need to use primitive types from a performance standpoint which sometimes means fixed point but often means floating point.

aidenn0 3y ago
Many simulations and gaming are both better off using fixed point. The range available with a 64-bit integer is staggering and precision varying with distance from an arbitrary origin is a bug, not a feature for those cases.
- gyrovagueGeist 3y ago
  This is absolutely not the case for most physical simulations where the significant digits are more important than fixed decimal accuracy.
  - aidenn0 3y ago
    Consider a simple newtonian physics simulation. What you say is probably true for both force and acceleration. It may be true for velocity, but is almost certainly not true for position. Consider an object with a very small velocity traveling away from the origin with no forces acting upon it. With floating-point arithmetic, it will eventually stop at some distance away from the origin. In general objects with small velocity will only move if they are close to the origin.
    - gyrovagueGeist 3y ago
      Sure, but most PIC methods are centered at their element or local domain. (You can likely even do the same with ewald/nbody). I definitely agree with you though if you need high accuracy on position when everything is on the same huge grid. A simulation that computes bulk properties by simulating particles directly is very rare. At least for me. Observable like temperature, electric/magnetic field strength, stress, velocity, etc are much more common outputs. These often all vary over many order of magnitude across a domain. Working with continuum mechanics, performing large inverted solves on matrices in fixed precision, or heck, even an FFT sounds terrifying to guard against overflow.
- thebooktocome 3y ago
  When exp(45) overflows that 64-bit integer doesn’t seem so impressive.
- Dylan16807 3y ago
  The range available with 54 bit fixed point is also staggering. And almost any code that would work with 54 bit fixed point works even better with 64 bit floating point, and the floating point version is much easier to code. So sure, floating point isn't optional when you start off assuming the same number of bits. But if you treat it as a small overhead to make code more robust to large numbers, easier to code, and often faster to run, then it looks a lot better.
  - aidenn0 3y ago
    Yes, the respective sins of both floating- and fixed-point math are significantly reduced when you have more bits (for a demo of this try using both at 16 bits; you will have to code very carefully and be very aware of the faults of each). For many situations though, I find the graceful degradation of floats to cause more subtle bugs, which can be a problem.
    - Dylan16807 3y ago
      I bet it's much easier to put range warnings into floating point than to switch to fixed point, and that the effect on bugs would be similar.
      - aidenn0 3y ago
        Range warnings are going to be calculation specific. Adding a non-zero fixed point number "y" N times to a fixed-point number "x" will result in either an overflow or x+N*y. If integer overflows trap (curse you Intel, C89), then repeated additions (important in many simulations) will either work as expected or crash. Floating point operations are (for practical purposes) highly privileged because of extremely mature hardware implementations. Hardware implementations that make other forms of calculations more tractable are possible (and have existed in the past) and should be considered when evaluating FPs fitness for purpose, otherwise we will be stuck in the IEEE local maximum forever.
        Dylan16807 3y ago
- constantcrying 3y ago
  Hilarious. Fixed point is awful it was thrown out the moment people could use floats on their computers. Fixed point numbers cause an enormous programmer overhead and do not fix the problems.
  - Karellen 3y ago
    > Fixed point numbers cause an enormous programmer overhead and do not fix the problems. How so? Is that because of inherent problems with the outcomes of fixed-point arithmetic, or are they just clumsy to use and no-one's written a decent library that makes dealing with fixed-point numbers straightforward?
    - magicalhippo 3y ago
      There are plenty of libraries which makes dealing with fixed-point relatively easy. I think GP is alluding to issues you get when you have a mix of scales. Like floating-point, fixed-point will have a limited precision range around the origin. In contrast to floating-point, the range for fixed-point is smaller but evenly spaced. Say you have 32:32 fixed-point. You can then represent numbers in multiple of ~0.2e-9. So if you need to calculate distances in nanometers, perhaps due to a short timestep in a simulation, you have hardly any precision to work with. The obvious way around this is to pull out a scale factor, say 1e-9 since you know you'll be in the nano-range. Now the numbers you're working with are back on the order of 1 with lots of precision, however you need to apply the scale factor when you actually want to use the number, and now you have to be careful not to overflow when multiplying two or more scaled numbers. This is part of the programmer overhead GP alluded to.
      - hurrrrrrrrrrrrr 3y ago
        > The obvious way around this is to pull out a scale factor, say 1e-9 since you know you'll be in the nano-range. Now the numbers you're working with are back on the order of 1 with lots of precision, however you need to apply the scale factor when you actually want to use the number, and now you have to be careful not to overflow when multiplying two or more scaled numbers. Do that rescaling automatically and have a system that just keeps track of the scale and you've just implemented floats. (:
lmm 3y ago
Occasionally yes. But the niche where you need something other than decimal bignums is narrow, and the niche where you need that but don't need to get into the weeds of custom float formats etc. is narrower still. IEEE 754 shouldn't be front-and-center in general-purpose programming languages.
- galangalalgol 3y ago
  I've never even heard of people using custom float formats outside of an fpga. And bignum seems useful for things that are not cpu-bound. If you need simd, or vectorization, including gpu acceleration, you use the format the hardware and compiler/optimizer support to meet your throughput needs. And on x86_64 aarch64 or spirv, thats a ieee floating point for better or worse.
  - lmm 3y ago
    > I've never even heard of people using custom float formats outside of an fpga. It's normal and expected for ML these days (not completely custom, but smaller than the IEEE floats). And I think for gaming graphics too. > If you need simd, or vectorization, including gpu acceleration, you use the format the hardware and compiler/optimizer support to meet your throughput needs. And on x86_64 aarch64 or spirv, thats a ieee floating point for better or worse. If you need to do it on the CPU, sure. (Even then, you probably don't want to use full IEEE754, you will likely disable subnormals for performance). If you're using the GPU then you'll likely be using something else. The niche that IEEE754 fits is pretty narrow - it works for non-HPC physics simulations because it was designed for that, but that's about it.

aidenn0 3y ago

Many simulations and gaming are both better off using fixed point. The range available with a 64-bit integer is staggering and precision varying with distance from an arbitrary origin is a bug, not a feature for those cases.

gyrovagueGeist 3y ago
This is absolutely not the case for most physical simulations where the significant digits are more important than fixed decimal accuracy.
- aidenn0 3y ago
  Consider a simple newtonian physics simulation. What you say is probably true for both force and acceleration. It may be true for velocity, but is almost certainly not true for position. Consider an object with a very small velocity traveling away from the origin with no forces acting upon it. With floating-point arithmetic, it will eventually stop at some distance away from the origin. In general objects with small velocity will only move if they are close to the origin.
  - gyrovagueGeist 3y ago
    Sure, but most PIC methods are centered at their element or local domain. (You can likely even do the same with ewald/nbody). I definitely agree with you though if you need high accuracy on position when everything is on the same huge grid. A simulation that computes bulk properties by simulating particles directly is very rare. At least for me. Observable like temperature, electric/magnetic field strength, stress, velocity, etc are much more common outputs. These often all vary over many order of magnitude across a domain. Working with continuum mechanics, performing large inverted solves on matrices in fixed precision, or heck, even an FFT sounds terrifying to guard against overflow.
thebooktocome 3y ago
When exp(45) overflows that 64-bit integer doesn’t seem so impressive.
Dylan16807 3y ago
The range available with 54 bit fixed point is also staggering. And almost any code that would work with 54 bit fixed point works even better with 64 bit floating point, and the floating point version is much easier to code. So sure, floating point isn't optional when you start off assuming the same number of bits. But if you treat it as a small overhead to make code more robust to large numbers, easier to code, and often faster to run, then it looks a lot better.
- aidenn0 3y ago
  Yes, the respective sins of both floating- and fixed-point math are significantly reduced when you have more bits (for a demo of this try using both at 16 bits; you will have to code very carefully and be very aware of the faults of each). For many situations though, I find the graceful degradation of floats to cause more subtle bugs, which can be a problem.
  - Dylan16807 3y ago
    I bet it's much easier to put range warnings into floating point than to switch to fixed point, and that the effect on bugs would be similar.
    - aidenn0 3y ago
      Range warnings are going to be calculation specific. Adding a non-zero fixed point number "y" N times to a fixed-point number "x" will result in either an overflow or x+N*y. If integer overflows trap (curse you Intel, C89), then repeated additions (important in many simulations) will either work as expected or crash. Floating point operations are (for practical purposes) highly privileged because of extremely mature hardware implementations. Hardware implementations that make other forms of calculations more tractable are possible (and have existed in the past) and should be considered when evaluating FPs fitness for purpose, otherwise we will be stuck in the IEEE local maximum forever.
      - Dylan16807 3y ago
constantcrying 3y ago
Hilarious. Fixed point is awful it was thrown out the moment people could use floats on their computers. Fixed point numbers cause an enormous programmer overhead and do not fix the problems.
- Karellen 3y ago
  > Fixed point numbers cause an enormous programmer overhead and do not fix the problems. How so? Is that because of inherent problems with the outcomes of fixed-point arithmetic, or are they just clumsy to use and no-one's written a decent library that makes dealing with fixed-point numbers straightforward?
  - magicalhippo 3y ago
    There are plenty of libraries which makes dealing with fixed-point relatively easy. I think GP is alluding to issues you get when you have a mix of scales. Like floating-point, fixed-point will have a limited precision range around the origin. In contrast to floating-point, the range for fixed-point is smaller but evenly spaced. Say you have 32:32 fixed-point. You can then represent numbers in multiple of ~0.2e-9. So if you need to calculate distances in nanometers, perhaps due to a short timestep in a simulation, you have hardly any precision to work with. The obvious way around this is to pull out a scale factor, say 1e-9 since you know you'll be in the nano-range. Now the numbers you're working with are back on the order of 1 with lots of precision, however you need to apply the scale factor when you actually want to use the number, and now you have to be careful not to overflow when multiplying two or more scaled numbers. This is part of the programmer overhead GP alluded to.
    - hurrrrrrrrrrrrr 3y ago
      > The obvious way around this is to pull out a scale factor, say 1e-9 since you know you'll be in the nano-range. Now the numbers you're working with are back on the order of 1 with lots of precision, however you need to apply the scale factor when you actually want to use the number, and now you have to be careful not to overflow when multiplying two or more scaled numbers. Do that rescaling automatically and have a system that just keeps track of the scale and you've just implemented floats. (:

lmm 3y ago

Occasionally yes. But the niche where you need something other than decimal bignums is narrow, and the niche where you need that but don't need to get into the weeds of custom float formats etc. is narrower still. IEEE 754 shouldn't be front-and-center in general-purpose programming languages.

galangalalgol 3y ago
I've never even heard of people using custom float formats outside of an fpga. And bignum seems useful for things that are not cpu-bound. If you need simd, or vectorization, including gpu acceleration, you use the format the hardware and compiler/optimizer support to meet your throughput needs. And on x86_64 aarch64 or spirv, thats a ieee floating point for better or worse.
- lmm 3y ago
  > I've never even heard of people using custom float formats outside of an fpga. It's normal and expected for ML these days (not completely custom, but smaller than the IEEE floats). And I think for gaming graphics too. > If you need simd, or vectorization, including gpu acceleration, you use the format the hardware and compiler/optimizer support to meet your throughput needs. And on x86_64 aarch64 or spirv, thats a ieee floating point for better or worse. If you need to do it on the CPU, sure. (Even then, you probably don't want to use full IEEE754, you will likely disable subnormals for performance). If you're using the GPU then you'll likely be using something else. The niche that IEEE754 fits is pretty narrow - it works for non-HPC physics simulations because it was designed for that, but that's about it.