11 ms·

Oof. Double-precision is only 2.5x better, which is less impressive than 20x for float. I still haven't found anything more cost-effective for double precision

by krapht 6y ago

Oof. Double-precision is only 2.5x better, which is less impressive than 20x for float. I still haven't found anything more cost-effective for double precision data processing (cost + dev time) than a rack full of used Xeons...

frankchn 6y ago
The double-precision number probably best represents the generational improvement. The 20x 32-bit floating point improvement is probably achieved by comparing doing full FP32 calculations on the previous generation vs doing TF32 calculations on Ampere. This would not be an apple-to-apple comparison as the TF32 result is less precise. That said, it is probably not terribly important for deep learning at least, given the success of BF16.
- frankchn 6y ago
  Actually it looks like the double precision number in general GPU usage only went up 25% (Volta did 7.8 TFLOPS). To get the 2.5x number, you need to use FP64 in conjunction with TensorCores, which then gets you 19.5 TFLOPS. Considering how big the die is (826mm^2 @ TSMC 7nm) and how many transistors there are, they really must have beefed up the TensorCores much more than the general compute units.
- randyrand 6y ago
  Wow. TF32 is only 19 bits. Thats some dubious marketing.
raverbashing 6y ago
I'm curious: why exactly do you need double precision digits? Not dismissing, just wondering what kind of application needs it.
- krapht 6y ago
  Physics simulations. There's a rule of thumb that to get an n-bit accurate result after a long chain of calculations, intermediate results should be stored with 2n bits. Often using the full dynamic range of a float is necessary because the magnitude of different physical phenomena varies so wildly. I guess people do store intermediate results in floats in order to take advantage of GPU acceleration. However, once you do that, you have to be careful in your programming and pay a lot more attention to underflow, overflow, and numerical accuracy. People who write scientific software usually also aren't experts in numerical analysis. Even if the code you write reliably works with floats, the library you use might not be. It's just a huge pain to make sure everything is accurate.
  - barbegal 6y ago
    I've always argued that if you are getting close to having to worry about underflow, overflow etc. then you have an ill-conditioned problem and just increasing the size of your intermediate results won't help you a huge amount because you need more precision from your inputs. There are very few fields where you need more than the 7 decimal digits afforded by floats. Maybe the only exceptions are in astrophysics.
    - auggierose 6y ago
      That argument would be wrong. Double can be a big help, for example when performing solid modelling operations on triangle meshes. Best would be actually exact arithmetic, but it being often too slow, Double is often good enough, while Float isn't.
    - evanb 6y ago
      Lattice QCD, especially near the physical point, has poorly-conditioned matrices that one wishes to solve Mx=b for. The state-of-the-art is that the [sparse] matrices can be as large as 4×3×(128^3 × 192) ~ 5e9 on a side. It's not so rare to find legitimately difficult problems in hard sciences.
    - jabl 6y ago
      Double precision has been the mainstay of scientific computing for decades, and no, it's not because all those scientists are dumb.
    - godelski 6y ago
      Pretty much any ODE solver you want double. All of CFD (fluid sims) and FEM (mechanics) use double. It can also be a big help in graphics. Basically, if you're doing fine meshing, doubles will help you. There's a balance between your meshing size as well. Your life is a whole lot easier and you'll get better answers if you just use double. Ask literally any computational {physicist,scientist,engineer}. We've all tried without double. Trust me, I'd love to have lower memory constraints.
    - fluffything 6y ago
- krastanov 6y ago
  Not OP, but like many other people I use linear algebra accelerators for scientific computing (in my case, simulating and controlling quantum-mechanical systems). We do need the precision if we want the solutions of our ODEs to converge.
QuixoticQuibit 6y ago
This thing is mainly targeted at AI workloads I assume, so double precision isn’t so interesting.
fourier_mode 6y ago
Titan V100's are pretty good for double precision, needs almost same operation intensity as Xeons to fully utilize them.

by krapht 6y ago

frankchn 6y ago

The double-precision number probably best represents the generational improvement. The 20x 32-bit floating point improvement is probably achieved by comparing doing full FP32 calculations on the previous generation vs doing TF32 calculations on Ampere. This would not be an apple-to-apple comparison as the TF32 result is less precise. That said, it is probably not terribly important for deep learning at least, given the success of BF16.

frankchn 6y ago
Actually it looks like the double precision number in general GPU usage only went up 25% (Volta did 7.8 TFLOPS). To get the 2.5x number, you need to use FP64 in conjunction with TensorCores, which then gets you 19.5 TFLOPS. Considering how big the die is (826mm^2 @ TSMC 7nm) and how many transistors there are, they really must have beefed up the TensorCores much more than the general compute units.
randyrand 6y ago
Wow. TF32 is only 19 bits. Thats some dubious marketing.

raverbashing 6y ago

I'm curious: why exactly do you need double precision digits? Not dismissing, just wondering what kind of application needs it.

krapht 6y ago
Physics simulations. There's a rule of thumb that to get an n-bit accurate result after a long chain of calculations, intermediate results should be stored with 2n bits. Often using the full dynamic range of a float is necessary because the magnitude of different physical phenomena varies so wildly. I guess people do store intermediate results in floats in order to take advantage of GPU acceleration. However, once you do that, you have to be careful in your programming and pay a lot more attention to underflow, overflow, and numerical accuracy. People who write scientific software usually also aren't experts in numerical analysis. Even if the code you write reliably works with floats, the library you use might not be. It's just a huge pain to make sure everything is accurate.
- barbegal 6y ago
  I've always argued that if you are getting close to having to worry about underflow, overflow etc. then you have an ill-conditioned problem and just increasing the size of your intermediate results won't help you a huge amount because you need more precision from your inputs. There are very few fields where you need more than the 7 decimal digits afforded by floats. Maybe the only exceptions are in astrophysics.
  - auggierose 6y ago
    That argument would be wrong. Double can be a big help, for example when performing solid modelling operations on triangle meshes. Best would be actually exact arithmetic, but it being often too slow, Double is often good enough, while Float isn't.
  - evanb 6y ago
    Lattice QCD, especially near the physical point, has poorly-conditioned matrices that one wishes to solve Mx=b for. The state-of-the-art is that the [sparse] matrices can be as large as 4×3×(128^3 × 192) ~ 5e9 on a side. It's not so rare to find legitimately difficult problems in hard sciences.
  - jabl 6y ago
    Double precision has been the mainstay of scientific computing for decades, and no, it's not because all those scientists are dumb.
  - godelski 6y ago
    Pretty much any ODE solver you want double. All of CFD (fluid sims) and FEM (mechanics) use double. It can also be a big help in graphics. Basically, if you're doing fine meshing, doubles will help you. There's a balance between your meshing size as well. Your life is a whole lot easier and you'll get better answers if you just use double. Ask literally any computational {physicist,scientist,engineer}. We've all tried without double. Trust me, I'd love to have lower memory constraints.
  - fluffything 6y ago
krastanov 6y ago
Not OP, but like many other people I use linear algebra accelerators for scientific computing (in my case, simulating and controlling quantum-mechanical systems). We do need the precision if we want the solutions of our ODEs to converge.

QuixoticQuibit 6y ago

This thing is mainly targeted at AI workloads I assume, so double precision isn’t so interesting.

fourier_mode 6y ago

Titan V100's are pretty good for double precision, needs almost same operation intensity as Xeons to fully utilize them.