3 ms·

> My primary task is FP64 throughput limited, and thus I am stuck on Titan V as it is ~6 times faster than the 4090 and 5 times faster than the 5090 is the only

> My primary task is FP64 throughput limited, and thus I am stuck on Titan V as it is ~6 times faster than the 4090 and 5 times faster than the 5090 is the only reason I don't have newer GPUS. interesting. Very interesting. Why fp64 as opposed to BF16? different sort of model? i don't even know where to find fp64 models (not that i've looked). also Bartowski may be on huggingface but they're also part of the LM Studio group, and frequently chat on that discord. actually, at least 3 of the main model converter / quant people are on that discord. I haven't got two 24GB cards, yet, but maybe soon, with the way people are hogging the 5000 series. edit: i realize that they're increasing the marketing FLOPS by halving the resolution, the current gen stuff is all "fast" at FP16 (or BF16 - brainfloat 16 bit). So when nvidia finishes and releases a card with double the FLOPS at 8 bit, will that card be 8 times slower at fp64?

nyrikki 2y ago
My primary task isn't ML, and 64bit is needed for numerical stability. For the Titan V, the F64 was 1/2 of F32, it was the only and last consumer generation to have that. For Titan RTX and newer NVIDIA cards, the ratio from F32 to F64 is typically 1/64th of FP32. So the Titan RTX, with 16 FP32 TFlop/s drops to 0.5 FP64 TFlops/s While the Titan V, starting at 15 FP32 TFlops/s still has 7.5 FP64 TFlops/s The 5090 TI has 104.9 FP16/32 TFlops/s, but only 1.64 FP64 TFlops/s. Basically Nvidia decided most people didn't need FP64, and chose to improve quantized performance instead. If you can run on a GPU, that Titan V has more 64bit Flops than even an AMD Ryzen Threadripper PRO 7995WX.
- genewitch 2y ago
  while researching this i discovered another fast fp64 card is the R9 280x by amd/ati. although the memory is weak, only 3GB! But i suppose if you need the numerical accuracy, there's always that, and those cards are like $40 (in the us, on ebay, sold listings), compared to $400 for the titan. if you need 4x the ram though i guess you're stuck paying 10x the price!

nyrikki 2y ago

My primary task isn't ML, and 64bit is needed for numerical stability. For the Titan V, the F64 was 1/2 of F32, it was the only and last consumer generation to have that. For Titan RTX and newer NVIDIA cards, the ratio from F32 to F64 is typically 1/64th of FP32. So the Titan RTX, with 16 FP32 TFlop/s drops to 0.5 FP64 TFlops/s While the Titan V, starting at 15 FP32 TFlops/s still has 7.5 FP64 TFlops/s The 5090 TI has 104.9 FP16/32 TFlops/s, but only 1.64 FP64 TFlops/s. Basically Nvidia decided most people didn't need FP64, and chose to improve quantized performance instead. If you can run on a GPU, that Titan V has more 64bit Flops than even an AMD Ryzen Threadripper PRO 7995WX.

genewitch 2y ago
while researching this i discovered another fast fp64 card is the R9 280x by amd/ati. although the memory is weak, only 3GB! But i suppose if you need the numerical accuracy, there's always that, and those cards are like $40 (in the us, on ebay, sold listings), compared to $400 for the titan. if you need 4x the ram though i guess you're stuck paying 10x the price!