3 ms·
Consumer GPUs can be used only for computations with single-precision floating numbers or with lower precision. Their native double-precision computation speed
by adrian_b 4y ago
Consumer GPUs can be used only for computations with single-precision floating numbers or with lower precision.
Their native double-precision computation speed is typically lower than that of the CPUs with which they are paired.
They are great for machine learning or graphics, but bad for CAD or physical simulations.
As a workaround for the non-support of double precision, it is possible to rewrite many algorithms to use a mixed precision or to implement higher precision operations with double-single, triple-single or quadruple-single numbers.
However, this greatly increases the complexity of porting most computations to a consumer GPU, so it is not normally worthwhile when compared to the alternative of just using a fast CPU.
More than a decade ago, NVIDIA has produced a huge amount of propaganda that presented CUDA as the future of all computing, which will soon replace all CPUs for any task requiring high throughput.
Nevertheless, a few years later NVIDIA has killed all that future lavishly presented by them, with their pricing and market segmentation strategies.
Datacenter GPUs have 2 to 4 times higher throughput than CPUs at equal power consumption, so they are the best choice for supercomputers or large datacenters.
On the other hand, the throughput per dollar of the datacenter GPUs is now lower than that of the CPUs, even if 5 years ago the relationship was reversed. Meanwhile, the price of CPU cores has been lowered by competition, while for datacenter GPUs the prices have been raised, by both NVIDIA and AMD.
So the datacenter GPUs are completely useless for small companies or individuals, for whom the acquisition price may be more important than the electricity expenses.
The consumer GPUs require too much effort to be used in any application for which single-precision is not good enough, so they remain great for the applications suited for them, but not good for many others.
What is annoying is that the crippling of the consumer GPUs started by NVIDIA cannot be justified as a cost-saving measure, it is a just a pure market segmentation method.
Reducing the throughput of double-precision from 1/2 of the single-precision throughput, to 1/8 of the single-precision throughput, would have already provided most of the area reduction in a GPU that can be obtained by removing completely the support for double-precision. A DP throughput of 1/8 would have still been competitive with CPUs and usable for some problems.
Reducing the DP throughput much more than that, to 1/32 of the SP throughput, like NVIDIA did, was just intended to make the DP useless, except for algorithm testing purposes, in order to push the NVIDIA customers towards the overpriced datacenter products.
AMD has also followed the NVIDIA strategy, replacing GCN, which had provided the last consumer GPUs with good DP capabilities, with the split by market RDNA and CDNA GPUs.
- dahart 4y agoI can’t think of a chip maker that doesn’t segment their products, I think you’re giving Nvidia too much credit to make it sound like they invented the practice. Same goes for suggesting AMD isn’t thinking for themselves, right? If doubles were a competitive advantage, certainly they’d jump on it, no? It seems like they do have a bit higher average double-to-single ratio. But maybe the real problem is that double precision just isn’t in that much demand, especially for GPUs with less memory? Maybe the market is doing the segmenting and not the manufacturers? BTW, didn’t prices come down in the last gen? Consumer did, and I don’t know about data center products, but you certainly can get newer GPUs with more mem and more single & double perf for less money than what @majke quoted for GV100. If you want a consumer GPU with DP perf, maybe the Titan V is what you want? Here’s a naive question: what is the expected ratio of double to single flops if you build the double precision hardware as much as possible out of single precision math units? So I mean given a certain single precision perf level, what double precision perf level do you get if you devote the absolute minimum amount of chip area to DP specific logic? I’m no hardware architecture expert, so maybe the question is weird, but I would speculate wildly that multipliers might be the limiting factor and that it would take 4 single precision multiplies to get 1 double precision result. So I’m wondering if a 1/4 ratio is the most ‘natural’ in some sense? Adds alone I could see being 1/2, but multiplies and other math, especially transcendentals seem like they’d bring down the average.
- adrian_b 4y agoIf you design in a CPU or GPU with only double-precision arithmetic units, they can also be used for single-precision operations, providing double throughput for them. This is what is done both in CPUs and in GPUs like the AMD CDNA GPUs and the top NVIDIA datacenter GPUs, i.e. A100 or V100. However, a double-precision multiplier has 4 times the area of a single-precision multiplier, not 2 times. So replacing a part of the DP ALUs with SP ALUs, will save area, due to the smaller multipliers. Exactly how much area is saved depends on what percentage of the total GPU area is occupied by the multipliers. Let's suppose that the multipliers occupy 32% of the area, even if this number must be greatly exaggerated, as besides the multipliers there are a many other circuits in an ALU, and there are many control circuits and a large number of registers associated with each ALU. Therefore the real percentage of area occupied by multipliers must be many times less than this. Under this assumption, removing all support for DP would save about 16% of the area, reducing the DP throughput to 1/4 would save about 8%, to 1/8 about 12%, to 1/16 about 14% and to 1/32 about 15%. Like I have said, the real savings must be much less, but approximately in the same ratios. I agree that everybody does market segmentation, but that does not make it less annoying, especially because I always fall in the smaller segment of the customers who are forced to pay excessive prices for features which in fact would be useful for much more customers, but those are not aware about what they should demand when buying something. In the case of NVIDIA, their market segmentation strategy is much more annoying than in the case of other companies, because I remember very clearly a large number of NVDIA presentations from approximately 2006 to 2010, in which they described a very different company future, about how they will provide great computing power for the masses. Then they have realized that there are more efficient ways to make money, so, starting with their Fermi GPUs, they have implemented rigid market segmentation policies. While I understand their reasons, I strongly despise company liars.