6 ms·

One thing I don't fully understand is why these accelerators all have relatively small memory sizes? Is it purely because of cost? The most expensive Tesla acc

One thing I don't fully understand is why these accelerators all have relatively small memory sizes? Is it purely because of cost? The most expensive Tesla accelerator I see in this list has 16 GB, which sounds like a lot, but it's 'only' twice as much as the consumer-level GPU I have in my PC at home, and it seems like a pretty paltry amount for large-scale simulations. I'm thinking about volumetric modeling applications, for example, where storing a dense voxel grid as a volume texture would easily run over 30 GB even for quite modest grid size/resolution (e.g. 2K x 2K x 2K grid). It's exactly these kinds of things where a GPU accelerator could shine, without the need to implement spare sampling solutions or accelerator data structures that would be complex and severely impact performance on GPU architectures.

ThrowawayR2 9y ago
If I understand it correctly, GPUs use specialized high-bandwidth memory that is more expensive instead of commodity memory. See https://en.wikipedia.org/wiki/GDDR5_SDRAM https://en.wikipedia.org/wiki/GDDR5_SDRAM.
- dkobran 9y ago
  HBM2 to be specific: https://en.wikipedia.org/wiki/High_Bandwidth_Memory#HBM_2 https://en.wikipedia.org/wiki/High_Bandwidth_Memory#HBM_2
Const-me 9y ago
> that would be complex and severely impact performance on GPU architectures Agree about the complexity. However, adaptive grids can be a huge win in terms of performance. The GPU doesn’t need to simulate the whole 8G grid nodes, it only simulates much fewer nodes, only in the areas of large gradients / higher frequencies of the simulated fields. Here’s an old SpaceX presentation about their GPU simulations of rockets: https://www.youtube.com/watch?v=vYA0f6R5KAI https://www.youtube.com/watch?v=vYA0f6R5KAI
- w0utert 9y ago
  That was very interesting and helpful, thanks!
Tuna-Fish 9y ago
It's not just the cost, at least not just the cost of the RAM. GPU ram is designed to be wider than desktop memory, meaning that you need more pins on the die for the same amount of memory. The IO in top end accelerators uses all the real estate available for pins, and the dies are already at the reticle limit, so to add memory you have to either: 1) Use desktop memory that allows more BW/pin but is also much slower per pin 2) Develop completely new memory just for your application 3) Put some kind of buffering between the chip and the ram, greatly increasing latency. None of these is currently that appealing for the manufacturers. HMC was originally going to do 3 by doing 2, but HMC appears stillborn, as all the momentum went to HBM instead, and HBM cannot increase the memory count per pin nearly as much as HMC could.
dkobran 9y ago
NVIDIA makes a chip called the P6000 that offers 24GB memory. It isn't high-throughput (HBM2) memory but regardless, the chip will benchmark relatively close to the Volta V100 and costs about half. Here's a deep learning benchmark for comparison: https://github.com/rejunity/cnn-benchmarks https://github.com/rejunity/cnn-benchmarks

ThrowawayR2 9y ago

If I understand it correctly, GPUs use specialized high-bandwidth memory that is more expensive instead of commodity memory. See https://en.wikipedia.org/wiki/GDDR5_SDRAM https://en.wikipedia.org/wiki/GDDR5_SDRAM.

dkobran 9y ago
HBM2 to be specific: https://en.wikipedia.org/wiki/High_Bandwidth_Memory#HBM_2 https://en.wikipedia.org/wiki/High_Bandwidth_Memory#HBM_2

Const-me 9y ago

> that would be complex and severely impact performance on GPU architectures Agree about the complexity. However, adaptive grids can be a huge win in terms of performance. The GPU doesn’t need to simulate the whole 8G grid nodes, it only simulates much fewer nodes, only in the areas of large gradients / higher frequencies of the simulated fields. Here’s an old SpaceX presentation about their GPU simulations of rockets: https://www.youtube.com/watch?v=vYA0f6R5KAI https://www.youtube.com/watch?v=vYA0f6R5KAI

w0utert 9y ago
That was very interesting and helpful, thanks!

Tuna-Fish 9y ago

It's not just the cost, at least not just the cost of the RAM. GPU ram is designed to be wider than desktop memory, meaning that you need more pins on the die for the same amount of memory. The IO in top end accelerators uses all the real estate available for pins, and the dies are already at the reticle limit, so to add memory you have to either: 1) Use desktop memory that allows more BW/pin but is also much slower per pin 2) Develop completely new memory just for your application 3) Put some kind of buffering between the chip and the ram, greatly increasing latency. None of these is currently that appealing for the manufacturers. HMC was originally going to do 3 by doing 2, but HMC appears stillborn, as all the momentum went to HBM instead, and HBM cannot increase the memory count per pin nearly as much as HMC could.

dkobran 9y ago

NVIDIA makes a chip called the P6000 that offers 24GB memory. It isn't high-throughput (HBM2) memory but regardless, the chip will benchmark relatively close to the Volta V100 and costs about half. Here's a deep learning benchmark for comparison: https://github.com/rejunity/cnn-benchmarks https://github.com/rejunity/cnn-benchmarks