3 ms·

> utilising a GPU acceleration GPUs are incredibly versatile. As CPUs struggle to add more cores instead we should shift software as much as possible on to the

> utilising a GPU acceleration GPUs are incredibly versatile. As CPUs struggle to add more cores instead we should shift software as much as possible on to the GPU and use the CPU as a coprocessor. At least the GPU should be given direct access to storage. Perhaps GPUs should come with 1-2 NVMe slots and upgradeable VRAM.

1oooqooq 3y ago
in this case the GPU is just to cheapen development/enable prototyping. Nothing in the real world will use it instead of a fpga or custom silicon. also, LOL at nokia making bank on their overly finicky 200Gpbs SFP and these guys "wires? meh"
- KeplerBoy 3y ago
  "Nothing in the real world will use it instead of a fpga or custom silicon." That's a bold claim and pretty wrong.
MisterTea 3y ago
GPU's suck at general loads. Let the CPU worry about the boring general nonsense like data shuffling and just hand the GPU massively parallel workloads like graphics, computations and so on. Let it do what it does best and let DMA handle the rest. Horses for courses.
- gumballindie 3y ago
  I am aware of their limitations. But I al wondering if there’s a way to make GPU cores behave like a single unit, or larger units like cpu cores, when needed - somehow to process data in a sequence of instructions, when needed and in parallel when not. Just thinking out loud. As I am playing around with GPUs for various reasons I am simply amazed at how versatile they are. One fun hack I wanted to make was a database function that would let me update database record on the GPU. One could implement the function as a kernel and have it update tens of thousands of records in one go. The bottleneck is reading data from disk, to cpu / ram, to gpu. Instead of there was a way to read data directly from disk, all records could be loaded in VRAM at once and data updated. I think for a good number of use cases databases are prime candidates for parallel processing. Each record being processed by a kernel. For gaming a significant bottleneck is moving texture and mesh data between disks, cpus, and the gpu. Essentially, I think the main component of a PC could be the GPU while the CPU is there to execute block operations - if there is no way to “fuse” gpu cores as per my first paragraph. I just think there’s a lot of performance left on the table otherwise. For ml inference at edge for instance all you need is a GPU with a network card and extremely fast storage. All of which are available but bottlenecked by the cpu.
xxpor 3y ago
If you need to do parallel matrix math, yeah GPUs are great. Turns out that describes a lot of scientific/graphics/signal processing workloads. But the majority of computing is just straight forward linear moving bits around.
juliangoldsmith 3y ago
GPUs are great for things that are parallelizable, but a lot of things aren't. Technologies like GPUDirect are interesting in that vein. Theoretically you can DMA data directly from an NVMe drive on another machine, across an Infiniband link, and into VRAM without the data ever touching the CPU.
- londons_explore 3y ago
  More importantly, why wasn't that possible in 1998? CPU independent DMA engines have existed since 1995 or earlier... DMA engines can typically be controlled by a queue of work, and can read/write to anything memory mapped. The work units themselves can be in memory mapped space of the GPU or network adapter, allowing those devices to queue up work too. And the DMA transfer can be triggered by any interrupt source too (initially intended for things like sound cards to have a tiny buffer be frequently refilled by the DMA engine without CPU intervention). Basically, hardware has supported this forever. It is software that is lagging.
- gumballindie 3y ago
  Every now and then there’s a comment on HN that either ruins my weekend or my budget. Yours does both. I have to get both types of devices now to play around with.
- KeplerBoy 3y ago
  It goes beyond that. You can DMA stuff directly from a NIC into VRAM. That's probably why Nvidia acquired Mellanox a few years ago.
topspin 3y ago
> upgradeable VRAM Consumer class GPUs don't have a lot of flexibility regarding VRAM size or type. GPUs have highly optimized, integrated memory controllers that are designed to use one or only a small number of distinct VRAM capacities and types. Enterprise class GPUs do have more options wrt VRAM, but it's still not a free-for-all like typical desktop/server RAM configurations. I suppose one can imagine this changing in the future, but the memory controller complexity and costs (die area, power consumption, etc.) would need to be greatly increased.

1oooqooq 3y ago

in this case the GPU is just to cheapen development/enable prototyping. Nothing in the real world will use it instead of a fpga or custom silicon. also, LOL at nokia making bank on their overly finicky 200Gpbs SFP and these guys "wires? meh"

KeplerBoy 3y ago
"Nothing in the real world will use it instead of a fpga or custom silicon." That's a bold claim and pretty wrong.

MisterTea 3y ago

GPU's suck at general loads. Let the CPU worry about the boring general nonsense like data shuffling and just hand the GPU massively parallel workloads like graphics, computations and so on. Let it do what it does best and let DMA handle the rest. Horses for courses.

gumballindie 3y ago
I am aware of their limitations. But I al wondering if there’s a way to make GPU cores behave like a single unit, or larger units like cpu cores, when needed - somehow to process data in a sequence of instructions, when needed and in parallel when not. Just thinking out loud. As I am playing around with GPUs for various reasons I am simply amazed at how versatile they are. One fun hack I wanted to make was a database function that would let me update database record on the GPU. One could implement the function as a kernel and have it update tens of thousands of records in one go. The bottleneck is reading data from disk, to cpu / ram, to gpu. Instead of there was a way to read data directly from disk, all records could be loaded in VRAM at once and data updated. I think for a good number of use cases databases are prime candidates for parallel processing. Each record being processed by a kernel. For gaming a significant bottleneck is moving texture and mesh data between disks, cpus, and the gpu. Essentially, I think the main component of a PC could be the GPU while the CPU is there to execute block operations - if there is no way to “fuse” gpu cores as per my first paragraph. I just think there’s a lot of performance left on the table otherwise. For ml inference at edge for instance all you need is a GPU with a network card and extremely fast storage. All of which are available but bottlenecked by the cpu.

xxpor 3y ago

If you need to do parallel matrix math, yeah GPUs are great. Turns out that describes a lot of scientific/graphics/signal processing workloads. But the majority of computing is just straight forward linear moving bits around.

juliangoldsmith 3y ago

GPUs are great for things that are parallelizable, but a lot of things aren't. Technologies like GPUDirect are interesting in that vein. Theoretically you can DMA data directly from an NVMe drive on another machine, across an Infiniband link, and into VRAM without the data ever touching the CPU.

londons_explore 3y ago
More importantly, why wasn't that possible in 1998? CPU independent DMA engines have existed since 1995 or earlier... DMA engines can typically be controlled by a queue of work, and can read/write to anything memory mapped. The work units themselves can be in memory mapped space of the GPU or network adapter, allowing those devices to queue up work too. And the DMA transfer can be triggered by any interrupt source too (initially intended for things like sound cards to have a tiny buffer be frequently refilled by the DMA engine without CPU intervention). Basically, hardware has supported this forever. It is software that is lagging.
gumballindie 3y ago
Every now and then there’s a comment on HN that either ruins my weekend or my budget. Yours does both. I have to get both types of devices now to play around with.
KeplerBoy 3y ago
It goes beyond that. You can DMA stuff directly from a NIC into VRAM. That's probably why Nvidia acquired Mellanox a few years ago.

topspin 3y ago

> upgradeable VRAM Consumer class GPUs don't have a lot of flexibility regarding VRAM size or type. GPUs have highly optimized, integrated memory controllers that are designed to use one or only a small number of distinct VRAM capacities and types. Enterprise class GPUs do have more options wrt VRAM, but it's still not a free-for-all like typical desktop/server RAM configurations. I suppose one can imagine this changing in the future, but the memory controller complexity and costs (die area, power consumption, etc.) would need to be greatly increased.