3 ms·

>enthusiast AI dev segment i think it isn't about enthusiast. To me it looks like Huang/NVDA is pushing further a small revolution using the opening provided b

by trhway 2y ago

>enthusiast AI dev segment i think it isn't about enthusiast. To me it looks like Huang/NVDA is pushing further a small revolution using the opening provided by the AI wave - up until now the GPU was add-on to the general computing core onto which that computing core offloaded some computing. With AI that offloaded computing becomes de-facto the main computing and Huang/NVDA is turning tables by making the CPU is just a small add-on on the GPU, with some general computing offloaded to that CPU. The CPU being located that "close" and with unified memory - that would stimulate development of parallelization for a lot of general computing so that it would be executed on GPU, very fast that way, instead of on the CPU. For example classic of enterprise computing - databases, the SQL ones - a lot, if not, with some work, everything, in these databases can be executed on GPU with a significant performance gain vs. CPU. Why it isn't happening today? Load/unload onto GPU eats into performance, complexity of having only some operations offloaded to GPU is very high in dev effort, etc. Streamlined development on a platform with unified memory will change it. That way Huang/NVDA may pull out rug from under the CPU-first platforms like AMD/INTC and would own both - new AI computing as well as significant share of the classic enterprise one.

tatersolid 2y ago
> these databases can be executed on GPU with a significant performance gain vs. CPU No, they can’t. GPU databases are niche products with severe limitations. GPUs are fast at massively parallel math problems, they anren’t useful for all tasks.
- trhway 2y ago
  >GPU databases are niche products with severe limitations. today. For the reasons like i mentioned. >GPUs are fast at massively parallel math problems, they anren’t useful for all tasks. GPU are fast at massively parallel tasks. Their memory bandwidth is 10x of that of the CPU for example. So, typical database operations, massively parallel in nature like join or filter, would run about that faster. Majority of computing can be parallelized and thus benefit from being executed on GPU (with unified memory of the practically usable for enterprise sizes like 128GB).
  - justincormack 2y ago
    The unified memory is no faster for the GPU than the CPU. So its not 10x the CPU. HBM on a GPU is much faster.
    - trhway 2y ago
      No. The unified memory on GB10 is much faster than regular RAM to CPU system: https://nvidianews.nvidia.com/news/nvidia-puts-grace-blackwell-on-every-desk-and-at-every-ai-developers-fingertips https://nvidianews.nvidia.com/news/nvidia-puts-grace-blackwe... "The GB10 Superchip enables Project DIGITS to deliver powerful performance using only a standard electrical outlet. Each Project DIGITS features 128GB of unified, coherent memory and up to 4TB of NVMe storage. With the supercomputer, developers can run up to 200-billion-parameter large language models to supercharge AI innovation." https://www.nvidia.com/en-us/data-center/grace-cpu-superchip/ https://www.nvidia.com/en-us/data-center/grace-cpu-superchip... "Grace is the first data center CPU to utilize server-class high-speed LPDDR5X memory with a wide memory subsystem that delivers up to 500GB/s of bandwidth " As far as i see it is about 4x of Zen 5.
  - menaerus 2y ago
    > So, typical database operations, massively parallel in nature like join or filter, would run about that faster. Given workload A how much of the total runtime JOIN or FILTER would take in contrast to the storage engine layer for example? My gut feeling tells me not much since to see the actual gain you'd need to be able to parallelize everything including the storage engine challenges. IIRC all the startups building databases around GPUs failed to deliver in the last ~10 years. All of them are shut down if I am not mistaken.
    - trhway 2y ago
      With cheap large RAMs and the SSD the storage has already became much less of an issue, especially when the database is primarily in-memory one. How about attaching SSD based storage to NVLink? :) Nvidia does have the direct to memory tech and uses wide buses, so i don't see any issue for them to direct attach arrays of SSD if they feel like it. >IIRC all the startups building databases around GPUs failed to deliver in the last ~10 years. All of them are shut down if I am not mistaken. As i already said - model of database offloading some ops to GPU with its separate memory isn't feasible, and those startups confirmed it. Especially when GPU would be 8-16GB while the main RAM can easily be 1-2TB with 100-200 CPU cores. With 128GB unified memory like on GB10 the situation looks completely different (that Nvidia allows only 2 to be connected by NVLink is just a market segmentation not a real technical limitation).

by trhway 2y ago

tatersolid 2y ago

> these databases can be executed on GPU with a significant performance gain vs. CPU No, they can’t. GPU databases are niche products with severe limitations. GPUs are fast at massively parallel math problems, they anren’t useful for all tasks.

trhway 2y ago
>GPU databases are niche products with severe limitations. today. For the reasons like i mentioned. >GPUs are fast at massively parallel math problems, they anren’t useful for all tasks. GPU are fast at massively parallel tasks. Their memory bandwidth is 10x of that of the CPU for example. So, typical database operations, massively parallel in nature like join or filter, would run about that faster. Majority of computing can be parallelized and thus benefit from being executed on GPU (with unified memory of the practically usable for enterprise sizes like 128GB).
- justincormack 2y ago
  The unified memory is no faster for the GPU than the CPU. So its not 10x the CPU. HBM on a GPU is much faster.
  - trhway 2y ago
    No. The unified memory on GB10 is much faster than regular RAM to CPU system: https://nvidianews.nvidia.com/news/nvidia-puts-grace-blackwell-on-every-desk-and-at-every-ai-developers-fingertips https://nvidianews.nvidia.com/news/nvidia-puts-grace-blackwe... "The GB10 Superchip enables Project DIGITS to deliver powerful performance using only a standard electrical outlet. Each Project DIGITS features 128GB of unified, coherent memory and up to 4TB of NVMe storage. With the supercomputer, developers can run up to 200-billion-parameter large language models to supercharge AI innovation." https://www.nvidia.com/en-us/data-center/grace-cpu-superchip/ https://www.nvidia.com/en-us/data-center/grace-cpu-superchip... "Grace is the first data center CPU to utilize server-class high-speed LPDDR5X memory with a wide memory subsystem that delivers up to 500GB/s of bandwidth " As far as i see it is about 4x of Zen 5.
- menaerus 2y ago
  > So, typical database operations, massively parallel in nature like join or filter, would run about that faster. Given workload A how much of the total runtime JOIN or FILTER would take in contrast to the storage engine layer for example? My gut feeling tells me not much since to see the actual gain you'd need to be able to parallelize everything including the storage engine challenges. IIRC all the startups building databases around GPUs failed to deliver in the last ~10 years. All of them are shut down if I am not mistaken.
  - trhway 2y ago
    With cheap large RAMs and the SSD the storage has already became much less of an issue, especially when the database is primarily in-memory one. How about attaching SSD based storage to NVLink? :) Nvidia does have the direct to memory tech and uses wide buses, so i don't see any issue for them to direct attach arrays of SSD if they feel like it. >IIRC all the startups building databases around GPUs failed to deliver in the last ~10 years. All of them are shut down if I am not mistaken. As i already said - model of database offloading some ops to GPU with its separate memory isn't feasible, and those startups confirmed it. Especially when GPU would be 8-16GB while the main RAM can easily be 1-2TB with 100-200 CPU cores. With 128GB unified memory like on GB10 the situation looks completely different (that Nvidia allows only 2 to be connected by NVLink is just a market segmentation not a real technical limitation).