4 ms·

I've only had a very slight dip into the water of the GPGPU world in the past 5 years, but it is obvious that Nvidia have the lion's share of the market when it

I've only had a very slight dip into the water of the GPGPU world in the past 5 years, but it is obvious that Nvidia have the lion's share of the market when it comes to hosted solutions. Still not sure why competitors don't have their GPU's offered in cloud services. At the time, it seemed the alternatives offered a more economical alternative. I was building a GPU version of Hashcash at the time fwiw.

fluffything 6y ago
> Still not sure why competitors don't have their GPU's offered in cloud services. AMD has many GPGPUs, but while they have kind of ok-ish raw numbers, their software stack makes quite poor use of them. Users care about perf/$, and nvidia is king there. For deep-learning and other apps, hardware without software is useless.
- JoeAltmaier 6y ago
  And if the population is not uniform (and it never is, as its a growing market and new resources have different feature sets) it can be hard to deploy some hardware-sensitive applications. We needed hardware support for encryption in our teleconferencing mix node. Had to contract specially, so we could deploy only on hardware that supported that (was modern enough).
Nokinside 6y ago
Nvidia has software advantage. Unlike with AMD, you have libraries and ways to build software that runs fast: CUDA, cuDNN.
- BadInformatics 6y ago
  They do, but (as described elsewhere in this thread), the level of polish isn't quite up to that of the CUDA stack. Add onto that the relatively abysmal developer evangelism from AMD and it's not surprising that adoption is so low...
- pjmlp 6y ago
  And good graphical debuggers as well.
staycoolboy 6y ago
There really aren't any serious competitors. AMD and Intel have GPU offerings, but they are too busy fighting for laptop, server, desktop market. Intel's graphics IP is just a value-add for chipsets to keep competitors out of the chipset market. Any new investment for startups is going toward ML acceleration, which is like GPGPU but with different topology for NN optimization. So to answer your Q: no one can compete with NVIDIA, so no one wants to invest there. IMHO, of course.
IdiocyInAction 6y ago
Nvidia has by far the best API/software support. CUDA is much nicer to use than OpenCL. Therefore, most software is written for Nvidia GPUs and it makes way more sense to host Nvidia.
- BadInformatics 6y ago
  > CUDA is much nicer to use than OpenCL I think this is also part of the problem. AMD has a nearly function-for-function compatible GPGPU stack with CUDA in ROCm/HIP that fixes many of the pitfalls in OpenCL, and even provides automated translation tools for CUDA developers switching over. However, this stack is so poorly documented and marketed that the mass majority of developers still associate AMD == OpenCL. Add onto that the lack of Windows, MacOS and Navi GPU support, and you've removed a ton of opportunities/incentives for folks to experiment with the platform.
slezyr 6y ago
It's simple - CUDA. A lot of software is CUDA-only like Meshroom[1] and other photogrammetry software. [1]: https://alicevision.org/#meshroom https://alicevision.org/#meshroom
dnautics 6y ago
I think the biggest problem for ml practitioners is that amd doesn't take the middle layer seriously. Last we tried, the and equivalent of cuda/cudnn (rocm) is riddled with stability bugs so it superficially works for quick tasks but if you need to do long runs it's an open question of whether or not it will work. This is really frustrating because in theory AMD gpgpus have a better architecture for ml. My guess is that it's also tough to get deals with AMD for their workstation class products which are competitive with Nvidia's rtx line, and the price/performance delta (both in terms of unit cost and power cost) is not huge. There are some consumer class amd gpus that have attractive price/performance, but they are being sunsetted and the future driver support is questionable. I looked at the Radeon 7, but it's bazel is a few millimeters out of pci spec so it literally doesn't fit in a 8u GPU chassis, and it's an architecture with an uncertain future.
ricardo81 6y ago
Given the replies, seems overwhelmingly it's about the software interface rather than the hardware. I'd got the impression that Vulkan would bridge that gap, but it's quite a low-level API and verbose. Would be interesting to hear comments on that, perhaps just for the case of the hosted GPU market being more competitive.
- BadInformatics 6y ago
  There have been a few discussions on what Vulkan needs to be competitive for GPGPU programming. https://news.ycombinator.com/item?id=22880502 https://news.ycombinator.com/item?id=22880502 comes to mind. I would also note that multi-gpu distributed synchronization (especially across nodes) is a big part of enterprise CUDA usage and doesn't seem to be well-supported in Vulkan.
- pjmlp 6y ago
  Not really, although off topic, one of the sessions at GTC 2020 was how Otoy gave up on Vulkan for their Octane Render, and decided to use CUDA instead, when Optix 7 added CUDA based rendering support. Vulkan also suffers from being too late to the game. NVidia moved CUDA into a polyglot runtime early in the game (around version 3 I think), and took a 10 year effort to redesign the hardware for optimal mapping to C++ memory model. Vulkan might eventually get it via SYCL, now that SYSCL has turned into backend agnostic mappings for C++ heterogeneous computing. But even if SYSCL turns out to be adopted across the industry, Vulkan will just be another backend alongside CUDA, Rom, FPGAs,....

fluffything 6y ago

> Still not sure why competitors don't have their GPU's offered in cloud services. AMD has many GPGPUs, but while they have kind of ok-ish raw numbers, their software stack makes quite poor use of them. Users care about perf/$, and nvidia is king there. For deep-learning and other apps, hardware without software is useless.

JoeAltmaier 6y ago
And if the population is not uniform (and it never is, as its a growing market and new resources have different feature sets) it can be hard to deploy some hardware-sensitive applications. We needed hardware support for encryption in our teleconferencing mix node. Had to contract specially, so we could deploy only on hardware that supported that (was modern enough).

Nokinside 6y ago

Nvidia has software advantage. Unlike with AMD, you have libraries and ways to build software that runs fast: CUDA, cuDNN.

BadInformatics 6y ago
They do, but (as described elsewhere in this thread), the level of polish isn't quite up to that of the CUDA stack. Add onto that the relatively abysmal developer evangelism from AMD and it's not surprising that adoption is so low...
pjmlp 6y ago
And good graphical debuggers as well.

staycoolboy 6y ago

There really aren't any serious competitors. AMD and Intel have GPU offerings, but they are too busy fighting for laptop, server, desktop market. Intel's graphics IP is just a value-add for chipsets to keep competitors out of the chipset market. Any new investment for startups is going toward ML acceleration, which is like GPGPU but with different topology for NN optimization. So to answer your Q: no one can compete with NVIDIA, so no one wants to invest there. IMHO, of course.

IdiocyInAction 6y ago

Nvidia has by far the best API/software support. CUDA is much nicer to use than OpenCL. Therefore, most software is written for Nvidia GPUs and it makes way more sense to host Nvidia.

BadInformatics 6y ago
> CUDA is much nicer to use than OpenCL I think this is also part of the problem. AMD has a nearly function-for-function compatible GPGPU stack with CUDA in ROCm/HIP that fixes many of the pitfalls in OpenCL, and even provides automated translation tools for CUDA developers switching over. However, this stack is so poorly documented and marketed that the mass majority of developers still associate AMD == OpenCL. Add onto that the lack of Windows, MacOS and Navi GPU support, and you've removed a ton of opportunities/incentives for folks to experiment with the platform.

slezyr 6y ago

It's simple - CUDA. A lot of software is CUDA-only like Meshroom[1] and other photogrammetry software. [1]: https://alicevision.org/#meshroom https://alicevision.org/#meshroom

dnautics 6y ago

I think the biggest problem for ml practitioners is that amd doesn't take the middle layer seriously. Last we tried, the and equivalent of cuda/cudnn (rocm) is riddled with stability bugs so it superficially works for quick tasks but if you need to do long runs it's an open question of whether or not it will work. This is really frustrating because in theory AMD gpgpus have a better architecture for ml. My guess is that it's also tough to get deals with AMD for their workstation class products which are competitive with Nvidia's rtx line, and the price/performance delta (both in terms of unit cost and power cost) is not huge. There are some consumer class amd gpus that have attractive price/performance, but they are being sunsetted and the future driver support is questionable. I looked at the Radeon 7, but it's bazel is a few millimeters out of pci spec so it literally doesn't fit in a 8u GPU chassis, and it's an architecture with an uncertain future.

ricardo81 6y ago

Given the replies, seems overwhelmingly it's about the software interface rather than the hardware. I'd got the impression that Vulkan would bridge that gap, but it's quite a low-level API and verbose. Would be interesting to hear comments on that, perhaps just for the case of the hosted GPU market being more competitive.

BadInformatics 6y ago
There have been a few discussions on what Vulkan needs to be competitive for GPGPU programming. https://news.ycombinator.com/item?id=22880502 https://news.ycombinator.com/item?id=22880502 comes to mind. I would also note that multi-gpu distributed synchronization (especially across nodes) is a big part of enterprise CUDA usage and doesn't seem to be well-supported in Vulkan.
pjmlp 6y ago
Not really, although off topic, one of the sessions at GTC 2020 was how Otoy gave up on Vulkan for their Octane Render, and decided to use CUDA instead, when Optix 7 added CUDA based rendering support. Vulkan also suffers from being too late to the game. NVidia moved CUDA into a polyglot runtime early in the game (around version 3 I think), and took a 10 year effort to redesign the hardware for optimal mapping to C++ memory model. Vulkan might eventually get it via SYCL, now that SYSCL has turned into backend agnostic mappings for C++ heterogeneous computing. But even if SYSCL turns out to be adopted across the industry, Vulkan will just be another backend alongside CUDA, Rom, FPGAs,....