4 ms·

Ask HN: Does Nvidia have any credible competition in the AI/DL space?

22 points by mmaunder 3y ago

PaulHoule 3y ago
For inference, yes. For training, no.
- tripplyons 3y ago
  Google trains on TPUs.
mepian 3y ago
My employer is getting closer: https://www.hpcwire.com/2023/06/28/mlperf-training-3-0-showcases-llm-nvidia-dominates-intel-habana-also-impress/ https://www.hpcwire.com/2023/06/28/mlperf-training-3-0-showc...
- p1esk 3y ago
  From your link: “Only two chips took on the LLM challenge – Nvidia’s H100 GPU and Intel/Habana’s Gaudi2 deep learning processor – each showcasing different strengths. Not surprisingly, the newer H100 was the clear performance winner.” I wouldn’t say Intel is getting close, and to actually beat Nvidia any competitor needs to be much better (not just “getting close”).
  - smoldesu 3y ago
    To beat Nvidia, you have to solve software. The rest is negotiable - weaker hardware can still be attractive with more competitive pricing or efficiency. Nvidia is definitely on top of their game right now, but only because they've gone all-in betting on ecosystem fragmentation (and keep getting proven right). Sufficient software investment could unseat them handily.
    - p1esk 3y ago
      Nvidia is definitely on top of their game right now Nvidia has been on top of their game since at least 2012, when I shopped for ML hardware for the first time. AMD had plenty of time to “solve software” - what happened?
      - smoldesu 3y ago
        > what happened? Key stakeholders in OpenCL started pulling out. The upcoming Khronos spec didn't have the entire industry on board, and both Microsoft and Apple pushed vendor-controlled GPU runtimes on consumer hardware. AMD didn't have the same support they did pre-2010, so a lot of their efforts fell apart. Again, Nvidia's key is that they thrive in this chaos. They want people to make half-assed Metal Performance Shaders and DirectML programs, because it makes their ecosystem seem so strong and intersectional by comparison. If Intel invests hard enough in the right software, that's half of the problem solved.
pizza 3y ago
Currently there’s lockin due to most things calling upon cuda on nvidia gpus. There are alternatives to cuda but they’re up and coming: triton, tvm, openvino, improved rocm, etc.
verdverm 3y ago
Cerebrus wafer Google TPU Not that these are for the average buyer
tikkun 3y ago
Some are here - some of these are things people are using today, some are available but don't have much user adoption, some are technically available but very hard to purchase or rent/use, and some aren't yet available: * Software: OpenAI's Triton (you might've noticed it mentioned in some of "TheBloke" model releases and as an option in the oobabooga text-generation-webui), Modular's Mojo (on top of MLIR), OctoML (from the creators of TVM), geohot's tiny corp, CUDA porting efforts, PyTorch as a way of reducing reliance on CUDA * Hardware: TPUs, Amazon Inferentia, Cloud companies working on chips (Microsoft Project Athena, AWS Tranium, TPU v5), chip startups (Cerebras, Tenstorrent), AMD's MI300A and MI300X, Tesla Dojo and D1, Meta's MTIA, Habana Gaudi, LLM ASICs I'm in the process of writing about this (I'll probably post in the hiring freelancer thread tomorrow, might like to find a freelancer to help me research and write it). The A/H100 with infiniband are still the most common request for startups doing LLM training though, I did some research and a writeup on that. If I'm missing or miscategorizing any above, please let me know.
- tikkun 3y ago
  Side note: The current angle I'm thinking about for the post would be to actually use them all. Take Llama 2, and see which software and hardware approaches we can get inference working on (would leave training to a follow-up post), write about how much of a hassle it is (to get access/to purchase/to rent, and to get running), and what the inference speed is like. That might be too ambitious though, I could see it taking a while. No points for companies that talk a big game but don't have a product that can actually be purchased/used, I think - they'd be relegated to a "things to watch for in future" section.
SirMaster 3y ago
I think AMD is at the very least, credible. https://wccftech.com/amd-instinct-mi250-boosted-ai-performance-pytorch-2-0-rocm-5-4-closes-in-on-nvidia-gpus/ https://wccftech.com/amd-instinct-mi250-boosted-ai-performan... I think they are better than people give them credit for. They seem to have a stigma against them. Nobody wants to make a large deployment of AMD hardware because there are of course risks involved and nobody wants to be the one to blame if it doesn't work out. I can't blame them, but that doesn't mean it wont potentially still work out. All it takes is for a big successful deployment I think.
- tikkun 3y ago
  > All it takes is for a big successful deployment I think. Yes - I think one company doing a big deployment of AMD GPUs for training and inference would be extremely confidence boosting for other companies. I wonder why AMD doesn't do it themselves.
- p1esk 3y ago
  If investors ask me whether to build AI supercomputer using AMD cards (rather than Nvidia), today I would say no.
  - tinktank 3y ago
    why?

22 points by mmaunder 3y ago

PaulHoule 3y ago

For inference, yes. For training, no.

tripplyons 3y ago
Google trains on TPUs.

mepian 3y ago

My employer is getting closer: https://www.hpcwire.com/2023/06/28/mlperf-training-3-0-showcases-llm-nvidia-dominates-intel-habana-also-impress/ https://www.hpcwire.com/2023/06/28/mlperf-training-3-0-showc...

p1esk 3y ago
From your link: “Only two chips took on the LLM challenge – Nvidia’s H100 GPU and Intel/Habana’s Gaudi2 deep learning processor – each showcasing different strengths. Not surprisingly, the newer H100 was the clear performance winner.” I wouldn’t say Intel is getting close, and to actually beat Nvidia any competitor needs to be much better (not just “getting close”).
- smoldesu 3y ago
  To beat Nvidia, you have to solve software. The rest is negotiable - weaker hardware can still be attractive with more competitive pricing or efficiency. Nvidia is definitely on top of their game right now, but only because they've gone all-in betting on ecosystem fragmentation (and keep getting proven right). Sufficient software investment could unseat them handily.
  - p1esk 3y ago
    Nvidia is definitely on top of their game right now Nvidia has been on top of their game since at least 2012, when I shopped for ML hardware for the first time. AMD had plenty of time to “solve software” - what happened?
    - smoldesu 3y ago
      > what happened? Key stakeholders in OpenCL started pulling out. The upcoming Khronos spec didn't have the entire industry on board, and both Microsoft and Apple pushed vendor-controlled GPU runtimes on consumer hardware. AMD didn't have the same support they did pre-2010, so a lot of their efforts fell apart. Again, Nvidia's key is that they thrive in this chaos. They want people to make half-assed Metal Performance Shaders and DirectML programs, because it makes their ecosystem seem so strong and intersectional by comparison. If Intel invests hard enough in the right software, that's half of the problem solved.

pizza 3y ago

Currently there’s lockin due to most things calling upon cuda on nvidia gpus. There are alternatives to cuda but they’re up and coming: triton, tvm, openvino, improved rocm, etc.

verdverm 3y ago

Cerebrus wafer Google TPU Not that these are for the average buyer

tikkun 3y ago

Some are here - some of these are things people are using today, some are available but don't have much user adoption, some are technically available but very hard to purchase or rent/use, and some aren't yet available: * Software: OpenAI's Triton (you might've noticed it mentioned in some of "TheBloke" model releases and as an option in the oobabooga text-generation-webui), Modular's Mojo (on top of MLIR), OctoML (from the creators of TVM), geohot's tiny corp, CUDA porting efforts, PyTorch as a way of reducing reliance on CUDA * Hardware: TPUs, Amazon Inferentia, Cloud companies working on chips (Microsoft Project Athena, AWS Tranium, TPU v5), chip startups (Cerebras, Tenstorrent), AMD's MI300A and MI300X, Tesla Dojo and D1, Meta's MTIA, Habana Gaudi, LLM ASICs I'm in the process of writing about this (I'll probably post in the hiring freelancer thread tomorrow, might like to find a freelancer to help me research and write it). The A/H100 with infiniband are still the most common request for startups doing LLM training though, I did some research and a writeup on that. If I'm missing or miscategorizing any above, please let me know.

tikkun 3y ago
Side note: The current angle I'm thinking about for the post would be to actually use them all. Take Llama 2, and see which software and hardware approaches we can get inference working on (would leave training to a follow-up post), write about how much of a hassle it is (to get access/to purchase/to rent, and to get running), and what the inference speed is like. That might be too ambitious though, I could see it taking a while. No points for companies that talk a big game but don't have a product that can actually be purchased/used, I think - they'd be relegated to a "things to watch for in future" section.

SirMaster 3y ago

I think AMD is at the very least, credible. https://wccftech.com/amd-instinct-mi250-boosted-ai-performance-pytorch-2-0-rocm-5-4-closes-in-on-nvidia-gpus/ https://wccftech.com/amd-instinct-mi250-boosted-ai-performan... I think they are better than people give them credit for. They seem to have a stigma against them. Nobody wants to make a large deployment of AMD hardware because there are of course risks involved and nobody wants to be the one to blame if it doesn't work out. I can't blame them, but that doesn't mean it wont potentially still work out. All it takes is for a big successful deployment I think.

tikkun 3y ago
> All it takes is for a big successful deployment I think. Yes - I think one company doing a big deployment of AMD GPUs for training and inference would be extremely confidence boosting for other companies. I wonder why AMD doesn't do it themselves.
p1esk 3y ago
If investors ask me whether to build AI supercomputer using AMD cards (rather than Nvidia), today I would say no.
- tinktank 3y ago
  why?