4 ms·

> AMD is so behind NVidia that it's not even funny. Do you really want all AI hardware and software dominated by a monopoly? We're not looking to "beat" Nvidia

> AMD is so behind NVidia that it's not even funny. Do you really want all AI hardware and software dominated by a monopoly? We're not looking to "beat" Nvidia, we are looking to offer a compelling alternative. MI300x is compelling. MI355x is even more compelling. If there is another company out there making a compelling product, send them my way!

doctorpangloss 2y ago
Time will tell, no? Transmeta shipped a lot of Crusoes. It was run by brilliant people. It was a “compelling alternative.” Maybe Cerebras is the Transmeta of this race, I don’t know. But. It’s not about making an alternative. It most definitely is about “beating” NVIDIA. Otherwise, you are just shoveling dollars - shareholders’, undercompensated employees at AMD and TSMC, etc. - to Meta, like everyone else.
- latchkey 2y ago
  The current ASIC's all fail in the memory game, they are not compelling. Cerebras is even more unavailable than AMD! > It most definitely is about “beating” NVIDIA. Hard disagree, but we are just going to have to agree to disagree on that.
cyberax 2y ago
It's not my job to reformat the entire AI market. I'm willing to try AMD, and I even built an AMD-based machine to experiment with AI workflows. So far it has been failing miserably. I don't care that MI300X is compelling when I can't make samples work both on my desktop and on a cloud-based MI300X. I don't care about their academic collaborations, I'm not in the business of producing papers. I'll just pay for H100 in the cloud to be sure that I will be able to run the resulting models on my 3090 locally and/or deploy to 4090 clusters. If AMD shows some sense, commits to long-term support for their hardware with reasonable feature-parity across multiple generations, I'll reconsider them. And AMD has a history of doing that! Their CPU division is _excellent_, they are renowned for having long-term support for motherboard socket types. I remember being able to buy a motherboard and then not worrying about upgrading the CPU for the next 3-4 years.
- latchkey 2y ago
  > I'm willing to try AMD, and I even built an AMD-based machine to experiment with AI workflows. So far it has been failing miserably. I don't care that MI300X is compelling when I can't make samples work both on my desktop and on a cloud-based MI300X. Anush was actively looking for feedback on this on github today... https://www.reddit.com/r/ROCm/comments/1i5aatx/rocm_feedback_for_amd/ https://www.reddit.com/r/ROCm/comments/1i5aatx/rocm_feedback... https://github.com/ROCm/ROCm/discussions/4276 https://github.com/ROCm/ROCm/discussions/4276
  - magic_at_nodai 2y ago
    I have quad w7900s under my desk that work well for workloads on my desktop that translate well to MI300x. There are some perf gaps with FAv2, and FP8 but otherwise I get a seamless experience. lmk if you have a pointer to any github issues for me to track down to make your experience better.
    - saagarjha 2y ago
      Do you not have (remote) access to a MI300X?
pjmlp 2y ago
People keep forgeting CUDA is not only about AI, graphics matter as well, as does being a polyglot ecosystem, the IDE integration, the graphical debugging tools, the libraries, having a memory model based on C++ memory model, and the last point is quite relevant, as NVidia employs a few key persons from C++ ecosystem that work on the ISO C++ standard (WG21).
- saagarjha 2y ago
  Does anyone use the fancy C++ memory model in their CUDA code? I thought they used the intrinsics and called it a day.
juliangoldsmith 2y ago
AMD's hardware might be compelling if it had good software support, but it doesn't. CUDA regularly breaks when I try to use Tensorflow on NVIDIA hardware already. Running a poorly-implemented clone of CUDA where even getting Pytorch running is a small miracle is going to be a hard sell. All AMD had to do was support open standards. They could have added OpenCL/SYCL/Vulkan Compute backends to Tensorflow and Pytorch and covered 80% of ML use cases. Instead of differentiating themselves with actual working software, they decided to become an inferior copy of NVIDIA. I recently switched from Tensorflow to Tinygrad for personal projects and haven't looked back. The performance is similar to Tensorflow with JIT [0]. The difference is that instead of spending 5 hours fixing things when NVIDIA's proprietary kernel modules update or I need a new box, it actually Just Works when I do "pip install tinygrad". 0: https://cprimozic.net/notes/posts/machine-learning-benchmarks-on-the-7900-xtx/ https://cprimozic.net/notes/posts/machine-learning-benchmark...
- latchkey 2y ago
  > AMD's hardware might be compelling if it had good software support, but it doesn't. CUDA regularly breaks when I try to use Tensorflow on NVIDIA hardware already. So it is all shit, but tinygrad saves the day?
  - juliangoldsmith 2y ago
    It works out of the box without jumping through any hoops, and the fact that it has an OpenCL backend means it can run on a wide variety of hardware. I don't know of any other autograd libraries with a non-CUDA backend, but I'd be interested to learn about them.

doctorpangloss 2y ago

Time will tell, no? Transmeta shipped a lot of Crusoes. It was run by brilliant people. It was a “compelling alternative.” Maybe Cerebras is the Transmeta of this race, I don’t know. But. It’s not about making an alternative. It most definitely is about “beating” NVIDIA. Otherwise, you are just shoveling dollars - shareholders’, undercompensated employees at AMD and TSMC, etc. - to Meta, like everyone else.

latchkey 2y ago
The current ASIC's all fail in the memory game, they are not compelling. Cerebras is even more unavailable than AMD! > It most definitely is about “beating” NVIDIA. Hard disagree, but we are just going to have to agree to disagree on that.

cyberax 2y ago

It's not my job to reformat the entire AI market. I'm willing to try AMD, and I even built an AMD-based machine to experiment with AI workflows. So far it has been failing miserably. I don't care that MI300X is compelling when I can't make samples work both on my desktop and on a cloud-based MI300X. I don't care about their academic collaborations, I'm not in the business of producing papers. I'll just pay for H100 in the cloud to be sure that I will be able to run the resulting models on my 3090 locally and/or deploy to 4090 clusters. If AMD shows some sense, commits to long-term support for their hardware with reasonable feature-parity across multiple generations, I'll reconsider them. And AMD has a history of doing that! Their CPU division is _excellent_, they are renowned for having long-term support for motherboard socket types. I remember being able to buy a motherboard and then not worrying about upgrading the CPU for the next 3-4 years.

latchkey 2y ago
> I'm willing to try AMD, and I even built an AMD-based machine to experiment with AI workflows. So far it has been failing miserably. I don't care that MI300X is compelling when I can't make samples work both on my desktop and on a cloud-based MI300X. Anush was actively looking for feedback on this on github today... https://www.reddit.com/r/ROCm/comments/1i5aatx/rocm_feedback_for_amd/ https://www.reddit.com/r/ROCm/comments/1i5aatx/rocm_feedback... https://github.com/ROCm/ROCm/discussions/4276 https://github.com/ROCm/ROCm/discussions/4276
- magic_at_nodai 2y ago
  I have quad w7900s under my desk that work well for workloads on my desktop that translate well to MI300x. There are some perf gaps with FAv2, and FP8 but otherwise I get a seamless experience. lmk if you have a pointer to any github issues for me to track down to make your experience better.
  - saagarjha 2y ago
    Do you not have (remote) access to a MI300X?

pjmlp 2y ago

People keep forgeting CUDA is not only about AI, graphics matter as well, as does being a polyglot ecosystem, the IDE integration, the graphical debugging tools, the libraries, having a memory model based on C++ memory model, and the last point is quite relevant, as NVidia employs a few key persons from C++ ecosystem that work on the ISO C++ standard (WG21).

saagarjha 2y ago
Does anyone use the fancy C++ memory model in their CUDA code? I thought they used the intrinsics and called it a day.

juliangoldsmith 2y ago

AMD's hardware might be compelling if it had good software support, but it doesn't. CUDA regularly breaks when I try to use Tensorflow on NVIDIA hardware already. Running a poorly-implemented clone of CUDA where even getting Pytorch running is a small miracle is going to be a hard sell. All AMD had to do was support open standards. They could have added OpenCL/SYCL/Vulkan Compute backends to Tensorflow and Pytorch and covered 80% of ML use cases. Instead of differentiating themselves with actual working software, they decided to become an inferior copy of NVIDIA. I recently switched from Tensorflow to Tinygrad for personal projects and haven't looked back. The performance is similar to Tensorflow with JIT [0]. The difference is that instead of spending 5 hours fixing things when NVIDIA's proprietary kernel modules update or I need a new box, it actually Just Works when I do "pip install tinygrad". 0: https://cprimozic.net/notes/posts/machine-learning-benchmarks-on-the-7900-xtx/ https://cprimozic.net/notes/posts/machine-learning-benchmark...

latchkey 2y ago
> AMD's hardware might be compelling if it had good software support, but it doesn't. CUDA regularly breaks when I try to use Tensorflow on NVIDIA hardware already. So it is all shit, but tinygrad saves the day?
- juliangoldsmith 2y ago
  It works out of the box without jumping through any hoops, and the fact that it has an OpenCL backend means it can run on a wide variety of hardware. I don't know of any other autograd libraries with a non-CUDA backend, but I'd be interested to learn about them.