3 ms·

Prior vendor relationships and standardization. It's too much of a PITA ($$$$ engineering time) for research SWEs to support and buyers also have to engage more

Prior vendor relationships and standardization. It's too much of a PITA ($$$$ engineering time) for research SWEs to support and buyers also have to engage more vendors to handle multiple stacks. Meta and Microsoft generally standardize their shit as much as possible. Meta's Reality Labs is sort-of a special case in that it does more "throwaway"-type stuff, but it still aims to standardize and not deviate that much because it slows things down.

ramesh31 2y ago
>It's too much of a PITA ($$$$ engineering time) for research SWEs to support and buyers also have to engage more vendors to handle multiple stacks. Meta and Microsoft generally standardize their shit as much as possible. Right, and they are standardizing on CUDA. Something no other company has any chance of matching for the forseeable future.
- hi-v-rocknroll 2y ago
  MAANG are willing to invest in ASIC engineering like Apple does but are willing to buy and rent things from others in the short-term. They will eventually wise-up, and design their own custom AI accelerators. Meta already designs and builds their own servers.
  - cma 2y ago
    Meta, Amazon, Google, Apple all already have custom accelerators, though some only for inference.
    - buildbot 2y ago
      Microsoft does too, Maia 100
- throw0101d 2y ago
  > Right, and they are standardizing on CUDA. How many people are writing to CUDA directly, and how many folks are using something like PyTorch front-end where the back-end can potentially be changed? * https://pytorch.org/blog/pytorch-for-amd-rocm-platform-now-available-as-python-package/ https://pytorch.org/blog/pytorch-for-amd-rocm-platform-now-a... * https://rocm.docs.amd.com/projects/install-on-linux/en/develop/how-to/3rd-party/pytorch-install.html https://rocm.docs.amd.com/projects/install-on-linux/en/devel... * https://dev-discuss.pytorch.org/t/opencl-backend-important-updates/845/8 https://dev-discuss.pytorch.org/t/opencl-backend-important-u...
  - pjmlp 2y ago
    There are tons of use cases for GPGPU, the compute world isn't only about Pytorch and Tensorflow. And even with middleware, turns out CUDA has the best debugging tooling, followed by DirectCompute/DirectML and Metal.
  - paulmd 2y ago
    sure, it's things like oneapi that have pluggable backends. spir-v/vulkan compute too. now, for how many of those things does AMD have a pluggable backend that actually works? none of them, unfortunately. Octane implemented on SPIR-V - on NVIDIA and Apple, because those were the runtimes that worked. AMD and Intel couldn't even compile it successfully (intel probably would today, I'd guess). Blender tried to support AMD for a long time, and the runtime was so buggy and defective they eventually pulled support from OpenCL entirely, since the OpenCL only ever ran properly on NVIDIA and Apple, and both of those have alternative interfaces that are cleaner and faster. The only point was supporting AMD, and AMD's OpenCL implementation is so buggy that it ended up being a de-facto AMD-specific codepath anyway. That's the problem, AMD's software stacks are fractally broken. Not just HIP and their oneAPI adapter, but the ROCm it runs on. Not just ROCm, but the OpenCL it runs on. Not just openCL, but the kernel scheduler it runs on. Everything is broken at every level even on supported hardware. Saying "it's just software" is a handwave. There are literally entire stacks here that are not fit-for-purpose and need to be substantially rebuilt before you can start building the layers above them. There isn't a stable enough basis to just implement an adapter for these pluggable frameworks, because the runtime and the kernel code are also broken.

ramesh31 2y ago

>It's too much of a PITA ($$$$ engineering time) for research SWEs to support and buyers also have to engage more vendors to handle multiple stacks. Meta and Microsoft generally standardize their shit as much as possible. Right, and they are standardizing on CUDA. Something no other company has any chance of matching for the forseeable future.

hi-v-rocknroll 2y ago
MAANG are willing to invest in ASIC engineering like Apple does but are willing to buy and rent things from others in the short-term. They will eventually wise-up, and design their own custom AI accelerators. Meta already designs and builds their own servers.
- cma 2y ago
  Meta, Amazon, Google, Apple all already have custom accelerators, though some only for inference.
  - buildbot 2y ago
    Microsoft does too, Maia 100
throw0101d 2y ago
> Right, and they are standardizing on CUDA. How many people are writing to CUDA directly, and how many folks are using something like PyTorch front-end where the back-end can potentially be changed? * https://pytorch.org/blog/pytorch-for-amd-rocm-platform-now-available-as-python-package/ https://pytorch.org/blog/pytorch-for-amd-rocm-platform-now-a... * https://rocm.docs.amd.com/projects/install-on-linux/en/develop/how-to/3rd-party/pytorch-install.html https://rocm.docs.amd.com/projects/install-on-linux/en/devel... * https://dev-discuss.pytorch.org/t/opencl-backend-important-updates/845/8 https://dev-discuss.pytorch.org/t/opencl-backend-important-u...
- pjmlp 2y ago
  There are tons of use cases for GPGPU, the compute world isn't only about Pytorch and Tensorflow. And even with middleware, turns out CUDA has the best debugging tooling, followed by DirectCompute/DirectML and Metal.
- paulmd 2y ago
  sure, it's things like oneapi that have pluggable backends. spir-v/vulkan compute too. now, for how many of those things does AMD have a pluggable backend that actually works? none of them, unfortunately. Octane implemented on SPIR-V - on NVIDIA and Apple, because those were the runtimes that worked. AMD and Intel couldn't even compile it successfully (intel probably would today, I'd guess). Blender tried to support AMD for a long time, and the runtime was so buggy and defective they eventually pulled support from OpenCL entirely, since the OpenCL only ever ran properly on NVIDIA and Apple, and both of those have alternative interfaces that are cleaner and faster. The only point was supporting AMD, and AMD's OpenCL implementation is so buggy that it ended up being a de-facto AMD-specific codepath anyway. That's the problem, AMD's software stacks are fractally broken. Not just HIP and their oneAPI adapter, but the ROCm it runs on. Not just ROCm, but the OpenCL it runs on. Not just openCL, but the kernel scheduler it runs on. Everything is broken at every level even on supported hardware. Saying "it's just software" is a handwave. There are literally entire stacks here that are not fit-for-purpose and need to be substantially rebuilt before you can start building the layers above them. There isn't a stable enough basis to just implement an adapter for these pluggable frameworks, because the runtime and the kernel code are also broken.