4 ms·

How about using some of that money to develop CUDA alternatives so everyone is not paying the Nvidia tax?

• lukan 2y ago
  It would be probably cheaper to negate some IP. There are quite some projects and initiatives to make CUDA code run on AMD for example, but as far as I know, they all stopped at some point, probably because of fear of being sued into oblivion.
• whimsicalism 2y ago
  It seems like rocm is already fully ready for transformer inference, so you are just referring to training?
  • janalsncm 2y ago
    ROCm is buggy and largely undocumented. That’s why we don’t use it.
    • latchkey 2y ago
      It is actively improving every day. https://news.ycombinator.com/item?id=41052750 https://news.ycombinator.com/item?id=41052750
• belter 2y ago
  Please start with the Windows Tax first for Linux users buying hardware...and the Apple Tax for Android users...
• zitterbewegung 2y ago
  Either you port Tensorflow (Apple)[1] or PyTorch to your platform or you allow CUDA to run on your hardware (AMD) [2]. Companies are incentives to not have NVIDIA having a monopoly but the thing is that CUDA is a huge moat due to compatibility of all frameworks and everyone knows it. Also, all of the cloud or on premises providers use NVIDIA regardless. [1] https://developer.apple.com/metal/tensorflow-plugin/ https://developer.apple.com/metal/tensorflow-plugin/ [2] https://www.xda-developers.com/nvidia-cuda-amd-zluda/ https://www.xda-developers.com/nvidia-cuda-amd-zluda/
  • TuringNYC 2y ago
    >> Either you port Tensorflow (Apple)[1] or PyTorch to your platform or you allow CUDA to run on your hardware (AMD) [2]. Companies are incentives to not have NVIDIA having a monopoly but the thing is that CUDA is a huge moat due to compatibility of all frameworks and everyone knows it. Also, all of the cloud or on premises providers use NVIDIA regardless. This never made sense to me -- Apple could easily hire top talent to write Apple Silicon bindings for these popular libraries. I work at a creative ad agency, we have tons of high end apple devices yet the neural cores sit unused most of the time.
    • jcheng 2y ago
      A lot of libraries seem to be working on Apple Silicon GPUs but not on ANE. I found this discussion interesting, seems like the ANE has a lot of limitations, is not well documented, and can only be used indirectly through Core ML. https://github.com/ggerganov/llama.cpp/discussions/336 https://github.com/ggerganov/llama.cpp/discussions/336
    • varelse 2y ago
      [dead]
• erickj 2y ago
  That's the kind of work that can come out of academia and open source communities when societies provide the resources required.
• latchkey 2y ago
  It is being done already... https://docs.scale-lang.com/ https://docs.scale-lang.com/
• dogcomplex 2y ago
  Or just develop the next wave of chips designed for specifically transformer-based architectures (and ternary computing), and bypass the needs for GPUs and CUDA altogether
  • Zambyte 2y ago
    That would be betting against other architectures like Mamba, which does not seem like an obviously good bet to make yet. Maybe it is though.
    • dogcomplex 2y ago
      You're right, there are a number of avenues that are viable alternatives to the gpu monopoly. I like the fact that these can be made with just mass-printed multiplication (and in ternary computing's case - addition) gates which require little more than 10 year old tech which is already widely distributed.