4 ms·

Some of us are running llama 405B locally already. All my GPUs are ancient Nvidai GPUs. IMO, the point of an open cuda is to force Nvidia to stop squeezing u

Some of us are running llama 405B locally already. All my GPUs are ancient Nvidai GPUs. IMO, the point of an open cuda is to force Nvidia to stop squeezing us. You get more performance for the buck for AMD. If I could run cuda on AMD, I would have bought new AMD gpus instead. Have enough people do that and Nvidia might take note and stop squeezing us for cash.

smokel 2y ago
> the point of an open cuda is to force Nvidia to stop squeezing us Nobody is forcing you to buy GPUs. Your logic is flawed in the sense that enough people could also simply write alternatives to Torch, which, by the way, is already open source.
- funcDropShadow 2y ago
  Nobody is forcing you to buy a computer. Nobody is forcing you to live under a roof. Nobody is forcing you to eat.
  - smokel 2y ago
    Sorry for the harsh comment. I just found it highly unlikely that Nvidia would change its ways due to this, and I don't really see how we're being "squeezed". Nvidia are delivering amazing products (as are AMD), and it is not going to be any cheaper this way. Building this kind of hardware is not something a hacker can do over the weekend.
    - Wytwwww 2y ago
      Nvidia is charging what they are entirely because there is no or very little competition. There isn't much else to it, if AMD/Intel caught up Nvidia would suddenly start selling their GPUs for way less...
      - deleted 2y ago
        [deleted]
      - deleted 2y ago
        [deleted]
    - tomooot 2y ago
      The squeeze is mostly within the segmentation of VRAM between products, it's basically a commodity and this week the spot price for 8GB of GDDR6 has varied from $1.30 to $3.50 [1]. Yet to get a card with 8GB more than one with comparable logical performance, you'd be looking at hundreds (or thousands in the case of "machine learning" cards) of dollars. [1] https://www.dramexchange.com/ https://www.dramexchange.com/
oaththrowaway 2y ago
What are you using P100s or something?
- segmondy 2y ago
  many 3090s
  - oaththrowaway 2y ago
    How is that ancient?

smokel 2y ago

> the point of an open cuda is to force Nvidia to stop squeezing us Nobody is forcing you to buy GPUs. Your logic is flawed in the sense that enough people could also simply write alternatives to Torch, which, by the way, is already open source.

funcDropShadow 2y ago
Nobody is forcing you to buy a computer. Nobody is forcing you to live under a roof. Nobody is forcing you to eat.
- smokel 2y ago
  Sorry for the harsh comment. I just found it highly unlikely that Nvidia would change its ways due to this, and I don't really see how we're being "squeezed". Nvidia are delivering amazing products (as are AMD), and it is not going to be any cheaper this way. Building this kind of hardware is not something a hacker can do over the weekend.
  - Wytwwww 2y ago
    Nvidia is charging what they are entirely because there is no or very little competition. There isn't much else to it, if AMD/Intel caught up Nvidia would suddenly start selling their GPUs for way less...
    - deleted 2y ago
      [deleted]
    - deleted 2y ago
      [deleted]
  - tomooot 2y ago
    The squeeze is mostly within the segmentation of VRAM between products, it's basically a commodity and this week the spot price for 8GB of GDDR6 has varied from $1.30 to $3.50 [1]. Yet to get a card with 8GB more than one with comparable logical performance, you'd be looking at hundreds (or thousands in the case of "machine learning" cards) of dollars. [1] https://www.dramexchange.com/ https://www.dramexchange.com/

oaththrowaway 2y ago

What are you using P100s or something?

segmondy 2y ago
many 3090s
- oaththrowaway 2y ago
  How is that ancient?