3 ms·

> The masses will be left with the NVidia monopoly, while large companies will be able to free themselves from that. My bet: if it really becomes clear what ca

> The masses will be left with the NVidia monopoly, while large companies will be able to free themselves from that. My bet: if it really becomes clear what capabilities an AI accelerator chip needs and lots of people want to run (or even train) AIs on their own computers, AI accelerators will appear at the market. This is how capitalism typically works. My further bet: these AI accelerators will initially come from China. Just look at the history of Bitcoin: initially the blocks were mined on CPUs, but then the miners switched to GPUs and "everybody" was complaining about increasing GPU prices because of all the Bitcoin mining. At some moment, Bitcoin mining ASICs appeared from China and after those spread, GPUs were not attractive anymore for Bitcoin mining (of course the cryptocurrency fans who bought the GPUs for mining attempted to use their investment for mining other cryptocurrencies).

brucethemoose2 3y ago
The capital costs are enormous, not even counting the CUDA moat. It takes years to start producing a big AI processor. Yet many startups and existing designers anticipated this demand correctly, years in advance, and they are all still kinda struggling. Nvidia is massively supply constrained. AI customers would be buying up MI250s, CS-2s, IPUs, Tenstorrent accelerators, Gaudi 2s and so on en masse if they wanted to... But they are not, and its not going to get any easier once the supply catches up. Unless there's a big one in stealth mode, I think we are stuck with the hardware companies we have.
- solardev 3y ago
  Is there not a distributed computing potential here like there was for crypto mining? Some sort of seti@home/boinc like setup where home users can donate or sell compute time?
  - latchkey 3y ago
    The capex/opex is different for ML/AI than it was for crypto mining. Totally different hardware profiles.
  - fragmede 3y ago
    you can setup a computer and sell time on it on a couple of saas platforms, but only for inference. for training, the slowness of the interconnect between nodes become a bottleneck.
    - solardev 3y ago
      I see, thanks for the explanation!
  - brucethemoose2 3y ago
    Yes, see projects like the AI Horde and Petals. I highly recommend the Horde in particular. Theres also some kind of actual AI crypto project that I wouldn't touch with a 10 foot pole. But ultimately, even if true distribution like Petals figures out the inefficiency (and thats hard), it had the same issue as non Nvidia hardware: its not turnkey.
- aleph_minus_one 3y ago
  > Yet many startups and existing designers anticipated this demand correctly, years in advance, and they are all still kinda struggling. As I already hinted in my post: I see a huge problem in the fact that in my opinion it still is not completely clear to this day which capabilities an AI accelerator really needs - too much is in my opinion still in a state of flux.
  - brucethemoose2 3y ago
    The answer is kinda "whatever Nvidia implements." Research papers literally build around their hardware capabilities. A good example of this is Intel canceling, and AMD sidelining, their unified memory CPU/GPU chips for AI. They are super useful!.. In theory. But actually, they totally useless because no one is programming frameworks with unified memory SoCs in mind, as Nvidia does not make something like that.
- jasonwatkinspdx 3y ago
  There's also potential IP issues.
- abstractcontrol 3y ago
  > Yet many startups and existing designers anticipated this demand correctly, years in advance, and they are all still kinda struggling. Nvidia is massively supply constrained. AI customers would be buying up MI250s, CS-2s, IPUs, Tenstorrent accelerators, Gaudi 2s and so on en masse if they wanted to... But they are not, and its not going to get any easier once the supply catches up. Can you order any of these devices online as a regular person? Anybody can order a $300 Nvidia GPU and program it. This is the reason why deep learning originated on the GPUs. Forget those other AI accelerators, even if you bought something like a consumer grade AMD GPU, you couldn't program it because it's restricted. The reason why Nvidia's competitors are struggling is because their hardware is either too expensive or hard to buy.
tomcam 3y ago
> My bet: if it really becomes clear what capabilities an AI accelerator chip needs and lots of people want to run (or even train) AIs on their own computers, AI accelerators will appear at the market. My bet: in 6 months jart will have models running on local or server, with support for all platforms and using only 88K of ram ;)

brucethemoose2 3y ago

The capital costs are enormous, not even counting the CUDA moat. It takes years to start producing a big AI processor. Yet many startups and existing designers anticipated this demand correctly, years in advance, and they are all still kinda struggling. Nvidia is massively supply constrained. AI customers would be buying up MI250s, CS-2s, IPUs, Tenstorrent accelerators, Gaudi 2s and so on en masse if they wanted to... But they are not, and its not going to get any easier once the supply catches up. Unless there's a big one in stealth mode, I think we are stuck with the hardware companies we have.

solardev 3y ago
Is there not a distributed computing potential here like there was for crypto mining? Some sort of seti@home/boinc like setup where home users can donate or sell compute time?
- latchkey 3y ago
  The capex/opex is different for ML/AI than it was for crypto mining. Totally different hardware profiles.
- fragmede 3y ago
  you can setup a computer and sell time on it on a couple of saas platforms, but only for inference. for training, the slowness of the interconnect between nodes become a bottleneck.
  - solardev 3y ago
    I see, thanks for the explanation!
- brucethemoose2 3y ago
  Yes, see projects like the AI Horde and Petals. I highly recommend the Horde in particular. Theres also some kind of actual AI crypto project that I wouldn't touch with a 10 foot pole. But ultimately, even if true distribution like Petals figures out the inefficiency (and thats hard), it had the same issue as non Nvidia hardware: its not turnkey.
aleph_minus_one 3y ago
> Yet many startups and existing designers anticipated this demand correctly, years in advance, and they are all still kinda struggling. As I already hinted in my post: I see a huge problem in the fact that in my opinion it still is not completely clear to this day which capabilities an AI accelerator really needs - too much is in my opinion still in a state of flux.
- brucethemoose2 3y ago
  The answer is kinda "whatever Nvidia implements." Research papers literally build around their hardware capabilities. A good example of this is Intel canceling, and AMD sidelining, their unified memory CPU/GPU chips for AI. They are super useful!.. In theory. But actually, they totally useless because no one is programming frameworks with unified memory SoCs in mind, as Nvidia does not make something like that.
jasonwatkinspdx 3y ago
There's also potential IP issues.
abstractcontrol 3y ago
> Yet many startups and existing designers anticipated this demand correctly, years in advance, and they are all still kinda struggling. Nvidia is massively supply constrained. AI customers would be buying up MI250s, CS-2s, IPUs, Tenstorrent accelerators, Gaudi 2s and so on en masse if they wanted to... But they are not, and its not going to get any easier once the supply catches up. Can you order any of these devices online as a regular person? Anybody can order a $300 Nvidia GPU and program it. This is the reason why deep learning originated on the GPUs. Forget those other AI accelerators, even if you bought something like a consumer grade AMD GPU, you couldn't program it because it's restricted. The reason why Nvidia's competitors are struggling is because their hardware is either too expensive or hard to buy.

tomcam 3y ago

> My bet: if it really becomes clear what capabilities an AI accelerator chip needs and lots of people want to run (or even train) AIs on their own computers, AI accelerators will appear at the market. My bet: in 6 months jart will have models running on local or server, with support for all platforms and using only 88K of ram ;)