4 ms·

>Can't they just hire all of nvidia's developers and pay them 5x as much? As time goes on I don’t see how you break the CUDA moat even if you had all of nvidia

>Can't they just hire all of nvidia's developers and pay them 5x as much? As time goes on I don’t see how you break the CUDA moat even if you had all of nvidia Al’s engineers. CUDA means you need everyone in AI to target your new (hopefully open) platform and that platform is faster than CUDA is. Given how most frameworks of the last 10 years have been optimized for CUDA you would need to turn around a global sized cruise ship. If Intel’s GPUs are only 3% faster, will that be enough to rewrite my entire software stack for something not CUDA? If intel opts for a translation layer, could they ever match nvidia’s performance?

HDThoreaun 3y ago
Well I'm not super experienced with GPU development but aren't most people using packages built on top of CUDA like pytroch etc? Would it be impossible to throw tons of resources at those packages so they handle whatever intel comes up with as well as they handle CUDA? If Intel is 10% slower but 50% cheaper and the open source stack you use has been heavily updated to work well with Intel drivers would that not be an enticing product?
- UncleOxidant 3y ago
  Intel's been trying this for several years now (OneAPI and OpenVINO), but so far they haven't gotten the traction. CUDA is just really entrenched at this point.
- michaelt 3y ago
  > aren't most people using packages built on top of CUDA like pytorch etc? Yes, and in fact both AMD and Intel have libraries. You can run Stable Diffusion and suchlike on AMD GPUs today, apparently. And you can export models from most ML frameworks to run in the browser, on phones and suchlike. > If Intel is 10% slower but 50% cheaper [...] would that not be an enticing product? Sometimes, yes. Some of the largest models apparently cost $600,000 in compute time to train [1], so halving that would be pretty appealing. However, part of the reason for nvidia's dominance is that if you're hiring an ML engineer for $160,000/year spending $1,600 to give them an RTX 4090 is chump change. [1] https://twitter.com/emostaque/status/1563870674111832066 https://twitter.com/emostaque/status/1563870674111832066

HDThoreaun 3y ago

Well I'm not super experienced with GPU development but aren't most people using packages built on top of CUDA like pytroch etc? Would it be impossible to throw tons of resources at those packages so they handle whatever intel comes up with as well as they handle CUDA? If Intel is 10% slower but 50% cheaper and the open source stack you use has been heavily updated to work well with Intel drivers would that not be an enticing product?

UncleOxidant 3y ago
Intel's been trying this for several years now (OneAPI and OpenVINO), but so far they haven't gotten the traction. CUDA is just really entrenched at this point.
michaelt 3y ago
> aren't most people using packages built on top of CUDA like pytorch etc? Yes, and in fact both AMD and Intel have libraries. You can run Stable Diffusion and suchlike on AMD GPUs today, apparently. And you can export models from most ML frameworks to run in the browser, on phones and suchlike. > If Intel is 10% slower but 50% cheaper [...] would that not be an enticing product? Sometimes, yes. Some of the largest models apparently cost $600,000 in compute time to train [1], so halving that would be pretty appealing. However, part of the reason for nvidia's dominance is that if you're hiring an ML engineer for $160,000/year spending $1,600 to give them an RTX 4090 is chump change. [1] https://twitter.com/emostaque/status/1563870674111832066 https://twitter.com/emostaque/status/1563870674111832066