4 ms·
Anybody spending tens of billions annually on Nvidia hardware is going to be willing to spend millions to port their software away from CUDA.
by bryanlarsen 2y ago
Anybody spending tens of billions annually on Nvidia hardware is going to be willing to spend millions to port their software away from CUDA.
- echelon 2y agoFor the average non-FAANG company, there's nothing to port to yet. We don't all have the luxury of custom TPUs.
- talldayo 2y agoTo slower hardware? What are they supposed to port to, ASICs?
- adgjlsfhk1 2y agoif the hardware is 30% slower and 2x cheaper, that's a pretty great deal.
- selectodude 2y agoPower density tends to be the limiting factor for this stuff, not money. If it's 30 percent slower per watt, it's useless.
- Wytwwww 2y agoThe ratio between power usage and GPU cost is very, very different than with CPUs, though. If you could save e.g. 20-30% of the purchase price that might make it worth it. e.g. you could run a H100 at 100% utilization 24/7 for 1 years at $0.4 per kWh (so assuming significant overhead for infrastructure etc.) and that would only cost ~10% of the purchase price of the GPU itself.
- wbl 2y agoPower usage cost isn't the money but the capacity and cooling.
- Wytwwww 2y agoYes, I know that. Hence I quadrupled the price of electricity or are you saying that the cost of capacity and cooling doesn't scale directly with power usage? We can increase that another 2x and the cost would still be relatively low compared to the price/deprecation of the GPU itself.
- pjmlp 2y agoFirst they need to support everything that CUDA is capable of in programing language portfolio, tooling and libraries.
- bryanlarsen 2y agoA typical LLM might use about 0.1% of CUDA. That's all that would have to be ported to get that LLM to work.
- pjmlp 2y agoWhich is missing the point why CUDA has won. Then again, maybe the goal is getting 0.1% of CUDA market share. /s
- its_down_again 2y agoIn the words of Gilfoyle-- I'll bite. Why has CUDA won?
- pjmlp 2y agoCUDA C++, CUDA Fortran, CUDA Anything PTX, plus libraries, IDE integration, GPU graphical debugging. Coupled with Khronos, Intel, AMD never delivering anything comparable with OpenCL, Apple losing interest after Khronos didn't took OpenCL into the direction they wanted, Google never adopting it favouring their Renderscript dialect.
- imtringued 2y agoNvidia has won because their compute drivers don't crash people's systems when they run e.g. Vulkan Compute. You are mostly listing irrelevant nice to have things that aren't deal breakers. AMD's consumer GPUs have a long history of being abandoned a year or two after release.
- pjmlp 2y agoCUDA C++, CUDA Fortran, CUDA Anything PTX, plus libraries, IDE integration, GPU graphical debugging, aren't only nice to have things.