4 ms·
I look at that inference engine as the most plausible current candidate for Nvidia AI moat drainer.
by TradingPlaces 3y ago
I look at that inference engine as the most plausible current candidate for Nvidia AI moat drainer.
- dandiep 3y agoCan you elaborate on this? Would love to understand how that would play out.
- TradingPlaces 3y agoMy view is that software is Nvidia’s moat. There is plenty of hardware that can do all this stuff cheaper, and more coming in the next couple of years. That’s not the issue. The issue is that strong hardware-software link that Nvidia has. The promise of the engine is the same as Lattner’s LLVM was for compilers: train on the back end of your choice, and run it on the front end of your choice. It is modular. That breaks the hardware-software link. https://www.modular.com/engine https://www.modular.com/engine Scroll down to the first flow chart. Note how they stuck Nvidia over to the bottom-right. That was intentional. ARM, the cheapest data center CPU hardware and the most prevalent at the edge, is in the middle. They launched with CPU support first, now doing GPUs. The main dividing line in LLMs and other large models is becoming giant commercial models running on GPU cloud vs open source at the edge or CPU cloud.