4 ms·

It feels really weird to see this when I keep seeing the CEO and CTO of HuggingFace push for local inference to win.

• airspresso 1mo ago
  Nvidia is also pushing for local inference IMHO. They want open models and competition in the model layer, not two big labs controlling all of it.
• DanielHB 1mo ago
  Why? If anything it makes complete sense: 1) Local inference means more general GPUs and less ASIC hardware. Only big companies can push ASIC because of the software required, meanwhile nVidia owns cuda which is the standard. 2) Local is less resource-efficient per chip (chips remaining idle much more, meaning more chips required). 3) End-customers have less bargaining power compared to hyper-scalers. Although this might change if customer hardware start behaving more like phones (SoC with everything packed in), but even then the SoC makers will likely still have less bargaining power than hyper-scalers. But then nVidia could potentially make the whole SoC too. So overall local-inference users = higher profit margins for nvidia. They much rather have every business on the globe buy one nvidia rack (or every laptop have a beefy GPU) than have 5-10 hyperscalers buy a few hundred thousand.
  • whizzter 1mo ago
    The hyperscalers were stable customers buying far more expensive equipment and NVidia has also made a bank selling a lot of auxillary hardware like Mellanox to AI datacenters. But the spending spree is probably coming to an end with the looming IPO's and NVidia is probably trying to hedge their bets by making themselves the sure bet once big-AI stops monopolizing RAM and everyone races to get their local setups.
• oersted 1mo ago
  You forget that Nvidia has also been the top consumer GPU player for 20 years.