3 ms·
> Nvidia seemed 'untouchable' for so long in this space that its nice to see things get shaken up. Did it? Both Mistral's LeChat (running on Cerebras) and Goo
by epolanski 2y ago
> Nvidia seemed 'untouchable' for so long in this space that its nice to see things get shaken up.
Did it?
Both Mistral's LeChat (running on Cerebras) and Google's Gemini (running on Tensors) have clearly showed ages ago Nvidia had no advantage at all in inference.
The hundreds of billions spent in hardware till now focused on training, but inference is in the long run gonna get the lion share of the work.
- wyager 2y ago> but inference is in the long run gonna get the lion share of the work. I'm not sure - might not the equilibrium state be that we are constantly fine-tuning models with the latest data (e.g. social media firehose)?
- NoahZuniga 2y agoHead of groq said that in his experience at google training was less than 10% of compute.
- lostmsu 2y agoIsn't Groq still more expensive than GPU-based providers?