3 ms·
I think that’s too much of a minority to hold up a market. The real key will be client side inference. And as more chips go the route of the Apple ones with on
by dagmx 3y ago
I think that’s too much of a minority to hold up a market.
The real key will be client side inference. And as more chips go the route of the Apple ones with onboard inference, I think it’ll be more appealing to the mass market.
Running LLMs locally on high end GPUs is going to always be niche.