4 ms·
Presumably at some point the rapid progress of models will plateau, at least insofar as a model could be frozen in time and remain economically useful for the e
by simondotau 4mo ago
Presumably at some point the rapid progress of models will plateau, at least insofar as a model could be frozen in time and remain economically useful for the expected life of hardware. Especially if it comes with compelling benefits e.g. dramatically lower latency and/or dramatically higher performance per watt.
If you can build chips that could run one specific LLM 100x faster than anything else, it would have a use case that nothing else could match.
- fragmede 4mo agohttps://www.cerebras.ai/ https://www.cerebras.ai/ is exactly that! Holy shit it's fast.
- otterley 4mo agoCerebras is not that. Cerebras isn’t tied to a particular model like Taalas is. The latter is even faster than Cerebras.
- jwHollister 4mo agoYeah at a certain point if inference is fast and cheap, do I care if it's the smartest? I can iterate in a tighter loop.
- lsaferite 4mo agoThose taalus chips apparently run at 1/10 the power as the current SOTA GPU setups. If they can execute even partially on their plan, it'll be a literal game changer.