3 ms·
I think it is too risky to build a company around the premise that someone won't soon solve the quadratic scaling issue. Especially, when that company involves
by bob1029 1y ago
I think it is too risky to build a company around the premise that someone won't soon solve the quadratic scaling issue. Especially, when that company involves creating ASICs.
E.g.: https://arxiv.org/abs/2312.00752 https://arxiv.org/abs/2312.00752
- qeternity 1y agoAttention is not the primary inference bottleneck. For each token you have to load all of the weights (or activated weights) from memory. This is why Cerebras is fast: they have huge memory bandwidth.
- Havoc 1y agoYeah also strikes me as quite risky. Their gear seems very focused on llama family specifically. Just takes one breakthrough and it's all different. See the recent diffusion style LLMs for example