3 ms·
> The question is if a llm will run with usable performance at that scale? For the self-attention mechanism, memory bandwidth requirements scale ~quadratically
by bob1029 2y ago
> The question is if a llm will run with usable performance at that scale?
For the self-attention mechanism, memory bandwidth requirements scale ~quadratically with the sequence length.
- kridsdale1 2y agoSomeone has got to be working on a better method than that. Hundreds of billions are at stake.
- deleted 2y ago[deleted]