3 ms·
If you're running a 1M context LLM I assume you are well past memory constraints since the weights alone are 10s of GBs.
by humansareok1 3y ago
If you're running a 1M context LLM I assume you are well past memory constraints since the weights alone are 10s of GBs.