3 ms·
This is exciting news, if the 8B numbers are true, we can already use model like Mixtral 8x7, even with a single GPU? But further into the development, we need
by karmasimida 3y ago
This is exciting news, if the 8B numbers are true, we can already use model like Mixtral 8x7, even with a single GPU?
But further into the development, we need comparison to large model sizes. 70B might be too much to ask, but 13B should be there at least.
- cjbprime 3y agoYou could already run Mixtral on the more expensive single consumer GPUs (with 24GB VRAM) before this paper, at e.g. 3-bits per weight.