2 ms·
>Has anybody figured out the hardware requirements yet? In 4bit 7B runs in 6GB of RAM at ChatGPT speeds on my old CPU. The hardware requirements are more or l
by MacsHeadroom 4y ago
>Has anybody figured out the hardware requirements yet?
In 4bit 7B runs in 6GB of RAM at ChatGPT speeds on my old CPU.
The hardware requirements are more or less identical to llama (the model is slightly larger, ~7.9B parameters vs ~6.7B).