3 ms·
In 4bit 7B requires 6GB of RAM and runs at ChatGPT speeds on CPU (with llama.cpp). The 15B model coming out soon will require 12GB of RAM and still run at good
by MacsHeadroom 4y ago
In 4bit 7B requires 6GB of RAM and runs at ChatGPT speeds on CPU (with llama.cpp).
The 15B model coming out soon will require 12GB of RAM and still run at good speeds on CPU.