3 ms·
How powerful of a computer does this need? It would be useful to see, for one thing, minimum RAM requirements for these models.
by skykooler 4y ago
How powerful of a computer does this need? It would be useful to see, for one thing, minimum RAM requirements for these models.
- spion 4y agollama.cpp needs 40GB for the 65B model (due to int4 quantization) RamNeeded(other_size) ~= 40GB * other_size/65B