4 ms·
Don't need a GPU to run the model, you can use your RAM and CPU, but it might be a bit slow
by janmo 3y ago
Don't need a GPU to run the model, you can use your RAM and CPU, but it might be a bit slow
- cmsj 3y agoIt's very slow, and for the 7b model you're still looking at a pretty hefty RAM hit whether it's CPU or GPU. The model download is something like 40GB.
- MacsHeadroom 3y agoThere's already support in llama.cpp. It runs faster than ChatGPT on my old laptop CPU.