3 ms·
Glad to be of help. Yea that is the repo. https://replicate.com/blog/run-llama-locally https://replicate.com/blog/run-llama-locally I found that guide here on
by hdjfkfbfbr 3y ago
Glad to be of help. Yea that is the repo.
https://replicate.com/blog/run-llama-locally https://replicate.com/blog/run-llama-locally
I found that guide here on hn.
I run it cpu only with 16 threads but yeah perf is good enough.
BTw my 6gb figure is me.measuring from htop so llama2 is likely less.
- Ms-J 3y agoThanks for the starting point. I'll give an update if I'm able to successfully run the other models. I hope it could help the community.