3 ms·

I also found this link ( https://blog.lastmileai.dev/run-llama-2-locally-in-7-lines-apple-silicon-mac-c3f46143f327 https://blog.lastmileai.dev/run-llama-2-local

I also found this link ( https://blog.lastmileai.dev/run-llama-2-locally-in-7-lines-apple-silicon-mac-c3f46143f327 https://blog.lastmileai.dev/run-llama-2-locally-in-7-lines-a...) but havent verified if it helps, but thanks for your reply I am taking a look now.

eurekin 3y ago
Just checked on a m1 w 32GB ram. 7b and 13b work as a charm. 70b doesn't fit, even the 2 bit version