4 ms·
LLaMa [1] works on Apple Mx processors due to unified memory. The 7B model works on a 8GB Apple MacBook Pro M1, ~40 seconds/token. [1] https://dev.l1x.be/posts
by ly3xqhl8g9 4y ago
LLaMa [1] works on Apple Mx processors due to unified memory. The 7B model works on a 8GB Apple MacBook Pro M1, ~40 seconds/token.
[1] https://dev.l1x.be/posts/2023/03/12/using-llama-with-m1-mac https://dev.l1x.be/posts/2023/03/12/using-llama-with-m1-mac
- recuter 4y agoI get 220ms/token on an M1 8GB Macbook Air, so ChatGPT speeds. on a 32GB M1 Max it babbles faster than my ex-gf.
- ly3xqhl8g9 4y agoWeird, I ran it alongside quite a few other processes, didn't expect to even be able to run and didn't bother to optimize it in any meaningful way.