3 ms·
For context, I got M2 Max MBP, 64 GB shared RAM, bought it March 2023 for $5-6K. Llama 3.2 1.0B - 650 t/s Phi 3.5 3.8B - 60 t/s. Llama 3.1 8.0B - 37 t/
by refulgentis 2y ago
For context, I got M2 Max MBP, 64 GB shared RAM, bought it March 2023 for $5-6K.
Llama 3.2 1.0B - 650 t/s
Phi 3.5 3.8B - 60 t/s.
Llama 3.1 8.0B - 37 t/s.
Mixtral 14.0B - 24 t/s.
Full GPU acceleration, using llama.cpp, just like LM Studio.