3 ms·
ah, got it. yeah the llama.cpp speeds I've seen are like 5-10 tokens/second on intel but I've been impressed with the relatively ancient hardware llama.cpp work
by rgbrgb 3y ago
ah, got it. yeah the llama.cpp speeds I've seen are like 5-10 tokens/second on intel but I've been impressed with the relatively ancient hardware llama.cpp works on (2017 iMac with 8GB RAM at 5.5 t/s with a 7B model!).
- cztomsik 3y agoit's ok, you can put the link back I don't mind the competition :) yes, I know it definitely works, so it has to be some stupid mistake but it's impossible to reproduce it without actual intel mac :-/ I had enough trouble with supporting monterey, which also does not work 100% AFAIK