3 ms·
I've only tried it on my M1, running Llama-3 via Ollama. It works, but it's slow to the point where it's not really usable. Maybe there are smaller models you
by mrweasel 2y ago
I've only tried it on my M1, running Llama-3 via Ollama. It works, but it's slow to the point where it's not really usable. Maybe there are smaller models you can run that will perform better.
- gkfasdfasdf 2y agoWhat size model did you try and how much memory does your M1 have? See my other comment, my experience has been that llama3 was very fast on an M1.
- mrweasel 2y agoI was just running: ollama run llama3, So that would be 8B parameters, on a 8GB M1 Air. Maybe it's just my expectations, but it seems rather slow to process queries. Depending on the prompt somewhere between the 10 - 40 tokens per second, but that very much depends on the prompt. My complaint is the time between the prompt is entered and output starts.