3 ms·
What size model did you try and how much memory does your M1 have? See my other comment, my experience has been that llama3 was very fast on an M1.
by gkfasdfasdf 2y ago
What size model did you try and how much memory does your M1 have? See my other comment, my experience has been that llama3 was very fast on an M1.
- mrweasel 2y agoI was just running: ollama run llama3, So that would be 8B parameters, on a 8GB M1 Air. Maybe it's just my expectations, but it seems rather slow to process queries. Depending on the prompt somewhere between the 10 - 40 tokens per second, but that very much depends on the prompt. My complaint is the time between the prompt is entered and output starts.