4 ms·
I was just running: ollama run llama3, So that would be 8B parameters, on a 8GB M1 Air. Maybe it's just my expectations, but it seems rather slow to process qu
by mrweasel 2y ago
I was just running: ollama run llama3, So that would be 8B parameters, on a 8GB M1 Air.
Maybe it's just my expectations, but it seems rather slow to process queries. Depending on the prompt somewhere between the 10 - 40 tokens per second, but that very much depends on the prompt.
My complaint is the time between the prompt is entered and output starts.