3 ms·
It's GPU performance. Spin up ollama and run some inference on your 5-year-old intel macbook. You won't see 4000x performance improvement (because performance
by condiment 1y ago
It's GPU performance.
Spin up ollama and run some inference on your 5-year-old intel macbook. You won't see 4000x performance improvement (because performance is bottlenecked outside of the GPU), but you might be in the right order of magnitude.
- jandrese 1y agoComparing GPU performance to some half decade old Intel IGP seems like lying with statistics. "Look how many times faster our car is![1]" [1] Compared to a paraplegic octogenarian in a broken wheelchair!"
- deleted 1y ago[deleted]
- School-Cotton 1y agoWell, Apple isn’t making that comparison, the OP was.
- blihp 1y agoNot possible given the anemic memory bandwidth [1]... you can scale up the compute all you want but if the memory doesn't scale up as well you're not going to see anywhere near those numbers. [1] The memory bandwidth is fine for CPU workloads, but not for GPU / NN workloads.