4 ms·
That's 10x speed increase. What's the secret behind apple M3? Faster clocked RAMs? Specific AI hardware?
by 3abiton 3y ago
That's 10x speed increase. What's the secret behind apple M3? Faster clocked RAMs? Specific AI hardware?
- bugglebeetle 3y agoUnified memory and optimizations in llama.cpp (which Ollama wraps).
- ithkuil 3y agoIs that using the GPU?
- bugglebeetle 3y agoIt can be variably configured. There are details in the repo, but llama.cpp makes use of Metal.