try oMLX or vMLX - both great projects that offer some amazing performance optimizations for Apple Silicon that utilize UMA and NVME caching efficiently. https

try oMLX or vMLX - both great projects that offer some amazing performance optimizations for Apple Silicon that utilize UMA and NVME caching efficiently. https://omlx.ai https://omlx.ai https://vmlx.net https://vmlx.net That said, it's been a few weeks since I've looked so maybe llama.cpp has those features now... they really do move that quickly.