4 ms·
Did you run it the best way possible? im no expert, but I understand it can affect inference time greatly (which format/engine is used)
by steinvakt2 1y ago
Did you run it the best way possible? im no expert, but I understand it can affect inference time greatly (which format/engine is used)
- pamelafox 1y agoI ran it via Ollama, which I assume uses the best way. Screenshot in my post here: https://bsky.app/profile/pamelafox.bsky.social/post/3lvobol3jfb2r https://bsky.app/profile/pamelafox.bsky.social/post/3lvobol3... I'm still wondering why my MPU usage was so low.. maybe Ollama isn't optimized for running it yet?
- wahnfrieden 1y agoMight need to wait on MLX