3 ms·

Related: live demo of DeepSeek v4 Flash running on my 128GB MacBook. Italian language with English subs. https://www.youtube.com/watch?v=todMmp6AGCE https://ww

by antirez 5mo ago

Related: live demo of DeepSeek v4 Flash running on my 128GB MacBook. Italian language with English subs. https://www.youtube.com/watch?v=todMmp6AGCE https://www.youtube.com/watch?v=todMmp6AGCE

dust42 5mo ago
For many models the performance of llama.cpp on Mac is 20-40% lower than MLX. Did you try MLX? At least on HF there are MLX 2-bit quants. Unfortunately I have only 64GB, so I can't test it.
- antirez 5mo ago
  I'm not using llama.cpp there, it's my inference engine that is DeepSeek v4 specific. The goal is to optimize it as much as possible.
  - oveja 5mo ago
    That's cool! I knew the name sounded familiar, thank you for SDS!