3 ms·
https://github.com/jundot/omlx https://github.com/jundot/omlx note: 27b is going to be slow; use the 35b MoE if you want decent token/sec speed.
by jonaustin 5mo ago
https://github.com/jundot/omlx https://github.com/jundot/omlx
note: 27b is going to be slow; use the 35b MoE if you want decent token/sec speed.
- dexterlagan 5mo agoMany of us tested 27B and 35B side by side, and the dense model is significantly smarter. It indeed is slower, but 35B makes a lot of mistakes 27B doesn't.