4 ms·
I'm actually confused by the post - indeed why choosing the 27B dense and the 35B MoE models? Given the structure of the post I honestly have the impression tha
by pizza234 3mo ago
I'm actually confused by the post - indeed why choosing the 27B dense and the 35B MoE models? Given the structure of the post I honestly have the impression that the Spark was purchased out of curiosity more than for a specific purpose (LLMs).
- biddit 3mo agoAh yeah, your reaction makes sense. In the circles I run in, there is a lot of hype around Sparks for inference, so my gut reaction is to respond with this type of warning. I did not intend to imply that the post author was advocating that they're great for inference, as they're obviously not.
- InTheArena 3mo agoThe 27b dense is _remarkably_ better at coding tasks. Especially at larger quanitizations (Q4 is pretty crap).