3 ms·
Interesting. I'd have guessed there would be meaningful opex benefits to serving smaller models.
by greyskull 4mo ago
Interesting. I'd have guessed there would be meaningful opex benefits to serving smaller models.
- deleted 4mo ago[deleted]
- mediaman 4mo agoWhat I've heard is that much of the model "intelligence" is a commingled bucket: although you can specialize specific knowledge somewhat, it's hard to specialize advanced reasoning to specific domains because so much of reasoning is a generalized capability that is not unique to, say, coding. It turns out coding has to do with a lot of the same reasoning needed in math or in legal analysis, even if the grammatical expression is different. This is less true of lower intelligence tasks. Classification requires a lot less reasoning capacity and so can be much smaller and more specialized.