3 ms·
> There is a reason why we hear less about this idea of smaller expert models, because large strong models to the tasks just as good. Smaller models are cheape
by tyre 2mo ago
> There is a reason why we hear less about this idea of smaller expert models, because large strong models to the tasks just as good.
Smaller models are cheaper, sometimes faster. I agree that the “we’re an LLM fine-tuned for X” hasn’t worked out because you can just train Claude to do X (and Anthropic will), but not burning Opus/Fable tokens on dumb-but-token-heavy tasks is good sense.
As we move from “integrate AI into Y” to “optimize the ROI on Y”, we’ll see more of this.
- kumama 2mo agocastform founder here. the roi optimization makes sense. i think there are lots of usecases for which even a 2% gain in accuracy can be quite useful. off the top of my head - high volume customer support. higher accuracy means fewer escalation, reducing labor costs - fraud detection. catching even one extra fraud attempt could mean a lot in savings - and ofc the classic ads use-case where at scale bps in improvement could mean millions in revenue :)
- simianwords 2mo agoWould a bigger model be able to beat yours if some effort were put into prompt?
- dkersten 2mo agoAt what cost though? “Beating” isn’t enough if it costs 100x or even 10x the amount.
- kumama 2mo agothe bigger model would still cost more :) at the same time, i see prompting as being orthogonal to post-training. i'd imagine post-training a smaller model with a better prompt would make it perform even better
- simianwords 2mo agoyes, can you show me tasks where this is true?