4 ms·
Are there significant usecases for the really small LLMs right now (<10b distills and such)? My impression so far was that the resulting models are unusably st
by myrmidon 9mo ago
Are there significant usecases for the really small LLMs right now (<10b distills and such)?
My impression so far was that the resulting models are unusably stupid, but maybe there are some specific tasks where they still perform acceptably?
- arkmm 9mo agoThey're still very good for finetuned classification, often 10-100x cheaper to run at similar or higher accuracy as a large model - but I think most people just prompt the large model unless they have high volume needs or need to self host.