3 ms·
How do we rate whether the smaller models are any good? How many questions do we need to ask it to know that it can be trusted and we didn't waste our time on i
by amelius 2y ago
How do we rate whether the smaller models are any good? How many questions do we need to ask it to know that it can be trusted and we didn't waste our time on it?
- mkl 2y agoYou should never completely trust any LLM. They all get things wrong, make things up, and have blind spots. They're any good if they help you for some of your particular uses (but may still fail badly for other uses).
- amelius 2y agoI think you didn't understand my question and maybe I phrased it poorly. The problem is not whether we should trust any deep learning model (the answer is indeed no). But the question is how we can find out if a model is any good before investing our time into that model. Each bad reply we get has a price, because it wastes our time. So, how can we compare models objectively without having to try them out ourselves first?
- abdullahkhalids 2y agoThere are leaderboards [1] that can provide a rough estimate of the relative capabilities of different models. [1] https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboar...