5 ms·
I find 7B models to be too stupid. They often respond with nonsense or fail to follow instructions.
by FooBarWidget 3y ago
I find 7B models to be too stupid. They often respond with nonsense or fail to follow instructions.
- cooper_ganglia 3y agoEven 65B models approach a level of being almost usable, but still fall short in my personal experience.
- cloudking 3y agoThis is why I'm not understanding the excitement around open source models, they pale in comparison to GPT-4 quality, so I have no use for them until we have something comparable.
- lettergram 3y agoWe utilize a LLama that frankly competes directly with gpt-4, to the point I can't tell which model we are using (we randomly switch between providers to ensure a robust backend). I think it likely depends on the use case, but many llama models can be returned and there are quite literally thousands of free versions available.
- WhrRTheBaboons 3y agoWhat is your use case?
- lettergram 3y agohttps://ipcopilot.ai https://ipcopilot.ai
- GaggiX 3y agoEven like OpenChat-3.5? (Probably the best 7B model out there) Demo: https://openchat.team/ https://openchat.team/ HuggingFace: https://huggingface.co/openchat/openchat_3.5 https://huggingface.co/openchat/openchat_3.5 On the LLM arena (blinded comparisons), it's the third best non-proprietary model: https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboar...
- sorokod 3y agoWhat is the sum of odd numbers in this set: 4, 7, 12, 1, 3 The sum of odd numbers in the given set is 4 + 7 + 1 = 12. Therefore, the answer is 12.
- all2 3y agoTechnically 3 is even. \s
- voiceblue 3y agoYou're using an LLM as if you don't understand how or why it works. Math is not what it's for. Here, look at what ChatGPT 4 does (for a case that hasn't been fed to it by a hundred other users yet): What is the sum of odd numbers in this set: 12345654321, 123456543212, 123456543213, 12345654324? Output only the sum, no code. The sum of the odd numbers in the set {12345654321, 123456543212, 123456543213, 12345654324} is 246913086434. What does that prove? Only that I am using an LLM poorly and/or do not understand what it is. Using OpenChat-3.5 for what LLMs are actually good at (e.g. asking it for shell commands to perform certain operations, getting some general information about a topic) seems to work surprisingly well for a 7B model.
- sorokod 3y agoNot sure what you mean by 'using'. The prompt is from one of the benchmarks used in scoring LLM models - don't recall which. Here is another from BBH [1]: >>> (-1 + 2 + 9 * 5) = ... (-1 + 2 + 9 * 5) = 43 [1] https://github.com/suzgunmirac/BIG-Bench-Hard/tree/main https://github.com/suzgunmirac/BIG-Bench-Hard/tree/main
- GaggiX 3y agoThis is even worse than the other, just straight up use a calculator.
- freilanzer 3y ago