5 ms·
There is a lot of hype around LLMs, but (BUT!) Mistral well deserves the hype. I use their original 7B model, as well as some derived models, all the time. I ca
by mark_l_watson 3y ago
There is a lot of hype around LLMs, but (BUT!) Mistral well deserves the hype. I use their original 7B model, as well as some derived models, all the time. I can’t wait to see what they release next (which I expect to be a commercial product, although the MoE model set they just released is free).
Another company worthy of some hype is 01.AI which released their Yi-34B model. I have been running Yi locally on my Mac (use “ ollama run yi:34b”) and it is amazing.
Hype away Mistral and 01.AI, hype away…
- jay-barronville 3y agoYou mind sharing what you find so amazing about Yi-34B? I haven’t had a chance to try it.
- mark_l_watson 3y agoI just installed it on my 32B Mac yesterday, first impressions: it does very well reasoning, it does very well answering general common sense world knowledge questions, and so far when it generates Python code, the code works and is well documented. I know this is just subjective, but I have been running a 30B model for a while in my Mac and Yi-34B just feels much better. With 4bit quantization, I can still run Emacs, terminal windows and a web browser with a few tabs without seeing much page faulting. Anyway, please try it and share a second opinion.
- brucethemoose2 3y agoThe 200K finetunes are also quite good at understanding their huge context.
- p1esk 3y agoHow do these small models compare to gpt4 for coding and technical questions? I noticed that gpt3.5 is practically useless to me (either wrong or too generic), while gpt4 provides a decent answer 80% of the time.
- modeless 3y agoThey are not close to GPT-4. Yet. But the rate of improvement is higher than I expected. I think there will be open source models at GPT-4 level that can run on consumer GPUs within a year or two. Possibly requiring some new techniques that haven't been invented yet. The rate of adoption of new techniques that work is incredibly fast. Of course, GPT-5 is expected soon, so there's a moving target. And I can't see myself using GPT-4 much after GPT-5 is available, if it represents a significant improvement. We are quite far from "good enough".
- vitorgrs 3y agoI believe one of the problems that OSS models need to solve, is... dataset. All of them lack a good and large dataset. And this is most noticiable if you ask anything that is not in English-American-ish.
- mattigames 3y agoMaybe it should be an independent model in charge only of converting your question to American English and back, instead of trying to make a single model speak all languages
- jitl 3y agoThere’s a lot of world knowledge that is just not present in an American English corpus. For example knowledge of world cuisine & culture. There’s precious few good English sources on Sichuan cooking.
- vitorgrs 3y agoI don't think this is a good idea. A good model if we are really aiming at anything that resembles AGI (or even a good LLM like GPT4) is a model that have world knowledge. The world is not just English.
- p1esk 3y agoI’m both excited and scared to think about this “significant improvement” over GPT-4. It can make our jobs a lot easier or it can take our jobs.
- dmos62 3y agoHow do you use these models? If you don't mind sharing. I use GPT-4 as an alternative to googling, haven't yet found a reason to switch to something else. I'll for example use it to learn about the history, architecture, cultural context, etc of a place when I'm visiting. I've found it very ergonomic for that.
- teaearlgraycold 3y agoI’ve use lm studio. It’s not reached peak user friendliness, but it’s a nice enough GUI. You’ll need to fiddle with resource allocation settings and select an optimally quantized model for best performance. But you can do all that in the UI.
- coconut08 3y agolm studio is an accessible simple way to use them. that said expecting them to be anywhere near as good as gpt-4 is going to lead to disappointment.
- davidkunz 3y agoI use them in my editor with my plugin https://github.com/David-Kunz/gen.nvim https://github.com/David-Kunz/gen.nvim
- loufe 3y agoIf you want to experiment Kobold.cpp is a great interface and goes a long distance to guarantee backwards compatibility of outdated model formats.
- yodsanklai 3y ago> I use their original 7B model, as well as some derived models, all the time. How does it compare to other models? and with chatgpt in particular?
- valval 3y agoNo comparison to be made.
- brucethemoose2 3y agoI concur, Yi 34B and Mistral 7B are fantastic. But you need to run the top Yi finetunes instead of the vanilla chat model. They are far better. I would recommend Xaboros/Cybertron, or my own merge of several models on huggingface if you want the long context Yi.