2 ms·

Is anyone actually using these small models for general purpose applications? Or are they just used to fine tune into narrowly useful specialized models? I kee

Is anyone actually using these small models for general purpose applications? Or are they just used to fine tune into narrowly useful specialized models? I keep seeing them pass each other on different benchmarks and leaderboards and I can't help but imagine that so many of them are only good at benchmarks and not much else. I haven't gotten a chance to play with this stuff yet so I have no basis to go off of, but I'd like to hear from anyone who's actually been impressed by any of these small models for general purpose tasks.

blurpesec 3y ago
Mistral-7B is surprisingly decent for general purpose small tasks. The more complex the task or the more specific the knowledge recall, the worse the performance since the smaller the models are - the less breadth they tend to have. But they're very nice for making PoCs on complex systems since they're near free to run.
- nwoli 3y ago
  Have you tried the 0.2 version they released yesterday? Curious if you’re seeing significant improvements
- MyFirstSass 3y ago
  There's even a new version of the Mistral 7b out there today that should be a lot better, v 0.2. The finetunes of 0.1 are already extremely impressive at general tasks. https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.2-GGUF https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.2-GGU...
nwhitehead 3y ago
I'm using the new generation of small models to do semantic search for music lyrics. The first step is giving the model the text and asking, "What is this text about?" Without fine-tuning and only minimal prompt engineering these models can understand most languages, pull out the most relevant phrases, and list major themes and speculative ideas of what the lyrics mean. I'm super impressed with the results. Reading the answers feels like grading ambitious undergraduate student essays (both the good and the bad).
davidkunz 3y ago
Yes, I'm using them in my text editor with my plugin [1] and it's already useful for everyday tasks like summarization, fixing grammar or making text more concise. [1]: https://github.com/David-Kunz/gen.nvim https://github.com/David-Kunz/gen.nvim
osmarks 3y ago
I use the Mistral-7B base model for a few things via fewshot prompts, but not really for anything production, just some fun toys.

blurpesec 3y ago

Mistral-7B is surprisingly decent for general purpose small tasks. The more complex the task or the more specific the knowledge recall, the worse the performance since the smaller the models are - the less breadth they tend to have. But they're very nice for making PoCs on complex systems since they're near free to run.

nwoli 3y ago
Have you tried the 0.2 version they released yesterday? Curious if you’re seeing significant improvements
MyFirstSass 3y ago
There's even a new version of the Mistral 7b out there today that should be a lot better, v 0.2. The finetunes of 0.1 are already extremely impressive at general tasks. https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.2-GGUF https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.2-GGU...

nwhitehead 3y ago

I'm using the new generation of small models to do semantic search for music lyrics. The first step is giving the model the text and asking, "What is this text about?" Without fine-tuning and only minimal prompt engineering these models can understand most languages, pull out the most relevant phrases, and list major themes and speculative ideas of what the lyrics mean. I'm super impressed with the results. Reading the answers feels like grading ambitious undergraduate student essays (both the good and the bad).

davidkunz 3y ago

Yes, I'm using them in my text editor with my plugin [1] and it's already useful for everyday tasks like summarization, fixing grammar or making text more concise. [1]: https://github.com/David-Kunz/gen.nvim https://github.com/David-Kunz/gen.nvim

osmarks 3y ago

I use the Mistral-7B base model for a few things via fewshot prompts, but not really for anything production, just some fun toys.