5 ms·

1. Mistral 2. Llama 2 3. Code Llama 4. Orca Mini 5. Vicuna What can I do with any of these models that won't result in 50% hallucinations/it recommending c

1. Mistral 2. Llama 2 3. Code Llama 4. Orca Mini 5. Vicuna What can I do with any of these models that won't result in 50% hallucinations/it recommending code with APIs that don't exist/it recommending basically regurgitated StackOverflow historical out of date answers (that it was trained on) for libraries that have had their versions/APIs change, etc? Can somebody share one real use case they are using any of these models for?

shironandonon_ 3y ago
Why don’t you wrap it with a verification system (eg: web scraper) and auto-regenerate / jailbreak any things you don’t like?
- MuffinFlavored 3y ago
  Because I pay $20/mo for GPT-4 and don't understand why anybody would run a "less-good" version locally that you can trust less/that has less functionality. That's why I wanted to try to understand, what am I missing about local-toy LLMs. How are they not just noise/nonsense generators?
  - jstarfish 3y ago
    Sometimes you just need to write creative nonsense. Emails, comments, stories, etc. Great for fiction since there are low stakes for errors. They're bad at generative tasks. Don't have it write code or scientific papers from scratch, but you can have it review anything you've written. You can also do summaries, keyword/entity extraction, and the like safely. Any reductive task works pretty well.
  - cellis 3y ago
    The first reason is because GPT-4 is censored for many types of tasks/topics.
  - tvararu 3y ago
    So do I, but on a flight 2 days ago, I forgot the name of a Ruby method, but knew what it does. I tried looking it up in Dash (offline rdocs) but didn’t find it. On a whim, I asked Zephyr 7B (Mistral based) “what’s the name of that Ruby method that does <insert code>” and it gave me 3 different correct ways of doing what I wanted, including the one I couldn’t remember. That was a real “oh wow” moment. So offline situations is the most likely use case for me.
  - ParetoOptimal 3y ago
    > don't understand why anybody would run a "less-good" version locally Privacy. If you use llm's on private documents you don't want others to see, you'll likely prefer local models strongly.
    - frankacter 3y ago
      How would you, in LM Studio or another local option, populate with your own documents? I don't see any option of applying knowledge via pdf/doc/txt.
      - ParetoOptimal 3y ago
        https://github.com/imartinez/privateGPT https://github.com/imartinez/privateGPT https://khoj.dev/ https://khoj.dev/
  - RecycledEle 3y ago
    > I pay $20/mo for GPT-4 and don't understand why anybody would run a "less-good" version locally that you can trust less/that has less functionality. Local might still be there after an online service is no longer available.
RecycledEle 3y ago
> What can I do with any of these models that won't result in 50% hallucinations There are many times when I am searching for a solution to a problem, and I would be perfectly happy with a possible answer I could test that has a 50% chance of being correct. Humans should test the output of all AIs. Humans should test the validity of everything we hear, see, and read.

shironandonon_ 3y ago

Why don’t you wrap it with a verification system (eg: web scraper) and auto-regenerate / jailbreak any things you don’t like?

MuffinFlavored 3y ago
Because I pay $20/mo for GPT-4 and don't understand why anybody would run a "less-good" version locally that you can trust less/that has less functionality. That's why I wanted to try to understand, what am I missing about local-toy LLMs. How are they not just noise/nonsense generators?
- jstarfish 3y ago
  Sometimes you just need to write creative nonsense. Emails, comments, stories, etc. Great for fiction since there are low stakes for errors. They're bad at generative tasks. Don't have it write code or scientific papers from scratch, but you can have it review anything you've written. You can also do summaries, keyword/entity extraction, and the like safely. Any reductive task works pretty well.
- cellis 3y ago
  The first reason is because GPT-4 is censored for many types of tasks/topics.
- tvararu 3y ago
  So do I, but on a flight 2 days ago, I forgot the name of a Ruby method, but knew what it does. I tried looking it up in Dash (offline rdocs) but didn’t find it. On a whim, I asked Zephyr 7B (Mistral based) “what’s the name of that Ruby method that does <insert code>” and it gave me 3 different correct ways of doing what I wanted, including the one I couldn’t remember. That was a real “oh wow” moment. So offline situations is the most likely use case for me.
- ParetoOptimal 3y ago
  > don't understand why anybody would run a "less-good" version locally Privacy. If you use llm's on private documents you don't want others to see, you'll likely prefer local models strongly.
  - frankacter 3y ago
    How would you, in LM Studio or another local option, populate with your own documents? I don't see any option of applying knowledge via pdf/doc/txt.
    - ParetoOptimal 3y ago
      https://github.com/imartinez/privateGPT https://github.com/imartinez/privateGPT https://khoj.dev/ https://khoj.dev/
- RecycledEle 3y ago
  > I pay $20/mo for GPT-4 and don't understand why anybody would run a "less-good" version locally that you can trust less/that has less functionality. Local might still be there after an online service is no longer available.

RecycledEle 3y ago

> What can I do with any of these models that won't result in 50% hallucinations There are many times when I am searching for a solution to a problem, and I would be perfectly happy with a possible answer I could test that has a 50% chance of being correct. Humans should test the output of all AIs. Humans should test the validity of everything we hear, see, and read.