6 ms·

Why don’t you wrap it with a verification system (eg: web scraper) and auto-regenerate / jailbreak any things you don’t like?

MuffinFlavored 3y ago
Because I pay $20/mo for GPT-4 and don't understand why anybody would run a "less-good" version locally that you can trust less/that has less functionality. That's why I wanted to try to understand, what am I missing about local-toy LLMs. How are they not just noise/nonsense generators?
- jstarfish 3y ago
  Sometimes you just need to write creative nonsense. Emails, comments, stories, etc. Great for fiction since there are low stakes for errors. They're bad at generative tasks. Don't have it write code or scientific papers from scratch, but you can have it review anything you've written. You can also do summaries, keyword/entity extraction, and the like safely. Any reductive task works pretty well.
- cellis 3y ago
  The first reason is because GPT-4 is censored for many types of tasks/topics.
- tvararu 3y ago
  So do I, but on a flight 2 days ago, I forgot the name of a Ruby method, but knew what it does. I tried looking it up in Dash (offline rdocs) but didn’t find it. On a whim, I asked Zephyr 7B (Mistral based) “what’s the name of that Ruby method that does <insert code>” and it gave me 3 different correct ways of doing what I wanted, including the one I couldn’t remember. That was a real “oh wow” moment. So offline situations is the most likely use case for me.
- ParetoOptimal 3y ago
  > don't understand why anybody would run a "less-good" version locally Privacy. If you use llm's on private documents you don't want others to see, you'll likely prefer local models strongly.
  - frankacter 3y ago
    How would you, in LM Studio or another local option, populate with your own documents? I don't see any option of applying knowledge via pdf/doc/txt.
    - ParetoOptimal 3y ago
      https://github.com/imartinez/privateGPT https://github.com/imartinez/privateGPT https://khoj.dev/ https://khoj.dev/
- RecycledEle 3y ago
  > I pay $20/mo for GPT-4 and don't understand why anybody would run a "less-good" version locally that you can trust less/that has less functionality. Local might still be there after an online service is no longer available.

MuffinFlavored 3y ago

Because I pay $20/mo for GPT-4 and don't understand why anybody would run a "less-good" version locally that you can trust less/that has less functionality. That's why I wanted to try to understand, what am I missing about local-toy LLMs. How are they not just noise/nonsense generators?

jstarfish 3y ago
Sometimes you just need to write creative nonsense. Emails, comments, stories, etc. Great for fiction since there are low stakes for errors. They're bad at generative tasks. Don't have it write code or scientific papers from scratch, but you can have it review anything you've written. You can also do summaries, keyword/entity extraction, and the like safely. Any reductive task works pretty well.
cellis 3y ago
The first reason is because GPT-4 is censored for many types of tasks/topics.
tvararu 3y ago
So do I, but on a flight 2 days ago, I forgot the name of a Ruby method, but knew what it does. I tried looking it up in Dash (offline rdocs) but didn’t find it. On a whim, I asked Zephyr 7B (Mistral based) “what’s the name of that Ruby method that does <insert code>” and it gave me 3 different correct ways of doing what I wanted, including the one I couldn’t remember. That was a real “oh wow” moment. So offline situations is the most likely use case for me.
ParetoOptimal 3y ago
> don't understand why anybody would run a "less-good" version locally Privacy. If you use llm's on private documents you don't want others to see, you'll likely prefer local models strongly.
- frankacter 3y ago
  How would you, in LM Studio or another local option, populate with your own documents? I don't see any option of applying knowledge via pdf/doc/txt.
  - ParetoOptimal 3y ago
    https://github.com/imartinez/privateGPT https://github.com/imartinez/privateGPT https://khoj.dev/ https://khoj.dev/
RecycledEle 3y ago
> I pay $20/mo for GPT-4 and don't understand why anybody would run a "less-good" version locally that you can trust less/that has less functionality. Local might still be there after an online service is no longer available.