4 ms·

You make it sound as if ChatGPT itself isnt severly limited. Vicuna and newer 13B models are quite close . And the uncensored models have a capability that Chat

by seydor 4y ago

You make it sound as if ChatGPT itself isnt severly limited. Vicuna and newer 13B models are quite close . And the uncensored models have a capability that ChatGPT will never have.

whimsicalism 4y ago
They just aren't quite close. Maybe you guys are asking shallow questions, but I'm asking subject matter questions and this is just not true. It would be great if we could have detailed QA evaluation to show this, but of course then the open source people would train their models on it as a fine-tuning datasaet.
- UncleEntity 4y ago
  Train it on specific subject matter and not waste space on things like “what year did Jack Nicholson beat on that dude’s car with a golf club?” I mean… the horror.
  - whimsicalism 4y ago
    That's just not how these models work.
    - taneq 4y ago
      You’ll have to forgive them, they’re probably trained mostly on HN posts. ;)
avereveard 4y ago
What are the newer models? I am testing them in batteries across complex tasks, so far vicuña is the most flexible but they all choke on reflective instructions (I. E. Knowledge to extract is not in the model but in the user text)
- ExxKA 4y ago
  I very interested in this usecase - where should I go on the interwebs?
  - avereveard 4y ago
    original paper: https://arxiv.org/abs/2205.00445 https://arxiv.org/abs/2205.00445 langchain agents are a good starting implementation. you can build your own prompt and get the ai to work by iself hallucinating tools, which may be cheaper to test out than going back and forth with an agent manager. not as accurate, but you can still extract useful work, i.e. https://i.imgur.com/AE4R3dR.png https://i.imgur.com/AE4R3dR.png (gpt-35-turbo is traditionally failing this task completely, prompt get it to work at it) these prompt all require the model to work off data within the prompt within the first shoot. model require a degree to introspection for that to work.
causi 4y ago
The way OpenAI has set their models up for prompting and follow-up is where their real advantage is. It's very hard to take a model you can run locally and just go "Hey, write me a powershell script to convert all these files into this format. Split them up so they're no more than an hour each. Ok, change it to also denoise them."
- thewataccount 4y ago
  Are you talking about the RLHF? That's where Vicuna/Alpaca and other finetuning help a lot. There's also already tools for conversation flows (which just means you prepend the conversation history to the prompt). I'm not saying the performance is nearly as good, but the actual workflow does already exist and is massively improving. The interesting part to me is that this finetuning can be done in a couple few hours on a consumer gpu (4090).

by seydor 4y ago

You make it sound as if ChatGPT itself isnt severly limited. Vicuna and newer 13B models are quite close . And the uncensored models have a capability that ChatGPT will never have.

whimsicalism 4y ago

They just aren't quite close. Maybe you guys are asking shallow questions, but I'm asking subject matter questions and this is just not true. It would be great if we could have detailed QA evaluation to show this, but of course then the open source people would train their models on it as a fine-tuning datasaet.

UncleEntity 4y ago
Train it on specific subject matter and not waste space on things like “what year did Jack Nicholson beat on that dude’s car with a golf club?” I mean… the horror.
- whimsicalism 4y ago
  That's just not how these models work.
  - taneq 4y ago
    You’ll have to forgive them, they’re probably trained mostly on HN posts. ;)

avereveard 4y ago

What are the newer models? I am testing them in batteries across complex tasks, so far vicuña is the most flexible but they all choke on reflective instructions (I. E. Knowledge to extract is not in the model but in the user text)

ExxKA 4y ago
I very interested in this usecase - where should I go on the interwebs?
- avereveard 4y ago
  original paper: https://arxiv.org/abs/2205.00445 https://arxiv.org/abs/2205.00445 langchain agents are a good starting implementation. you can build your own prompt and get the ai to work by iself hallucinating tools, which may be cheaper to test out than going back and forth with an agent manager. not as accurate, but you can still extract useful work, i.e. https://i.imgur.com/AE4R3dR.png https://i.imgur.com/AE4R3dR.png (gpt-35-turbo is traditionally failing this task completely, prompt get it to work at it) these prompt all require the model to work off data within the prompt within the first shoot. model require a degree to introspection for that to work.

causi 4y ago

The way OpenAI has set their models up for prompting and follow-up is where their real advantage is. It's very hard to take a model you can run locally and just go "Hey, write me a powershell script to convert all these files into this format. Split them up so they're no more than an hour each. Ok, change it to also denoise them."

thewataccount 4y ago
Are you talking about the RLHF? That's where Vicuna/Alpaca and other finetuning help a lot. There's also already tools for conversation flows (which just means you prepend the conversation history to the prompt). I'm not saying the performance is nearly as good, but the actual workflow does already exist and is massively improving. The interesting part to me is that this finetuning can be done in a couple few hours on a consumer gpu (4090).