4 ms·

Can you elaborate on how they are not like ChatGPT? I was looking into GPT-JT (built on top of GPT J you mentioned). If I spend the time to actually finetune an

Can you elaborate on how they are not like ChatGPT? I was looking into GPT-JT (built on top of GPT J you mentioned). If I spend the time to actually finetune and run inference with one of these models, am I likely going to be disappointed at the results?

speedgoose 4y ago
It depends what you use it for. If it’s to classify text and you can fine tune it, it’s probably good enough. For following instructions, ChatGPT is a lot better but GPT J did relatively well if given enough examples on simple tasks. For a chatbot, it’s not really useable.
smoldesu 4y ago
Maybe? GPT-J is closer to the AI-Dungeon model of intelligence. It's able to fill in the blank after what you type, but it's hysterically bad at answering precise questions (to the point that I had to nerf it for fun to see how stupid the output could get). It will handle basic natural language and context clues just fine. It's just not very fast, and the generations probably won't be as thorough as ChatGPT.
- baobabKoodaa 4y ago
  Am I going to be able to fine tune GPT-J or GPT-JT on consumer hardware?
  - deleted 4y ago
    [deleted]
  - simonw 4y ago
    When you say "fine tune" here what are you looking to do? The impression I've got is that fine tuning large language models is mostly useful for very simple tasks, such as training a spam or categorization filter. If you're looking to take a model and then e.g. train it on a few thousand additional pages of documentation in order to get it to answer project-specific questions, I've got the impression that fine tuning isn't actually a useful way to achieve that (I'd love to be proven wrong about this). Instead, people are getting good results with "retrieval augmented generation" - where you first run a search (or an embeddings-based semantic search) against your docs to find relevant snippets, then feed them to the large language model as part of a glued together prompt. I wrote about my explorations of this technique here - plenty of other people have written about this too: https://simonwillison.net/2023/Jan/13/semantic-search-answers/ https://simonwillison.net/2023/Jan/13/semantic-search-answer...
    - baobabKoodaa 4y ago
      > When you say "fine tune" here what are you looking to do? As an example of fine tuning, I might take a pretrained model and then continue training it with a custom dataset that is tailored to a specific text generation task (not classification). Here is an example of a custom dataset that I might fine tune on: https://github.com/baobabKoodaa/future/blob/8d2ae91e6a6f00c7c47ac7e8dc24fb9271b13a49/public/index.html#L487 https://github.com/baobabKoodaa/future/blob/8d2ae91e6a6f00c7... I would like the LLM to generate fictional text in the same style as the fine tuned dataset.
      - simonw 4y ago
        I've not yet managed to convince myself if fine tuning LLMs works for that kind of example. Have you tried fine tuning GPT3 via the OpenAI APIs for this? It should only cost a few dollars for that smaller set of examples, and it would at least help demonstrate it it's possible to get the results you want with the current best-in-class language model before you try to run that against a smaller model that you can fit on your own hardware.
        baobabKoodaa 4y ago
        > Have you tried fine tuning GPT3 via the OpenAI APIs for this I haven't. That's not a bad idea. > it would at least help demonstrate it it's possible to get the results you want with the current best-in-class language model before you try to run that against a smaller model that you can fit on your own hardware The dataset you saw was (mostly) generated with ChatGPT and davinci-002, by using prompt engineering instead of fine tuning. So it's definitely possible to produce good results like this (though no judgment here on the question of prompt engineering vs fine tuning).
      - simonw 4y ago
        Wow, this is really fun: I pasted your prompt from https://raw.githubusercontent.com/baobabKoodaa/future/8d2ae91e6a6f00c7c47ac7e8dc24fb9271b13a49/prompt-instructions.txt https://raw.githubusercontent.com/baobabKoodaa/future/8d2ae9... into ChatGPT and now it's acting exactly like I'm stuck in a weird future.

speedgoose 4y ago

It depends what you use it for. If it’s to classify text and you can fine tune it, it’s probably good enough. For following instructions, ChatGPT is a lot better but GPT J did relatively well if given enough examples on simple tasks. For a chatbot, it’s not really useable.

smoldesu 4y ago

Maybe? GPT-J is closer to the AI-Dungeon model of intelligence. It's able to fill in the blank after what you type, but it's hysterically bad at answering precise questions (to the point that I had to nerf it for fun to see how stupid the output could get). It will handle basic natural language and context clues just fine. It's just not very fast, and the generations probably won't be as thorough as ChatGPT.

baobabKoodaa 4y ago
Am I going to be able to fine tune GPT-J or GPT-JT on consumer hardware?
- deleted 4y ago
  [deleted]
- simonw 4y ago
  When you say "fine tune" here what are you looking to do? The impression I've got is that fine tuning large language models is mostly useful for very simple tasks, such as training a spam or categorization filter. If you're looking to take a model and then e.g. train it on a few thousand additional pages of documentation in order to get it to answer project-specific questions, I've got the impression that fine tuning isn't actually a useful way to achieve that (I'd love to be proven wrong about this). Instead, people are getting good results with "retrieval augmented generation" - where you first run a search (or an embeddings-based semantic search) against your docs to find relevant snippets, then feed them to the large language model as part of a glued together prompt. I wrote about my explorations of this technique here - plenty of other people have written about this too: https://simonwillison.net/2023/Jan/13/semantic-search-answers/ https://simonwillison.net/2023/Jan/13/semantic-search-answer...
  - baobabKoodaa 4y ago
    > When you say "fine tune" here what are you looking to do? As an example of fine tuning, I might take a pretrained model and then continue training it with a custom dataset that is tailored to a specific text generation task (not classification). Here is an example of a custom dataset that I might fine tune on: https://github.com/baobabKoodaa/future/blob/8d2ae91e6a6f00c7c47ac7e8dc24fb9271b13a49/public/index.html#L487 https://github.com/baobabKoodaa/future/blob/8d2ae91e6a6f00c7... I would like the LLM to generate fictional text in the same style as the fine tuned dataset.
    - simonw 4y ago
      I've not yet managed to convince myself if fine tuning LLMs works for that kind of example. Have you tried fine tuning GPT3 via the OpenAI APIs for this? It should only cost a few dollars for that smaller set of examples, and it would at least help demonstrate it it's possible to get the results you want with the current best-in-class language model before you try to run that against a smaller model that you can fit on your own hardware.
      - baobabKoodaa 4y ago
        > Have you tried fine tuning GPT3 via the OpenAI APIs for this I haven't. That's not a bad idea. > it would at least help demonstrate it it's possible to get the results you want with the current best-in-class language model before you try to run that against a smaller model that you can fit on your own hardware The dataset you saw was (mostly) generated with ChatGPT and davinci-002, by using prompt engineering instead of fine tuning. So it's definitely possible to produce good results like this (though no judgment here on the question of prompt engineering vs fine tuning).
    - simonw 4y ago
      Wow, this is really fun: I pasted your prompt from https://raw.githubusercontent.com/baobabKoodaa/future/8d2ae91e6a6f00c7c47ac7e8dc24fb9271b13a49/prompt-instructions.txt https://raw.githubusercontent.com/baobabKoodaa/future/8d2ae9... into ChatGPT and now it's acting exactly like I'm stuck in a weird future.