3 ms·

Am I going to be able to fine tune GPT-J or GPT-JT on consumer hardware?

deleted 4y ago
[deleted]
simonw 4y ago
When you say "fine tune" here what are you looking to do? The impression I've got is that fine tuning large language models is mostly useful for very simple tasks, such as training a spam or categorization filter. If you're looking to take a model and then e.g. train it on a few thousand additional pages of documentation in order to get it to answer project-specific questions, I've got the impression that fine tuning isn't actually a useful way to achieve that (I'd love to be proven wrong about this). Instead, people are getting good results with "retrieval augmented generation" - where you first run a search (or an embeddings-based semantic search) against your docs to find relevant snippets, then feed them to the large language model as part of a glued together prompt. I wrote about my explorations of this technique here - plenty of other people have written about this too: https://simonwillison.net/2023/Jan/13/semantic-search-answers/ https://simonwillison.net/2023/Jan/13/semantic-search-answer...
- baobabKoodaa 4y ago
  > When you say "fine tune" here what are you looking to do? As an example of fine tuning, I might take a pretrained model and then continue training it with a custom dataset that is tailored to a specific text generation task (not classification). Here is an example of a custom dataset that I might fine tune on: https://github.com/baobabKoodaa/future/blob/8d2ae91e6a6f00c7c47ac7e8dc24fb9271b13a49/public/index.html#L487 https://github.com/baobabKoodaa/future/blob/8d2ae91e6a6f00c7... I would like the LLM to generate fictional text in the same style as the fine tuned dataset.
  - simonw 4y ago
    I've not yet managed to convince myself if fine tuning LLMs works for that kind of example. Have you tried fine tuning GPT3 via the OpenAI APIs for this? It should only cost a few dollars for that smaller set of examples, and it would at least help demonstrate it it's possible to get the results you want with the current best-in-class language model before you try to run that against a smaller model that you can fit on your own hardware.
    - baobabKoodaa 4y ago
      > Have you tried fine tuning GPT3 via the OpenAI APIs for this I haven't. That's not a bad idea. > it would at least help demonstrate it it's possible to get the results you want with the current best-in-class language model before you try to run that against a smaller model that you can fit on your own hardware The dataset you saw was (mostly) generated with ChatGPT and davinci-002, by using prompt engineering instead of fine tuning. So it's definitely possible to produce good results like this (though no judgment here on the question of prompt engineering vs fine tuning).
  - simonw 4y ago
    Wow, this is really fun: I pasted your prompt from https://raw.githubusercontent.com/baobabKoodaa/future/8d2ae91e6a6f00c7c47ac7e8dc24fb9271b13a49/prompt-instructions.txt https://raw.githubusercontent.com/baobabKoodaa/future/8d2ae9... into ChatGPT and now it's acting exactly like I'm stuck in a weird future.
    - baobabKoodaa 4y ago
      Glad to hear you like it! I should probably note that the way I used it was by appending a few selected examples to the prompt. Having some examples improves the quality a lot. (I'm on mobile now so can't easily link but you can find the prompt examples starting from server.js)
- bzmrgonz 4y ago
  This is very interesting, I think what the world needs, is the inference/deductive reasoning/critical thinking kung-fu in an AI, which can be trained on "bespoke" body of knowledge/datalake.

deleted 4y ago

[deleted]

When you say "fine tune" here what are you looking to do? The impression I've got is that fine tuning large language models is mostly useful for very simple tasks, such as training a spam or categorization filter. If you're looking to take a model and then e.g. train it on a few thousand additional pages of documentation in order to get it to answer project-specific questions, I've got the impression that fine tuning isn't actually a useful way to achieve that (I'd love to be proven wrong about this). Instead, people are getting good results with "retrieval augmented generation" - where you first run a search (or an embeddings-based semantic search) against your docs to find relevant snippets, then feed them to the large language model as part of a glued together prompt. I wrote about my explorations of this technique here - plenty of other people have written about this too: https://simonwillison.net/2023/Jan/13/semantic-search-answers/ https://simonwillison.net/2023/Jan/13/semantic-search-answer...

baobabKoodaa 4y ago
> When you say "fine tune" here what are you looking to do? As an example of fine tuning, I might take a pretrained model and then continue training it with a custom dataset that is tailored to a specific text generation task (not classification). Here is an example of a custom dataset that I might fine tune on: https://github.com/baobabKoodaa/future/blob/8d2ae91e6a6f00c7c47ac7e8dc24fb9271b13a49/public/index.html#L487 https://github.com/baobabKoodaa/future/blob/8d2ae91e6a6f00c7... I would like the LLM to generate fictional text in the same style as the fine tuned dataset.
- simonw 4y ago
  I've not yet managed to convince myself if fine tuning LLMs works for that kind of example. Have you tried fine tuning GPT3 via the OpenAI APIs for this? It should only cost a few dollars for that smaller set of examples, and it would at least help demonstrate it it's possible to get the results you want with the current best-in-class language model before you try to run that against a smaller model that you can fit on your own hardware.
  - baobabKoodaa 4y ago
    > Have you tried fine tuning GPT3 via the OpenAI APIs for this I haven't. That's not a bad idea. > it would at least help demonstrate it it's possible to get the results you want with the current best-in-class language model before you try to run that against a smaller model that you can fit on your own hardware The dataset you saw was (mostly) generated with ChatGPT and davinci-002, by using prompt engineering instead of fine tuning. So it's definitely possible to produce good results like this (though no judgment here on the question of prompt engineering vs fine tuning).
- simonw 4y ago
  Wow, this is really fun: I pasted your prompt from https://raw.githubusercontent.com/baobabKoodaa/future/8d2ae91e6a6f00c7c47ac7e8dc24fb9271b13a49/prompt-instructions.txt https://raw.githubusercontent.com/baobabKoodaa/future/8d2ae9... into ChatGPT and now it's acting exactly like I'm stuck in a weird future.
  - baobabKoodaa 4y ago
    Glad to hear you like it! I should probably note that the way I used it was by appending a few selected examples to the prompt. Having some examples improves the quality a lot. (I'm on mobile now so can't easily link but you can find the prompt examples starting from server.js)
bzmrgonz 4y ago
This is very interesting, I think what the world needs, is the inference/deductive reasoning/critical thinking kung-fu in an AI, which can be trained on "bespoke" body of knowledge/datalake.