3 ms·
I've not yet managed to convince myself if fine tuning LLMs works for that kind of example. Have you tried fine tuning GPT3 via the OpenAI APIs for this? It sh
by simonw 4y ago
I've not yet managed to convince myself if fine tuning LLMs works for that kind of example.
Have you tried fine tuning GPT3 via the OpenAI APIs for this? It should only cost a few dollars for that smaller set of examples, and it would at least help demonstrate it it's possible to get the results you want with the current best-in-class language model before you try to run that against a smaller model that you can fit on your own hardware.
- baobabKoodaa 4y ago> Have you tried fine tuning GPT3 via the OpenAI APIs for this I haven't. That's not a bad idea. > it would at least help demonstrate it it's possible to get the results you want with the current best-in-class language model before you try to run that against a smaller model that you can fit on your own hardware The dataset you saw was (mostly) generated with ChatGPT and davinci-002, by using prompt engineering instead of fine tuning. So it's definitely possible to produce good results like this (though no judgment here on the question of prompt engineering vs fine tuning).