5 ms·

What kind of data or data formatting do you need to fine-tune GPT-4o? I'd love to throw a bunch of documentation at it and let it learn, but I don't have the r

• simonw 2y ago
  Fine-tuning generally isn't an effective way to add extra knowledge from things documentation - my understanding is that the vast amounts of knowledge in the original training data tend to overwhelm any extra knowledge you try to add by fine-tuning. OpenAI's documentation has good examples of how the data should be formatted (and when it's appropriate to fine-tune): https://platform.openai.com/docs/guides/fine-tuning/fine-tuning-examples https://platform.openai.com/docs/guides/fine-tuning/fine-tun...
  • qudat 2y ago
    does the same result apply to LoRA?
    • qeternity 2y ago
      Fine tuning is most often performed by training a LoRA. It's almost certainly what OAI are doing, as they can inference many lightweight LoRA in parallel atop the same foundation model.
• floam 2y ago
  https://platform.openai.com/docs/guides/fine-tuning https://platform.openai.com/docs/guides/fine-tuning