5 ms·
How do they know GPT-4 received no specialized financial training?
by jebarker 3y ago
How do they know GPT-4 received no specialized financial training?
- CharlesW 3y agoMeaning, they used the same generalized foundation model that all of us have access to, with no special fine-tuning, no retrieval-augmented generation, etc.
- jebarker 3y agoI don't understand your point. To me GPT-4 is not a foundation model, it's been highly tuned for the chat task. Nobody outside of OpenAI knows what that fine-tuning really involved. So it's impossible to say how much finance specific data it was trained on (in pre-training or fine-tuning) or whether finance specific tasks were involved in fine-tuning.
- CharlesW 3y ago> To me GPT-4 is not a foundation model… It is. https://www.adalovelaceinstitute.org/resource/foundation-models-explainer/ https://www.adalovelaceinstitute.org/resource/foundation-mod...
- deleted 3y ago[deleted]
- jebarker 3y agoWhat I was meaning was that ChatGPT is not a foundation model since it's been fine-tuned. Although the definition in the link is sufficiently broad you could choose to include it. I can't tell from the OpenAI docs whether it's possible to access GPT-4 without the ChatGPT fine-tuning. If so, that'd make this result more meaningful. Otherwise, I just don't think you can draw any great conclusions from this.
- doctorpangloss 3y agoThe instruction fine tuning is what manifests knowledge and reasoning.
- rmbyrro 3y agoGPT is general purpose, it's not fine tuned for specific topics. A fine tuned model is tuned to a specific subject.