3 ms·
Another dumb question: is it possible to "fine-tune" this model with a "corpus" of specialized data/jargon in another language ? I tried to do this with OpenAI
by huqedato 4y ago
Another dumb question: is it possible to "fine-tune" this model with a "corpus" of specialized data/jargon in another language ?
I tried to do this with OpenAI API in order to improve the response on a specialized field (pediatric oncology) in another language (Italian) but the results were mixed (not to say disappointing).
- CGamesPlay 4y agoI imagine the path to doing this, assuming the model already has good Italian, is to train it on all educational materials that go into getting a degree in pediatric oncology in Italian. This would help the model to actually understand the jargon, by tying it together with its baseline knowledge of Italian.
- harisec 4y agoThis is a common misunderstanding; fine-tuning a model does not mean teaching the model new information. Fine-tuning is used to adapt the model to perform better in a specific task or domain using the information it already has, in a specialized way (like a chatbot for question/answer interactions). Training a LLM on new data is extremely expensive so it's not possible.
- huqedato 4y agoOk then. So how to teach the LLM new information/jargon ? I repeat: I tried to do this with GPT (fine-tuning and embeddings) and I got no improvements on jargon and re-training is out of reach, of course.