3 ms·
This makes me wonder about training an LLM on one language and then fine tuning it for another. If you train over only, say, JavaScript, and then finetune for C
by foota 3y ago
This makes me wonder about training an LLM on one language and then fine tuning it for another. If you train over only, say, JavaScript, and then finetune for C, I imagine it will be quite bad at writing safe code, even if it makes the code look like C, because it didn't have to learn about freeing and such.
Similarly, would it pick up patterns from one language and keep then in the other? Maybe an LLM trained on Kotlin would be more likely to write functional code finetuned.
- fragmede 3y agoGiven that dataset anomalies can result in LLM output corruption, I'm not convinced that cross-training like that would even work.