3 ms·
Anecdata, but I did some continued pretraining on a toy LLM using machine-translated data; of the original dataset. Performance improved across all benchmarks;
by dannyw 2y ago
Anecdata, but I did some continued pretraining on a toy LLM using machine-translated data; of the original dataset.
Performance improved across all benchmarks; in English (the original language).
- benmanns 2y agoAm I understanding correctly? You look an English dataset, trained an LLM, machine translated the English dataset to e.g. Spanish, continued training the model, and performance for queries in English improved? That’s really interesting.