3 ms·
Idiot question: if I have access to sentence-by-sentence professionally-translated text of foreign-language-to-English in gigantic quantities, and I fed the or
by handelaar 3y ago
Idiot question: if I have access to sentence-by-sentence professionally-translated text of foreign-language-to-English in gigantic quantities, and I fed the originals as prompts and the translations as completions...
... would I be likely to get anything useful if I then fed it new prompts in a similar style? Or would it just generate gibberish?
- seanthemon 3y agoIndeed, it sounds like you have what's called fine tuned data (given an input, here's the output), there's loads of info both here on HN about fine tuning and on youtube's huggingface channels Note if you have sufficient data, look into existing models on huggingface, you may find a smaller, faster and more open (licencing-wise) model that you can fine tune to get the results you want - Llama is hot, but not a catch-all for all tasks (as no model should be) Happy inferring!
- zakki 3y agoHow to know if the data is sufficient?
- seanthemon 3y agoIt's more about quality vs sufficiency - you can have a relatively small but accurate and wide ranging dataset, this is better than an inaccurate huge dataset
- nl 3y agoIf you have that much data you can build your own model that can be much smaller and faster. A simple version is a beginner tutorial: https://pytorch.org/tutorials/beginner/translation_transformer.html https://pytorch.org/tutorials/beginner/translation_transform...