4 ms·
I hope they also build a variant dataset that includes high-quality translations and enough material in other languages. Even with the limited non-english conte
by Zuiii 4y ago
I hope they also build a variant dataset that includes high-quality translations and enough material in other languages. Even with the limited non-english content that slipped through LLaMA's trainig dataset, LLaMA still shows strong (albeit useless) evidence that it would have excelled in expressing knowledge coherently in other languages in my testing.
Obviously this would increase training costs substantially, so I understand why more languages are not included in the base dataset.
- crakenzak 4y agohmm, could increasing the amount of data in other languages potentially improve performance within the english portion of the model? from it building deeper understandings of concepts that are found within other languages.