4 ms·
You can have small languages, sure, but then you run into awkward extended clarifying clauses. The thing that makes languages difficult is that almost all vocab
by cjohnson318 2y ago
You can have small languages, sure, but then you run into awkward extended clarifying clauses. The thing that makes languages difficult is that almost all vocabulary is sparse. The "Top N" words in a language are always pronouns, prepositions, articles, and the conjugations of the top 12 or so verbs: to be, to have, to do, to go, to come, to say, to give, etc. This is the reason that "Top N Words of Language X" and "Learn the Top 50% of Words in Language X" listicles/videos are always disappointing.
- nine_k 2y agoBut they seem to use much wider grammars, because their (synthetic) dataset is a bunch of coherent stories at the level of 3-4 y.o. children. I would consider the "Simple English Wikipedia" the next training set / benchmark.
- Pikamander2 2y agoThere's an oddly relevant skit of this concept in the American version of The Office: https://www.youtube.com/watch?v=_K-L9uhsBLM https://www.youtube.com/watch?v=_K-L9uhsBLM