2 ms·
It really depends on what dialects of Arabic you are looking at and what your goals are. My starting dataset is very small. I study classical Arabic, so I am on
by tboyd47 8y ago
It really depends on what dialects of Arabic you are looking at and what your goals are. My starting dataset is very small. I study classical Arabic, so I am only looking at fus-ha or MSA (not quite the same but very close), and I only pulled a small subset of the total number of words because this started out as just a toy project. The total number of fus-ha words in use today is probably in the low millions. But if you extend to all dialects of Arabic and all time periods then you may reach half a billion to a billion words. If you go from written Arabic to spoken Arabic, then there's no telling how big the input set is. Practically infinite at that point.
For the purpose of just using the language, the morphological rules are well-understood. One of the most popular dictionaries (Hans Wehr) is arranged not in alphabetical order, but by root. And there are many online lexicons with morphological metadata as listed in other comments here. So you're right, machine learning is not necessary here (but that's not to say it couldn't be done). This is mainly just for fun and learning.
Now, if you were to add all dialects of Arabic, then you may have a use case for ML...