4 ms·
Funny you should mention that, in about a week I'm planning to make public a frequency list collected from a corpus of over 40k Spanish news articles (I've been
by wkrause 9y ago
Funny you should mention that, in about a week I'm planning to make public a frequency list collected from a corpus of over 40k Spanish news articles (I've been working on Langliter for a long time). Since I'm finding the lemma of every word in the corpus and I'm tagging each word with its PoS, it's actually a really valuable list. I plan to update the list quarterly as my corpus grows.
I'm currently only using it to calculate a modified version of the Dale Chall readability score. Figured people could make good use of it and it would be a good way to drive people the the site.
- pugio 9y agoThat would be absolutely amazing. I can't wait for the German addition to the site (current learning project). How much hand tuning does this process require? If you had a corpus of German news articles, for instance, could you just run the software and receive a sorted frequency list?
- wkrause 9y agoJust letting you know that I've uploaded the first frequency list. Just Spanish at the moment, but will be adding more over time. https://langliter.com/blog/frequency_lists/ https://langliter.com/blog/frequency_lists/
- deleted 9y ago[deleted]
- wkrause 9y agoI benefit from the fact that articles are processed and tagged at the time of collection. So getting this list is just a matter of looping over the structured data that make up the articles I send down to clients.