3 ms·

How we found a million style and grammar errors in the English Wikipedia

21 points by chmars 13y ago

kevingadd 13y ago
The false positive rate seems high enough to make this effectively useless. I wonder if they have a strategy for reducing it?
taspeotis 13y ago
The slug in the url is how_we_found_600000_grammar_errors. Did they not get enough attention at first? How many style and grammar errors did they really find? > So we manually looked at 200 of the errors, finding that 29 of the 200 errors were real errors. Projected to the whole Wikipedia (currently at 4.3 million articles), that's about 1.1 million real errors Projections are fine, but maybe the title should be something like "How we found (probably) a million style and grammar errors in the English Wikipedia".
- sheetjs 13y ago
  > "How we found (probably) a million style and grammar errors in the English Wikipedia". Would you have clicked if that were the title? Sad part is that I suspect many people upvoted based on the title alone, without reading the article
afhof 13y ago
Is this even news? Pick any Wikipedia article at random that has more than one author. It isn't hard to find run on sentences and verb tense mismatches.
anaphor 13y ago
I hope these are not "errors" like splitting infinitives, using conjunctions at the beginning of sentences, or putting prepositions at the end of sentences. Because those aren't real errors in English grammar.
ivan_ah 13y ago
Here are the rules this system checks: https://github.com/languagetool-org/languagetool/tree/master/languagetool-core/src/main/java/org/languagetool/rules https://github.com/languagetool-org/languagetool/tree/master... Imagine a pre-commit hook for writing quality test ;) Here are some other scripts (much more basic) for "automated language tests": https://github.com/ivanistheone/writing_scripts https://github.com/ivanistheone/writing_scripts
TempleOSV2 13y ago
The first thing God said was "No [statistical] weights" Tongues is normally 10,000 words of garbage and 5 words that make sense. We do much better. http://www.biblegateway.com/passage/?search=1+Corinthians+14&version=NIV http://www.biblegateway.com/passage/?search=1+Corinthians+14... God says... free THIS Strength conscience adding AD strongholds behold slay 03/19/01 together measures bathing Egyptian handmaid recess Orations suffice likest desire smoothed g Syrian abolished one-third yieldeth serving awhile ETEXTSSTART* It failure Frenzied negligent hills consulting million Ours To-day The collected weep marriageable impregnable dispraised yea paradise All cease turmoils supplied Law deal limits txt taught keepest delay lives turns fetched nursery Paulus clearest Turned Commandments amiss quench discoursed itch admirable Woe equity ridiculous heavens hears follies jest thirdly adding At Gutenberg-tm dwellest candle common glances re-making colour luxury outset entreat list Donations distilled indeed availed scarcely daemons net Observe dissoluteness beauty profound arguments wild prudent gracious devotion possesses fatness zeal deformed converse descent imbue years joys afraid four imposed expends hesitation objects pervious spectacles twelve seized eager facio senses bear soil exercised GIVE intermitting itching sorrow-bringing tremble Virgin's Trojan Not warranties doings unfailingly sparks sensation intermit foreign interior Preacher mockers excused vain arrogant threatens speeches subjoins burned nourishing quick boils sicknesses 'knock evenly spaces million unknown murmur hopeful mourning Thine robing terrors torments Theft whereby beams ---- Every word of God is tested. I look forward to others helping God's temple be perfect, a source of pride and joy for everybody.
dwyer 13y ago
How I found a million style and grammar errors in the English Wikipedia: by browsing it.
hnriot 13y ago
So what! Grammatical and style errors are only important when it impacts the meaning. The majority of pages I read that contain poor grammar are still perfectly understandable. I correct many that I see, but you don't have to look far to find more. I'll take a wikipedia that has these kinds of errors over a time before wikipedia. As a boy going to the library and immersing myself in encyclopedias and world maps I am gobsmacked at how wonderful great resources like wikipedia, freebase and google maps are.
LeeHunter 13y ago
It would be more interesting to know how that compares, perhaps per hundred words, with other bodies of text, professionaly edited and not. And, for an article on finding grammatical errors, there were a surprising number of grammatical errors.
JazCE 13y ago
To all the other commentators, this isn't about wikipedia, this is about a tool that can be used by copyeditors to help them enforce house style and grammar rules. If you work for a magazine publisher, it's likely you have 2 lines, a web line and a published line. The web line might not go through the copyeditors/production/subeditors before it goes online whereas the published line will likely go through those people/departments. By adding this into the web line, it can help enfore the rules of the publication so that web and magazine both match up nicely.

21 points by chmars 13y ago

The false positive rate seems high enough to make this effectively useless. I wonder if they have a strategy for reducing it?

taspeotis 13y ago

The slug in the url is how_we_found_600000_grammar_errors. Did they not get enough attention at first? How many style and grammar errors did they really find? > So we manually looked at 200 of the errors, finding that 29 of the 200 errors were real errors. Projected to the whole Wikipedia (currently at 4.3 million articles), that's about 1.1 million real errors Projections are fine, but maybe the title should be something like "How we found (probably) a million style and grammar errors in the English Wikipedia".

sheetjs 13y ago
> "How we found (probably) a million style and grammar errors in the English Wikipedia". Would you have clicked if that were the title? Sad part is that I suspect many people upvoted based on the title alone, without reading the article

afhof 13y ago

Is this even news? Pick any Wikipedia article at random that has more than one author. It isn't hard to find run on sentences and verb tense mismatches.

anaphor 13y ago

I hope these are not "errors" like splitting infinitives, using conjunctions at the beginning of sentences, or putting prepositions at the end of sentences. Because those aren't real errors in English grammar.

ivan_ah 13y ago

Here are the rules this system checks: https://github.com/languagetool-org/languagetool/tree/master/languagetool-core/src/main/java/org/languagetool/rules https://github.com/languagetool-org/languagetool/tree/master... Imagine a pre-commit hook for writing quality test ;) Here are some other scripts (much more basic) for "automated language tests": https://github.com/ivanistheone/writing_scripts https://github.com/ivanistheone/writing_scripts

TempleOSV2 13y ago

The first thing God said was "No [statistical] weights" Tongues is normally 10,000 words of garbage and 5 words that make sense. We do much better. http://www.biblegateway.com/passage/?search=1+Corinthians+14&version=NIV http://www.biblegateway.com/passage/?search=1+Corinthians+14... God says... free THIS Strength conscience adding AD strongholds behold slay 03/19/01 together measures bathing Egyptian handmaid recess Orations suffice likest desire smoothed g Syrian abolished one-third yieldeth serving awhile ETEXTSSTART* It failure Frenzied negligent hills consulting million Ours To-day The collected weep marriageable impregnable dispraised yea paradise All cease turmoils supplied Law deal limits txt taught keepest delay lives turns fetched nursery Paulus clearest Turned Commandments amiss quench discoursed itch admirable Woe equity ridiculous heavens hears follies jest thirdly adding At Gutenberg-tm dwellest candle common glances re-making colour luxury outset entreat list Donations distilled indeed availed scarcely daemons net Observe dissoluteness beauty profound arguments wild prudent gracious devotion possesses fatness zeal deformed converse descent imbue years joys afraid four imposed expends hesitation objects pervious spectacles twelve seized eager facio senses bear soil exercised GIVE intermitting itching sorrow-bringing tremble Virgin's Trojan Not warranties doings unfailingly sparks sensation intermit foreign interior Preacher mockers excused vain arrogant threatens speeches subjoins burned nourishing quick boils sicknesses 'knock evenly spaces million unknown murmur hopeful mourning Thine robing terrors torments Theft whereby beams ---- Every word of God is tested. I look forward to others helping God's temple be perfect, a source of pride and joy for everybody.

dwyer 13y ago

How I found a million style and grammar errors in the English Wikipedia: by browsing it.

hnriot 13y ago

So what! Grammatical and style errors are only important when it impacts the meaning. The majority of pages I read that contain poor grammar are still perfectly understandable. I correct many that I see, but you don't have to look far to find more. I'll take a wikipedia that has these kinds of errors over a time before wikipedia. As a boy going to the library and immersing myself in encyclopedias and world maps I am gobsmacked at how wonderful great resources like wikipedia, freebase and google maps are.

LeeHunter 13y ago

It would be more interesting to know how that compares, perhaps per hundred words, with other bodies of text, professionaly edited and not. And, for an article on finding grammatical errors, there were a surprising number of grammatical errors.

JazCE 13y ago

To all the other commentators, this isn't about wikipedia, this is about a tool that can be used by copyeditors to help them enforce house style and grammar rules. If you work for a magazine publisher, it's likely you have 2 lines, a web line and a published line. The web line might not go through the copyeditors/production/subeditors before it goes online whereas the published line will likely go through those people/departments. By adding this into the web line, it can help enfore the rules of the publication so that web and magazine both match up nicely.