5 ms·

LanguageTool – FOSS Style and Grammar Checker for 25 Languages

84 points by jka 5y ago

pella 5y ago
github: https://github.com/languagetool-org/languagetool https://github.com/languagetool-org/languagetool languages: https://dev.languagetool.org/languages https://dev.languagetool.org/languages Improving LanguageTool: https://languagetool.org/dev https://languagetool.org/dev
lockyc 5y ago
I have recently rolled this out for our users. We have found it to be very good, cheaper than Grammerly. Works with most Google Workspace apps, unlike Grammerly. I didn't realise it was open source until now
- scim-knox-twox 5y ago
  And support many more languages than Grammerly.
- sleightofmind 5y ago
  Love your ironic sense of humor -- intentionally misspelling Grammarly in a comment on a style and grammar checker. Too rich!
msla 5y ago
I wonder how many varieties of various languages it will recognize, and how many it will flag as erroneous. For example, is Catalan a variety of Spanish, a language, or erroneous Castilian? How about Gascon vis-à-vis French? I also wonder whether AAVE will be recognized on a level with Indian English, for example.
- sohkamyung 5y ago
  I installed LanguageTool as a browser add-on in Firefox and took a look at its settings. It does have a "Language Variants" setting, where one of the options is "Interpret Catalan as ...", and the options are "Catalan" or "Catalan (Valencian)"
smaudet 5y ago
I find it more interesting the lack of comments than anything else - I'm not sure this software is a good idea, but regardless, nobody uses long form messaging anymore? I.e. if you write anything, you likely already have a solution you are content with, or you use twitter or YouTube comments and largely couldn't care less...
melony 5y ago
It is interesting that the fundamental parts of grammar checking are still driven by rules engines and GOFAI expert systems. Even the gramamarly core is written in common lisp as a GOFAI app. Most of the modern attention/transformer NLPs are used to supplement the front end tasks like dependency parsing.
- solarmist 5y ago
  I'm not sure I believe this represents state of the art in this area. I think they just have put enough hours into writing rules it does a passable job for the biggest languages it supports. If you get enough rules it can look like progress. I looked at the engine for the Japanese parsing and it looks like they are parsing Japanese in fundamentally the same way they are English and it's has been known for around 20 years that is insufficient. Japanese requires tagging be done in order to tokenize it not the reverse as is done with English. Take a look at spaCy if you're interested in what's possible nowadays, but it's not an end user product. Edit: it's not quite that bad they're using GoSen which stopped being used only about 10 years ago.
  - itazula 5y ago
    Yes, I put in Japanese that was nonsense, and it passed with flying colors.
    - solarmist 5y ago
      Yeah, I figured. That's why there are almost no NLP tools on the market that support Asian languages, in particular Japanese, Korean, and somewhat Chinese (Chinese is its own thing because of the large number of engineers it has to throw at problems).
- lewisjoe 5y ago
  I'm part of the team that makes Zoho Writer's grammar checker[1] and we did go with a modern neural-net based NLP technique to detect grammatical errors. It had a huge win in terms of the number of errors it caught, but we could never tell the user what exactly is wrong with that sentence, because.. well, deep learning. So we've taken a hybrid approach where a modern AI flags errors and then run it through a rule engine to try and figure out what exactly is wrong. This way we get the best of both worlds. That lack of explanation is also why Grammarly is sticking with GOFAI systems perhaps. But the future of this problem area is certainly AI augmented with rule engines. [1] - https://www.zoho.com/writer/free-grammar-checker.html https://www.zoho.com/writer/free-grammar-checker.html
scim-knox-twox 5y ago
I'm using LT every day, can't complain. One of my “must have” tools.
blopker 5y ago
It's great, I've been running the self-hosted version for a few months now. I run it using a container[0] on a $10/mo Digital Ocean VPS using Dokku[1], along with other services. The main downside is that it's a bit of a memory hog, possible because it's written in Java? Otherwise, I haven't had to mess with it after the initial set up. I like that I'm not sending everything I write to another 3rd party service I don't control. As for the plugin, it definitely catches more issues than the stock browser spellchecks. The main issue I have with it (maybe someone can point me in the right direction) is that it always tries to autodetect the language. This is fine for longer texts, but often fails on shorter strings like headlines. This leaves my words highlighted red because it thinks I'm writing bad German, or Swedish (which is fair). I haven't been able to figure out how to force it to only use US English. [0]: https://hub.docker.com/r/erikvl87/languagetool https://hub.docker.com/r/erikvl87/languagetool [1]: https://dokku.com/ https://dokku.com/
- jcelerier 5y ago
  I don't understand, why are you running it on a separate server ? You can just install it on your computer and it will likely be faster and work also when you don't have internet (and free). IDK about other OSes but on archlinux it's just a package to install.
xvilka 5y ago
I wish it would have been native tool, not Java. It would be much lighter and easier to use then.
leobg 5y ago
Can anybody tell me what these tools are used for? Where’s the need? Who are the people using them? Thanks.
- mhitza 5y ago
  I use a similar tool (Grammarly) for all business related communication, as a non-native English speaker, to avoid simple grammar mistakes. For example, I often make the were/where, it's/its typos because, due to my native language, I think of words phonetically while I type them out, even if English isn't. It also highlights very long sentences, which reminds me to rephrase and break them up for legibility.

84 points by jka 5y ago

github: https://github.com/languagetool-org/languagetool https://github.com/languagetool-org/languagetool languages: https://dev.languagetool.org/languages https://dev.languagetool.org/languages Improving LanguageTool: https://languagetool.org/dev https://languagetool.org/dev

lockyc 5y ago

I have recently rolled this out for our users. We have found it to be very good, cheaper than Grammerly. Works with most Google Workspace apps, unlike Grammerly. I didn't realise it was open source until now

scim-knox-twox 5y ago
And support many more languages than Grammerly.
sleightofmind 5y ago
Love your ironic sense of humor -- intentionally misspelling Grammarly in a comment on a style and grammar checker. Too rich!

msla 5y ago

I wonder how many varieties of various languages it will recognize, and how many it will flag as erroneous. For example, is Catalan a variety of Spanish, a language, or erroneous Castilian? How about Gascon vis-à-vis French? I also wonder whether AAVE will be recognized on a level with Indian English, for example.

sohkamyung 5y ago
I installed LanguageTool as a browser add-on in Firefox and took a look at its settings. It does have a "Language Variants" setting, where one of the options is "Interpret Catalan as ...", and the options are "Catalan" or "Catalan (Valencian)"

smaudet 5y ago

I find it more interesting the lack of comments than anything else - I'm not sure this software is a good idea, but regardless, nobody uses long form messaging anymore? I.e. if you write anything, you likely already have a solution you are content with, or you use twitter or YouTube comments and largely couldn't care less...

melony 5y ago

It is interesting that the fundamental parts of grammar checking are still driven by rules engines and GOFAI expert systems. Even the gramamarly core is written in common lisp as a GOFAI app. Most of the modern attention/transformer NLPs are used to supplement the front end tasks like dependency parsing.

solarmist 5y ago
I'm not sure I believe this represents state of the art in this area. I think they just have put enough hours into writing rules it does a passable job for the biggest languages it supports. If you get enough rules it can look like progress. I looked at the engine for the Japanese parsing and it looks like they are parsing Japanese in fundamentally the same way they are English and it's has been known for around 20 years that is insufficient. Japanese requires tagging be done in order to tokenize it not the reverse as is done with English. Take a look at spaCy if you're interested in what's possible nowadays, but it's not an end user product. Edit: it's not quite that bad they're using GoSen which stopped being used only about 10 years ago.
- itazula 5y ago
  Yes, I put in Japanese that was nonsense, and it passed with flying colors.
  - solarmist 5y ago
    Yeah, I figured. That's why there are almost no NLP tools on the market that support Asian languages, in particular Japanese, Korean, and somewhat Chinese (Chinese is its own thing because of the large number of engineers it has to throw at problems).
lewisjoe 5y ago
I'm part of the team that makes Zoho Writer's grammar checker[1] and we did go with a modern neural-net based NLP technique to detect grammatical errors. It had a huge win in terms of the number of errors it caught, but we could never tell the user what exactly is wrong with that sentence, because.. well, deep learning. So we've taken a hybrid approach where a modern AI flags errors and then run it through a rule engine to try and figure out what exactly is wrong. This way we get the best of both worlds. That lack of explanation is also why Grammarly is sticking with GOFAI systems perhaps. But the future of this problem area is certainly AI augmented with rule engines. [1] - https://www.zoho.com/writer/free-grammar-checker.html https://www.zoho.com/writer/free-grammar-checker.html

scim-knox-twox 5y ago

I'm using LT every day, can't complain. One of my “must have” tools.

blopker 5y ago

It's great, I've been running the self-hosted version for a few months now. I run it using a container[0] on a $10/mo Digital Ocean VPS using Dokku[1], along with other services. The main downside is that it's a bit of a memory hog, possible because it's written in Java? Otherwise, I haven't had to mess with it after the initial set up. I like that I'm not sending everything I write to another 3rd party service I don't control. As for the plugin, it definitely catches more issues than the stock browser spellchecks. The main issue I have with it (maybe someone can point me in the right direction) is that it always tries to autodetect the language. This is fine for longer texts, but often fails on shorter strings like headlines. This leaves my words highlighted red because it thinks I'm writing bad German, or Swedish (which is fair). I haven't been able to figure out how to force it to only use US English. [0]: https://hub.docker.com/r/erikvl87/languagetool https://hub.docker.com/r/erikvl87/languagetool [1]: https://dokku.com/ https://dokku.com/

jcelerier 5y ago
I don't understand, why are you running it on a separate server ? You can just install it on your computer and it will likely be faster and work also when you don't have internet (and free). IDK about other OSes but on archlinux it's just a package to install.

xvilka 5y ago

I wish it would have been native tool, not Java. It would be much lighter and easier to use then.

leobg 5y ago

Can anybody tell me what these tools are used for? Where’s the need? Who are the people using them? Thanks.

mhitza 5y ago
I use a similar tool (Grammarly) for all business related communication, as a non-native English speaker, to avoid simple grammar mistakes. For example, I often make the were/where, it's/its typos because, due to my native language, I think of words phonetically while I type them out, even if English isn't. It also highlights very long sentences, which reminds me to rephrase and break them up for legibility.