5 ms·

One of the projects I'd love to develop is an automated peer editor for student essays. My wife is an english teacher and a large percentage of her time is tak

One of the projects I'd love to develop is an automated peer editor for student essays. My wife is an english teacher and a large percentage of her time is taken up by grading papers. A large percentage of that time is then spent marking up grammar and spelling. What I envision is a website that handles that grammar/spelling bit. More importantly, I'd like it as a tool that the students use freely prior to submitting their essays to the teacher. I want them to have immediate feedback on how to improve the grammar in their essays, so they can iterate and learn. By the time the essays reach the teacher, the teacher should only have to grade for content, composition, style, plagiarism, citations, etc. Hopefully this also helps to reduce the amount of grammar that needs to be taught in-class, freeing time for more meaningful discussions. The problem is that while I have knowledge and experience in the computer vision side of machine learning, I lack experience in NLP. And to the best of my knowledge NLP as a field has not come as far as vision, to the extent that such an automated editor would have too many mistakes. To be student facing it would need to be really accurate. On top of that it wouldn't be dealing with well formed input. The input by definition is adversarial. So unlike SyntaxNet which is built to deal with comprehensible sentences, this tool would need to deal with incomprehensible sentences. According to the link, SyntaxNet only gets 90% accuracy on random sentences from the web. That said, I might give SyntaxNet a try. The idea would be to use SyntaxNet to extract meaning from a broken sentence, and then work backwards from the meaning to identify how the sentence can be modified to better match that meaning. Thank you Google for contributing this tool to the community at large.

tvural 10y ago
"A large percentage of that time is then spent marking up grammar and spelling." As an aside, I don't think this is the optimal way to teach people how to write. What were the ideas in those papers? How were they organized? Do the student's arguments make sense? I think that's what most students spend most of their time thinking about when writing an essay, and it can be a bit demoralizing to see the teacher care just as much about whether the grammar was right. Most students can fix grammar mistakes relatively easily once they notice them anyway.
- seanmcdirmid 10y ago
  > Most students can fix grammar mistakes relatively easily once they notice them anyway. You would be surprised! Especially with people whose first languages aren't English. Having something to provide feedback on grammar problems early on would be really useful. Of course, once they get grammar down, the next step is rhythm and flow, as well as reducing redundancy, and the biggest problem, as you say, is always the story, but you have to peel through lots of grammar problems before you get to that point. I edit a lot of research papers for my Chinese peers (most have PhDs, I work in a China-based research lab, so that isn't weird), so I'm pretty clear on the problems.
- narrator 10y ago
  I don't think I remember actually being taught how to write in primary school. I think they should reteach grammar in high school from the beginning. Most people's brains just can't pick up a systematic treatment of some of the finer points when younger. I went and did all the grammar quizzes over at http://grammar.ccc.commnet.edu/grammar/quiz_list.htm http://grammar.ccc.commnet.edu/grammar/quiz_list.htm a couple of years ago. It improved my writing a lot, and only took a weekend.
  - seanmcdirmid 10y ago
    I honestly didn't learn how to write until I got to graduate school. My very patient adviser had to beat it into my skull. Writing is a very practical art...you need to practice at it a lot vs. studying and memorizing rules, and my pre-grad school education didn't really force that.
  - knight17 10y ago
    Can you tell what other resources you've used to learn, what practices were helpful in studying grammar, what was not helpful in your study of grammar?
kafkaesq 10y ago
Not sure how they work exactly, but have you looked at http://noredink.com http://noredink.com (and as another commenter mentioned, http://grammarly.com/ http://grammarly.com/)? I'd be interested in your thoughts.
- fpgaminer 10y ago
  I've bounced my idea off my wife before and asked whether such a thing existed. I do recall her mentioning things like noredink and commenting that they weren't a match for one reason or another (the specific reasons escape me at the moment). Visiting noredink.com now ... I can't actually figure out what it is the site offers from a cursory glance, so I can't even begin to figure out whether it matches my idea and what issues it may have had that excluded it from her classroom. I'll bounce grammarly off her later though, in case that's new.
ilyaeck 10y ago
SyntaxNet is, by definition, for syntactic analysis - it would likely not help you much with semantics, to extract meaning. It could maybe help you automatically determine is a sentence is grammatically correct, though.
- dscpls 10y ago
  syntactic analysis is generally a precursor to semantic analysis EDIT: but sure, this is only the first step and semantic parsing is far from solved
schoen 10y ago
I think this is still risky if used in a context where the student might think that the computer is somehow always right. Great English writers often deliberately use sentence fragments or puns, or use a word with a nonstandard part-of-speech interpretation (especially using a noun as a verb). They may also sometimes use a sentence that's difficult for readers to parse and then explain the ambiguity after the fact. If a teacher gave students a grammar-checking tool to check their writing, they might assume that the tool knew better than they did, which is only sometimes true.
- wmeredith 10y ago
  Those great writers that break all the rules still know them. "Learn the rules like a pro, so you can break them like an artist." -Picaso
  - erikpukinskis 10y ago
    This is often repeated, but there's no evidence it's true. Many great writers had no formal training. "And that quote is almost certainly made up." - Willem Shakespere
    - DougMerritt 10y ago
      Knowing the rules is not the same thing as having formal training. With then-innovated art like Cubism, which in some sense broke earlier rules, the point is that those artists, like Picasso, were able to do representative art in correct proportions -- they were just going beyond that. This is a nontrivial issue, because there are always students who think they can skip learning boring mundane old fashioned art and go straight to something like cubism, but in 99% of cases that doesn't work well at all compared with learning "the rules" first. With writing, many great writers have broken "the rules" with punctuation, spelling, grammar, etc. But the important thing is that they do so on purpose. Whereas if one doesn't know the rules in the first place, one doesn't have the choice of whether to follow them or break them. Such a person will always break those rules they don't know (subconscious knowledge counts btw) -- but not for aesthetic reasons, only out of ignorance.
danappelxx 10y ago
Doesn't Grammarly[0] already do this? It analyzes the input for common grammar mistakes and proposes ways to fix them. As a student, I occasionally use Grammarly to proofread a paper for me, and it has worked pretty well so far. [0]: http://grammarly.com http://grammarly.com
fernly 10y ago
Such a checker could be a boon for students as well as instructors, but take note of this near the end of the article, > This suggests that we are approaching human performance—but only on well-formed text. It may fall down on exactly the bad writing you want to process. GIGO?

tvural 10y ago

"A large percentage of that time is then spent marking up grammar and spelling." As an aside, I don't think this is the optimal way to teach people how to write. What were the ideas in those papers? How were they organized? Do the student's arguments make sense? I think that's what most students spend most of their time thinking about when writing an essay, and it can be a bit demoralizing to see the teacher care just as much about whether the grammar was right. Most students can fix grammar mistakes relatively easily once they notice them anyway.

seanmcdirmid 10y ago
> Most students can fix grammar mistakes relatively easily once they notice them anyway. You would be surprised! Especially with people whose first languages aren't English. Having something to provide feedback on grammar problems early on would be really useful. Of course, once they get grammar down, the next step is rhythm and flow, as well as reducing redundancy, and the biggest problem, as you say, is always the story, but you have to peel through lots of grammar problems before you get to that point. I edit a lot of research papers for my Chinese peers (most have PhDs, I work in a China-based research lab, so that isn't weird), so I'm pretty clear on the problems.
narrator 10y ago
I don't think I remember actually being taught how to write in primary school. I think they should reteach grammar in high school from the beginning. Most people's brains just can't pick up a systematic treatment of some of the finer points when younger. I went and did all the grammar quizzes over at http://grammar.ccc.commnet.edu/grammar/quiz_list.htm http://grammar.ccc.commnet.edu/grammar/quiz_list.htm a couple of years ago. It improved my writing a lot, and only took a weekend.
- seanmcdirmid 10y ago
  I honestly didn't learn how to write until I got to graduate school. My very patient adviser had to beat it into my skull. Writing is a very practical art...you need to practice at it a lot vs. studying and memorizing rules, and my pre-grad school education didn't really force that.
- knight17 10y ago
  Can you tell what other resources you've used to learn, what practices were helpful in studying grammar, what was not helpful in your study of grammar?

kafkaesq 10y ago

Not sure how they work exactly, but have you looked at http://noredink.com http://noredink.com (and as another commenter mentioned, http://grammarly.com/ http://grammarly.com/)? I'd be interested in your thoughts.

fpgaminer 10y ago
I've bounced my idea off my wife before and asked whether such a thing existed. I do recall her mentioning things like noredink and commenting that they weren't a match for one reason or another (the specific reasons escape me at the moment). Visiting noredink.com now ... I can't actually figure out what it is the site offers from a cursory glance, so I can't even begin to figure out whether it matches my idea and what issues it may have had that excluded it from her classroom. I'll bounce grammarly off her later though, in case that's new.

ilyaeck 10y ago

SyntaxNet is, by definition, for syntactic analysis - it would likely not help you much with semantics, to extract meaning. It could maybe help you automatically determine is a sentence is grammatically correct, though.

dscpls 10y ago
syntactic analysis is generally a precursor to semantic analysis EDIT: but sure, this is only the first step and semantic parsing is far from solved

schoen 10y ago

I think this is still risky if used in a context where the student might think that the computer is somehow always right. Great English writers often deliberately use sentence fragments or puns, or use a word with a nonstandard part-of-speech interpretation (especially using a noun as a verb). They may also sometimes use a sentence that's difficult for readers to parse and then explain the ambiguity after the fact. If a teacher gave students a grammar-checking tool to check their writing, they might assume that the tool knew better than they did, which is only sometimes true.

wmeredith 10y ago
Those great writers that break all the rules still know them. "Learn the rules like a pro, so you can break them like an artist." -Picaso
- erikpukinskis 10y ago
  This is often repeated, but there's no evidence it's true. Many great writers had no formal training. "And that quote is almost certainly made up." - Willem Shakespere
  - DougMerritt 10y ago
    Knowing the rules is not the same thing as having formal training. With then-innovated art like Cubism, which in some sense broke earlier rules, the point is that those artists, like Picasso, were able to do representative art in correct proportions -- they were just going beyond that. This is a nontrivial issue, because there are always students who think they can skip learning boring mundane old fashioned art and go straight to something like cubism, but in 99% of cases that doesn't work well at all compared with learning "the rules" first. With writing, many great writers have broken "the rules" with punctuation, spelling, grammar, etc. But the important thing is that they do so on purpose. Whereas if one doesn't know the rules in the first place, one doesn't have the choice of whether to follow them or break them. Such a person will always break those rules they don't know (subconscious knowledge counts btw) -- but not for aesthetic reasons, only out of ignorance.

danappelxx 10y ago

Doesn't Grammarly[0] already do this? It analyzes the input for common grammar mistakes and proposes ways to fix them. As a student, I occasionally use Grammarly to proofread a paper for me, and it has worked pretty well so far. [0]: http://grammarly.com http://grammarly.com

fernly 10y ago

Such a checker could be a boon for students as well as instructors, but take note of this near the end of the article, > This suggests that we are approaching human performance—but only on well-formed text. It may fall down on exactly the bad writing you want to process. GIGO?