4 ms·

I don’t really see a problem with this - if someone does their due diligence and reads a paper thoroughly to provide feedback, and passes their draft through an

I don’t really see a problem with this - if someone does their due diligence and reads a paper thoroughly to provide feedback, and passes their draft through an LLM to make it easier to read or sound better, that’s fine; great even. I guess it does make it easier for people to be lazy and get away with it. That being said, I’d be surprised if ChatGPT/Claude/Gemini actually provided _zero_ useful feedback. Not exactly the same, but instead of my co-workers just pasting “LGTM” on everything, I’d probably prefer an LLM review.

barbazoo 3y ago
It’s like grammarly for people that don’t want to pay for grammarly?
- eyegor 3y ago
  I'd argue llms can do a much better job, because they can also emulate styles in addition to cleaning up grammar. "that's too formal", "that's too casual", "rephrase this more concisely", etc. I've run into plenty of drafts or arxiv publications that are annoying to read because the sentence structure is unnatural or overly verbose.
al_borland 3y ago
It may start out that way, if we assume the best of people, but if it does a decent enough job, they will be less and less review as time goes on. Most people can't even be bothered to read a function they copy off StackOverflow to understand what it does, it's not a stretch to believe they aren't going to bother reading a whole paper. >Not exactly the same, but instead of my co-workers just pasting “LGTM” on everything, I’d probably prefer an LLM review. You could do your own LLM review before sending it to your co-workers, if you'd like that kind of review. It may catch basic syntax stuff or suggest ways to write something differently, but it won't catch bigger issues of logic or "is this a good idea". I'll admit to a lot of LGTM reviews going around in my team, but last week I saw someone make something extreme dangerous and the review stopped it from rolling out to prod. I am almost 100% certain an LLM would have said LGTM, as it was written fine and did exactly what it said it would do... it was just a horrible idea to do that thing.

barbazoo 3y ago

It’s like grammarly for people that don’t want to pay for grammarly?

eyegor 3y ago
I'd argue llms can do a much better job, because they can also emulate styles in addition to cleaning up grammar. "that's too formal", "that's too casual", "rephrase this more concisely", etc. I've run into plenty of drafts or arxiv publications that are annoying to read because the sentence structure is unnatural or overly verbose.

al_borland 3y ago

It may start out that way, if we assume the best of people, but if it does a decent enough job, they will be less and less review as time goes on. Most people can't even be bothered to read a function they copy off StackOverflow to understand what it does, it's not a stretch to believe they aren't going to bother reading a whole paper. >Not exactly the same, but instead of my co-workers just pasting “LGTM” on everything, I’d probably prefer an LLM review. You could do your own LLM review before sending it to your co-workers, if you'd like that kind of review. It may catch basic syntax stuff or suggest ways to write something differently, but it won't catch bigger issues of logic or "is this a good idea". I'll admit to a lot of LGTM reviews going around in my team, but last week I saw someone make something extreme dangerous and the review stopped it from rolling out to prod. I am almost 100% certain an LLM would have said LGTM, as it was written fine and did exactly what it said it would do... it was just a horrible idea to do that thing.