4 ms·

Sounds a bit scary from a correctness standpoint. I have to wonder if such paper authors have the necessary language skills to ascertain whether their LLM of ch

Sounds a bit scary from a correctness standpoint. I have to wonder if such paper authors have the necessary language skills to ascertain whether their LLM of choice didn't e.g. add in any nuance that isn't correct or intended. I do think it's possible they do, I myself am a foreign speaker who would consider doing something like this, and I think I could notice if that happens. But then I think I'm also well beyond the level where this would be a necessity for me, rather than just a convenience and an additional polishing step.

diggan 1y ago
Wouldn't it be the same if they use the "Author Services" as with an LLM? Essentially "How do you evaluate something that might go above your head?" remains the same regardless of the tool/service used.
- perching_aix 1y ago
  I don't think modeling this as Author Services = good, LLM = bad (or vice versa) makes for a fruitful conversation, and that (modeling it as such) wasn't my intention either. I'd definitely expect Author Services to do a better job though, even just based on this much description.
  - diggan 1y ago
    > I'd definitely expect Author Services to do a better job though, even just based on this much description Yeah, me too, but the claim was if "paper authors have the necessary language skills to ascertain" the correctness of the translation. Regardless of how good the LLM or Author Services are, the problem remains the same.
    - perching_aix 1y ago
      Right, that's true. Also scary in its own way. I'm not too big a fan of natural languages - something I voice quite frequently these days I notice :)
- AlienRobot 1y ago
  The main worry, in my opinion, is an issue of responsibility. If the author services makes a mistake, a person will be blamed for it. But when an LLM makes a mistake, people tend to shift the responsibility from the person that used the LLM to the LLM itself. People don't simply want to shift the effort of a task away from themselves, they also want to shift away the responsibility for doing it wrong.
  - diggan 1y ago
    > If the author services makes a mistake, a person will be blamed for it Wouldn't the paper authors be "blamed" (held responsible) for it, rather than the "Author Services"? Ultimately, they're responsible for the quality of their paper. Just like I'm responsible for making sure my taxes are paid correctly, even if I use an account, tax lawyer and financial advisor for the whole shebang. If anything is incorrect, I'll be held responsible.
    - AlienRobot 1y ago
      Yes, but this will only happen after a mistake occurs. What I mean is that if it's your responsibility to do something right, it acts a deterrent that motivates you to make sure of it. LLM's allow people to defer this responsibility to the LLM, so they may avoid making sure of things because they let the LLM take upon their responsibility.
      - diggan 1y ago
        If anything, wouldn't it be the opposite? When you offload something to a human, at least you can say "it was their fault!" and point to an actual name, or similar. But you can't use a tool and then say "it was their fault!" and point to the screen, it's not another entity like the "Author Services" is.
        AlienRobot 1y ago
        It's the same thing, but normally there wouldn't be "a human" because this chain of delegations stopped on you. Now you can just open ChatGPT.
mihaaly 1y ago
I believe both arguments assume honest contributors, which may not be completely true. I have no reference but vague recollection of doctored data, made up results (basically cheating) due to publication pressure or just dishonesty/lazyness plaguing the reliability of scientific publications. Some may even be happy of the twisted reality or halucinations producing novel sounding results advancing the scientific career similar to those occured in the past. Reproducibility crisis is their friend to get away with it for a long enough time to have the personal gain harvested.
- perching_aix 1y ago
  The reason I "assume honest contributors" is because assuming anything else produces trivial conclusions. Obviously someone who just wants to pump out something, anything, won't care about the accuracy of delivery, nor will anyone be impacted by the lack of it (as the paper was rubbish from the get-go). I just don't find that whole can of worms relevant here. If I really think about it, I guess I can see it being relevant like how typesetting software is, for making a paper feel more serious than it really is. Not really the angle I was going for though.

diggan 1y ago

Wouldn't it be the same if they use the "Author Services" as with an LLM? Essentially "How do you evaluate something that might go above your head?" remains the same regardless of the tool/service used.

perching_aix 1y ago
I don't think modeling this as Author Services = good, LLM = bad (or vice versa) makes for a fruitful conversation, and that (modeling it as such) wasn't my intention either. I'd definitely expect Author Services to do a better job though, even just based on this much description.
- diggan 1y ago
  > I'd definitely expect Author Services to do a better job though, even just based on this much description Yeah, me too, but the claim was if "paper authors have the necessary language skills to ascertain" the correctness of the translation. Regardless of how good the LLM or Author Services are, the problem remains the same.
  - perching_aix 1y ago
    Right, that's true. Also scary in its own way. I'm not too big a fan of natural languages - something I voice quite frequently these days I notice :)
AlienRobot 1y ago
The main worry, in my opinion, is an issue of responsibility. If the author services makes a mistake, a person will be blamed for it. But when an LLM makes a mistake, people tend to shift the responsibility from the person that used the LLM to the LLM itself. People don't simply want to shift the effort of a task away from themselves, they also want to shift away the responsibility for doing it wrong.
- diggan 1y ago
  > If the author services makes a mistake, a person will be blamed for it Wouldn't the paper authors be "blamed" (held responsible) for it, rather than the "Author Services"? Ultimately, they're responsible for the quality of their paper. Just like I'm responsible for making sure my taxes are paid correctly, even if I use an account, tax lawyer and financial advisor for the whole shebang. If anything is incorrect, I'll be held responsible.
  - AlienRobot 1y ago
    Yes, but this will only happen after a mistake occurs. What I mean is that if it's your responsibility to do something right, it acts a deterrent that motivates you to make sure of it. LLM's allow people to defer this responsibility to the LLM, so they may avoid making sure of things because they let the LLM take upon their responsibility.
    - diggan 1y ago
      If anything, wouldn't it be the opposite? When you offload something to a human, at least you can say "it was their fault!" and point to an actual name, or similar. But you can't use a tool and then say "it was their fault!" and point to the screen, it's not another entity like the "Author Services" is.
      - AlienRobot 1y ago
        It's the same thing, but normally there wouldn't be "a human" because this chain of delegations stopped on you. Now you can just open ChatGPT.

mihaaly 1y ago

I believe both arguments assume honest contributors, which may not be completely true. I have no reference but vague recollection of doctored data, made up results (basically cheating) due to publication pressure or just dishonesty/lazyness plaguing the reliability of scientific publications. Some may even be happy of the twisted reality or halucinations producing novel sounding results advancing the scientific career similar to those occured in the past. Reproducibility crisis is their friend to get away with it for a long enough time to have the personal gain harvested.

perching_aix 1y ago
The reason I "assume honest contributors" is because assuming anything else produces trivial conclusions. Obviously someone who just wants to pump out something, anything, won't care about the accuracy of delivery, nor will anyone be impacted by the lack of it (as the paper was rubbish from the get-go). I just don't find that whole can of worms relevant here. If I really think about it, I guess I can see it being relevant like how typesetting software is, for making a paper feel more serious than it really is. Not really the angle I was going for though.