3 ms·

Something you've gotta understand is that the majority of English-language scientific journal articles are written by authors who aren't native English speakers

Something you've gotta understand is that the majority of English-language scientific journal articles are written by authors who aren't native English speakers/writers. > https://ncses.nsf.gov/pubs/nsb202333/publication-output-by-region-country-or-economy-and-by-scientific-field https://ncses.nsf.gov/pubs/nsb202333/publication-output-by-r... In the past, just after they submit a poorly-written paper, the sleazeballs at Wiley/Elsevier/Springer would "encourage" said authors to employ their "Author Services" who would edit and rewrite their paper for them, and resubmit it for publication. This didn't come cheap. Today, everybody just uses LLMs. LLMs are masterful translators of language and concepts. It's a win-win for everybody but Author Services. Admittedly, word choice and sentence structure are much more limited. (This is not necessarily a bad thing. LLM-written scientific papers can be clearer, and are often more tidy, than human-written scientific papers.) I don't like seeing LLM-written text in journalism, literature, and other media, but I don't mind it so much in scientific literature -- if there's no fraud involved, that is.

leakycap 1y ago
> It's a win-win for everybody but Author Services. Having work that can be edited by a human who understands the concepts –or at least the words— shows a great depth of usefulness to the work itself. Running your rough draft in one language through an LLM so it appears, at surface glance, to be similar to a publication that a team of humans had been involved in doesn't actually provide value to the author or the reader. Now the author has a paper they cannot say they wrote (or indeed, say who wrote it) and the reader cannot assume that what they read was intended to mean what it now means in English. If anything, people who write papers in another language to be published in English should be leaning more than ever on human editors and quality assurance measures, not slop machines that hallucinate with good grammar.
- DontBreakAlex 1y ago
  I find it quite insulting that you seem to think that non-native english speakers are incapable of reading the outputs of LLMs to asses if it still means what they intended to say.
  - perching_aix 1y ago
    Why? I don't have academic paper publishing peers with bad language skills, but I do have colleagues with bad language skills, and the misunderstandings and petty catfights they get themselves into over poorly worded sentences, missing linguistic cues, and misinterpretations, is utterly bonkers. All otherwise perfectly smart capable people, they just happen to have this as a gap in their skillset. And no, they don't notice if transformative details get added in or left out.
    - leakycap 1y ago
      > I do have colleagues with bad language skills, and the misunderstandings and petty catfights they get themselves into over poorly worded sentences, missing linguistic cues, and misinterpretations, is utterly bonkers. Is this a widespread systemic issue within the organization, or do you work somewhere large enough that it is easy to find examples like this due to the number of people involved? If it is the former, I would not want to work somewhere that people get into petty catfights over editing and have no abiity to write a sentence or understand linguistic cues. I don't remember working anywhere I would describe in the way you do in your second paragraph. > And no, they don't notice if transformative details get added in or left out. I guess I don't have to tell you not to select them as the people to review your work output? Again, all the examples I'm reading make me think it would be beneficial for folks to include competent team members or external support for projects that will be published in a language you don't speak natively.
      - perching_aix 1y ago
        > Is this a widespread systemic issue within the organization, or do you work somewhere large enough that it is easy to find examples like this due to the number of people involved? Can't tell you for sure (would require me to have comprehensive knowledge of the language skills around the company). I do know a few folks with proper language skills, but they're a rarity (and I treasure them greatly). Could definitely be just my neck of the woods in the company being like this. > If it is the former, I would not want to work somewhere [like that where] (...) Yeah, it's not great. The way I solved this was by simply not caring and just talking to them in proper English, hammering them until they provide me (and each other) with enough cross-verifiable information that thus definitely cannot be wrong (or will be wrong in a very defendable way), with an additional serving of double-triple checking everything. Some are annoyed by this, others appreciate it. Such is life I suppose. > I guess I don't have to tell you not to select them as the people to review your work output? I don't really have a choice. I think you might misunderstand what it is that I deliver though. I work with cloud technologies, so while I do sometimes deliver technical writing, most of my output is configuration changes and code. When I speak of language barrier issues, that's about chat, email, and ticket communications. I think that's plenty bad enough to have these kinds of troubles in though, especially when it's managers who are having difficulties.
- A_D_E_P_T 1y ago
  I think you overestimate the extent of Author Services' involvement. It's near nil; there's no back-and-forth; you just send them your article and they send back a version with fewer grammatical/textual errors and correct formatting for the journal. It's a lot like an LLM, just slower and considerably more expensive. Also, you severely underestimate the ability of the authors to edit and verify the correctness of their own work. They wrote it in the first place, after all. They know, better than anybody else, what's important to convey. What's more, this editing step happens before peer review, so if the paper is junk or has glaring errors it'll probably be turned down by most respectable publications. Even the marginal publications won't publish stuff that's really bad.
  - leakycap 1y ago
    You're pointing out issues with one departmental service provided by one publishing company. I'm pointing out issues with using any LLM. If you don't like Author Services, use someone else. Involve a coauthor. This is not even a remotely hard problem to solve.
    - auggierose 1y ago
      Yes, it is not hard at all to solve. Just use an LLM.
- raincole 1y ago
  In my (non-English-speaking) country, paying for an editing service before one submits a paper was very common. Now it's all AI edited. That's not the interesting part though. The interesting part is that the biggest editing service provider here explicitly states it's okay for the clients to send the papers through AI first then buy their service for 'additional human editing.' They even offer cheaper price for papers that are already edited by AI. The irony is not lost on me.
- jug 1y ago
  Don't LLM's have enough insight in at least the very concepts here? o3 is considered PhD level on this. Maybe not pure math solving since LLM's suck on that, but I wouldn't be quick to doubt the capacity of understanding the advanced concepts and typical wording. Also, this is hopefully not passing through LLM's completely unvetted, but as a crutch to get 80-90% there with minimal effort.
perching_aix 1y ago
Sounds a bit scary from a correctness standpoint. I have to wonder if such paper authors have the necessary language skills to ascertain whether their LLM of choice didn't e.g. add in any nuance that isn't correct or intended. I do think it's possible they do, I myself am a foreign speaker who would consider doing something like this, and I think I could notice if that happens. But then I think I'm also well beyond the level where this would be a necessity for me, rather than just a convenience and an additional polishing step.
- diggan 1y ago
  Wouldn't it be the same if they use the "Author Services" as with an LLM? Essentially "How do you evaluate something that might go above your head?" remains the same regardless of the tool/service used.
  - perching_aix 1y ago
    I don't think modeling this as Author Services = good, LLM = bad (or vice versa) makes for a fruitful conversation, and that (modeling it as such) wasn't my intention either. I'd definitely expect Author Services to do a better job though, even just based on this much description.
    - diggan 1y ago
      > I'd definitely expect Author Services to do a better job though, even just based on this much description Yeah, me too, but the claim was if "paper authors have the necessary language skills to ascertain" the correctness of the translation. Regardless of how good the LLM or Author Services are, the problem remains the same.
      - perching_aix 1y ago
        Right, that's true. Also scary in its own way. I'm not too big a fan of natural languages - something I voice quite frequently these days I notice :)
  - 1y ago
dev_l1x_be 1y ago
Absolutely. I think, even for non scientific journals the LLMs have a net good effect on expressiveness and clarity when instructed like that.
einpoklum 1y ago
> LLMs are masterful translators of language and concepts No, they are not. They will - speaking generally - try to produce what seems like a typical translation; which is often the same thing, but not always.
- diggan 1y ago
  They're pretty good at translation, to be honest. Transformers was initially invented/discovered in order to build something better for doing translations with Google Translate, so it's hardly a surprise they are pretty good at it.

leakycap 1y ago

> It's a win-win for everybody but Author Services. Having work that can be edited by a human who understands the concepts –or at least the words— shows a great depth of usefulness to the work itself. Running your rough draft in one language through an LLM so it appears, at surface glance, to be similar to a publication that a team of humans had been involved in doesn't actually provide value to the author or the reader. Now the author has a paper they cannot say they wrote (or indeed, say who wrote it) and the reader cannot assume that what they read was intended to mean what it now means in English. If anything, people who write papers in another language to be published in English should be leaning more than ever on human editors and quality assurance measures, not slop machines that hallucinate with good grammar.

DontBreakAlex 1y ago
I find it quite insulting that you seem to think that non-native english speakers are incapable of reading the outputs of LLMs to asses if it still means what they intended to say.
- perching_aix 1y ago
  Why? I don't have academic paper publishing peers with bad language skills, but I do have colleagues with bad language skills, and the misunderstandings and petty catfights they get themselves into over poorly worded sentences, missing linguistic cues, and misinterpretations, is utterly bonkers. All otherwise perfectly smart capable people, they just happen to have this as a gap in their skillset. And no, they don't notice if transformative details get added in or left out.
  - leakycap 1y ago
    > I do have colleagues with bad language skills, and the misunderstandings and petty catfights they get themselves into over poorly worded sentences, missing linguistic cues, and misinterpretations, is utterly bonkers. Is this a widespread systemic issue within the organization, or do you work somewhere large enough that it is easy to find examples like this due to the number of people involved? If it is the former, I would not want to work somewhere that people get into petty catfights over editing and have no abiity to write a sentence or understand linguistic cues. I don't remember working anywhere I would describe in the way you do in your second paragraph. > And no, they don't notice if transformative details get added in or left out. I guess I don't have to tell you not to select them as the people to review your work output? Again, all the examples I'm reading make me think it would be beneficial for folks to include competent team members or external support for projects that will be published in a language you don't speak natively.
    - perching_aix 1y ago
      > Is this a widespread systemic issue within the organization, or do you work somewhere large enough that it is easy to find examples like this due to the number of people involved? Can't tell you for sure (would require me to have comprehensive knowledge of the language skills around the company). I do know a few folks with proper language skills, but they're a rarity (and I treasure them greatly). Could definitely be just my neck of the woods in the company being like this. > If it is the former, I would not want to work somewhere [like that where] (...) Yeah, it's not great. The way I solved this was by simply not caring and just talking to them in proper English, hammering them until they provide me (and each other) with enough cross-verifiable information that thus definitely cannot be wrong (or will be wrong in a very defendable way), with an additional serving of double-triple checking everything. Some are annoyed by this, others appreciate it. Such is life I suppose. > I guess I don't have to tell you not to select them as the people to review your work output? I don't really have a choice. I think you might misunderstand what it is that I deliver though. I work with cloud technologies, so while I do sometimes deliver technical writing, most of my output is configuration changes and code. When I speak of language barrier issues, that's about chat, email, and ticket communications. I think that's plenty bad enough to have these kinds of troubles in though, especially when it's managers who are having difficulties.
A_D_E_P_T 1y ago
I think you overestimate the extent of Author Services' involvement. It's near nil; there's no back-and-forth; you just send them your article and they send back a version with fewer grammatical/textual errors and correct formatting for the journal. It's a lot like an LLM, just slower and considerably more expensive. Also, you severely underestimate the ability of the authors to edit and verify the correctness of their own work. They wrote it in the first place, after all. They know, better than anybody else, what's important to convey. What's more, this editing step happens before peer review, so if the paper is junk or has glaring errors it'll probably be turned down by most respectable publications. Even the marginal publications won't publish stuff that's really bad.
- leakycap 1y ago
  You're pointing out issues with one departmental service provided by one publishing company. I'm pointing out issues with using any LLM. If you don't like Author Services, use someone else. Involve a coauthor. This is not even a remotely hard problem to solve.
  - auggierose 1y ago
    Yes, it is not hard at all to solve. Just use an LLM.
raincole 1y ago
In my (non-English-speaking) country, paying for an editing service before one submits a paper was very common. Now it's all AI edited. That's not the interesting part though. The interesting part is that the biggest editing service provider here explicitly states it's okay for the clients to send the papers through AI first then buy their service for 'additional human editing.' They even offer cheaper price for papers that are already edited by AI. The irony is not lost on me.
jug 1y ago
Don't LLM's have enough insight in at least the very concepts here? o3 is considered PhD level on this. Maybe not pure math solving since LLM's suck on that, but I wouldn't be quick to doubt the capacity of understanding the advanced concepts and typical wording. Also, this is hopefully not passing through LLM's completely unvetted, but as a crutch to get 80-90% there with minimal effort.

perching_aix 1y ago

Sounds a bit scary from a correctness standpoint. I have to wonder if such paper authors have the necessary language skills to ascertain whether their LLM of choice didn't e.g. add in any nuance that isn't correct or intended. I do think it's possible they do, I myself am a foreign speaker who would consider doing something like this, and I think I could notice if that happens. But then I think I'm also well beyond the level where this would be a necessity for me, rather than just a convenience and an additional polishing step.

diggan 1y ago
Wouldn't it be the same if they use the "Author Services" as with an LLM? Essentially "How do you evaluate something that might go above your head?" remains the same regardless of the tool/service used.
- perching_aix 1y ago
  I don't think modeling this as Author Services = good, LLM = bad (or vice versa) makes for a fruitful conversation, and that (modeling it as such) wasn't my intention either. I'd definitely expect Author Services to do a better job though, even just based on this much description.
  - diggan 1y ago
    > I'd definitely expect Author Services to do a better job though, even just based on this much description Yeah, me too, but the claim was if "paper authors have the necessary language skills to ascertain" the correctness of the translation. Regardless of how good the LLM or Author Services are, the problem remains the same.
    - perching_aix 1y ago
      Right, that's true. Also scary in its own way. I'm not too big a fan of natural languages - something I voice quite frequently these days I notice :)
- 1y ago

dev_l1x_be 1y ago

Absolutely. I think, even for non scientific journals the LLMs have a net good effect on expressiveness and clarity when instructed like that.

einpoklum 1y ago

> LLMs are masterful translators of language and concepts No, they are not. They will - speaking generally - try to produce what seems like a typical translation; which is often the same thing, but not always.

diggan 1y ago
They're pretty good at translation, to be honest. Transformers was initially invented/discovered in order to build something better for doing translations with Google Translate, so it's hardly a surprise they are pretty good at it.