5 ms·

> If these are the only errors, we are not troubled. However: we do not know if these are the only errors, they are merely a signature that the paper was submit

by neilv 9mo ago

> If these are the only errors, we are not troubled. However: we do not know if these are the only errors, they are merely a signature that the paper was submitted without being thoroughly checked for hallucinations. They are a signature that some LLM was used to generate parts of the paper and the responsible authors used this LLM without care. I am troubled by people using an LLM at all to write academic research papers. It's a shoddy, irresponsible way to work. And also plagiarism, when you claim authorship of it. I'd see a failure of the 'author' to catch hallucinations, to be more like a failure to hide evidence of misconduct. If academic venues are saying that using an LLM to write your papers is OK ("so long as you look it over for hallucinations"?), then those academic venues deserve every bit of operational pain and damaged reputation that will result.

mapontosevenths 9mo ago
> It's a shoddy, irresponsible way to work. And also plagiarism, when you claim authorship of it. It reminds me of kids these days and their fancy calculators! Those new fangled doohickeys just aren't reliable, and the kids never realize that they won't always have a calculator on them! Everyone should just do it the good old fashioned way with slide rules! Or these darn kids and their unreliable sources like Wikipedia! Everyone knows that you need a nice solid reliable source that's made out of dead trees and fact checked but up to 3 paid professionals!
- neilv 9mo ago
  Annoying dismissal. In an academic paper, you condense a lot of thinking and work, into a writeup. Why would you blow off the writeup part, and impose AI slop upon the reviewers and the research community?
  - HKH2 9mo ago
    I don't necessarily disagree, but researchers are not required to be good communicators. An academic can lead their field and be a terrible lecturer. A specialist can let a generalist help explain concepts for them. They should still review the final result though. There is no excuse for not doing that.
    - dasyud 9mo ago
      I disagree here. A good researcher has to be a good communicator. I am not saying that it is necessarily the case that you don't understand the topic if you cannot explain it well enough to someone new, but it is essential to communicate to have a good exchange of ideas with others, and consequently, become a better researcher. This is one of the skills you learn in a PhD program.
      - HKH2 9mo ago
        That is how it should be, yes. Do PhDs always meet that standard though? No.
- foxes 9mo ago
  Im really not motivated by this argument; it seems a false equivalence. Its not merely a spell checker or removing some tedium. As a professional mathematician I used wikipedia all the time to lookup quick facts before verifying it myself or elsewhere. A calculator well; I can use an actual programming language. Up until this point neither of those tools were asvertised or used by people to entirely replace human input.
  - sodapopcan 9mo ago
    I hate to sound like a 19 year old on Reddit but: AI People: "AI is a completely unprecedented technology where its introduction is unlike the introduction of any other transformative technology in history! We must treat it totally differently!" Also AI People: "You're worried about nothing, this is just like when people were worried about the internet."
    - foxes 9mo ago
      Well the internet has definitely changed things; but also it wasnt initially controlled by a bunch of megacorps with the same level of power and centralisation today.
      - sodapopcan 9mo ago
        :pointing-up-emoji:
    - mikkupikku 9mo ago
      The internet analogy is apt because it was in fact a massive bubble, but that bubble popping didn't mean the tech went away. Same will happen again, which is a point both extremes miss. One would have you believe there is no bubble and you should dump all your money into this industry, while the other would have us believe that once the bubble pops all this AI stuff will be debunked and discarded as useless scamware.
  - ekidd 9mo ago
    There are some interesting possibilities for LLMs in math, especially in terms of generating machine-checked proofs using languages like Lean. But this is a supplement to the actual result, where the LLM would actually be adding a more rigorous version of a human's argument with all the boring steps included. In a few cases, I see Terrance Tao has pointed out examples LLMs actually finding proofs of open problems unassisted. Not necessarily problems anyone cared deeply about. But there's still the fact that if the proof holds, then it's valid no matter who or what came up with it. So it's complicated I guess?
- api 9mo ago
  One issue with this analogy is that calculators really are precise when used correctly. LLMs are not. I do think they can be used in research but not without careful checking. In my own work I’ve found them most useful as search aids and brainstorming sounding boards.
  - ragnarok451 9mo ago
    One issue with this analogy is that paper encyclopedias really are precise when used correctly. Wikipedia is not. I do think it can be used in research but not without careful checking. In my own work I've found it most useful as a search aid and for brainstorming. ^ this same comment 10 years ago
    - mapontosevenths 9mo ago
      > I do think it can be used in research but not without careful checking. This is really just restating what I already said in this thread, but you're right. That's because wikipedia isn't a primary source and was never, ever meant to be. You are SUPPOSED to go read it then click through to the primary sources and cite those. Lots of people use it incorrectly and get bad results because they still haven't realized this... all these years later. Same thing with treating stochastic LLM's like sources of truth and knowledge. Those folks are just doing it wrong.
    - mikkupikku 9mo ago
      Paper encyclopedias were neither precise nor accurate. You could count on them to give you ballpark figures most of the time, but certainly not precise answers. And that's assuming the set was new, but in reality most encyclopedias ever encountered by people in reality were several years old at least. I remember the encyclopedia set I had access to in the 90s was written before the USSR fell..
  - mapontosevenths 9mo ago
    > I do think they can be used in research but not without careful checking. Of course you are right. It is the same with all tools, calculators included, if you use them improperly you get poor results. In this case they're stochastic, which isn't something people are used to happening with computers yet. You have to understand that and learn how to use them or you will get poor results.
- usefulcat 9mo ago
  I doubt that it's common for anyone to read a research paper and then question whether the researcher's calculator was working reliably. Sure, maybe someday LLMs will be able to report facts in a mostly reliable fashion (like a typical calculator), but we're definitely not even close to that yet, so until we are the skepticism is very much warranted. Especially when the details really do matter, as in scientific research.
  - mapontosevenths 9mo ago
    > whether the researcher's calculator was working reliably. LLM's do not work reliably, that's not their purpose. If you use them that way it's akin to using a butter knife as a screwdriver. You might get away with it once or twice, but then you slip and stab yourself. Better to go find screwdriver if you need reliable.
  - westurner 9mo ago
    > I doubt that it's common for anyone to read a research paper and then question whether the researcher's calculator was working reliably Reproducibility and repeatability in the sciences? Replication crisis > Causes > Problems with the publication system in science > Mathematical errors; Causes > Questionable research practices > In AI research, Remedies > [..., open science, reproducible workflows, disclosure, ] https://en.wikipedia.org/wiki/Replication_crisis#Mathematical_errors https://en.wikipedia.org/wiki/Replication_crisis#Mathematica... Already verifiable proofs are too impossibly many pages for human review There are "verify each Premise" and "verify the logical form of the Argument" (P therefore Q) steps that still the model doesn't do for the user. For your domain, how insufficient is the output given process as a prompt like: Identify hallucinations from models prior to (date in the future) Check each sentence of this: ```{...}``` Research ScholarlyArticles (and then their Datasets) which support and which reject your conclusions. Critically review findings and controls. Suggest code to write to apply data science principles to proving correlative and causative relations given already-collected observations. Design experiment(s) given the scientific method to statistically prove causative (and also correlative) relations Identify a meta-analytic workflow (process, tools, schema, and maybe code) for proving what is suggested by this chat
- andrepd 9mo ago
  > Those new fangled doohickeys just aren't reliable Except they are (unlike a chatbot, a calculator is perfectly deterministic), and the unreliability of LLMs is one of their most, if not the most, widespread target of criticism. Low effort doesn't even begin to describe your comment.
  - mapontosevenths 9mo ago
    > Except they are (unlike a chatbot, a calculator is perfectly deterministic) LLM's are supposed to be stochastic. That is not a bug, I can see why you find that disappointing but it's just the reality of the tool. However, as I mentioned elsewhere calculators also have bugs and those bugs make their way into scientific research all the time. Floating point errors are particularly common, as are order of operations problems because physical devices get it wrong frequently and are hard to patch. Worse, they are not SUPPOSED TO BE stochastic so when they fail nobody notices until it's far too late. [0 - PDF] Further, spreadsheets are no better, for example a scan of ~3,600 genomics papers found that about 1 in 5 had gene‑name errors (e.g., SEPT2 → “2‑Sep”) because that's how Excel likes to format things.[1] Again, this is much worse than a stochastic machine doing it's stochastic job... because it's not SUPPOSED to be random, it's just broken and on a truly massive scale. [0] https://ttu-ir.tdl.org/server/api/core/bitstreams/7fce5b73-18c9-4d5c-af68-57915cb9dabe/content https://ttu-ir.tdl.org/server/api/core/bitstreams/7fce5b73-1... [1]https://www.washingtonpost.com/news/wonk/wp/2016/08/26/an-alarming-number-of-scientific-papers-contain-excel-errors/ https://www.washingtonpost.com/news/wonk/wp/2016/08/26/an-al...
    - raddan 9mo ago
      That’s a strange argument. There are plenty of stochastic processes that have perfectly acceptable guarantees. A good example is Karger’s min-cut algorithm. You might not know what you get on any given single run, but you know EXACTLY what you’re going to get when you crank up the number of trials. Nobody can tell you what you are going to get when you run an LLM once. Nobody can tell you what you’re going to get when you run it N times. There are, in fact, no guarantees at all. Nobody even really knows why it can solve some problems and why it can’t solve other except maybe it memorized the answer at some point. But this is not how they are marketed. They are marketed as wondrous inventions that can SOLVE EVERYTHING. This is obviously not true. You can verify it yourself, with a simple deterministic problem: generate an arithmetic expression of length N. As you increase N, the probability that an LLM can solve it drops to zero. Ok, fine. This kind of problem is not a good fit for an LLM. But which is? And after you’ve found a problem that seems like a good fit, how do you know? Did you test it systematically? The big LLM vendors are fudging the numbers. They’re testing on the training set, they’re using ad hoc measurements and so on. But don’t take my word for it. There’s lots of great literature out there that probes the eccentricities of these models; for some reason this work rarely makes its way into the HN echo chamber. Now I’m not saying these things are broken and useless. Far from it. I use them every day. But I don’t trust anything they produce, because there are no guarantees, and I have been burned many times. If you have not been burned, you’re either exceptionally lucky, you are asking it to solve homework assignments, or you are ignoring the pain. Excel bugs are not the same thing. Most of those problems can be found trivially. You can find them because Excel is a language with clear rules (just not clear to those particular users). The problem with Excel is that people aren’t looking for bugs.
derefr 9mo ago
I would argue that an LLM is a perfectly sensible tool for structure-preserving machine translation from another language to English. (Where by "another language", you could also also substitute "very poor/non-fluent English." Though IMHO that's a bit silly, even though it's possible; there's little sense in writing in a language you only half know, when you'd get a less-lossy result from just writing in your native tongue, and then having it translate from that.) Google Translate et al were never good enough at this task to actually allow people to use the results for anything professional. Previous tools were limited to getting a rough gloss of what words in another language mean. But LLMs can be used in this way, and are being used in this way; and this is increasingly allowing non-English-fluent academics to publish papers in English-language journals (thus engaging with the English-language academic community), where previously those academics they may have felt "stuck" publishing in what few journals exist for their discipline in their own language. Would you call the use of LLMs for translation "shoddy" or "irresponsible"? To me, it'd be no more and no less "shoddy" or "irresponsible" than it would be to hire a freelance human translator to translate the paper for you. (In fact, the human translator might be a worse idea, as LLMs are more likely to understand how to translate the specific academic jargon of your discipline than a randomly-selected human translator would be.)
- neilv 9mo ago
  Good point. There may be a place for LLMs for science writing translation (hopefully not adding nor subtracting anything) when you're not fluent in the language of a venue. You need a way to validate the correctness of the translation, and to be able to stand behind whatever the translation says. And the translation should be disclosed on the paper.
- gus_massa 9mo ago
  Autotranslating technical texts is very hard. After the translation, you muct check that all the technical words were translated correctly, instead of a fancy synonym that does not make sense. (A friend has an old book translated a long time ago (by a human) from Russian to Spanish. Instead of "complex numbers", the book calls them "complicated numbers". :) )
bjourne 9mo ago
There are legitimate, non-cheating ways to use LLMs for writing. I often use the wrong verb forms ("They synthesizes the ..."), write "though" when it should be "although", and forget to comma-separate clauses. LLMs are perfect for that. Generating text from scratch, however, is wrong.
- thaumasiotes 9mo ago
  > I often ... write "though" when it should be "although" That is a purely imaginary "error". Anywhere you can use 'although', you are free to use 'though' instead.
  - bjourne 9mo ago
    Yeah, but you cannot use although anywhere you can use though, though.
    - thaumasiotes 9mo ago
      That's true, but the one-way substitutability still means there is no such thing as "writ[ing] 'though' when it should be 'although'".
- rustystump 9mo ago
  I do similar proofing (esp spelling) but u need to be very careful as it will nudge u to specific styles that rob originality.
- bloppe 9mo ago
  I agree, but I don't think any of the broadly acceptable uses would result in easily identifiable flaws like those in the post, especially hallucinated URLs.
aydyn 9mo ago
>also plagiarism To me, this is a reminder of how much of a specific minority this forum is. Nobody I know in real life, personally or at work, has expressed this belief. I have literally only ever encountered this anti-AI extremism (extremism in the non-pejorative sense) in places like reddit and here. Clearly, the authors in NeurIPS don't agree that using an LLM to help write is "plagiarism", and I would trust their opinions far more than some random redditor.
- fingerlocks 9mo ago
  I find that hard to believe. Every creative professional that I know shares this sentiment. That’s several graphic designers at big tech companies, one person in print media, and one visual effects artist in the film industry. And once you include many of their professional colleagues that becomes a decent sample size.
  - jama211 9mo ago
    Graphic design is a completely different kettle of fish. Comparing it to academic paper writing is disingenuous.
    - kortilla 9mo ago
      The thread is about not knowing anyone at all who thinks AI is plagiarizing.
      - jama211 9mo ago
        Yes, plagiarising text based content. No one in this thread meant graphics.
- jama211 9mo ago
  Yup, and no matter how flimsy an anti-ai article is, it will skyrocket to the top of HN because of it. It makes sense though, HN users are the most likely to feel threatened by LLMs, and therefore are more likely to be anxious about them. I don’t love ai either, but that’s the truth.
thomasahle 9mo ago
> And also plagiarism, when you claim authorship of it. I don't actually mind putting Claude as a co-author on my github commits. But for papers there are usually so many tools involved. It would be crowded to include each of Claude, Gemini, Codex, Mathematica, Grammarly, Translate etc. as co-authors, even though I used all of them for some parts. Maybe just having a "tools used" section could work?
- the__alchemist 9mo ago
  I suspect the parent post was concerned about plagiarizing the author of training data; not software tools.
piyh 9mo ago
>I am troubled by people using an LLM at all to write academic research papers. I'm an outsider to the academic system. I have cool projects that I feel push some niche application to SOTA in my tiny little domain, which is publishable based on many of the papers I've read. If I can build a system that does a thing, I can benchmark and prove it's better than previous papers, my main blocker is getting all my work and information into the "Arxiv PDF" format and tone. Seems like a good use of LLMs to me.

by neilv 9mo ago

mapontosevenths 9mo ago

> It's a shoddy, irresponsible way to work. And also plagiarism, when you claim authorship of it. It reminds me of kids these days and their fancy calculators! Those new fangled doohickeys just aren't reliable, and the kids never realize that they won't always have a calculator on them! Everyone should just do it the good old fashioned way with slide rules! Or these darn kids and their unreliable sources like Wikipedia! Everyone knows that you need a nice solid reliable source that's made out of dead trees and fact checked but up to 3 paid professionals!

neilv 9mo ago
Annoying dismissal. In an academic paper, you condense a lot of thinking and work, into a writeup. Why would you blow off the writeup part, and impose AI slop upon the reviewers and the research community?
- HKH2 9mo ago
  I don't necessarily disagree, but researchers are not required to be good communicators. An academic can lead their field and be a terrible lecturer. A specialist can let a generalist help explain concepts for them. They should still review the final result though. There is no excuse for not doing that.
  - dasyud 9mo ago
    I disagree here. A good researcher has to be a good communicator. I am not saying that it is necessarily the case that you don't understand the topic if you cannot explain it well enough to someone new, but it is essential to communicate to have a good exchange of ideas with others, and consequently, become a better researcher. This is one of the skills you learn in a PhD program.
    - HKH2 9mo ago
      That is how it should be, yes. Do PhDs always meet that standard though? No.
foxes 9mo ago
Im really not motivated by this argument; it seems a false equivalence. Its not merely a spell checker or removing some tedium. As a professional mathematician I used wikipedia all the time to lookup quick facts before verifying it myself or elsewhere. A calculator well; I can use an actual programming language. Up until this point neither of those tools were asvertised or used by people to entirely replace human input.
- sodapopcan 9mo ago
  I hate to sound like a 19 year old on Reddit but: AI People: "AI is a completely unprecedented technology where its introduction is unlike the introduction of any other transformative technology in history! We must treat it totally differently!" Also AI People: "You're worried about nothing, this is just like when people were worried about the internet."
  - foxes 9mo ago
    Well the internet has definitely changed things; but also it wasnt initially controlled by a bunch of megacorps with the same level of power and centralisation today.
    - sodapopcan 9mo ago
      :pointing-up-emoji:
  - mikkupikku 9mo ago
    The internet analogy is apt because it was in fact a massive bubble, but that bubble popping didn't mean the tech went away. Same will happen again, which is a point both extremes miss. One would have you believe there is no bubble and you should dump all your money into this industry, while the other would have us believe that once the bubble pops all this AI stuff will be debunked and discarded as useless scamware.
- ekidd 9mo ago
  There are some interesting possibilities for LLMs in math, especially in terms of generating machine-checked proofs using languages like Lean. But this is a supplement to the actual result, where the LLM would actually be adding a more rigorous version of a human's argument with all the boring steps included. In a few cases, I see Terrance Tao has pointed out examples LLMs actually finding proofs of open problems unassisted. Not necessarily problems anyone cared deeply about. But there's still the fact that if the proof holds, then it's valid no matter who or what came up with it. So it's complicated I guess?
api 9mo ago
One issue with this analogy is that calculators really are precise when used correctly. LLMs are not. I do think they can be used in research but not without careful checking. In my own work I’ve found them most useful as search aids and brainstorming sounding boards.
- ragnarok451 9mo ago
  One issue with this analogy is that paper encyclopedias really are precise when used correctly. Wikipedia is not. I do think it can be used in research but not without careful checking. In my own work I've found it most useful as a search aid and for brainstorming. ^ this same comment 10 years ago
  - mapontosevenths 9mo ago
    > I do think it can be used in research but not without careful checking. This is really just restating what I already said in this thread, but you're right. That's because wikipedia isn't a primary source and was never, ever meant to be. You are SUPPOSED to go read it then click through to the primary sources and cite those. Lots of people use it incorrectly and get bad results because they still haven't realized this... all these years later. Same thing with treating stochastic LLM's like sources of truth and knowledge. Those folks are just doing it wrong.
  - mikkupikku 9mo ago
    Paper encyclopedias were neither precise nor accurate. You could count on them to give you ballpark figures most of the time, but certainly not precise answers. And that's assuming the set was new, but in reality most encyclopedias ever encountered by people in reality were several years old at least. I remember the encyclopedia set I had access to in the 90s was written before the USSR fell..
- mapontosevenths 9mo ago
  > I do think they can be used in research but not without careful checking. Of course you are right. It is the same with all tools, calculators included, if you use them improperly you get poor results. In this case they're stochastic, which isn't something people are used to happening with computers yet. You have to understand that and learn how to use them or you will get poor results.
usefulcat 9mo ago
I doubt that it's common for anyone to read a research paper and then question whether the researcher's calculator was working reliably. Sure, maybe someday LLMs will be able to report facts in a mostly reliable fashion (like a typical calculator), but we're definitely not even close to that yet, so until we are the skepticism is very much warranted. Especially when the details really do matter, as in scientific research.
- mapontosevenths 9mo ago
  > whether the researcher's calculator was working reliably. LLM's do not work reliably, that's not their purpose. If you use them that way it's akin to using a butter knife as a screwdriver. You might get away with it once or twice, but then you slip and stab yourself. Better to go find screwdriver if you need reliable.
- westurner 9mo ago
  > I doubt that it's common for anyone to read a research paper and then question whether the researcher's calculator was working reliably Reproducibility and repeatability in the sciences? Replication crisis > Causes > Problems with the publication system in science > Mathematical errors; Causes > Questionable research practices > In AI research, Remedies > [..., open science, reproducible workflows, disclosure, ] https://en.wikipedia.org/wiki/Replication_crisis#Mathematical_errors https://en.wikipedia.org/wiki/Replication_crisis#Mathematica... Already verifiable proofs are too impossibly many pages for human review There are "verify each Premise" and "verify the logical form of the Argument" (P therefore Q) steps that still the model doesn't do for the user. For your domain, how insufficient is the output given process as a prompt like: Identify hallucinations from models prior to (date in the future) Check each sentence of this: ```{...}``` Research ScholarlyArticles (and then their Datasets) which support and which reject your conclusions. Critically review findings and controls. Suggest code to write to apply data science principles to proving correlative and causative relations given already-collected observations. Design experiment(s) given the scientific method to statistically prove causative (and also correlative) relations Identify a meta-analytic workflow (process, tools, schema, and maybe code) for proving what is suggested by this chat
andrepd 9mo ago
> Those new fangled doohickeys just aren't reliable Except they are (unlike a chatbot, a calculator is perfectly deterministic), and the unreliability of LLMs is one of their most, if not the most, widespread target of criticism. Low effort doesn't even begin to describe your comment.
- mapontosevenths 9mo ago
  > Except they are (unlike a chatbot, a calculator is perfectly deterministic) LLM's are supposed to be stochastic. That is not a bug, I can see why you find that disappointing but it's just the reality of the tool. However, as I mentioned elsewhere calculators also have bugs and those bugs make their way into scientific research all the time. Floating point errors are particularly common, as are order of operations problems because physical devices get it wrong frequently and are hard to patch. Worse, they are not SUPPOSED TO BE stochastic so when they fail nobody notices until it's far too late. [0 - PDF] Further, spreadsheets are no better, for example a scan of ~3,600 genomics papers found that about 1 in 5 had gene‑name errors (e.g., SEPT2 → “2‑Sep”) because that's how Excel likes to format things.[1] Again, this is much worse than a stochastic machine doing it's stochastic job... because it's not SUPPOSED to be random, it's just broken and on a truly massive scale. [0] https://ttu-ir.tdl.org/server/api/core/bitstreams/7fce5b73-18c9-4d5c-af68-57915cb9dabe/content https://ttu-ir.tdl.org/server/api/core/bitstreams/7fce5b73-1... [1]https://www.washingtonpost.com/news/wonk/wp/2016/08/26/an-alarming-number-of-scientific-papers-contain-excel-errors/ https://www.washingtonpost.com/news/wonk/wp/2016/08/26/an-al...
  - raddan 9mo ago
    That’s a strange argument. There are plenty of stochastic processes that have perfectly acceptable guarantees. A good example is Karger’s min-cut algorithm. You might not know what you get on any given single run, but you know EXACTLY what you’re going to get when you crank up the number of trials. Nobody can tell you what you are going to get when you run an LLM once. Nobody can tell you what you’re going to get when you run it N times. There are, in fact, no guarantees at all. Nobody even really knows why it can solve some problems and why it can’t solve other except maybe it memorized the answer at some point. But this is not how they are marketed. They are marketed as wondrous inventions that can SOLVE EVERYTHING. This is obviously not true. You can verify it yourself, with a simple deterministic problem: generate an arithmetic expression of length N. As you increase N, the probability that an LLM can solve it drops to zero. Ok, fine. This kind of problem is not a good fit for an LLM. But which is? And after you’ve found a problem that seems like a good fit, how do you know? Did you test it systematically? The big LLM vendors are fudging the numbers. They’re testing on the training set, they’re using ad hoc measurements and so on. But don’t take my word for it. There’s lots of great literature out there that probes the eccentricities of these models; for some reason this work rarely makes its way into the HN echo chamber. Now I’m not saying these things are broken and useless. Far from it. I use them every day. But I don’t trust anything they produce, because there are no guarantees, and I have been burned many times. If you have not been burned, you’re either exceptionally lucky, you are asking it to solve homework assignments, or you are ignoring the pain. Excel bugs are not the same thing. Most of those problems can be found trivially. You can find them because Excel is a language with clear rules (just not clear to those particular users). The problem with Excel is that people aren’t looking for bugs.

derefr 9mo ago

I would argue that an LLM is a perfectly sensible tool for structure-preserving machine translation from another language to English. (Where by "another language", you could also also substitute "very poor/non-fluent English." Though IMHO that's a bit silly, even though it's possible; there's little sense in writing in a language you only half know, when you'd get a less-lossy result from just writing in your native tongue, and then having it translate from that.) Google Translate et al were never good enough at this task to actually allow people to use the results for anything professional. Previous tools were limited to getting a rough gloss of what words in another language mean. But LLMs can be used in this way, and are being used in this way; and this is increasingly allowing non-English-fluent academics to publish papers in English-language journals (thus engaging with the English-language academic community), where previously those academics they may have felt "stuck" publishing in what few journals exist for their discipline in their own language. Would you call the use of LLMs for translation "shoddy" or "irresponsible"? To me, it'd be no more and no less "shoddy" or "irresponsible" than it would be to hire a freelance human translator to translate the paper for you. (In fact, the human translator might be a worse idea, as LLMs are more likely to understand how to translate the specific academic jargon of your discipline than a randomly-selected human translator would be.)

neilv 9mo ago
Good point. There may be a place for LLMs for science writing translation (hopefully not adding nor subtracting anything) when you're not fluent in the language of a venue. You need a way to validate the correctness of the translation, and to be able to stand behind whatever the translation says. And the translation should be disclosed on the paper.
gus_massa 9mo ago
Autotranslating technical texts is very hard. After the translation, you muct check that all the technical words were translated correctly, instead of a fancy synonym that does not make sense. (A friend has an old book translated a long time ago (by a human) from Russian to Spanish. Instead of "complex numbers", the book calls them "complicated numbers". :) )

bjourne 9mo ago

There are legitimate, non-cheating ways to use LLMs for writing. I often use the wrong verb forms ("They synthesizes the ..."), write "though" when it should be "although", and forget to comma-separate clauses. LLMs are perfect for that. Generating text from scratch, however, is wrong.

thaumasiotes 9mo ago
> I often ... write "though" when it should be "although" That is a purely imaginary "error". Anywhere you can use 'although', you are free to use 'though' instead.
- bjourne 9mo ago
  Yeah, but you cannot use although anywhere you can use though, though.
  - thaumasiotes 9mo ago
    That's true, but the one-way substitutability still means there is no such thing as "writ[ing] 'though' when it should be 'although'".
rustystump 9mo ago
I do similar proofing (esp spelling) but u need to be very careful as it will nudge u to specific styles that rob originality.
bloppe 9mo ago
I agree, but I don't think any of the broadly acceptable uses would result in easily identifiable flaws like those in the post, especially hallucinated URLs.

aydyn 9mo ago

>also plagiarism To me, this is a reminder of how much of a specific minority this forum is. Nobody I know in real life, personally or at work, has expressed this belief. I have literally only ever encountered this anti-AI extremism (extremism in the non-pejorative sense) in places like reddit and here. Clearly, the authors in NeurIPS don't agree that using an LLM to help write is "plagiarism", and I would trust their opinions far more than some random redditor.

fingerlocks 9mo ago
I find that hard to believe. Every creative professional that I know shares this sentiment. That’s several graphic designers at big tech companies, one person in print media, and one visual effects artist in the film industry. And once you include many of their professional colleagues that becomes a decent sample size.
- jama211 9mo ago
  Graphic design is a completely different kettle of fish. Comparing it to academic paper writing is disingenuous.
  - kortilla 9mo ago
    The thread is about not knowing anyone at all who thinks AI is plagiarizing.
    - jama211 9mo ago
      Yes, plagiarising text based content. No one in this thread meant graphics.
jama211 9mo ago
Yup, and no matter how flimsy an anti-ai article is, it will skyrocket to the top of HN because of it. It makes sense though, HN users are the most likely to feel threatened by LLMs, and therefore are more likely to be anxious about them. I don’t love ai either, but that’s the truth.

thomasahle 9mo ago

> And also plagiarism, when you claim authorship of it. I don't actually mind putting Claude as a co-author on my github commits. But for papers there are usually so many tools involved. It would be crowded to include each of Claude, Gemini, Codex, Mathematica, Grammarly, Translate etc. as co-authors, even though I used all of them for some parts. Maybe just having a "tools used" section could work?

the__alchemist 9mo ago
I suspect the parent post was concerned about plagiarizing the author of training data; not software tools.

piyh 9mo ago

>I am troubled by people using an LLM at all to write academic research papers. I'm an outsider to the academic system. I have cool projects that I feel push some niche application to SOTA in my tiny little domain, which is publishable based on many of the papers I've read. If I can build a system that does a thing, I can benchmark and prove it's better than previous papers, my main blocker is getting all my work and information into the "Arxiv PDF" format and tone. Seems like a good use of LLMs to me.