4 ms·

I'm amazed that such a simple method of detection worked so flawlessly for so many people. This would not work for those who merely used LLMs to help pinpoint s

I'm amazed that such a simple method of detection worked so flawlessly for so many people. This would not work for those who merely used LLMs to help pinpoint strengths and weaknesses in the paper; there are separate techniques to judge that. Instead, it only detects those who quite literally copied and pasted the LLM output as a review. It's incredible how so many people thought it was fair that their paper should be assessed by human reviewers alone, and yet would not extend the same courtesy to others.

everdrive 7mo ago
Generally speaking people have worse impulse control than they believe they do. Once you give a tool that does most of the work for you, very very few people will actually be able to use that tool in truly enriching ways. The majority of people (even the smart ones) will weaken over time and take shortcuts.
- hodgehog11 7mo ago
  That's an excellent point. It seems likely they thought they could operate as a proper reviewer, but when the deadline came, they took the shortcut they knew they were not supposed to take. It really does sound like an addiction when you put it this way.
- jacquesm 7mo ago
  I have a very simple solution to this but it is a bit expensive. I run two laptops, one that I talk to an LLM on and another where I do all my work and which is my main machine. The LLM is strictly there in a consulting role, I've done some coding experiments as well (see previous comments) but nothing that stood out to me as a major improvement. The trick is: I can't cut-and-paste between the two machines. So there is never even a temptation to do so and I can guarantee that my writing or other professional output will never be polluted. Because like you I'm well aware of that poor impulse control factor and I figured the only way to really solve this is to make sure it can not happen.
  - jjgreen 7mo ago
    You could ssh in to the "dirty" machine ... just sayin'
    - jacquesm 7mo ago
      Yes, I could. But I've purposefully made linking the two quite hard.
      - jjgreen 7mo ago
        People do similar things installing an app locking their phones so they don't spend all day on them, then "oh just this once", ...
  - manbash 7mo ago
    This somewhat of the equivalent of "quitting cold turkey", in the sense that you remove the temptation from your reach. The problem is that it's just much easier to un-quit and run the LLM in the same laptop you work on. It's just so very tempting.
    - jacquesm 7mo ago
      I think that's the only way to deal with such temptations. Kidding yourself that you are strong enough to do it 'just once' or that you can handle the temptation is foolish and will only lead to predictable outcomes. I have a similar policy to smoking, drugs, alcohol and so on, I just don't want the temptation. It helps to have seen lots of people who thought they were smart enough eventually go under (but the price is pretty high). Oh, and LLMs are of course geared to pull you in further, they are on a continuous upsell salespitch. Drug pushers could learn a thing or two from them.
  - everdrive 7mo ago
    This is a nice solution, but I think it speaks to just how enticing the problem is. This is the sort of tactic someone with a gambling addiction would employ. I don't say that to be rude to you: I've had to do similar things with regard to addicting infinite-scroll internet sites, and I definitely give in more than I'd like.
- retsibsi 7mo ago
  I think you're framing this behaviour too generously. Laziness is one thing, lack of integrity is another, and this seems to be a straightforward case of cheating and lying.
  - everdrive 7mo ago
    I think it's just numbers. When one person errs it's a fault of character. When most people err, we call it a systematic fault. Why are most people overweight for the first time in history? Do most people lack the good character to restrict their diet? You could argue yes, however appeals to character won't actually solve the problem.
bonoboTP 7mo ago
I'm not surprised at all. The ML research community isn't a community any more, it's turned into a dog-eat-dog low-trust fierce competition. So much more people, papers, churn, that everyone is just fending for themselves. Any moment that you charitably spend on community service can be felt as a moment you take away from the next project, jeopardizing the next paper, getting scooped, delaying your graduation, your contract, your funding, your visa, your residence permit, your industry plans etc. It's a machine. I don't think people outside the phd system really understand the incentives involved. People are offered very little slack in this system. It's sink or swim, with very little instruction or scientific culture or integrity getting passed on. The PhD students see their supervisors cut corners all the time too, authorship bullshit jockeying even in big name labs etc. People I talked to are quite disillusioned, expect their work to have little impact and get superseded by a new better model in a few months so it's all about who can grind faster, who can twist the benchmarks into showing a minimal improvement etc. And the starry eyed novices get slapped by reality into thinking this way fairly early. To be clear this is not an excuse but an explanation why I am not surprised.
- matusp 7mo ago
  And the real punchline is that the deluge of papers barely matters, as the academic field is barely moving, and the most interesting innovations are happening on the product side.
  - bonoboTP 7mo ago
    I disagree with this. Usually the products are based on published research. This is not easily seen by the enthusiast power user base. Of course it's only a small fraction of all papers that end up actually being used. Most are mainly about advancing careers and strengthening CVs.
    - nis0s 7mo ago
      One key reason you’re wrong is that many interesting things aren’t even getting published, they’re on the DL for years and eventually make it to public spheres and products. Academia is just a daycare at this point, and many labs shouldn’t exists or get funding. The people who move the field aren’t necessarily the ones with the most citations, they’re usually hard at work in places that don’t publish at all.
jacquesm 7mo ago
This is 'spam' all over again. Before spam every email was valuable and required some attention. It was a better version of paper mail in that it was faster and cheaper. But then the spam thing happened and suddenly being 'faster and cheaper' was no longer an advantage, it was a massive drawback. But by then there was no way back. I think LLMs will do the same with text in general. By making the production of text faster and cheaper the value of all text will diminish, quite probably to something very close to the energy value of the bits that carry the data.

everdrive 7mo ago

Generally speaking people have worse impulse control than they believe they do. Once you give a tool that does most of the work for you, very very few people will actually be able to use that tool in truly enriching ways. The majority of people (even the smart ones) will weaken over time and take shortcuts.

hodgehog11 7mo ago
That's an excellent point. It seems likely they thought they could operate as a proper reviewer, but when the deadline came, they took the shortcut they knew they were not supposed to take. It really does sound like an addiction when you put it this way.
jacquesm 7mo ago
I have a very simple solution to this but it is a bit expensive. I run two laptops, one that I talk to an LLM on and another where I do all my work and which is my main machine. The LLM is strictly there in a consulting role, I've done some coding experiments as well (see previous comments) but nothing that stood out to me as a major improvement. The trick is: I can't cut-and-paste between the two machines. So there is never even a temptation to do so and I can guarantee that my writing or other professional output will never be polluted. Because like you I'm well aware of that poor impulse control factor and I figured the only way to really solve this is to make sure it can not happen.
- jjgreen 7mo ago
  You could ssh in to the "dirty" machine ... just sayin'
  - jacquesm 7mo ago
    Yes, I could. But I've purposefully made linking the two quite hard.
    - jjgreen 7mo ago
      People do similar things installing an app locking their phones so they don't spend all day on them, then "oh just this once", ...
- manbash 7mo ago
  This somewhat of the equivalent of "quitting cold turkey", in the sense that you remove the temptation from your reach. The problem is that it's just much easier to un-quit and run the LLM in the same laptop you work on. It's just so very tempting.
  - jacquesm 7mo ago
    I think that's the only way to deal with such temptations. Kidding yourself that you are strong enough to do it 'just once' or that you can handle the temptation is foolish and will only lead to predictable outcomes. I have a similar policy to smoking, drugs, alcohol and so on, I just don't want the temptation. It helps to have seen lots of people who thought they were smart enough eventually go under (but the price is pretty high). Oh, and LLMs are of course geared to pull you in further, they are on a continuous upsell salespitch. Drug pushers could learn a thing or two from them.
- everdrive 7mo ago
  This is a nice solution, but I think it speaks to just how enticing the problem is. This is the sort of tactic someone with a gambling addiction would employ. I don't say that to be rude to you: I've had to do similar things with regard to addicting infinite-scroll internet sites, and I definitely give in more than I'd like.
retsibsi 7mo ago
I think you're framing this behaviour too generously. Laziness is one thing, lack of integrity is another, and this seems to be a straightforward case of cheating and lying.
- everdrive 7mo ago
  I think it's just numbers. When one person errs it's a fault of character. When most people err, we call it a systematic fault. Why are most people overweight for the first time in history? Do most people lack the good character to restrict their diet? You could argue yes, however appeals to character won't actually solve the problem.

bonoboTP 7mo ago

I'm not surprised at all. The ML research community isn't a community any more, it's turned into a dog-eat-dog low-trust fierce competition. So much more people, papers, churn, that everyone is just fending for themselves. Any moment that you charitably spend on community service can be felt as a moment you take away from the next project, jeopardizing the next paper, getting scooped, delaying your graduation, your contract, your funding, your visa, your residence permit, your industry plans etc. It's a machine. I don't think people outside the phd system really understand the incentives involved. People are offered very little slack in this system. It's sink or swim, with very little instruction or scientific culture or integrity getting passed on. The PhD students see their supervisors cut corners all the time too, authorship bullshit jockeying even in big name labs etc. People I talked to are quite disillusioned, expect their work to have little impact and get superseded by a new better model in a few months so it's all about who can grind faster, who can twist the benchmarks into showing a minimal improvement etc. And the starry eyed novices get slapped by reality into thinking this way fairly early. To be clear this is not an excuse but an explanation why I am not surprised.

matusp 7mo ago
And the real punchline is that the deluge of papers barely matters, as the academic field is barely moving, and the most interesting innovations are happening on the product side.
- bonoboTP 7mo ago
  I disagree with this. Usually the products are based on published research. This is not easily seen by the enthusiast power user base. Of course it's only a small fraction of all papers that end up actually being used. Most are mainly about advancing careers and strengthening CVs.
  - nis0s 7mo ago
    One key reason you’re wrong is that many interesting things aren’t even getting published, they’re on the DL for years and eventually make it to public spheres and products. Academia is just a daycare at this point, and many labs shouldn’t exists or get funding. The people who move the field aren’t necessarily the ones with the most citations, they’re usually hard at work in places that don’t publish at all.

jacquesm 7mo ago

This is 'spam' all over again. Before spam every email was valuable and required some attention. It was a better version of paper mail in that it was faster and cheaper. But then the spam thing happened and suddenly being 'faster and cheaper' was no longer an advantage, it was a massive drawback. But by then there was no way back. I think LLMs will do the same with text in general. By making the production of text faster and cheaper the value of all text will diminish, quite probably to something very close to the energy value of the bits that carry the data.