3 ms·

> Is it standard practice for authors to have to defend their submissions via interview like this? If not, why not? It isn't, but maybe it should be. For post-

> Is it standard practice for authors to have to defend their submissions via interview like this? If not, why not? It isn't, but maybe it should be. For post-grad qualifications oral defense is standard, and I didn't mind defending my central thesis then, and won't mind now.

doc_ick 16d ago
Not in a direct interview style, but most us conferences can request additional information or feedback. If they conditionally accept or reject a paper, that conditional relies on feedback from the author(s). Interviews like this are interesting, but in no way can scale to the infinite paper slop conferences are facing.
- lelanthran 16d ago
  > Not in a direct interview style, but most us conferences can request additional information or feedback. If they conditionally accept or reject a paper, that conditional relies on feedback from the author(s). Requesting feedback is useless, as the article points out - the "authors" could not answer basic questions during the interview, but after the interview were able to send full explanations to the interviewer. If you have indirect feedback ("please answer these questions we have") the "author" will simply feed it into an LLM and send the results back. You need to get the author to do an oral defense to verify that they wrote the paper. This is the main problem with AI generated output, whether it's a research paper, a blog, an email, a comment on a forum, similar: the value in knowing that a human wrote $X sends a signal - that the human understands what it is they wrote, even if they misunderstand the concepts. When you get a message from someone who is a "I only used an LLM to clean it up, the thoughts are all mine"[1] person, you cannot engage with them, because they may not understand the message they transmitted, and so any human engaging with them is only burning their own time for no gain. When you get a message from a real person, you get not only the message, you also get a signal about their understanding. That signal is missing in AI generated messages. ========================== [1] Sure, buddy. We believe you /s.
  - CPLX 16d ago
    > I only used an LLM to clean it up, the thoughts are all mine" I mean, it seems to me like that should be nearly everybody at this point, right? There are at least some elements of using LLMs that are the equivalent of a spell check, like asking to make sure that links work and that references are to the thing they're supposed to be, and so on. I feel like the only difference at this point is between people who do that and say that they do, and people who do that and don't say that they do.
    - lelanthran 16d ago
      > I feel like the only difference at this point is between people who do that and say that they do, and people who do that and don't say that they do. No. Did you "clean this up" with an LLM before posting it? I hate to go there, but how is that argument different from a thief who argues "Everyone steals. The only difference is some of them claim they do, and some claim they don't"? > I mean, it seems to me like that should be nearly everybody at this point, right? There are at least some elements of using LLMs that are the equivalent of a spell check, like asking to make sure that links work and that references are to the thing they're supposed to be, and so on. Right. Maybe. The problem is that when material has all the tells of LLM generation, do you expect the reader to figure out if the thoughts were the authors or introduced by the LLM? The value in a human directly communicating with you is that you understand what they are saying; if they misunderstand, you spot their misunderstanding. If they are talking at cross-purposes, you know you are arguing past each other. If they agree with you, you know they agree with you. That signal is missing when the communication is LLM generated; you message said "Not X, Not Y, just Z", but I can't tell if you even know what X, Y and Z are. f I ask you to give me a definition for them so I can ensure we aren't talking about different things, the LLM response will be "X is $SOMETHING", but I still can't tell if you think that X is $SOMETHING or if you still think that X is $SOMETHING_ELSE while your LLM and I agree that X is $SOMETHING. Communication from a person tells us something about that person, outside of the message being communicated. If that communication is relayed via an non-invested third party, the missing signal prevents any communication.
      - 16d ago

doc_ick 16d ago

Not in a direct interview style, but most us conferences can request additional information or feedback. If they conditionally accept or reject a paper, that conditional relies on feedback from the author(s). Interviews like this are interesting, but in no way can scale to the infinite paper slop conferences are facing.

lelanthran 16d ago
> Not in a direct interview style, but most us conferences can request additional information or feedback. If they conditionally accept or reject a paper, that conditional relies on feedback from the author(s). Requesting feedback is useless, as the article points out - the "authors" could not answer basic questions during the interview, but after the interview were able to send full explanations to the interviewer. If you have indirect feedback ("please answer these questions we have") the "author" will simply feed it into an LLM and send the results back. You need to get the author to do an oral defense to verify that they wrote the paper. This is the main problem with AI generated output, whether it's a research paper, a blog, an email, a comment on a forum, similar: the value in knowing that a human wrote $X sends a signal - that the human understands what it is they wrote, even if they misunderstand the concepts. When you get a message from someone who is a "I only used an LLM to clean it up, the thoughts are all mine"[1] person, you cannot engage with them, because they may not understand the message they transmitted, and so any human engaging with them is only burning their own time for no gain. When you get a message from a real person, you get not only the message, you also get a signal about their understanding. That signal is missing in AI generated messages. ========================== [1] Sure, buddy. We believe you /s.
- CPLX 16d ago
  > I only used an LLM to clean it up, the thoughts are all mine" I mean, it seems to me like that should be nearly everybody at this point, right? There are at least some elements of using LLMs that are the equivalent of a spell check, like asking to make sure that links work and that references are to the thing they're supposed to be, and so on. I feel like the only difference at this point is between people who do that and say that they do, and people who do that and don't say that they do.
  - lelanthran 16d ago
    > I feel like the only difference at this point is between people who do that and say that they do, and people who do that and don't say that they do. No. Did you "clean this up" with an LLM before posting it? I hate to go there, but how is that argument different from a thief who argues "Everyone steals. The only difference is some of them claim they do, and some claim they don't"? > I mean, it seems to me like that should be nearly everybody at this point, right? There are at least some elements of using LLMs that are the equivalent of a spell check, like asking to make sure that links work and that references are to the thing they're supposed to be, and so on. Right. Maybe. The problem is that when material has all the tells of LLM generation, do you expect the reader to figure out if the thoughts were the authors or introduced by the LLM? The value in a human directly communicating with you is that you understand what they are saying; if they misunderstand, you spot their misunderstanding. If they are talking at cross-purposes, you know you are arguing past each other. If they agree with you, you know they agree with you. That signal is missing when the communication is LLM generated; you message said "Not X, Not Y, just Z", but I can't tell if you even know what X, Y and Z are. f I ask you to give me a definition for them so I can ensure we aren't talking about different things, the LLM response will be "X is $SOMETHING", but I still can't tell if you think that X is $SOMETHING or if you still think that X is $SOMETHING_ELSE while your LLM and I agree that X is $SOMETHING. Communication from a person tells us something about that person, outside of the message being communicated. If that communication is relayed via an non-invested third party, the missing signal prevents any communication.
    - 16d ago