22 ms·

Could a large language model help with this? Like, train it on papers, then feed it your rejected draft and ask it to generate an improved draft that sounded m

Could a large language model help with this? Like, train it on papers, then feed it your rejected draft and ask it to generate an improved draft that sounded more like the training data? The idea would be to use the expected phrasings and idioms, format, etc. Of course, all content would have to be checked, as LLMs are prone to BS. But if the problems were mostly that it didn't follow the conventions of that academic subgenre, this seems exactly what an LLM could do. If the problem was more about e.g. not citing previous work, the LLM probably wouldn't do a good job there.

thomasmg 4y ago
Actually I think it would help quite a lot in what I struggled with! > as LLMs are prone to BS I agree. The output sounds very convincing, and on a high-level it might make sense. If there are 3 layers: (a) high-level idea, (b) more detailed ideas, and (c) choosing the right words, tone, syntax, grammar, then LLMs seem to be very good in (a) and (c), but quite bad at (b). For somebody not familiar with the topic, it's hard to detect this.
- gcr 4y ago
  There are also good ways of using LLMs to write better papers that don't even require the LLM to contribute to the text! Consider feeding your outline through the LLM alongside prompts like: - "What's missing from this draft that a reader would expect to find in a top-tier conference paper?" - "Do you see any problems with the {experiments,discussion,background,related work} section?" - "What previous work should this paper cite that an expert would find relevant in this field?" (be sure to double-check the sources, LLMs tend to hallucinate with prompts like this) - "What points is an expert in the field likely to raise during peer review?" - "Which parts should be rephrased to make the paper sound more natural to a native English speaker?" As an occasional peer reviewer, I could imagine conferences having a problem with GPT-generated paper submissions clogging up an already highly competitive acceptance pipeline. In case conferences start using detection tools to filter such manuscripts (and I'm torn on whether they should tbh), you can still use ChatGPT like a "friendly, expert colleague" who offers suggestions that you then draft yourself. I think the application of ChatGPT that I like the most is where ChatGPT is a "friendly, expert editor that always has time to give you suggestions and offer advice," tightening the feedback loop between author and editor to help the author improve their writing. Think of it like a "human-learner-in-the-loop" AI system rather than a "replaces-a-human" AI system.

thomasmg 4y ago

Actually I think it would help quite a lot in what I struggled with! > as LLMs are prone to BS I agree. The output sounds very convincing, and on a high-level it might make sense. If there are 3 layers: (a) high-level idea, (b) more detailed ideas, and (c) choosing the right words, tone, syntax, grammar, then LLMs seem to be very good in (a) and (c), but quite bad at (b). For somebody not familiar with the topic, it's hard to detect this.

gcr 4y ago
There are also good ways of using LLMs to write better papers that don't even require the LLM to contribute to the text! Consider feeding your outline through the LLM alongside prompts like: - "What's missing from this draft that a reader would expect to find in a top-tier conference paper?" - "Do you see any problems with the {experiments,discussion,background,related work} section?" - "What previous work should this paper cite that an expert would find relevant in this field?" (be sure to double-check the sources, LLMs tend to hallucinate with prompts like this) - "What points is an expert in the field likely to raise during peer review?" - "Which parts should be rephrased to make the paper sound more natural to a native English speaker?" As an occasional peer reviewer, I could imagine conferences having a problem with GPT-generated paper submissions clogging up an already highly competitive acceptance pipeline. In case conferences start using detection tools to filter such manuscripts (and I'm torn on whether they should tbh), you can still use ChatGPT like a "friendly, expert colleague" who offers suggestions that you then draft yourself. I think the application of ChatGPT that I like the most is where ChatGPT is a "friendly, expert editor that always has time to give you suggestions and offer advice," tightening the feedback loop between author and editor to help the author improve their writing. Think of it like a "human-learner-in-the-loop" AI system rather than a "replaces-a-human" AI system.