3 ms·

Unless you’re very weak-willed, I wouldn’t worry about independence. LLMs make mistakes all the time, they are nothing like an all-knowing oracle nor are they g

Unless you’re very weak-willed, I wouldn’t worry about independence. LLMs make mistakes all the time, they are nothing like an all-knowing oracle nor are they going to replace humans despite the absurd fantasies of LLM fans and those with a vested interest. Plagiarism is an interesting point, though honestly I think it would be fairly easy to work out who had published similar research if the LLM gives you an idea - personally I see this as the weakest argument against using them, as long as you are strict about attribution - all work like this depends heavily on the research of others - the LLM is just a tool to aid that research IMO. Opsec is a fair point, and it might be worth avoiding the completely amoral OpenAI at this point for that reason. There are open models though.

achierius 28d ago
> despite the absurd fantasies of LLM fans Absurd fantasies like solving a Millennium Prize Problem? How is this not prima facie absurd? And if that's come to pass, why should we believe your bar for anything else?
- grey-area 28d ago
  Yes absurd fantasies like it did that without human help and guidance. OpenAI found out there was a solution and humans attempted to brute force generating all possible solutions while threatening the mathematicians involved (generous interpretation), or stole some ideas and took shortcuts to know where to look and brute forced it so they could claim credit. Neither looks good for OpenAI or those who support them.
  - achierius 19d ago
    And yet they solved it. You don't think mathematicians have been trying to brute force a counterexample for decades now? Only with AI did it actually work. Even the researchers that got scooped were using AI to drive the work -- not a dig at them, they were certainly wronged here, but just to point out that AI was essential in this. Yes there was human involvement. But that does ~nothing to change the fact that this is an incredible leap forward, and one that should be viewed with deep concern.
YeGoblynQueenne 28d ago
Thanks. Regarding plagiarism, I agree that it is possible to check; I don't know how easy it is. But I do note that the mathematicians who complained that their work was plagiarised by AI did not seem to realise that they, themselves, were using a plagiarism machine and were instead quite comfortable admitting that, yeah, we worked our Euler result out together with Sol, Claude and the gang. To clarify, they do attribute the original ideas to Cordoba and Martinez-Zoroa, but they don't seem to acknowledge that their result which they say was achieved "with a great deal of help from LLMs" is also derivative of others' work. Independence is about avoiding making errors because of the influence of error-prone models. I guess I didn't explain it well.
- grey-area 28d ago
  I think plagiarism machine is a stretch personally in this domain. In art or writing I could see it persuasively argued (see attempts to generate famous books or imitate illustrators). Academics and scientists build on the work of many others and always have - their work is not possible without using other’s work. Attribution is a problem here but I don’t think a new way to reference many others’ research and combine it in novel ways is the problem or should be rejected on that basis alone.

achierius 28d ago

> despite the absurd fantasies of LLM fans Absurd fantasies like solving a Millennium Prize Problem? How is this not prima facie absurd? And if that's come to pass, why should we believe your bar for anything else?

grey-area 28d ago
Yes absurd fantasies like it did that without human help and guidance. OpenAI found out there was a solution and humans attempted to brute force generating all possible solutions while threatening the mathematicians involved (generous interpretation), or stole some ideas and took shortcuts to know where to look and brute forced it so they could claim credit. Neither looks good for OpenAI or those who support them.
- achierius 19d ago
  And yet they solved it. You don't think mathematicians have been trying to brute force a counterexample for decades now? Only with AI did it actually work. Even the researchers that got scooped were using AI to drive the work -- not a dig at them, they were certainly wronged here, but just to point out that AI was essential in this. Yes there was human involvement. But that does ~nothing to change the fact that this is an incredible leap forward, and one that should be viewed with deep concern.

YeGoblynQueenne 28d ago

Thanks. Regarding plagiarism, I agree that it is possible to check; I don't know how easy it is. But I do note that the mathematicians who complained that their work was plagiarised by AI did not seem to realise that they, themselves, were using a plagiarism machine and were instead quite comfortable admitting that, yeah, we worked our Euler result out together with Sol, Claude and the gang. To clarify, they do attribute the original ideas to Cordoba and Martinez-Zoroa, but they don't seem to acknowledge that their result which they say was achieved "with a great deal of help from LLMs" is also derivative of others' work. Independence is about avoiding making errors because of the influence of error-prone models. I guess I didn't explain it well.

grey-area 28d ago
I think plagiarism machine is a stretch personally in this domain. In art or writing I could see it persuasively argued (see attempts to generate famous books or imitate illustrators). Academics and scientists build on the work of many others and always have - their work is not possible without using other’s work. Attribution is a problem here but I don’t think a new way to reference many others’ research and combine it in novel ways is the problem or should be rejected on that basis alone.