4 ms·

Copied text from Wikipedia somehow seems different from deriving the nobel prize winning process from a dataset that didn't have the answer in it. And downvote

Copied text from Wikipedia somehow seems different from deriving the nobel prize winning process from a dataset that didn't have the answer in it. And downvotes. The article presents this as a magic box capable of novel work, and some distance down says it found the answers on Wikipedia. Those are different things, whether you're excited about LLMs or not.

Obscurity4340 3y ago
[flagged]
devmor 3y ago
Definitely different things but I suppose “machine that finds related prior research for me effortlessly” is still quite useful.
- obmelvin 3y ago
  Yes, missing prior work at best can impact your review, and at worst can mean you waste a significant amount of research time
striking 3y ago
> In less than four minutes, Coscientist had designed an accurate procedure for producing the required reactions using chemicals provided by the team. When it sought to carry out its procedure in the physical world with robots, it made a mistake in the code it wrote to control a device that heats and shakes liquid samples. Without prompting from humans, Coscientist spotted the problem, referred back to the technical manual for the device, corrected its code and tried again. What does it matter that it looked up the reactions on Wikipedia? They're building a "coscientist", not an AI-based Nobel prize winning researcher.
- Filligree 3y ago
  > What does it matter that it looked up the reactions on Wikipedia? They're building a "coscientist", not an AI-based Nobel prize winning researcher. Well, quite. This is attempting to automate the boring, time-consuming and dangerous part of chemical research. We should be applauding.
  - gabegomes 3y ago
    You got it! That’s part of what we did, but so much more… Coscientist democratizes access to fields that one is not a deep specialist in, allowing for people to merge different ideas with ease. Biologists can do cutting-edge chemistry, chemists can try the latest in materials science, and so on.
exit 3y ago
"a magic box capable of novel work" or a box whose content can be cleanly swapped out with future iterations which already demonstrate the potential for novel discovery: https://thenextweb.com/news/deepminds-ai-finds-solution-to-decades-old-math-problem https://thenextweb.com/news/deepminds-ai-finds-solution-to-d... "Researchers claim it is the first time an LLM has made a novel scientific discovery"
- riku_iki 3y ago
  > "Researchers claim it is the first time an LLM has made a novel scientific discovery" LLM didn't make discovery on its own in that work, it played role in one step of unknown importance, other steps were lots of manual coding and lots of CPUs to brute force solution.
teepo 3y ago
I think the point was that the application includes the source attribute of the data.
- dylan604 3y ago
  so crediting prior work is the novel thing done by the AI?
bglazer 3y ago
You should read the actual paper before dismissing it. Fig 6 and the surrounding discussion shows that their system is able to iteratively optimize a reaction by adding steps and changing reaction conditions. Based on how poorly the model does on its first try, it seems unlikely that it had already just memorized the best reaction. So, they show that the model can try things, observe results, and try again. Also their model converts the "wikipedia answer" into steps that they run on a robot, which hasn't been done before for this specific reaction.
mym1990 3y ago
Novel work can often be accomplished by taking two existing methods and implementing parts of both, it doesn't have to be a eureka moment that stems from nothing. In this case, seeing what went wrong in one experiment, seeing what went right in another, combining the best of both, and re-implementing...is novel work.

Obscurity4340 3y ago

[flagged]

devmor 3y ago

Definitely different things but I suppose “machine that finds related prior research for me effortlessly” is still quite useful.

obmelvin 3y ago
Yes, missing prior work at best can impact your review, and at worst can mean you waste a significant amount of research time

striking 3y ago

> In less than four minutes, Coscientist had designed an accurate procedure for producing the required reactions using chemicals provided by the team. When it sought to carry out its procedure in the physical world with robots, it made a mistake in the code it wrote to control a device that heats and shakes liquid samples. Without prompting from humans, Coscientist spotted the problem, referred back to the technical manual for the device, corrected its code and tried again. What does it matter that it looked up the reactions on Wikipedia? They're building a "coscientist", not an AI-based Nobel prize winning researcher.

Filligree 3y ago
> What does it matter that it looked up the reactions on Wikipedia? They're building a "coscientist", not an AI-based Nobel prize winning researcher. Well, quite. This is attempting to automate the boring, time-consuming and dangerous part of chemical research. We should be applauding.
- gabegomes 3y ago
  You got it! That’s part of what we did, but so much more… Coscientist democratizes access to fields that one is not a deep specialist in, allowing for people to merge different ideas with ease. Biologists can do cutting-edge chemistry, chemists can try the latest in materials science, and so on.

exit 3y ago

"a magic box capable of novel work" or a box whose content can be cleanly swapped out with future iterations which already demonstrate the potential for novel discovery: https://thenextweb.com/news/deepminds-ai-finds-solution-to-decades-old-math-problem https://thenextweb.com/news/deepminds-ai-finds-solution-to-d... "Researchers claim it is the first time an LLM has made a novel scientific discovery"

riku_iki 3y ago
> "Researchers claim it is the first time an LLM has made a novel scientific discovery" LLM didn't make discovery on its own in that work, it played role in one step of unknown importance, other steps were lots of manual coding and lots of CPUs to brute force solution.

teepo 3y ago

I think the point was that the application includes the source attribute of the data.

dylan604 3y ago
so crediting prior work is the novel thing done by the AI?

bglazer 3y ago

You should read the actual paper before dismissing it. Fig 6 and the surrounding discussion shows that their system is able to iteratively optimize a reaction by adding steps and changing reaction conditions. Based on how poorly the model does on its first try, it seems unlikely that it had already just memorized the best reaction. So, they show that the model can try things, observe results, and try again. Also their model converts the "wikipedia answer" into steps that they run on a robot, which hasn't been done before for this specific reaction.

mym1990 3y ago

Novel work can often be accomplished by taking two existing methods and implementing parts of both, it doesn't have to be a eureka moment that stems from nothing. In this case, seeing what went wrong in one experiment, seeing what went right in another, combining the best of both, and re-implementing...is novel work.