4 ms·

I will respectfully disagree. All "new" ideas come from old ideas. AI is a tool to access old ideas with speed and with new perspectives that hasn't been avail

I will respectfully disagree. All "new" ideas come from old ideas. AI is a tool to access old ideas with speed and with new perspectives that hasn't been available up until now. Innovation is in the cracks: recognition of holes, intersections, tangents, etc. on old ideas. It has bent said that innovation is done on the shoulders of giants. So AI can be an express elevator up to an army of giant's shoulders? It all depends on how you use the tools.

gametorch 1y ago
Exactly! Can you imagine if we applied the same gatekeeping logic to science? Imagine you weren't allowed to use someone else's scientific work or any derivative of it. We would make no progress. The only legitimate defense I have ever seen here revolves around IP and copyright infringement, which I couldn't care less about.
alfalfasprout 1y ago
Access old ideas? Yes. With new perspectives? Not necessarily. An LLM may be able to assist in interpreting data with new perspectives but in practice they're still fairly bad at greenfield work. As with most things, the truth lies somewhere in the middle. LLMs can be helpful as a way of accelerating certain kinds and certain aspects of research but not others.
- stevep98 1y ago
  > Access old ideas? Yes. With new perspectives? I wonder if we can mine patent databases for old ideas that never worked out in the past, but now are more useful. Perhaps due to modern machining or newer materials or just new applications of the idea.
deleted 1y ago
[deleted]
deleted 1y ago
[deleted]
bcrosby95 1y ago
The article is discussing working in AI innovation vs focusing on getting more and better data. And while there have been key breakthroughs in new ideas, one of the best ways to increase the performance of these systems is getting more and better data. And how many people think data is the primary avenue to improvement. It reminds me of an AI talk a few decades ago, about how the cycle goes: more data -> more layers -> repeat... Anyways, I'm not sure how your comment relates to these two avenues of improvement.
jjtheblunt 1y ago
> I will respectfully disagree. All "new" ideas come from old ideas. The insight into the structure of the benzene ring famously came in a dream, hadn't been seen before, but was imagined as a snake bitings its own tail.
- troupo 1y ago
  And as we all know, it came in a dream to a complete novice in chemistry with zero knowledge of any old ideas in chemistry: https://en.wikipedia.org/wiki/August_Kekul%C3%A9 https://en.wikipedia.org/wiki/August_Kekul%C3%A9 --- start quote --- The empirical formula for benzene had been long known, but its highly unsaturated structure was a challenge to determine. Archibald Scott Couper in 1858 and Joseph Loschmidt in 1861 suggested possible structures that contained multiple double bonds or multiple rings, but the study of aromatic compounds was in its earliest years, and too little evidence was then available to help chemists decide on any particular structure. More evidence was available by 1865, especially regarding the relationships of aromatic isomers. [ Kekule claimed to have had the dream in 1865 ] --- end quote --- The dream claim came from Kekule himself 25 years after his proposal that he had to modify 10 years after he proposed it.
baxtr 1y ago
Imagine a human had read every book/publication in every field of knowledge that mankind has ever produced AND couldn’t come up with anything entirely new. Hard to imagine.
- mdaniel 1y ago
  My hypothesis of the mismatch is centered around "read" - I think that when you wrote it, and when others similarly think about that scenario, the surprise is because our version of "read" is the implied "read and internalized" or at bare minimum "read for comprehension" but as very best I can tell the LLM's version is "encoded tokens into vector space" and not "encoded into semantic graph" I welcome the hair-splittery that is sure to follow about what it means to "understand" anything
  - 8n4vidtmkvmk 1y ago
    That's the point, isn't it? The missing link. AIs can't yet truly comprehend, or internalize, or whatever you want to call it. That's probably equivalent to AGI or singularity. We're not there yet. Feeding copious amounts of data into existing architecture won't get us there either. A human with all that data, if it could fit in their brain, would likely come up with something interesting. Even then... I'm not entirely sure it's so simple. I'd wager most of us have enough knowledge in our brains today to come up with something if we applied ourselves, but ideas don't spontaneously appear just because the knowledge is there. What if we take our AI models and force them to continuously try making connections between unlikely things? The novel stuff is likely in the parts that don't already have strong connections because research is lacking but could. But how would it evaluate what's interesting?
- hugh-avherald 1y ago
  It is possible that such a human wouldn't come up with anything new, even if they could.
- melagonster 1y ago
  The difficult part is proposing an experiment to check a new idea.

gametorch 1y ago

Exactly! Can you imagine if we applied the same gatekeeping logic to science? Imagine you weren't allowed to use someone else's scientific work or any derivative of it. We would make no progress. The only legitimate defense I have ever seen here revolves around IP and copyright infringement, which I couldn't care less about.

alfalfasprout 1y ago

Access old ideas? Yes. With new perspectives? Not necessarily. An LLM may be able to assist in interpreting data with new perspectives but in practice they're still fairly bad at greenfield work. As with most things, the truth lies somewhere in the middle. LLMs can be helpful as a way of accelerating certain kinds and certain aspects of research but not others.

stevep98 1y ago
> Access old ideas? Yes. With new perspectives? I wonder if we can mine patent databases for old ideas that never worked out in the past, but now are more useful. Perhaps due to modern machining or newer materials or just new applications of the idea.

deleted 1y ago

[deleted]

deleted 1y ago

[deleted]

bcrosby95 1y ago

The article is discussing working in AI innovation vs focusing on getting more and better data. And while there have been key breakthroughs in new ideas, one of the best ways to increase the performance of these systems is getting more and better data. And how many people think data is the primary avenue to improvement. It reminds me of an AI talk a few decades ago, about how the cycle goes: more data -> more layers -> repeat... Anyways, I'm not sure how your comment relates to these two avenues of improvement.

jjtheblunt 1y ago

> I will respectfully disagree. All "new" ideas come from old ideas. The insight into the structure of the benzene ring famously came in a dream, hadn't been seen before, but was imagined as a snake bitings its own tail.

troupo 1y ago
And as we all know, it came in a dream to a complete novice in chemistry with zero knowledge of any old ideas in chemistry: https://en.wikipedia.org/wiki/August_Kekul%C3%A9 https://en.wikipedia.org/wiki/August_Kekul%C3%A9 --- start quote --- The empirical formula for benzene had been long known, but its highly unsaturated structure was a challenge to determine. Archibald Scott Couper in 1858 and Joseph Loschmidt in 1861 suggested possible structures that contained multiple double bonds or multiple rings, but the study of aromatic compounds was in its earliest years, and too little evidence was then available to help chemists decide on any particular structure. More evidence was available by 1865, especially regarding the relationships of aromatic isomers. [ Kekule claimed to have had the dream in 1865 ] --- end quote --- The dream claim came from Kekule himself 25 years after his proposal that he had to modify 10 years after he proposed it.

baxtr 1y ago

Imagine a human had read every book/publication in every field of knowledge that mankind has ever produced AND couldn’t come up with anything entirely new. Hard to imagine.

mdaniel 1y ago
My hypothesis of the mismatch is centered around "read" - I think that when you wrote it, and when others similarly think about that scenario, the surprise is because our version of "read" is the implied "read and internalized" or at bare minimum "read for comprehension" but as very best I can tell the LLM's version is "encoded tokens into vector space" and not "encoded into semantic graph" I welcome the hair-splittery that is sure to follow about what it means to "understand" anything
- 8n4vidtmkvmk 1y ago
  That's the point, isn't it? The missing link. AIs can't yet truly comprehend, or internalize, or whatever you want to call it. That's probably equivalent to AGI or singularity. We're not there yet. Feeding copious amounts of data into existing architecture won't get us there either. A human with all that data, if it could fit in their brain, would likely come up with something interesting. Even then... I'm not entirely sure it's so simple. I'd wager most of us have enough knowledge in our brains today to come up with something if we applied ourselves, but ideas don't spontaneously appear just because the knowledge is there. What if we take our AI models and force them to continuously try making connections between unlikely things? The novel stuff is likely in the parts that don't already have strong connections because research is lacking but could. But how would it evaluate what's interesting?
hugh-avherald 1y ago
It is possible that such a human wouldn't come up with anything new, even if they could.
melagonster 1y ago
The difficult part is proposing an experiment to check a new idea.