29 ms·

There's a breakthrough that I've been waiting for that I haven't heard anything about: when will an AI agent (probably a language model) discover something scie

There's a breakthrough that I've been waiting for that I haven't heard anything about: when will an AI agent (probably a language model) discover something scientific that humans had not at the time it was trained. What if there was a math proof, physics interaction, ... that emerged from the model's approximation of our world? Right now, the state of the art AlphaZero models can destroy humans at Go. But what if the machine learning models could teach us things about how Go works that humans have not yet discovered.

SemanticStrengh 4y ago
Narrow deep learning ai is generally not suited for this. However automated theorem provers are a thing and have proven major conjectures/theorems that weren't solved by humans before. E.g. The four color problem IIRC. Although the best results are generally obtained with semi-automated theorems provers But still, this is not cleverness, this just show that raw bruteforce + a few tricks can solve a few problems, by generating proofs of multiple terabytes(yes this is absurd scaling). The asymmetry between compute power and computer lack of intelligence is remarkable. https://en.m.wikipedia.org/wiki/Automated_theorem_proving https://en.m.wikipedia.org/wiki/Automated_theorem_proving
hans1729 4y ago
It very likely already did, specifically in Go. The problem is that humans would still be required to comprehend what they are seeing :-) letting agents develop strategies in an unsupervised manner has already yielded strategies we haven’t figured out ourselves. Other examples that come to mind are video compression (see twominutepapers) and proteine folding! Think about it like this: if the domain of a problem we want AI to solve is so complex that we can barely formulate the question, how could we be confident that we can understand 100% of the answer we get? “Here, gpu, make sense of this 20-dimensional problem my brain can’t even approximately visualize!”
axg11 4y ago
You are describing most successful machine learning models. Take AlphaFold, it has surely discovered relationships that govern protein folding better than any human has ever previously understood.
- nightski 4y ago
  That's not really science though. Science requires developing a hypothesis. I've yet to see an AI do this. Not trying to raise the bar or anything, just saying let's not call it scientific.
  - tazjin 4y ago
    Developing a hypothesis is not the same as expressing one, though.
  - refulgentis 4y ago
    Folding proteins is a scientific endeavour. You are raising the bar, thought I understand you wish you weren't—metaphysical questions about science, a requirement for the AI to explain any results in natural language of meatsuits, and referring to the first step of a grade school analysis of the _scientific method_ (forming a hypothesis) as a requirement for _science_, all raise the bar.
    - nightski 4y ago
      I don't feel it was I who raised the bar. The parent citing "scientific" did. Science is a human construct in and of itself. If it is going to do science, then it needs to do so as we have defined as "the scientific method". But I would be satisfied if it developed a hypothesis in any language (including mathematics), not necessarily natural language.
jasonwatkinspdx 4y ago
In both Go and Chess the strong engines available now have already shown us human strategy was off the mark somewhat. Said simply, humans put too much weight on margin of victory vs probability of victory. In these games the difference means humans favor maintaining a material advantage more than the engines, who are more likely to trade material for a positional advantage.
bogwog 4y ago
There was a link posted here recently that talked about an AI which could identify race from xrays while trained experts could not. Maybe it discovered some new science to pull that off?

SemanticStrengh 4y ago

Narrow deep learning ai is generally not suited for this. However automated theorem provers are a thing and have proven major conjectures/theorems that weren't solved by humans before. E.g. The four color problem IIRC. Although the best results are generally obtained with semi-automated theorems provers But still, this is not cleverness, this just show that raw bruteforce + a few tricks can solve a few problems, by generating proofs of multiple terabytes(yes this is absurd scaling). The asymmetry between compute power and computer lack of intelligence is remarkable. https://en.m.wikipedia.org/wiki/Automated_theorem_proving https://en.m.wikipedia.org/wiki/Automated_theorem_proving

hans1729 4y ago

It very likely already did, specifically in Go. The problem is that humans would still be required to comprehend what they are seeing :-) letting agents develop strategies in an unsupervised manner has already yielded strategies we haven’t figured out ourselves. Other examples that come to mind are video compression (see twominutepapers) and proteine folding! Think about it like this: if the domain of a problem we want AI to solve is so complex that we can barely formulate the question, how could we be confident that we can understand 100% of the answer we get? “Here, gpu, make sense of this 20-dimensional problem my brain can’t even approximately visualize!”

axg11 4y ago

You are describing most successful machine learning models. Take AlphaFold, it has surely discovered relationships that govern protein folding better than any human has ever previously understood.

nightski 4y ago
That's not really science though. Science requires developing a hypothesis. I've yet to see an AI do this. Not trying to raise the bar or anything, just saying let's not call it scientific.
- tazjin 4y ago
  Developing a hypothesis is not the same as expressing one, though.
- refulgentis 4y ago
  Folding proteins is a scientific endeavour. You are raising the bar, thought I understand you wish you weren't—metaphysical questions about science, a requirement for the AI to explain any results in natural language of meatsuits, and referring to the first step of a grade school analysis of the _scientific method_ (forming a hypothesis) as a requirement for _science_, all raise the bar.
  - nightski 4y ago
    I don't feel it was I who raised the bar. The parent citing "scientific" did. Science is a human construct in and of itself. If it is going to do science, then it needs to do so as we have defined as "the scientific method". But I would be satisfied if it developed a hypothesis in any language (including mathematics), not necessarily natural language.

jasonwatkinspdx 4y ago

In both Go and Chess the strong engines available now have already shown us human strategy was off the mark somewhat. Said simply, humans put too much weight on margin of victory vs probability of victory. In these games the difference means humans favor maintaining a material advantage more than the engines, who are more likely to trade material for a positional advantage.

bogwog 4y ago

There was a link posted here recently that talked about an AI which could identify race from xrays while trained experts could not. Maybe it discovered some new science to pull that off?