5 ms·

Bad title, this was a hybrid ML/human effort, not a ML only achievement. From the article: "What’s most exciting to me is modelling new modes of human–machine

Bad title, this was a hybrid ML/human effort, not a ML only achievement. From the article: "What’s most exciting to me is modelling new modes of human–machine collaboration,” Ellenberg adds. “I don’t look to use these as a replacement for human mathematicians, but as a force multiplier.”

SirMaster 3y ago
It's almost like saying human+calculator beats human. Haven't mathematicians been using complex computer modeling to help solve unsolved math problems since computers have existed? And havent those computers basically always beat out a human alone? So isn't this news just that the mathematicians now have a newer and better computer model to help them solve their problems? Seems like evolution, not revolution.
- fasterik 3y ago
  If I understand correctly, they're using an LLM to write a series of computer programs exploring a large solution space and feeding the output of those programs into a separate validation program written by a human. To take the calculator analogy, it's more like having the human give a set of constraints on a solution and having the calculator decide what buttons to push. Real progress is always incremental. I wouldn't be surprised if 5-10 years from now we have similar kinds of systems discovering new materials or new candidates for dark matter.
  - hitchstory 3y ago
    This sounds like TDD but with the humans writing the test and the machine writing the code.
fasterik 3y ago
Even Nature has clickbait titles now. I agree with the quote. The ability of AI to augment human capabilities has way more potential than the more speculative ideas about artificial general intelligence. This is why I'm not very sympathetic to the skepticism toward deep learning and LLMs as "not intelligence", "not real AI", "stochastic parrots", etc. Who cares whether or not these systems are generally intelligent agents, if they have the potential to increase the scientific output of humanity by even 10 or 20%?
- lainga 3y ago
  Nobody's asked me to donate all my money to Yudkowsky or else suffer the scientific output of humanity increasing by 10 or 20%.
ummonk 3y ago
No it’s not a hybrid effort (except insomuch as LLMs are reliant on human generated data for training). They’re simply saying that the code created by the LLM can be examined and potentially understood by humans.
- caddemon 3y ago
  Their best reported results are a hybrid effort though, here is one of the authors of the paper describing how they used programs generated by the LLM to extract their own insights that then refined future iterations of their workflow: https://x.com/matejbalog/status/1735331210140819938?s=20 https://x.com/matejbalog/status/1735331210140819938?s=20 It can work by itself too but it is unclear at a glance how well since the main focus of the paper is the new mathematical benchmarks they achieved, i.e. their best results. Will have to read the paper more closely to say anything with high confidence, but based on their summary I'd guess the human in the loop part was pretty important here.

SirMaster 3y ago

It's almost like saying human+calculator beats human. Haven't mathematicians been using complex computer modeling to help solve unsolved math problems since computers have existed? And havent those computers basically always beat out a human alone? So isn't this news just that the mathematicians now have a newer and better computer model to help them solve their problems? Seems like evolution, not revolution.

fasterik 3y ago
If I understand correctly, they're using an LLM to write a series of computer programs exploring a large solution space and feeding the output of those programs into a separate validation program written by a human. To take the calculator analogy, it's more like having the human give a set of constraints on a solution and having the calculator decide what buttons to push. Real progress is always incremental. I wouldn't be surprised if 5-10 years from now we have similar kinds of systems discovering new materials or new candidates for dark matter.
- hitchstory 3y ago
  This sounds like TDD but with the humans writing the test and the machine writing the code.

fasterik 3y ago

Even Nature has clickbait titles now. I agree with the quote. The ability of AI to augment human capabilities has way more potential than the more speculative ideas about artificial general intelligence. This is why I'm not very sympathetic to the skepticism toward deep learning and LLMs as "not intelligence", "not real AI", "stochastic parrots", etc. Who cares whether or not these systems are generally intelligent agents, if they have the potential to increase the scientific output of humanity by even 10 or 20%?

lainga 3y ago
Nobody's asked me to donate all my money to Yudkowsky or else suffer the scientific output of humanity increasing by 10 or 20%.

ummonk 3y ago

No it’s not a hybrid effort (except insomuch as LLMs are reliant on human generated data for training). They’re simply saying that the code created by the LLM can be examined and potentially understood by humans.

caddemon 3y ago
Their best reported results are a hybrid effort though, here is one of the authors of the paper describing how they used programs generated by the LLM to extract their own insights that then refined future iterations of their workflow: https://x.com/matejbalog/status/1735331210140819938?s=20 https://x.com/matejbalog/status/1735331210140819938?s=20 It can work by itself too but it is unclear at a glance how well since the main focus of the paper is the new mathematical benchmarks they achieved, i.e. their best results. Will have to read the paper more closely to say anything with high confidence, but based on their summary I'd guess the human in the loop part was pretty important here.