4 ms·

https://twitter.com/gfodor/status/1735348301812383906 https://twitter.com/gfodor/status/1735348301812383906 >If DeepMind just definitively proved neural networ

https://twitter.com/gfodor/status/1735348301812383906 https://twitter.com/gfodor/status/1735348301812383906 >If DeepMind just definitively proved neural networks can generate genuinely new knowledge then it’s the most important discovery since fire. If this were actually the case why wouldn't everyone be talking about this? I am impressed it was done on Palm 2 given that's less advanced than GPT-4 and Gemini. Will be wild to see what the next few generations of models can do utilizing methods like this.

dougmwne 3y ago
I said “WOW!” out loud. An LLM can discover a new solution in high dimensional geometry that hasn’t advanced in 20 years!? That goes way beyond glueing little bits of plagiarized training data together in a plausible way. This suggest that there are hidden depths to LLMs’ capabilities if we can just figure out how to prompt and evaluate them correctly. This significantly broke my expectations. Who knows what discovery could be hiding behind the next prompt and random seed.
- bendergarcia 3y ago
  It’s kind of like humans: seed of two people and a random interest to pursue, what could they do?!? It makes poverty and children dying unnecessarily even more depressing.
- riku_iki 3y ago
  > An LLM can discover a new solution in high dimensional geometry that hasn’t advanced in 20 years!? LLM + brute force coded by humans.
  - dougmwne 3y ago
    And that not only created a human interpretable method, but beat out 20 years of mathematicians working on a high profile open question. Who’s to say our brains themselves aren’t brute force solution searchers?
    - riku_iki 3y ago
      > Who’s to say our brains themselves aren’t brute force solution searchers? yes, its just much slower (many trillions times) for numbers crunching.
- nybsjytm 3y ago
  > An LLM can discover a new solution in high dimensional geometry that hasn’t advanced in 20 years!? That goes way beyond glueing little bits of plagiarized training data together in a plausible way. The first advance in 20 years was by Fred Tyrrell last year https://arxiv.org/abs/2209.10045 https://arxiv.org/abs/2209.10045, who showed that the combinatorics quantity in question is between 2.218 and 2.756, improving the previous lower bound of 2.2174. DeepMind has now shown that the number is between 2.2202 and 2.756. That's why the DeepMind authors describe it as "the largest increase in the size of cap sets in the past 20 years," not as the only increase. The arithmetic is that 2.2202-2.218 is larger than 2.218-2.2174. (If you consider this a very meaningful comparison to make, you might work for DeepMind.)
- a_wild_dandan 3y ago
  > That goes way beyond glueing little bits of plagiarized training data together Moreover, LLMs almost ALWAYS extrapolate, and never interpolate. They don't regurgitate training data. Doing so is virtually impossible. An LLM's input (AND feature) space is enormous. Hundreds or thousands of dimensions. 3D space isn't like 50D or 5,000D space. The space is so combinatorially vast that basically no two points are neighbors. You cannot take your input and "pick something nearby" to a past training example. There IS NO nearby. No convex hull to walk around in. This "curse of dimensionality" wrecks arguments that these models only produce "in distribution" responses. They overwhelmingly can't! (Check out the literature of LeCun et al. for more rigor re. LLM extrapolation.) LLMs are creative. They work. They push into new areas daily. This reality won't change regardless of how weirdly, desperately the "stochastic parrot" people wish it were otherwise. At this point they're just denialists pushing goalposts around. Don't let 'em get to you!
  - jcgrillo 3y ago
    Do you have a specific reference? I've mostly ignored LLMs until now because it seemed like the violent failure mode (confident + competent + wrong) renders them incapable of being a useful tool[1]. However this application, combined with the dimensionality idea, has me interested. I do wish the authors of the work referenced here made it more clear what, if anything, the LLM is doing here. It's not clear to me it confers some advantage over a more normal genetic programming approach to these particular problems. [1] in the sense that useful, safe tools degrade predictably. An airplane which stalls violently and in an unrecoverable manner doesn't get mass-produced. A circular saw which disintegrates when the blade binds throwing shrapnel into its operator's body doesn't pass QA. Etc.
    - a_wild_dandan 3y ago
      "Learning in High Dimension Always Amounts to Extrapolation" [1] [1] https://arxiv.org/abs/2110.09485 https://arxiv.org/abs/2110.09485
      - jcgrillo 3y ago
Imnimo 3y ago
The heavy lifting here is done by the evolutionary algorithm. The LLM is just being asked "propose some reasonable edits to these 20 lines of python" to replace a random mutation operator. It feels generous to credit the neural network with the knowledge generation here. It's also highly dependent on the nature of the problem, even beyond the need to have the "hard to generate, easy to evaluate" structure. You have to be able to decompose the problem in such a way that a very short Python function is all that you want to evolve.
- famouswaffles 3y ago
  There's nothing generous about it. It wouldn't be possible without the LLM. Traditional Computational solvers fall well short as they say and demonstrate. >The LLM is just being asked "propose some reasonable edits to these 20 lines of python" to replace a random mutation operator. Hahaha "Just".
  - Imnimo 3y ago
    We can see from the ablations that simply asking the LLM to generate a large number of potential solutions without the evolutionary search performs terribly. It's certainly true that the LLM mutator produces more reasonable edits than a random mutator. But the value the LLM is bringing is that it knows how to produce reasonable-looking programs, not that it has some special understanding of bin packing or capset finding. The only thing the LLM sees is two previously generated code samples, labeled <function>_v0 and <function>_v1 and then a header for <function>_v2 which it fills in. Look at Extended Data Fig. 1.
    - famouswaffles 3y ago
      With the current competency or state of the models today, both are necessary, nobody is denying that. It doesn't take a genius however to see who the more valuable contributor to the system is. The authors say as much when they anticipate much of any improvement to come from capabilities of better models than any change to the search algorithm. Gpt-4 may well have reduced the search time by half and so on. >But the value the LLM is bringing is that it knows how to produce reasonable-looking programs, not that it has some special understanding of bin packing or capset finding. So it is a general problem solver then, great. That's what we want.
famouswaffles 3y ago
Neural networks have been able to "generate new knowledge" for a long time. So have LLMs, https://www.nature.com/articles/s41587-022-01618-2 https://www.nature.com/articles/s41587-022-01618-2
empath-nirvana 3y ago
In some sense, it's not especially interesting because millions of programmers are doing this with copilot every day. Like I get that in a lot of cases it's _just_ applying common knowledge to new domains, but it is novel code for the most part.
fatherzine 3y ago
Don't Look Up
lossolo 3y ago
From the paper: We note that FunSearch currently works best for problems having the following characteristics: a) availability of an efficient evaluator; b) a “rich” scoring feedback quantifying the improvements (as opposed to a binary signal); c) ability to provide a skeleton with an isolated part to be evolved. For example, the problem of generating proofs for theorems [52–54] falls outside this scope, since it is unclear how to provide a rich enough scoring signal.
didibus 3y ago
In this example, it's relatively limited I feel to finding new algorithms/functions. Which is great, but compared to the discovery of fire, and tons of things in between, like say electricity, it wouldn't feel in the same ballpark at all.

dougmwne 3y ago

I said “WOW!” out loud. An LLM can discover a new solution in high dimensional geometry that hasn’t advanced in 20 years!? That goes way beyond glueing little bits of plagiarized training data together in a plausible way. This suggest that there are hidden depths to LLMs’ capabilities if we can just figure out how to prompt and evaluate them correctly. This significantly broke my expectations. Who knows what discovery could be hiding behind the next prompt and random seed.

bendergarcia 3y ago
It’s kind of like humans: seed of two people and a random interest to pursue, what could they do?!? It makes poverty and children dying unnecessarily even more depressing.
riku_iki 3y ago
> An LLM can discover a new solution in high dimensional geometry that hasn’t advanced in 20 years!? LLM + brute force coded by humans.
- dougmwne 3y ago
  And that not only created a human interpretable method, but beat out 20 years of mathematicians working on a high profile open question. Who’s to say our brains themselves aren’t brute force solution searchers?
  - riku_iki 3y ago
    > Who’s to say our brains themselves aren’t brute force solution searchers? yes, its just much slower (many trillions times) for numbers crunching.
nybsjytm 3y ago
> An LLM can discover a new solution in high dimensional geometry that hasn’t advanced in 20 years!? That goes way beyond glueing little bits of plagiarized training data together in a plausible way. The first advance in 20 years was by Fred Tyrrell last year https://arxiv.org/abs/2209.10045 https://arxiv.org/abs/2209.10045, who showed that the combinatorics quantity in question is between 2.218 and 2.756, improving the previous lower bound of 2.2174. DeepMind has now shown that the number is between 2.2202 and 2.756. That's why the DeepMind authors describe it as "the largest increase in the size of cap sets in the past 20 years," not as the only increase. The arithmetic is that 2.2202-2.218 is larger than 2.218-2.2174. (If you consider this a very meaningful comparison to make, you might work for DeepMind.)
a_wild_dandan 3y ago
> That goes way beyond glueing little bits of plagiarized training data together Moreover, LLMs almost ALWAYS extrapolate, and never interpolate. They don't regurgitate training data. Doing so is virtually impossible. An LLM's input (AND feature) space is enormous. Hundreds or thousands of dimensions. 3D space isn't like 50D or 5,000D space. The space is so combinatorially vast that basically no two points are neighbors. You cannot take your input and "pick something nearby" to a past training example. There IS NO nearby. No convex hull to walk around in. This "curse of dimensionality" wrecks arguments that these models only produce "in distribution" responses. They overwhelmingly can't! (Check out the literature of LeCun et al. for more rigor re. LLM extrapolation.) LLMs are creative. They work. They push into new areas daily. This reality won't change regardless of how weirdly, desperately the "stochastic parrot" people wish it were otherwise. At this point they're just denialists pushing goalposts around. Don't let 'em get to you!
- jcgrillo 3y ago
  Do you have a specific reference? I've mostly ignored LLMs until now because it seemed like the violent failure mode (confident + competent + wrong) renders them incapable of being a useful tool[1]. However this application, combined with the dimensionality idea, has me interested. I do wish the authors of the work referenced here made it more clear what, if anything, the LLM is doing here. It's not clear to me it confers some advantage over a more normal genetic programming approach to these particular problems. [1] in the sense that useful, safe tools degrade predictably. An airplane which stalls violently and in an unrecoverable manner doesn't get mass-produced. A circular saw which disintegrates when the blade binds throwing shrapnel into its operator's body doesn't pass QA. Etc.
  - a_wild_dandan 3y ago
    "Learning in High Dimension Always Amounts to Extrapolation" [1] [1] https://arxiv.org/abs/2110.09485 https://arxiv.org/abs/2110.09485
    - jcgrillo 3y ago

Imnimo 3y ago

The heavy lifting here is done by the evolutionary algorithm. The LLM is just being asked "propose some reasonable edits to these 20 lines of python" to replace a random mutation operator. It feels generous to credit the neural network with the knowledge generation here. It's also highly dependent on the nature of the problem, even beyond the need to have the "hard to generate, easy to evaluate" structure. You have to be able to decompose the problem in such a way that a very short Python function is all that you want to evolve.

famouswaffles 3y ago
There's nothing generous about it. It wouldn't be possible without the LLM. Traditional Computational solvers fall well short as they say and demonstrate. >The LLM is just being asked "propose some reasonable edits to these 20 lines of python" to replace a random mutation operator. Hahaha "Just".
- Imnimo 3y ago
  We can see from the ablations that simply asking the LLM to generate a large number of potential solutions without the evolutionary search performs terribly. It's certainly true that the LLM mutator produces more reasonable edits than a random mutator. But the value the LLM is bringing is that it knows how to produce reasonable-looking programs, not that it has some special understanding of bin packing or capset finding. The only thing the LLM sees is two previously generated code samples, labeled <function>_v0 and <function>_v1 and then a header for <function>_v2 which it fills in. Look at Extended Data Fig. 1.
  - famouswaffles 3y ago
    With the current competency or state of the models today, both are necessary, nobody is denying that. It doesn't take a genius however to see who the more valuable contributor to the system is. The authors say as much when they anticipate much of any improvement to come from capabilities of better models than any change to the search algorithm. Gpt-4 may well have reduced the search time by half and so on. >But the value the LLM is bringing is that it knows how to produce reasonable-looking programs, not that it has some special understanding of bin packing or capset finding. So it is a general problem solver then, great. That's what we want.

famouswaffles 3y ago

Neural networks have been able to "generate new knowledge" for a long time. So have LLMs, https://www.nature.com/articles/s41587-022-01618-2 https://www.nature.com/articles/s41587-022-01618-2

empath-nirvana 3y ago

In some sense, it's not especially interesting because millions of programmers are doing this with copilot every day. Like I get that in a lot of cases it's _just_ applying common knowledge to new domains, but it is novel code for the most part.

fatherzine 3y ago

Don't Look Up

lossolo 3y ago

From the paper: We note that FunSearch currently works best for problems having the following characteristics: a) availability of an efficient evaluator; b) a “rich” scoring feedback quantifying the improvements (as opposed to a binary signal); c) ability to provide a skeleton with an isolated part to be evolved. For example, the problem of generating proofs for theorems [52–54] falls outside this scope, since it is unclear how to provide a rich enough scoring signal.

didibus 3y ago

In this example, it's relatively limited I feel to finding new algorithms/functions. Which is great, but compared to the discovery of fire, and tons of things in between, like say electricity, it wouldn't feel in the same ballpark at all.