3 ms·

There is a caveat to this. Strawberry/Q* relies on elements similar to an element of AlphaZero to find "strategies" suitable for a problem. That takes it furter

There is a caveat to this. Strawberry/Q* relies on elements similar to an element of AlphaZero to find "strategies" suitable for a problem. That takes it furter from "next-word-prediction" than current models, and improves quality of the output. The downside is that this requires more compute during inference. That makes it too expensive to deploy directly. Still, at least to some extent, this could allow a larger model to achieve similar performance to a Strawberry enhanced GPT-4o by adding more parameters, without the impact on speed and compute cost. Humans often do the same. While we first learn some topic, we often first use conscious reasoning (which has elements of a tree-search) to find a way to solve it. But if we practice enough times, it becomes "muscle memory".

JumpCrisscross 2y ago
> Strawberry/Q relies on elements similar to an element of AlphaZero to find "strategies" suitable for a problem. That takes it furter from "next-word-prediction" than current models, and improves quality of the output* Genuine question: is this independently substantiated? Or Altmanspeak?
- mjburgess 2y ago
  The whole comment is "altmanspeak", which we might more directly call, "ideological descriptions of AI consistent with increasing stock prices"
- trashtester 2y ago
  The specifics are kept secret, but all the labs appear to work on some variant. It seems to be related to the DeepMind reference in [1] and most of [2]. [1] https://en.wikipedia.org/wiki/Q-learning https://en.wikipedia.org/wiki/Q-learning [2] https://arxiv.org/pdf/2403.09629 https://arxiv.org/pdf/2403.09629
  - fire_lake 2y ago
    This sounds like chess AI
    - trashtester 2y ago
      More generally, it's part of what we call reasoning. As opposed to do what first comes to mind, which would be similar to what regular LLM's have been doing.
skywhopper 2y ago
You have far too much confidence in the idea that LLMs are anything like human brains. It’s next to meaningless to try to draw parallels between the two things. Your assertion that “conscious reasoning” “has elements of a tree-search” is just completely made up. And the idea that human learning is at all similar to what LLM training is doing is completely divorced from reality.
- mewpmewp2 2y ago
  But how do you reason? Because I definitely do brute force tree search in my brain to solve all sorts of problems. E.g. let's imagine system design or some programming problem. Based on my past experience or what I've read in general, my brain brings up potential solutions in theory. To me it's similar to embeddings search. It will try to pattern match the solutions. And the embeddings are in a tree or graph shape, where you narrow down constantly. My brain then would start to evaluate the solutions in the order of likelihood that they fit the pattern according to my intuition. Basically, I personally do see how an LLM that has certain chain of reasoning, algorithmically or otherwise built in, could represent my approach to problem solving. Because my problem solving can definitely be represented by a continuous flow of words. I don't think current LLMs are exactly capable of that, because they would make too many mistakes somewhere and potentially get stuck, but I can't say they wouldn't be able to do that with more scale, when those mistakes get ironed out due to more ability to do nuanced things.
  - a_wild_dandan 2y ago
    > But how do you reason? My theory: reasoning is the application of analogies. Here `analogy = memorization + pattern matching`. Pattern matching is just an associative memory query; remembering an example with fuzzy enough details to apply generally. Analogies are self-similar/recursive and sometimes transcend contexts -- they're useful for thinking, for thinking about thinking, etc. Analogies pervade our language, our thoughts, our approach to novel problems, and the cached solutions to trivial/practiced problems. There is no qualitative distinction between System 1 & System 2 thinking. It's a spectrum, quantitative. The more analogy steps required, the more thinking done, and the more System 2-ish that thoughts feel. This (probably wrong) theory has many consequences: 1. LLMs are truly intelligent, albeit barely. They rely heavily on memorization rather than much analogy matching. Their depth of memorized knowledge can somewhat substitute/mislead us to misjudge that intelligence. 2. The validation set grokking phenomenon occurs when a new analogy is internalized. 3. Scale really is enough. The scaling laws will continue holding with astonishing accuracy. 4. MCTS heuristics are useful inductive biases to vastly lower compute at the expense of model flexibility. Eventually, The Bitter Lesson will come for MCTS too. I have much more to say about this bonkers theory, but I fear already sounding like a ranting madman. Anyway, just my 2c.
thesz 2y ago
> Still, at least to some extent, this could allow a larger model to achieve similar performance to a Strawberry enhanced GPT-4o by adding more parameters, without the impact on speed and compute cost. I see a contradiction here, do you?

JumpCrisscross 2y ago

> Strawberry/Q relies on elements similar to an element of AlphaZero to find "strategies" suitable for a problem. That takes it furter from "next-word-prediction" than current models, and improves quality of the output* Genuine question: is this independently substantiated? Or Altmanspeak?

mjburgess 2y ago
The whole comment is "altmanspeak", which we might more directly call, "ideological descriptions of AI consistent with increasing stock prices"
trashtester 2y ago
The specifics are kept secret, but all the labs appear to work on some variant. It seems to be related to the DeepMind reference in [1] and most of [2]. [1] https://en.wikipedia.org/wiki/Q-learning https://en.wikipedia.org/wiki/Q-learning [2] https://arxiv.org/pdf/2403.09629 https://arxiv.org/pdf/2403.09629
- fire_lake 2y ago
  This sounds like chess AI
  - trashtester 2y ago
    More generally, it's part of what we call reasoning. As opposed to do what first comes to mind, which would be similar to what regular LLM's have been doing.

skywhopper 2y ago

You have far too much confidence in the idea that LLMs are anything like human brains. It’s next to meaningless to try to draw parallels between the two things. Your assertion that “conscious reasoning” “has elements of a tree-search” is just completely made up. And the idea that human learning is at all similar to what LLM training is doing is completely divorced from reality.

mewpmewp2 2y ago
But how do you reason? Because I definitely do brute force tree search in my brain to solve all sorts of problems. E.g. let's imagine system design or some programming problem. Based on my past experience or what I've read in general, my brain brings up potential solutions in theory. To me it's similar to embeddings search. It will try to pattern match the solutions. And the embeddings are in a tree or graph shape, where you narrow down constantly. My brain then would start to evaluate the solutions in the order of likelihood that they fit the pattern according to my intuition. Basically, I personally do see how an LLM that has certain chain of reasoning, algorithmically or otherwise built in, could represent my approach to problem solving. Because my problem solving can definitely be represented by a continuous flow of words. I don't think current LLMs are exactly capable of that, because they would make too many mistakes somewhere and potentially get stuck, but I can't say they wouldn't be able to do that with more scale, when those mistakes get ironed out due to more ability to do nuanced things.
- a_wild_dandan 2y ago
  > But how do you reason? My theory: reasoning is the application of analogies. Here `analogy = memorization + pattern matching`. Pattern matching is just an associative memory query; remembering an example with fuzzy enough details to apply generally. Analogies are self-similar/recursive and sometimes transcend contexts -- they're useful for thinking, for thinking about thinking, etc. Analogies pervade our language, our thoughts, our approach to novel problems, and the cached solutions to trivial/practiced problems. There is no qualitative distinction between System 1 & System 2 thinking. It's a spectrum, quantitative. The more analogy steps required, the more thinking done, and the more System 2-ish that thoughts feel. This (probably wrong) theory has many consequences: 1. LLMs are truly intelligent, albeit barely. They rely heavily on memorization rather than much analogy matching. Their depth of memorized knowledge can somewhat substitute/mislead us to misjudge that intelligence. 2. The validation set grokking phenomenon occurs when a new analogy is internalized. 3. Scale really is enough. The scaling laws will continue holding with astonishing accuracy. 4. MCTS heuristics are useful inductive biases to vastly lower compute at the expense of model flexibility. Eventually, The Bitter Lesson will come for MCTS too. I have much more to say about this bonkers theory, but I fear already sounding like a ranting madman. Anyway, just my 2c.

thesz 2y ago

> Still, at least to some extent, this could allow a larger model to achieve similar performance to a Strawberry enhanced GPT-4o by adding more parameters, without the impact on speed and compute cost. I see a contradiction here, do you?