4 ms·

Feels like, but isn't. When you are reasoning things out, there is a brain with state that is actively modeling the problem. AI does no such thing, it produces

Feels like, but isn't. When you are reasoning things out, there is a brain with state that is actively modeling the problem. AI does no such thing, it produces text and then uses that text to condition the next text. If it isn't written, it does not exist. Put another way, LLMs are good at talking like they are thinking. That can get you pretty far, but it is not reasoning.

double0jimb0 1y ago
So exactly what language/paradigm is this brain modeling the problem within?
- phailhaus 1y ago
  We literally don't know. We don't understand how the brain stores concepts. It's not necessarily language: there are people that do not have an internal monologue, and yet they are still capable of higher level thinking.
  - chrisweekly 1y ago
    Rilke: "There is a depth of thought untouched by words, and deeper still a depth of formless feeling untouched by thought."
  - adastra22 1y ago
    I am one of those for what it’s worth. I struggle to put my thoughts into words, as it does not come naturally to me. When I think internally, I do not use language at all, a lot of of the time. Drive my wife crazy as my answer to her questions are always really slow and considered. I have the first think what thought I want to convey, and then think “how do I translate this into words?”
Enginerrrd 1y ago
The transformer architecture absolutely keeps state information "in its head" so to speak as it produces the next word prediction, and uses that information in its compute. It's true that if it's not producing text, there is no thinking involved, but it is absolutely NOT clear that the attention block isn't holding state and modeling something as it works to produce text predictions. In fact, I can't think of a way to define it that would make that untrue... unless you mean that there isn't a system wherein something like attention is updating/computing and the model itself chooses when to make text predictions. That's by design, but what you're arguing doesn't really follow. Now, whether what the model is thinking about inside that attention block matches up exactly or completely with the text it's producing as generated context is probably at least a little dubious, and its unlikely to be a complete representation regardless.
- dmacfour 1y ago
  > The transformer architecture absolutely keeps state information "in its head" so to speak as it produces the next word prediction, and uses that information in its compute. How so? Transformers are state space models.

double0jimb0 1y ago

So exactly what language/paradigm is this brain modeling the problem within?

phailhaus 1y ago
We literally don't know. We don't understand how the brain stores concepts. It's not necessarily language: there are people that do not have an internal monologue, and yet they are still capable of higher level thinking.
- chrisweekly 1y ago
  Rilke: "There is a depth of thought untouched by words, and deeper still a depth of formless feeling untouched by thought."
- adastra22 1y ago
  I am one of those for what it’s worth. I struggle to put my thoughts into words, as it does not come naturally to me. When I think internally, I do not use language at all, a lot of of the time. Drive my wife crazy as my answer to her questions are always really slow and considered. I have the first think what thought I want to convey, and then think “how do I translate this into words?”

Enginerrrd 1y ago

The transformer architecture absolutely keeps state information "in its head" so to speak as it produces the next word prediction, and uses that information in its compute. It's true that if it's not producing text, there is no thinking involved, but it is absolutely NOT clear that the attention block isn't holding state and modeling something as it works to produce text predictions. In fact, I can't think of a way to define it that would make that untrue... unless you mean that there isn't a system wherein something like attention is updating/computing and the model itself chooses when to make text predictions. That's by design, but what you're arguing doesn't really follow. Now, whether what the model is thinking about inside that attention block matches up exactly or completely with the text it's producing as generated context is probably at least a little dubious, and its unlikely to be a complete representation regardless.

dmacfour 1y ago
> The transformer architecture absolutely keeps state information "in its head" so to speak as it produces the next word prediction, and uses that information in its compute. How so? Transformers are state space models.