4 ms·

Pen and paper? LLMs are literally a computer program that cannot compute.

by _1tem 2y ago

carlmr 2y ago
The comparison makes sense though. We're trying to build an simulated brain. We want to create a brain that can think about math. And chain of thought is kind of like giving that brain some scratch space to figure out the problem. This simulated brain can't access multiplication instructions on the CPU directly. It has to do the computation via it's simulated neurons interacting. This is why it's not so surprising that this is an issue.
- ulbu 2y ago
  Does it have an understanding of the strict rules that govern the problem and that it needs to produce a result that is in total accordance to them? (In accordance which is not 100%, but boolean) i.e., can it apply a function over a sentence? I don’t know, that’s why I ask.
  - ThunderSizzle 2y ago
    The answer is sometimes. Typically it'll forget rules you've given it by the time it might be useful because of the memory limit of LLMs. Either way, you basically need to know it's hallucinating to you so you can keep applying more rules.
- namaria 2y ago
  LLMs are not simulating brains in any capacity. The words 'neural network' shouldn't be taken at face value. A single human neuron can take quite a few 'neurons' and layers to simulate as a 'neural network'.
  - carlmr 2y ago
    Sure, but the basic idea of firing neurons is there, and the connection of these "neurons" to a neural network like an LLM does not allow the network to perform computations directly. The level of detail of the simulation has little bearing on this. And in fact whether you call it a simulation or something else doesn't matter either. Understanding that the LLM does not compute by using the CPU or GPU directly is what's necessary to understand why computation is hard for LLMs.
moi2388 2y ago
But it can call into systems that can do compute. Do you think your inner monologue is any different? Because it sure as hell isn’t the same system as the one doing math, or recognising faces, or storing or retrieving memories, to name a few

Pen and paper? LLMs are literally a computer program that cannot compute.

by _1tem 2y ago

Pen and paper? LLMs are literally a computer program that cannot compute.

The comparison makes sense though. We're trying to build an simulated brain. We want to create a brain that can think about math. And chain of thought is kind of like giving that brain some scratch space to figure out the problem. This simulated brain can't access multiplication instructions on the CPU directly. It has to do the computation via it's simulated neurons interacting. This is why it's not so surprising that this is an issue.

ulbu 2y ago
Does it have an understanding of the strict rules that govern the problem and that it needs to produce a result that is in total accordance to them? (In accordance which is not 100%, but boolean) i.e., can it apply a function over a sentence? I don’t know, that’s why I ask.
- ThunderSizzle 2y ago
  The answer is sometimes. Typically it'll forget rules you've given it by the time it might be useful because of the memory limit of LLMs. Either way, you basically need to know it's hallucinating to you so you can keep applying more rules.
namaria 2y ago
LLMs are not simulating brains in any capacity. The words 'neural network' shouldn't be taken at face value. A single human neuron can take quite a few 'neurons' and layers to simulate as a 'neural network'.
- carlmr 2y ago
  Sure, but the basic idea of firing neurons is there, and the connection of these "neurons" to a neural network like an LLM does not allow the network to perform computations directly. The level of detail of the simulation has little bearing on this. And in fact whether you call it a simulation or something else doesn't matter either. Understanding that the LLM does not compute by using the CPU or GPU directly is what's necessary to understand why computation is hard for LLMs.

moi2388 2y ago

But it can call into systems that can do compute. Do you think your inner monologue is any different? Because it sure as hell isn’t the same system as the one doing math, or recognising faces, or storing or retrieving memories, to name a few