4 ms·
> One major reason it doesn't qualify is that it is impossible for it to learn from its input. It's right there in the name: generative pre-trained transformer
by PartiallyTyped 4y ago
> One major reason it doesn't qualify is that it is impossible for it to learn from its input. It's right there in the name: generative pre-trained transformer
Both chatGPT and GPT models exhibit in-context learning, just as you do. The difference is that during your context updates modifications to your synapses occur as a consequence of your brain's activations.
Let's entertain a hypothetical scenario where I am a 5D creature, and you are (3 Space + 1 Time)D human.
I could go back to any point in your time and have any conversation that I wish with you. Each time I do that, I am effectively resetting your brain's state to that particular point in time.
The analog here is that for ChatGPT and the likes, time does not exist in the manner that it does for us, they are purely abstract in that they don't even need physical form to exist, it's a collection of ones and zeros that we store in such form because it is convenient.
You can very well store the parameters of the GPT models in any other medium capable of holding information and actually execute the model.
You can even compute gradient updates, or hook the models up to external data sources.
In the same vein, you "could" do the same for any human brain no matter how computationally prohibitive it may be.
You'd still classify the human brain as a general intelligence. Furthermore, we classify humans with long-term amnesia or inability to form new memories generally intelligent even though they only exhibit in-context learning.
- Karunamon 4y agoInteresting analogy… Actually a damn good one because I'm having a hard time finding how it leaks :) I have to ask what definition(s) of intelligence we are using here. There are multiple, and GPT satisfies some to some degree but not others. Emotion and self-awareness can probably be written off entirely. Some experimenting shows that it still sucks at logic when presented with a pristine problem (something that definitely is not in its training set); and I would say this is a requirement of a general intelligence, being able to take the knowledge it has and speculatively applying that to things it has no knowledge of using induction/deduction.
- PartiallyTyped 4y agoAs I have said in a parallel chain, ChatGPT has been able to solve programming assignments for 15-17 year olds written in Greek. The P it has seen those tasks in Greek before is approaching zero imho. > being able to take the knowledge it has and speculatively applying that to things it has no knowledge of using induction/deduction. One could argue that its 'confidence' is an example of speculatively applying what it knows. The problem is that the inference machinery behind it is quite weak.