4 ms·

This article lines up well with my feelings on the matter. In general, people seem to understate the emergent behaviours of ML models, while overstating the uni

This article lines up well with my feelings on the matter. In general, people seem to understate the emergent behaviours of ML models, while overstating the uniqueness of human intelligence. I think a lot of this is down to the fact that although both systems exhibit a form of intelligence, they’re very different. LLMs deliver mastery of natural language that would normally be a signal for a highly intelligent human. While in other ways they’re less intelligent than a cat. So it’s not ‘human like intelligence’ but it is a form of intelligence and the reality is no one would have predicted the behaviours we are seeing. So it seems silly to pretend we can know for certain how it achieves its results. For human intelligence, do we assume cave men had theory of mind at the level of modern day humans? Or did language have to develop first? Our intelligence is built on previous generations, and most of us just ‘interpolate’ within that to a large extent. We behave on occasion like ‘stochastic parrots’ too, mindlessly repeating some new term or phrase we’ve started hearing on Hacker News (why? It just felt like the ‘right thing’ to say). Human intelligence is the working example that combinations of atoms built into large networks have emergent properties. I’m sure our artificial networks won’t behave qualitatively like the human one as they continue to develop, but I think the burden of proof is on those that suggest we can know what ultimately is and isn’t possible.

the_gipsy 4y ago
Maybe it is human like intelligence already. Maybe our internal monologue is just a better trained and refined ChatGPT. And maybe that is all the magic that is necessary for this holy grail of consciousness, there is no quantum brain, no nothing. Just a stream of the next word that says that we are there, therefore we are. That is what scares me.
- glenstein 4y ago
  Well I think that understates the actual vastness and complexity and actual 'magic' that's embodied in a system capable of such a thing as thinking. And I do think that so many people are motivated to dispute this precisely because it feels scary that we might merely be such a thing, and we need to reserve some extra special thing, some form of magic, in order to differentiate ourselves as special. Darwin's Dangerous idea by Daniel Dennett is one of my favorite books because it tackles this very idea.
- espadrine 4y ago
  > Maybe it is human like intelligence already. It objectively is not. First, intelligence (human or otherwise) is typically defined as the sum of value functions of a given policy across all states, divided by an exponential of the complexity of the environment[0] (page 23). The GPT-4 technical report[1] specifically points out that it is unable to make money when given an initial amount and AWS access (appendix H § 2.9). A human can. Second, let's look at it mechanically. It is very easy to anthropomorphize machines, but they are just built differently. Neurons fundamentally are not approximated by self-attention and it is highly unlikely that they improve through backpropagation. In terms of behaviour, we can and do notice differences. Transformers can, and do, make logical mistakes; when asked to immediately answer a complex question, they can only think through it for a fixed amount of time, and thus fail to give an accurate answer. More importantly, they just don't care about their (or your) own survival; they can give lethal advice. All it takes is to find a prompt for which the most common completion is lethal, and they will just complete that. Removing those completions is a whack-a-mole in RLHF: each that gets popular on Twitter will be fixed, but there are more, and GPT just won't take extra time to think just because a life is on the line. [0]: https://arxiv.org/pdf/0712.3329.pdf https://arxiv.org/pdf/0712.3329.pdf [1]: https://cdn.openai.com/papers/gpt-4.pdf https://cdn.openai.com/papers/gpt-4.pdf
  - im3w1l 4y ago
    That isn't true at all. Maybe some narrow field defines it in that way but people in general do not. Traditionally intelligence has been fuzzily defined as that mental thing which humans have that other things do not. If you go back in time you will see that people have associated it with both our ability to breath and later the beating of the heart. We have looked at animals and said we can count and they cannot that is the nature of intelligence. We told ourselves that we have a complex language and they do not and that is the nature of intelligence. We have noticed that we use tools and they do not and that is the nature of intelligence. We have seen that we can play chess and they cannot and that is the nature of intelligence. Now with the advent of computers they have increasingly forced us to reconsider these things. Computers can play chess better than us so that is not intelligence. They are as verbal as us so that is not intelligence. Humorously, because of this reevaluation and redrawing of the boundaries we now seem to feel more of a kinship with animals. But anyway, intelligence is clearly a complex mixture of many things, and crucially those aspects must be integrated and be able to feed into and support each other.
glenstein 4y ago
>LLMs deliver mastery of natural language that would normally be a signal for a highly intelligent human. While in other ways they’re less intelligent than a cat. Just as an example to illustrate your point, yesterday saw a Twitter meme that had multiple overlapping Venn diagrams, where Chicago was not only in a Venn diagram for a type of deep dish pizza, but is also a city, and is also a play, and also a format for writing academic citations, and so there were several different Venn diagrams all overlapping with Chicago, and then those the Venn Diagrams overlapped with each other in humorous ways. I saw that there was an opportunity to add one additional Venn diagram on top of all of the others if it were possible to find any possible connection between 'Sicilian' and 'AMA'. I tried asking chat GPT if it could think of one, and it wasn't able to. So then I stepped back a step, and asked if it could think of a Venn diagram that would include both chicago, and rent, under any understanding of those respective two terms. But it couldn't do that either, although they can both be understood to be Broadway plays.
- ivxvm 4y ago
  Maybe it simply doesn't understand Venn diagrams well - after all, it's a visual concept.
  - glenstein 4y ago
    My thinking is that it relates to organizing concepts into categories, which can challenge a system to prove how deep it is capable of going into abstract associations, and searching through a full range of possible abstract associations. I suspect ChatGPT probably can be reasonably effective at understanding, or pretending to understand numerous other visual concepts.
- ilaksh 4y ago
  Did you try asking GPT-4? It has a significantly higher IQ.
  - glenstein 4y ago
    Yeah, it was GPT-4.
fasterik 4y ago
I agree that we don't know what is possible in hypothetical future models, but claims about understanding in the current models are overblown (at least by people like the article author). For example, I gave GPT-4 some chess games and asked it to evaluate the final move of each game as good or bad and explain its evaluation. It clearly had some understanding of the board state and what makes a move good or bad, but the answers were mostly plausible-sounding nonsense, far worse than what a novice chess player would come up with. I have to assume that its level of understanding of other topics isn't any different; it's just easier to expose the flaws with something that has objectively true and false answers.
- peterfication 4y ago
  This is probably something that will be fixed with an OpenAI to "some chess AI"-plugin. Then for the current model it's just enough to understand that it's a chess problem.
  - fasterik 4y ago
    Maybe, but my point is that if it's this bad at understanding chess, which a fairly simple and objective domain, why should we have confidence it has understanding in other domains where it's harder to prove wrong answers wrong? We aren't going to be able to come up with specialized plugins that fix its mistakes in every conceivable domain. The amount of trust people are putting in the current models seems to indicate that they think it's roughly at a human level of understanding, which doesn't seem to be the case. And if we are actually trying to solve the problem of general intelligence, we can't just sweep things under the rug and say "plugins will fix it".

the_gipsy 4y ago

Maybe it is human like intelligence already. Maybe our internal monologue is just a better trained and refined ChatGPT. And maybe that is all the magic that is necessary for this holy grail of consciousness, there is no quantum brain, no nothing. Just a stream of the next word that says that we are there, therefore we are. That is what scares me.

glenstein 4y ago
Well I think that understates the actual vastness and complexity and actual 'magic' that's embodied in a system capable of such a thing as thinking. And I do think that so many people are motivated to dispute this precisely because it feels scary that we might merely be such a thing, and we need to reserve some extra special thing, some form of magic, in order to differentiate ourselves as special. Darwin's Dangerous idea by Daniel Dennett is one of my favorite books because it tackles this very idea.
espadrine 4y ago
> Maybe it is human like intelligence already. It objectively is not. First, intelligence (human or otherwise) is typically defined as the sum of value functions of a given policy across all states, divided by an exponential of the complexity of the environment[0] (page 23). The GPT-4 technical report[1] specifically points out that it is unable to make money when given an initial amount and AWS access (appendix H § 2.9). A human can. Second, let's look at it mechanically. It is very easy to anthropomorphize machines, but they are just built differently. Neurons fundamentally are not approximated by self-attention and it is highly unlikely that they improve through backpropagation. In terms of behaviour, we can and do notice differences. Transformers can, and do, make logical mistakes; when asked to immediately answer a complex question, they can only think through it for a fixed amount of time, and thus fail to give an accurate answer. More importantly, they just don't care about their (or your) own survival; they can give lethal advice. All it takes is to find a prompt for which the most common completion is lethal, and they will just complete that. Removing those completions is a whack-a-mole in RLHF: each that gets popular on Twitter will be fixed, but there are more, and GPT just won't take extra time to think just because a life is on the line. [0]: https://arxiv.org/pdf/0712.3329.pdf https://arxiv.org/pdf/0712.3329.pdf [1]: https://cdn.openai.com/papers/gpt-4.pdf https://cdn.openai.com/papers/gpt-4.pdf
- im3w1l 4y ago
  That isn't true at all. Maybe some narrow field defines it in that way but people in general do not. Traditionally intelligence has been fuzzily defined as that mental thing which humans have that other things do not. If you go back in time you will see that people have associated it with both our ability to breath and later the beating of the heart. We have looked at animals and said we can count and they cannot that is the nature of intelligence. We told ourselves that we have a complex language and they do not and that is the nature of intelligence. We have noticed that we use tools and they do not and that is the nature of intelligence. We have seen that we can play chess and they cannot and that is the nature of intelligence. Now with the advent of computers they have increasingly forced us to reconsider these things. Computers can play chess better than us so that is not intelligence. They are as verbal as us so that is not intelligence. Humorously, because of this reevaluation and redrawing of the boundaries we now seem to feel more of a kinship with animals. But anyway, intelligence is clearly a complex mixture of many things, and crucially those aspects must be integrated and be able to feed into and support each other.

glenstein 4y ago

>LLMs deliver mastery of natural language that would normally be a signal for a highly intelligent human. While in other ways they’re less intelligent than a cat. Just as an example to illustrate your point, yesterday saw a Twitter meme that had multiple overlapping Venn diagrams, where Chicago was not only in a Venn diagram for a type of deep dish pizza, but is also a city, and is also a play, and also a format for writing academic citations, and so there were several different Venn diagrams all overlapping with Chicago, and then those the Venn Diagrams overlapped with each other in humorous ways. I saw that there was an opportunity to add one additional Venn diagram on top of all of the others if it were possible to find any possible connection between 'Sicilian' and 'AMA'. I tried asking chat GPT if it could think of one, and it wasn't able to. So then I stepped back a step, and asked if it could think of a Venn diagram that would include both chicago, and rent, under any understanding of those respective two terms. But it couldn't do that either, although they can both be understood to be Broadway plays.

ivxvm 4y ago
Maybe it simply doesn't understand Venn diagrams well - after all, it's a visual concept.
- glenstein 4y ago
  My thinking is that it relates to organizing concepts into categories, which can challenge a system to prove how deep it is capable of going into abstract associations, and searching through a full range of possible abstract associations. I suspect ChatGPT probably can be reasonably effective at understanding, or pretending to understand numerous other visual concepts.
ilaksh 4y ago
Did you try asking GPT-4? It has a significantly higher IQ.
- glenstein 4y ago
  Yeah, it was GPT-4.

fasterik 4y ago

I agree that we don't know what is possible in hypothetical future models, but claims about understanding in the current models are overblown (at least by people like the article author). For example, I gave GPT-4 some chess games and asked it to evaluate the final move of each game as good or bad and explain its evaluation. It clearly had some understanding of the board state and what makes a move good or bad, but the answers were mostly plausible-sounding nonsense, far worse than what a novice chess player would come up with. I have to assume that its level of understanding of other topics isn't any different; it's just easier to expose the flaws with something that has objectively true and false answers.

peterfication 4y ago
This is probably something that will be fixed with an OpenAI to "some chess AI"-plugin. Then for the current model it's just enough to understand that it's a chess problem.
- fasterik 4y ago
  Maybe, but my point is that if it's this bad at understanding chess, which a fairly simple and objective domain, why should we have confidence it has understanding in other domains where it's harder to prove wrong answers wrong? We aren't going to be able to come up with specialized plugins that fix its mistakes in every conceivable domain. The amount of trust people are putting in the current models seems to indicate that they think it's roughly at a human level of understanding, which doesn't seem to be the case. And if we are actually trying to solve the problem of general intelligence, we can't just sweep things under the rug and say "plugins will fix it".