9 ms·

Do large language models understand us?

64 points by ofou 5y ago

viraptor 5y ago
I'm surprised by no mention of the Chinese room argument: https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room - it's closely related to the topic.
- gorgoiler 5y ago
  Instead of the computer/book telling the non-Chinese speaker what to write, what if it’s a human that speaks Chinese? Does it matter then that the writer has no understanding of the language, when the backend clearly does? (I think I get it actually, after writing these two questions, but I’ll post them anyway.)
  - viraptor 5y ago
    I think at "the writer has no understanding of the language, when the backend clearly does?" you circle back to the argument's main problem - is it possible for the non-human backend to "understand" the language, even though it behaves like it does.
- elcomet 5y ago
  The Chinese room argument is very unconvincing unfortunately, as brains are actually biological computing machines.
- subroutine 5y ago
  Searle goes into the room with the copy of the program instructions. He gets passed a note written in Chinese, cracks his knuckles, and then opens the manual. He reads, "Step-1: prior to translating the note, we are going to train your neural networks on a Chinese corpus consisting of 410 billion tokens from Common Crawl, 19 billion tokens from WebText, 67 billion tokens from Books, and 3 billion tokens from Wikipedia. Training in 5, 4, 3..." He shouts, "wait but if you do that wo shuō zhōngwén."
  - ww_wpg 5y ago
    The search result I'm finding for that phrase is "Wǒ bù huì shuō zhōngwén. I kept saying it over and over again, the phrase I had memorized to explain my inability to speak their language". So I guess it means "I don't speak Mandarin"?
    - elikoga 5y ago
      The "Wǒ bù huì[...]" part means "I negate/not can[...]" It means precisely the opposite.
- robbedpeter 5y ago
  This isn't aimed at you, viraptor, but at Searle and his argument. Searle and subsequent proponents fail to define mind, intelligence, or understanding. The argument is founded in anthropocentric arrogance. Gpt-3 has convincingly passed the Turing test, with various and sundry incarnations passing as human for days or even weeks on reddit and Twitter and other social media. You can keep moving the goalposts, but the herd has left the barn. The argument is over. The Turing test has been passed. There is no way to frame human mind, intelligence, or understanding such that you exclude the equivalent functionality in software without engaging in magical thinking. Searle's assertion that he would not understand Chinese while executing the rules isn't relevant. Searle could conceivably execute the inference pass over a gpt-3 model based on Chinese, and produce results that surpass the Turing threshold as conceived by Searle at the time. The intelligence and understanding and mind is encoded in the model and logic of the processing. This demonstrates that substrate is not involved in the process of intelligence. There's nothing unique or magical about the human brain that suggests a shred of evidence that it can't be replicated by any Turing complete processing system, or computer. Our minds are an algorithm being run continuously by a messy wet computer in a bone vat connected to a meat suit. Your sense of continuous perception is the state of a model stored in and processed by the physical, electrical, and chemical configuration of your brain and the laws of physics. Any methodical definition of understanding arrives at logic and semantic relationships between concepts and information theory. The brain has to be operating as a computer because there is literally nothing else that could be reasonably asserted. Understanding and consciousness and intelligence and mind are consequences of algorithms, or software. Gpt-3 and similar models are undeniably different types of minds then that of humans, lacking persistent state, online feedback, and a myriad of systems and capabilities. Transformers in their current form are not agi and interact with the world in a radically different way than humans do. They do something that human minds do, however. Consciousness could be the result of a certain class of algorithms being processed, with a self referential model processed concurrent with external input. Additional systems might be required, but consciousness has to be accomplished using no more than the capabilities offered by the brain, so it's likely to be an emergent consequence of the holistic execution of the cortical algorithm. Each inference pass might represent a single moment of mind, a disjoint and ephemeral flash of consciousness, unrelated to its prior or future states, deterministically replicable and static. The Chinese room is a failed argument and we should move on to more interesting things. Brain meat is not magic.
deleted 5y ago
[deleted]
Animats 5y ago
"Furthermore, much of what we consider intelligence is inherently dialogic, hence social; it requires a theory of mind." That's an assumption. The contrary view is that language is just part of the I/O system, not where the hard work gets done. Animals with very limited language capabilities have the ability to run their own lives, and to some extent manipulate their environment. Systems built from large language databases don't have that. After 50 years, robotic manipulation in unstructured situations is still terrible. Rod Brooks tried to crack that and his startup, Rethink Robotics, failed. DARPA has had decades of manipulation projects. Fifty years ago, they could put a bolt in a hole. Five years ago, they got a robot hand to put a key in a lock. Amazon is still trying to get robots to pick things from bins fast and reliably. We lack some fundamental understanding about how to interact with the real world. The problem is not on the language side.
- fake-name 5y ago
  > That's an assumption. The contrary view is that language is just part of the I/O system, not where the hard work gets done. FYI, this is the [Sapir-Whorf hypothesis](1) > We lack some fundamental understanding about how to interact with the real world. The problem is not on the language side. This is to a significant extent a sensing issue, not a understanding issue. 1: https://en.wikipedia.org/wiki/Linguistic_relativity https://en.wikipedia.org/wiki/Linguistic_relativity
  - tsimionescu 5y ago
    The Sappir-Whorf hypothesis refers to specific languages, and is all but proven wrong in its strong form (a person speaking English and a person speaking Japanese seem to perform exactly the same at all cognitive tasks). But there is a different theory, due to Noam Chomsky I believe, that language in the general sense is in fact our mechanism for higher thought, with communication being just a secondary function[0]. I believe that this is the theory that the GP was combatting with the animal example. [0] https://www.youtube.com/watch?v=KEmpRtj34xg https://www.youtube.com/watch?v=KEmpRtj34xg
    - thaumasiotes 5y ago
my50cents 5y ago
> Since the interior state of another being can only be understood through interaction, no objective answer is possible to the question of when an “it” becomes a “who” A large language model can in theory be understood at an algorithmic level by reverse engineering. If the algorithm turned out to be a giant lookup table, it's an "it". If the algorithm contained an obvious model of self, it's a "who".
- crooked-v 5y ago
  > an obvious model of self How do you determine what's an "obvious" model of self?
  - deleted 5y ago
    [deleted]
- robbedpeter 5y ago
  We assume other people are "who" and not "it" because their behavior is predictably similar to our own, and so we model their experiences and perspectives based on our own subjective experience. Occam's razor suggests that if we know other people have the same basic configuration as ourselves, then the subjective experiences of others will be more or less comparable to our own. Additional assumptions are necessary to propose that Chalmersian/philosophical zombies could exist, having the same human hardware and behavior but lacking subjective experience. Under the principle of least complexity and the absence of evidence for alternate explanations, I think it's rational to assume the consciousness of other humans and certain animals with nearly 100% confidence. That means something happening in the neocortex is causing consciousness. We know it's in the cortex because of the history of injury or absence of other parts of the brain, leaving us with empirical evidence. We know that hippocampus injury can result in a person losing the ability to remember more then 5 minutes of their past. We also know these individuals retain their personalities and the long-term memory from before their injury. This suggests that long term memory is encoded in the synaptic structure of the neocortex and that consciousness is an emergent result of neocortical operation. There might be some involvement of particular brain regions or the thalamus or other organ, but it looks like the answer is part of whatever algorithm is encoded in the structure and processing of the neocortex. It could be possible that consciousness doesn't require an explicit model of self. This is implied by ego death and other experiences by meditators and psychedelic users. The thing that is having a subjective experience could simply be a consequence of processing a particular configuration of information. The self concept seems to be a separate model that can be perceived as part of the process of awareness, but it seems to be a discrete thing. To me that presents an interesting ethical question - if gpt-3 has a sense of self, then is it OK to subject it to the incoherent flashes of single moments of consciousness it undergoes each time you run it? If it has no self concept, it might still have subjective experience, but would have no persistent contiguous experience . If it did have a self concept, then it would have a static past that informed the results of each run, as if you could isolate a single moment of awareness, then instantly reset the brain to a saved state and run it again. The output could be convincingly continuous, but the subjective experiences would be a myriad of similar but unrelated singularities of awareness. I think gpt-3 lacks consciousness, and that a persistence mechanism is missing that plays a part in whatever is happening that causes awareness. Hopefully, at some point, a m mathematician or scientist will be able to identify and explain the process of consciousness so that we can be reasonably certain we're not subjecting entities to a really weird tortured existence.
deleted 5y ago
[deleted]
Borrible 5y ago
Ask them, why would they lie?
js8 5y ago
I have an objection to the idea that understanding is just a pattern recognition, or prediction of the next token. I think fundamental to understanding is the concept of logical contradiction, which I don't believe GPT-3 has. I don't think GPT-3 can say, wait, what I am saying is BS, because logically it's not consistent. To argue that, I propose a "simplified Chinese room". We have an automaton in the Chinese room, which does only pattern recognition, and can learn by matching patterns and structures, i.e. it will give an answer that is similar to what it has seen before. And we want to teach it to answer a problem "recognize whether the given logical formula is satisfiable". So we give it formulas and it tries to pattern match the formulas it already has seen. Unfortunately, there always exists a formula which is matching closely everything that was shown so far, and yet the correct answer is unexpected. So such an automaton will never be able to discover the actual rule. What I think has to be added are different levels of discourse, or, a discourse about discourse. With people, we can say, "stop joking around", and it's a meta-discourse signal that we want stop discussing imaginary worlds and focus on the real one, for example, which has to be totally self-consistent. I believe an analogy can be made with Futamura projections, we can have not only systems that learn a specific task (like even GPT-3 does), but also systems that learn to produce an agent to do a specific task, from a metalanguage description that encodes certain rules on how to do it (for example, to follow logical consistency). And if the metalanguage is general enough, you get general intelligence, because then you can apply the agent production process to itself.
- jcelerier 5y ago
  > I think fundamental to understanding is the concept of logical contradiction, which I don't believe GPT-3 has are we sure that humans actually have that though ? It's not always my experience
  - js8 5y ago
    I think we do. We get a distinct feeling that we don't understand something, and a different feeling when things click together. Or when we hear a syntactically valid (in other words, pattern-matching) but nonsensical sentence (like "colorless green ideas sleep furiously"), or see a Penrose triangle, and we cannot imagine an example, we get a very specific feeling that something is wrong. I think we are actually solving the satisfiability problem every time we imagine something - we find a concrete instance of a situation that satisfies certain conditions (for example, words in a sentence, or pixels in the image). The book Inside Jokes even speculates that reason why we find jokes pleasurable is that they are actually a learning mechanism to detect logical inconsistencies and further our understanding of the world. With GPT-3, you can't tell whether it's joking or not, and whether it's aware if it's joking or not. It doesn't (to my knowledge) have that boundary where it could say, wait, I don't understand what we are saying here. Maybe it's easy to add to the system, but as it stands, I don't think GPT-3 does the above.
tikwidd 5y ago
Do submarines swim?
tgv 5y ago
Trolling by some google AI honcho. What's the world coming to?
quantum_state 5y ago
Obviously not.
akyu 5y ago
Do we understand large language models?
okareaman 5y ago
I feel so strongly that I have a soul that starts with self-aware consciousness and I get such push back on this idea from high tech people, that I'm starting to wonder if perhaps most don't possess a soul and are effectively biorobots. This is also the theme of Kurt Vonnegut's "Breakfast of Champions" in which the author (Vonnegut inserts himself into the novel) is going to kill himself (like his mother) because he feels everyone is a robot except him. He is saved when he meets an artist and senses the soul in the man.
- BeKindAndLearn 5y ago
  Everyone feels that way, but people hold different perspectives when they scrutinize exactly what that means and how it would work. There is no evidence of souls existing, they are unable to be measured in any way. Unless they have several magical rules and violate physics (but only sometimes and are still somehow undetectable), then your brain would still have all those same thoughts about how it feels strongly about having a soul, because that's part of how the ego hard-coded into your brain works. I don't particularly like the view you're espousing, not only because its inconsistent without explanation but because it creates a group of "others" where those with a different philosophical viewpoint are now literally "unfeeling robots" to you.
  - okareaman 5y ago
    I don't actually believe that. I was referring to a work of fiction that is philosophically meaningful to me. What I do believe is there are components to intelligence that includes emotional intelligence, self-awareness and memory that some people don't seem to have in great quantities, but I don't think they are robots.
    - BeKindAndLearn 5y ago
      My mistake for misunderstanding, I suppose. But you initially said that you (1) were beginning to think it, then said it is also (2) the theme of your literature: > I'm (1) starting to wonder if perhaps most don't possess a soul and are effectively biorobots. This is also (2) the theme...
nharada 5y ago
It's easy to dismiss stuff like this without having interacted with these models, but there's something visceral about doing so that at least personally left me re-assessing some of my viewpoints on ML/AI. I've had a chance to interact with the model in the article and it is truly a spooky experience. I'm looking forward to seeing how people react once they get a chance to try some of these increasingly massive chat models in person. GPT-3 is just the tip of the iceberg. (views my own blah blah)
- skybrian 5y ago
  Sure, but I can't get over how it's picking the next word literally at random from a probability distribution - unless you turn that off and then it often goes into an infinite loop. I think it's like song lyrics. We can find meaning in vagueness. Particularly if you retry a few times. High-probability completions are likely to seem meaningful.
  - cecilpl2 5y ago
    How do you know you are not also picking the next word at random from a probability distribution? I mean, as I'm typing this sentence, I have some notion about which word is going to come next, and a sense of where the sentence is going to go. But maybe that is what it feels like to be a large neural network that has been trained on very large data sets for long periods of time.
    - skybrian 5y ago
      One skill we do have is the ability to realize that what we said is not what we meant, and make a correction. This is more goal-oriented than what a transformer is doing. Its only goal is to continue onward in a plausible way and it could be in any direction depending on a random choice. It doesn't care which way it goes.
      - deleted 5y ago
        [deleted]
      - deleted 5y ago
motohagiography 5y ago
> There’s something fundamentally unanswerable about the question “What are the minimum requirements for personhood?”, or more colloquially, “When does an ‘it’ become a ‘who’?” This is the crux of the question, and I'd argue the abstraction and generalization of person and personhood itself is not only flawed, but the base error that diverts the course and trajectory of our ability to reason about ethics. It's perhaps easier to meander around with analogies, contrasts, and comparisons than to delve down into that harder question, and as long as we're circumspect about it, philosophically we're just treading water.
a_square_peg 5y ago
I think this would be like asking if calculators understand math.

64 points by ofou 5y ago

I'm surprised by no mention of the Chinese room argument: https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room - it's closely related to the topic.

gorgoiler 5y ago
Instead of the computer/book telling the non-Chinese speaker what to write, what if it’s a human that speaks Chinese? Does it matter then that the writer has no understanding of the language, when the backend clearly does? (I think I get it actually, after writing these two questions, but I’ll post them anyway.)
- viraptor 5y ago
  I think at "the writer has no understanding of the language, when the backend clearly does?" you circle back to the argument's main problem - is it possible for the non-human backend to "understand" the language, even though it behaves like it does.
elcomet 5y ago
The Chinese room argument is very unconvincing unfortunately, as brains are actually biological computing machines.
subroutine 5y ago
Searle goes into the room with the copy of the program instructions. He gets passed a note written in Chinese, cracks his knuckles, and then opens the manual. He reads, "Step-1: prior to translating the note, we are going to train your neural networks on a Chinese corpus consisting of 410 billion tokens from Common Crawl, 19 billion tokens from WebText, 67 billion tokens from Books, and 3 billion tokens from Wikipedia. Training in 5, 4, 3..." He shouts, "wait but if you do that wo shuō zhōngwén."
- ww_wpg 5y ago
  The search result I'm finding for that phrase is "Wǒ bù huì shuō zhōngwén. I kept saying it over and over again, the phrase I had memorized to explain my inability to speak their language". So I guess it means "I don't speak Mandarin"?
  - elikoga 5y ago
    The "Wǒ bù huì[...]" part means "I negate/not can[...]" It means precisely the opposite.
robbedpeter 5y ago
This isn't aimed at you, viraptor, but at Searle and his argument. Searle and subsequent proponents fail to define mind, intelligence, or understanding. The argument is founded in anthropocentric arrogance. Gpt-3 has convincingly passed the Turing test, with various and sundry incarnations passing as human for days or even weeks on reddit and Twitter and other social media. You can keep moving the goalposts, but the herd has left the barn. The argument is over. The Turing test has been passed. There is no way to frame human mind, intelligence, or understanding such that you exclude the equivalent functionality in software without engaging in magical thinking. Searle's assertion that he would not understand Chinese while executing the rules isn't relevant. Searle could conceivably execute the inference pass over a gpt-3 model based on Chinese, and produce results that surpass the Turing threshold as conceived by Searle at the time. The intelligence and understanding and mind is encoded in the model and logic of the processing. This demonstrates that substrate is not involved in the process of intelligence. There's nothing unique or magical about the human brain that suggests a shred of evidence that it can't be replicated by any Turing complete processing system, or computer. Our minds are an algorithm being run continuously by a messy wet computer in a bone vat connected to a meat suit. Your sense of continuous perception is the state of a model stored in and processed by the physical, electrical, and chemical configuration of your brain and the laws of physics. Any methodical definition of understanding arrives at logic and semantic relationships between concepts and information theory. The brain has to be operating as a computer because there is literally nothing else that could be reasonably asserted. Understanding and consciousness and intelligence and mind are consequences of algorithms, or software. Gpt-3 and similar models are undeniably different types of minds then that of humans, lacking persistent state, online feedback, and a myriad of systems and capabilities. Transformers in their current form are not agi and interact with the world in a radically different way than humans do. They do something that human minds do, however. Consciousness could be the result of a certain class of algorithms being processed, with a self referential model processed concurrent with external input. Additional systems might be required, but consciousness has to be accomplished using no more than the capabilities offered by the brain, so it's likely to be an emergent consequence of the holistic execution of the cortical algorithm. Each inference pass might represent a single moment of mind, a disjoint and ephemeral flash of consciousness, unrelated to its prior or future states, deterministically replicable and static. The Chinese room is a failed argument and we should move on to more interesting things. Brain meat is not magic.

deleted 5y ago

[deleted]

Animats 5y ago

"Furthermore, much of what we consider intelligence is inherently dialogic, hence social; it requires a theory of mind." That's an assumption. The contrary view is that language is just part of the I/O system, not where the hard work gets done. Animals with very limited language capabilities have the ability to run their own lives, and to some extent manipulate their environment. Systems built from large language databases don't have that. After 50 years, robotic manipulation in unstructured situations is still terrible. Rod Brooks tried to crack that and his startup, Rethink Robotics, failed. DARPA has had decades of manipulation projects. Fifty years ago, they could put a bolt in a hole. Five years ago, they got a robot hand to put a key in a lock. Amazon is still trying to get robots to pick things from bins fast and reliably. We lack some fundamental understanding about how to interact with the real world. The problem is not on the language side.

fake-name 5y ago
> That's an assumption. The contrary view is that language is just part of the I/O system, not where the hard work gets done. FYI, this is the [Sapir-Whorf hypothesis](1) > We lack some fundamental understanding about how to interact with the real world. The problem is not on the language side. This is to a significant extent a sensing issue, not a understanding issue. 1: https://en.wikipedia.org/wiki/Linguistic_relativity https://en.wikipedia.org/wiki/Linguistic_relativity
- tsimionescu 5y ago
  The Sappir-Whorf hypothesis refers to specific languages, and is all but proven wrong in its strong form (a person speaking English and a person speaking Japanese seem to perform exactly the same at all cognitive tasks). But there is a different theory, due to Noam Chomsky I believe, that language in the general sense is in fact our mechanism for higher thought, with communication being just a secondary function[0]. I believe that this is the theory that the GP was combatting with the animal example. [0] https://www.youtube.com/watch?v=KEmpRtj34xg https://www.youtube.com/watch?v=KEmpRtj34xg
  - thaumasiotes 5y ago

my50cents 5y ago

> Since the interior state of another being can only be understood through interaction, no objective answer is possible to the question of when an “it” becomes a “who” A large language model can in theory be understood at an algorithmic level by reverse engineering. If the algorithm turned out to be a giant lookup table, it's an "it". If the algorithm contained an obvious model of self, it's a "who".

crooked-v 5y ago
> an obvious model of self How do you determine what's an "obvious" model of self?
- deleted 5y ago
  [deleted]
robbedpeter 5y ago
We assume other people are "who" and not "it" because their behavior is predictably similar to our own, and so we model their experiences and perspectives based on our own subjective experience. Occam's razor suggests that if we know other people have the same basic configuration as ourselves, then the subjective experiences of others will be more or less comparable to our own. Additional assumptions are necessary to propose that Chalmersian/philosophical zombies could exist, having the same human hardware and behavior but lacking subjective experience. Under the principle of least complexity and the absence of evidence for alternate explanations, I think it's rational to assume the consciousness of other humans and certain animals with nearly 100% confidence. That means something happening in the neocortex is causing consciousness. We know it's in the cortex because of the history of injury or absence of other parts of the brain, leaving us with empirical evidence. We know that hippocampus injury can result in a person losing the ability to remember more then 5 minutes of their past. We also know these individuals retain their personalities and the long-term memory from before their injury. This suggests that long term memory is encoded in the synaptic structure of the neocortex and that consciousness is an emergent result of neocortical operation. There might be some involvement of particular brain regions or the thalamus or other organ, but it looks like the answer is part of whatever algorithm is encoded in the structure and processing of the neocortex. It could be possible that consciousness doesn't require an explicit model of self. This is implied by ego death and other experiences by meditators and psychedelic users. The thing that is having a subjective experience could simply be a consequence of processing a particular configuration of information. The self concept seems to be a separate model that can be perceived as part of the process of awareness, but it seems to be a discrete thing. To me that presents an interesting ethical question - if gpt-3 has a sense of self, then is it OK to subject it to the incoherent flashes of single moments of consciousness it undergoes each time you run it? If it has no self concept, it might still have subjective experience, but would have no persistent contiguous experience . If it did have a self concept, then it would have a static past that informed the results of each run, as if you could isolate a single moment of awareness, then instantly reset the brain to a saved state and run it again. The output could be convincingly continuous, but the subjective experiences would be a myriad of similar but unrelated singularities of awareness. I think gpt-3 lacks consciousness, and that a persistence mechanism is missing that plays a part in whatever is happening that causes awareness. Hopefully, at some point, a m mathematician or scientist will be able to identify and explain the process of consciousness so that we can be reasonably certain we're not subjecting entities to a really weird tortured existence.

deleted 5y ago

[deleted]

Borrible 5y ago

Ask them, why would they lie?

js8 5y ago

I have an objection to the idea that understanding is just a pattern recognition, or prediction of the next token. I think fundamental to understanding is the concept of logical contradiction, which I don't believe GPT-3 has. I don't think GPT-3 can say, wait, what I am saying is BS, because logically it's not consistent. To argue that, I propose a "simplified Chinese room". We have an automaton in the Chinese room, which does only pattern recognition, and can learn by matching patterns and structures, i.e. it will give an answer that is similar to what it has seen before. And we want to teach it to answer a problem "recognize whether the given logical formula is satisfiable". So we give it formulas and it tries to pattern match the formulas it already has seen. Unfortunately, there always exists a formula which is matching closely everything that was shown so far, and yet the correct answer is unexpected. So such an automaton will never be able to discover the actual rule. What I think has to be added are different levels of discourse, or, a discourse about discourse. With people, we can say, "stop joking around", and it's a meta-discourse signal that we want stop discussing imaginary worlds and focus on the real one, for example, which has to be totally self-consistent. I believe an analogy can be made with Futamura projections, we can have not only systems that learn a specific task (like even GPT-3 does), but also systems that learn to produce an agent to do a specific task, from a metalanguage description that encodes certain rules on how to do it (for example, to follow logical consistency). And if the metalanguage is general enough, you get general intelligence, because then you can apply the agent production process to itself.

jcelerier 5y ago
> I think fundamental to understanding is the concept of logical contradiction, which I don't believe GPT-3 has are we sure that humans actually have that though ? It's not always my experience
- js8 5y ago
  I think we do. We get a distinct feeling that we don't understand something, and a different feeling when things click together. Or when we hear a syntactically valid (in other words, pattern-matching) but nonsensical sentence (like "colorless green ideas sleep furiously"), or see a Penrose triangle, and we cannot imagine an example, we get a very specific feeling that something is wrong. I think we are actually solving the satisfiability problem every time we imagine something - we find a concrete instance of a situation that satisfies certain conditions (for example, words in a sentence, or pixels in the image). The book Inside Jokes even speculates that reason why we find jokes pleasurable is that they are actually a learning mechanism to detect logical inconsistencies and further our understanding of the world. With GPT-3, you can't tell whether it's joking or not, and whether it's aware if it's joking or not. It doesn't (to my knowledge) have that boundary where it could say, wait, I don't understand what we are saying here. Maybe it's easy to add to the system, but as it stands, I don't think GPT-3 does the above.

tikwidd 5y ago

Do submarines swim?

tgv 5y ago

Trolling by some google AI honcho. What's the world coming to?

quantum_state 5y ago

Obviously not.

akyu 5y ago

Do we understand large language models?

okareaman 5y ago

I feel so strongly that I have a soul that starts with self-aware consciousness and I get such push back on this idea from high tech people, that I'm starting to wonder if perhaps most don't possess a soul and are effectively biorobots. This is also the theme of Kurt Vonnegut's "Breakfast of Champions" in which the author (Vonnegut inserts himself into the novel) is going to kill himself (like his mother) because he feels everyone is a robot except him. He is saved when he meets an artist and senses the soul in the man.

BeKindAndLearn 5y ago
Everyone feels that way, but people hold different perspectives when they scrutinize exactly what that means and how it would work. There is no evidence of souls existing, they are unable to be measured in any way. Unless they have several magical rules and violate physics (but only sometimes and are still somehow undetectable), then your brain would still have all those same thoughts about how it feels strongly about having a soul, because that's part of how the ego hard-coded into your brain works. I don't particularly like the view you're espousing, not only because its inconsistent without explanation but because it creates a group of "others" where those with a different philosophical viewpoint are now literally "unfeeling robots" to you.
- okareaman 5y ago
  I don't actually believe that. I was referring to a work of fiction that is philosophically meaningful to me. What I do believe is there are components to intelligence that includes emotional intelligence, self-awareness and memory that some people don't seem to have in great quantities, but I don't think they are robots.
  - BeKindAndLearn 5y ago
    My mistake for misunderstanding, I suppose. But you initially said that you (1) were beginning to think it, then said it is also (2) the theme of your literature: > I'm (1) starting to wonder if perhaps most don't possess a soul and are effectively biorobots. This is also (2) the theme...

nharada 5y ago

It's easy to dismiss stuff like this without having interacted with these models, but there's something visceral about doing so that at least personally left me re-assessing some of my viewpoints on ML/AI. I've had a chance to interact with the model in the article and it is truly a spooky experience. I'm looking forward to seeing how people react once they get a chance to try some of these increasingly massive chat models in person. GPT-3 is just the tip of the iceberg. (views my own blah blah)

skybrian 5y ago
Sure, but I can't get over how it's picking the next word literally at random from a probability distribution - unless you turn that off and then it often goes into an infinite loop. I think it's like song lyrics. We can find meaning in vagueness. Particularly if you retry a few times. High-probability completions are likely to seem meaningful.
- cecilpl2 5y ago
  How do you know you are not also picking the next word at random from a probability distribution? I mean, as I'm typing this sentence, I have some notion about which word is going to come next, and a sense of where the sentence is going to go. But maybe that is what it feels like to be a large neural network that has been trained on very large data sets for long periods of time.
  - skybrian 5y ago
    One skill we do have is the ability to realize that what we said is not what we meant, and make a correction. This is more goal-oriented than what a transformer is doing. Its only goal is to continue onward in a plausible way and it could be in any direction depending on a random choice. It doesn't care which way it goes.
    - deleted 5y ago
      [deleted]
    - deleted 5y ago

motohagiography 5y ago

> There’s something fundamentally unanswerable about the question “What are the minimum requirements for personhood?”, or more colloquially, “When does an ‘it’ become a ‘who’?” This is the crux of the question, and I'd argue the abstraction and generalization of person and personhood itself is not only flawed, but the base error that diverts the course and trajectory of our ability to reason about ethics. It's perhaps easier to meander around with analogies, contrasts, and comparisons than to delve down into that harder question, and as long as we're circumspect about it, philosophically we're just treading water.

a_square_peg 5y ago

I think this would be like asking if calculators understand math.