5 ms·

> I think ChatGPT is good evidence of this. What evidence do we have that this isn't how intelligence works? What evidence do we have that this is how intellig

> I think ChatGPT is good evidence of this. What evidence do we have that this isn't how intelligence works? What evidence do we have that this is how intelligence works?

dr_dshiv 3y ago
A transformative aspect of human verbal intelligence involves dealing with concepts and their combinations intersection. ChatGPT does this very well. I think we can agree that ChatGPT provides intelligent completions for an astonishing range of human concepts? It seems appropriate to describe what ChatGPT understands and what it doesn’t understand through evals or assessments (in the same way that we can use assessments to determine what a student understands or doesn’t). So if we have to call it “computational understanding”, fine —- but clearly ChatGPT understands an incredible range of concepts and their combinations.
- andrepd 3y ago
  > ChatGPT does this very well Lol, this is precisely what chatgpt doesn't do well at all! It fails to understand simple concepts, logical inference, simple math, etc.
  - dr_dshiv 3y ago
    It’s terrible and math and logic, but ChatGPT is amazing at concepts—that’s why it is so powerful. It doesn’t work programmatically—that’s why it fails at logic. But it can reason inductively very very well. Do you have an example besides logic/math where it doesn’t understand simple concepts?
    - andrepd 3y ago
      What do you mean exactly "besides math/logic"? Because logic underpins everything we think. For example it cannot identify musical chords because despite (I presume) ample training material including explanations of how exactly this works, it cannot reasonable represent this as an abstract rigorous rule, as humans do. So I ask what is C E G and it tells me C major correctly, as it presumably appears many times throughout the training set, yet I ask F Ab Db and it does not tell me Db major, because it did not understand the rules at all. In a sense though, this is just a logic problem.
      - dr_dshiv 3y ago
        Sure. Let’s take quantum theory. There are lots of concepts that are based in math but can be reasoned about non-mathematically. The reason that chatGPT can write quantum computer programs to in any domain (despite the lack of existing programs!) is because it can deal with the concepts of quantum computing and the concepts in a domain (eg, predicting housing prices) and align them. Very little of human reasoning is based on logic and math.
        automatic6131 3y ago
        Now you are making things up
        dr_dshiv 3y ago
        Want to be specific?
        mnky9800n 3y ago
        Adding quantum to whatever you are doing always makes it better!
        andrepd 3y ago
        >There are lots of concepts that are based in math but can be reasoned about non-mathematically. Can you be more specific? I literally don't know what you mean. What can you say about quantum mechanics that is not mathematical or logical in nature? Barring metaphysical issues of interpretation, which I assume is not what you mean.
        dr_dshiv 3y ago
        Rarely do I think responding with chatGPT is appropriate, but this is one of those times. * Of course! When discussing concepts from quantum mechanics without getting into the mathematical details, we can focus on the general ideas and principles that underlie the theory. Here are some key concepts in quantum mechanics that can be explained in a non-mathematical way: 1. Superposition: In quantum mechanics, particles can exist in multiple states simultaneously, until they are measured. This is called superposition. It's like a coin spinning in the air, being both heads and tails at the same time, until it lands and shows one face. 2. Wave-particle duality: Particles like electrons, photons, and others exhibit both wave-like and particle-like properties. This means they can sometimes behave as particles, and at other times, as waves. This dual nature has been experimentally demonstrated through phenomena like the double-slit experiment. 3. Quantum entanglement: When two particles become entangled, their properties become correlated, regardless of the distance between them. If you measure one of the entangled particles, you'll immediately know the state of the other, even if they are light-years apart. This phenomenon is often referred to as "spooky action at a distance." 4. Heisenberg's uncertainty principle: This principle states that we cannot simultaneously know the exact position and momentum of a particle. The more precisely we know one of these properties, the less precisely we can know the other. This inherent uncertainty is a fundamental aspect of quantum mechanics. 5. Quantum tunneling: In quantum mechanics, particles can "tunnel" through barriers that would be insurmountable in classical physics. This is because the particle's wave function, which describes its probable location, can extend beyond the barrier, allowing the particle to appear on the other side. 6. Quantum superposition of states: Quantum systems can exist in multiple states at once, and when you measure a property of the system, it "collapses" into one of the possible states. This is a fundamental difference between quantum and classical mechanics, where systems have definite properties even before measurement. These concepts can be discussed and reasoned about without delving into the complex mathematical equations that govern quantum mechanics. While a mathematical understanding is necessary for rigorous study and application of the theory, non-mathematical discussions can still provide valuable insights into the strange and fascinating world of quantum mechanics.* QED
    - chimprich 3y ago
      > Do you have an example besides logic/math where it doesn’t understand simple concepts? All the time. It often fails to understand simple concepts. It doesn't really seem to understand anything. For example, try to get it to write some code for a program in a moderately obscure programming language. It's terrible: it will confidently produce stuff, but make errors all over the place. It's unable to understand that it doesn't know the language, and it doesn't know how to ask the right questions to improve. It doesn't have a good model of what it's trying to do, or what you're trying to do. If you point out problems it'll happily try again and repeat the same errors over and over again. What it does is intuit an answer based on the data it's already seen. It's amazingly good at identifying, matching, and combining abstractions that it's already been trained on. This is often good enough for simple tasks, because it has been trained on so much of the world's output that it can frequently map a request to learned concepts, but it's basically a glorified Markov model when it comes to genuinely new or obscure stuff. It's a big step forward, but I think the current approach has a ceiling.
      - dr_dshiv 3y ago
        Sure, so what are the specific concepts it doesn’t understand? I don’t think its ability to program in an obscure program is really a great test. That’s a matter of syntax more than semantics, no? Novel conceptual blends are where it excels. Yes, it needs to understand the concepts involved to blend them —but humans need that too.
        chimprich 3y ago
        I think you missed my point. It's understandable that it doesn't know how to program in a moderately obscure language. But the model doesn't understand that it doesn't. The specific concepts it doesn't understand are understanding what it is, its limitations, and what it's being asked to do. It doesn't seem to have any "meta" understanding. It's subconscious thought only. If I asked a human to program in a language they didn't understand, they'd say they couldn't, or they'd ask for further instructions, or some reference to the documentation, or they'd suggest asking someone else to do it, or they'd eventually figure out how to write in the language by experimenting on small programs and gradually writing more complex ones. GPT4 and friends "just" take an input that seems like it could plausibly answer the request. If it gets it wrong then it just has another go using the same generative technique as before with whatever extra direction the human decides to give it. It doesn't think about the problem. ("just" doing a lot of work in the above sentence: what it does is seriously impressive! But it still seems to be well behind humans in capability.)
    - AnimalMuppet 3y ago
      No, it's amazing at words. Humans use words to encode concepts, but ChatGPT doesn't get the concepts at all - just words and their relationships. To the extent that humans have encoded the concepts into words, and that text is in the training set, to that degree ChatGPT can work with the words in a way that is at least somewhat true to the concepts encoded in them. But it doesn't actually understand any of the concepts - just words and their relationships.
      - dr_dshiv 3y ago
        This is the debate, isn’t it? I think if we create tests for understanding and deliver them to people, we will find variations in what people understand. I think we will find the same for chatGPT. But I suspect your notion of understanding is not measurable, is it? For you, chatGPT lacks something essential such that it is incapable of understanding, no matter the test. Or do you have a way to measure this without appeal to consciousness or essentialism?
        AnimalMuppet 3y ago
        Well, consciousness is part of the question, isn't it? We know that we are conscious (even if we can't precisely define what that means). Is ChatGPT conscious? I'm pretty sure the answer is no, but how do you prove it? Does understanding require consciousness? Maybe yes, for the kind of understanding I'm thinking of, but I'm not certain of that. How do you measure understanding? You step a bit outside the training set, and see if whoever (or whatever) being tested can apply what it has learned in that somewhat novel situation. That's hard when ChatGPT has been trained on the entire internet. But to the degree we can test it, ChatGPT often falls down horribly. (It even falls down on things that should be within its training set.) So we conclude that it doesn't actually understand.
        mnky9800n 3y ago
        Another part of all this is chatgpt was trained on the entire internet and still does a mediocre job when its doing well. That's an amazing amount of resources required to do all that to arrive at it being able to do what it does writing codes or whatever when a person typically requires a couple slices of pizza and general access to the internet which that person has Read almost none of. How come humans are so efficient when these computers are using enormous amounts of energy? When will we have a computer that only requires a few slices of pizza worth of energy to get to the next step?
  - throwawayadvsec 3y ago
    GPT is actually EXTREMELY good at math compared to an average human who is not using a calculator. Ask a random person stuff like whats 465*42/12 and you'll get ridiculous answers, and LLM can use tools now. I'd argue that it's actually superior to humans in that regard
    - IIAOPSW 3y ago
      Ask a person on the street and they will either say "I don't know", "I don't have time for this", or on rare occasion you'll find some nerd who starts juggling numbers outloud eventually reaching some rational terminus (at worst with an error in recall but not in principle along the way).

> I think ChatGPT is good evidence of this. What evidence do we have that this isn't how intelligence works? What evidence do we have that this is how intelligence works?

dr_dshiv 3y ago

A transformative aspect of human verbal intelligence involves dealing with concepts and their combinations intersection. ChatGPT does this very well. I think we can agree that ChatGPT provides intelligent completions for an astonishing range of human concepts? It seems appropriate to describe what ChatGPT understands and what it doesn’t understand through evals or assessments (in the same way that we can use assessments to determine what a student understands or doesn’t). So if we have to call it “computational understanding”, fine —- but clearly ChatGPT understands an incredible range of concepts and their combinations.

andrepd 3y ago
> ChatGPT does this very well Lol, this is precisely what chatgpt doesn't do well at all! It fails to understand simple concepts, logical inference, simple math, etc.
- dr_dshiv 3y ago
  It’s terrible and math and logic, but ChatGPT is amazing at concepts—that’s why it is so powerful. It doesn’t work programmatically—that’s why it fails at logic. But it can reason inductively very very well. Do you have an example besides logic/math where it doesn’t understand simple concepts?
  - andrepd 3y ago
    What do you mean exactly "besides math/logic"? Because logic underpins everything we think. For example it cannot identify musical chords because despite (I presume) ample training material including explanations of how exactly this works, it cannot reasonable represent this as an abstract rigorous rule, as humans do. So I ask what is C E G and it tells me C major correctly, as it presumably appears many times throughout the training set, yet I ask F Ab Db and it does not tell me Db major, because it did not understand the rules at all. In a sense though, this is just a logic problem.
    - dr_dshiv 3y ago
      Sure. Let’s take quantum theory. There are lots of concepts that are based in math but can be reasoned about non-mathematically. The reason that chatGPT can write quantum computer programs to in any domain (despite the lack of existing programs!) is because it can deal with the concepts of quantum computing and the concepts in a domain (eg, predicting housing prices) and align them. Very little of human reasoning is based on logic and math.
      - automatic6131 3y ago
        Now you are making things up
        dr_dshiv 3y ago
        Want to be specific?
        mnky9800n 3y ago
        Adding quantum to whatever you are doing always makes it better!
      - andrepd 3y ago
        >There are lots of concepts that are based in math but can be reasoned about non-mathematically. Can you be more specific? I literally don't know what you mean. What can you say about quantum mechanics that is not mathematical or logical in nature? Barring metaphysical issues of interpretation, which I assume is not what you mean.
        dr_dshiv 3y ago
        Rarely do I think responding with chatGPT is appropriate, but this is one of those times. * Of course! When discussing concepts from quantum mechanics without getting into the mathematical details, we can focus on the general ideas and principles that underlie the theory. Here are some key concepts in quantum mechanics that can be explained in a non-mathematical way: 1. Superposition: In quantum mechanics, particles can exist in multiple states simultaneously, until they are measured. This is called superposition. It's like a coin spinning in the air, being both heads and tails at the same time, until it lands and shows one face. 2. Wave-particle duality: Particles like electrons, photons, and others exhibit both wave-like and particle-like properties. This means they can sometimes behave as particles, and at other times, as waves. This dual nature has been experimentally demonstrated through phenomena like the double-slit experiment. 3. Quantum entanglement: When two particles become entangled, their properties become correlated, regardless of the distance between them. If you measure one of the entangled particles, you'll immediately know the state of the other, even if they are light-years apart. This phenomenon is often referred to as "spooky action at a distance." 4. Heisenberg's uncertainty principle: This principle states that we cannot simultaneously know the exact position and momentum of a particle. The more precisely we know one of these properties, the less precisely we can know the other. This inherent uncertainty is a fundamental aspect of quantum mechanics. 5. Quantum tunneling: In quantum mechanics, particles can "tunnel" through barriers that would be insurmountable in classical physics. This is because the particle's wave function, which describes its probable location, can extend beyond the barrier, allowing the particle to appear on the other side. 6. Quantum superposition of states: Quantum systems can exist in multiple states at once, and when you measure a property of the system, it "collapses" into one of the possible states. This is a fundamental difference between quantum and classical mechanics, where systems have definite properties even before measurement. These concepts can be discussed and reasoned about without delving into the complex mathematical equations that govern quantum mechanics. While a mathematical understanding is necessary for rigorous study and application of the theory, non-mathematical discussions can still provide valuable insights into the strange and fascinating world of quantum mechanics.* QED
  - chimprich 3y ago
    > Do you have an example besides logic/math where it doesn’t understand simple concepts? All the time. It often fails to understand simple concepts. It doesn't really seem to understand anything. For example, try to get it to write some code for a program in a moderately obscure programming language. It's terrible: it will confidently produce stuff, but make errors all over the place. It's unable to understand that it doesn't know the language, and it doesn't know how to ask the right questions to improve. It doesn't have a good model of what it's trying to do, or what you're trying to do. If you point out problems it'll happily try again and repeat the same errors over and over again. What it does is intuit an answer based on the data it's already seen. It's amazingly good at identifying, matching, and combining abstractions that it's already been trained on. This is often good enough for simple tasks, because it has been trained on so much of the world's output that it can frequently map a request to learned concepts, but it's basically a glorified Markov model when it comes to genuinely new or obscure stuff. It's a big step forward, but I think the current approach has a ceiling.
    - dr_dshiv 3y ago
      Sure, so what are the specific concepts it doesn’t understand? I don’t think its ability to program in an obscure program is really a great test. That’s a matter of syntax more than semantics, no? Novel conceptual blends are where it excels. Yes, it needs to understand the concepts involved to blend them —but humans need that too.
      - chimprich 3y ago
        I think you missed my point. It's understandable that it doesn't know how to program in a moderately obscure language. But the model doesn't understand that it doesn't. The specific concepts it doesn't understand are understanding what it is, its limitations, and what it's being asked to do. It doesn't seem to have any "meta" understanding. It's subconscious thought only. If I asked a human to program in a language they didn't understand, they'd say they couldn't, or they'd ask for further instructions, or some reference to the documentation, or they'd suggest asking someone else to do it, or they'd eventually figure out how to write in the language by experimenting on small programs and gradually writing more complex ones. GPT4 and friends "just" take an input that seems like it could plausibly answer the request. If it gets it wrong then it just has another go using the same generative technique as before with whatever extra direction the human decides to give it. It doesn't think about the problem. ("just" doing a lot of work in the above sentence: what it does is seriously impressive! But it still seems to be well behind humans in capability.)
  - AnimalMuppet 3y ago
    No, it's amazing at words. Humans use words to encode concepts, but ChatGPT doesn't get the concepts at all - just words and their relationships. To the extent that humans have encoded the concepts into words, and that text is in the training set, to that degree ChatGPT can work with the words in a way that is at least somewhat true to the concepts encoded in them. But it doesn't actually understand any of the concepts - just words and their relationships.
    - dr_dshiv 3y ago
      This is the debate, isn’t it? I think if we create tests for understanding and deliver them to people, we will find variations in what people understand. I think we will find the same for chatGPT. But I suspect your notion of understanding is not measurable, is it? For you, chatGPT lacks something essential such that it is incapable of understanding, no matter the test. Or do you have a way to measure this without appeal to consciousness or essentialism?
      - AnimalMuppet 3y ago
        Well, consciousness is part of the question, isn't it? We know that we are conscious (even if we can't precisely define what that means). Is ChatGPT conscious? I'm pretty sure the answer is no, but how do you prove it? Does understanding require consciousness? Maybe yes, for the kind of understanding I'm thinking of, but I'm not certain of that. How do you measure understanding? You step a bit outside the training set, and see if whoever (or whatever) being tested can apply what it has learned in that somewhat novel situation. That's hard when ChatGPT has been trained on the entire internet. But to the degree we can test it, ChatGPT often falls down horribly. (It even falls down on things that should be within its training set.) So we conclude that it doesn't actually understand.
        mnky9800n 3y ago
        Another part of all this is chatgpt was trained on the entire internet and still does a mediocre job when its doing well. That's an amazing amount of resources required to do all that to arrive at it being able to do what it does writing codes or whatever when a person typically requires a couple slices of pizza and general access to the internet which that person has Read almost none of. How come humans are so efficient when these computers are using enormous amounts of energy? When will we have a computer that only requires a few slices of pizza worth of energy to get to the next step?
- throwawayadvsec 3y ago
  GPT is actually EXTREMELY good at math compared to an average human who is not using a calculator. Ask a random person stuff like whats 465*42/12 and you'll get ridiculous answers, and LLM can use tools now. I'd argue that it's actually superior to humans in that regard
  - IIAOPSW 3y ago
    Ask a person on the street and they will either say "I don't know", "I don't have time for this", or on rare occasion you'll find some nerd who starts juggling numbers outloud eventually reaching some rational terminus (at worst with an error in recall but not in principle along the way).