3 ms·

At least you know I am human. As for the rest, hard to say, because we really don’t know what puts the G in the I inside the coconut. Hard to measure internal

At least you know I am human. As for the rest, hard to say, because we really don’t know what puts the G in the I inside the coconut. Hard to measure internal thought and understanding. Hard for you to even prove I have it.

LoganDark 4y ago
> At least you know I am human. ...we technically don't always assume that people we speak to on the internet identify human, since we don't ourselves, but we at least know that all host bodies (that we know of) are physically human. And we also know that all consciousnesses we've spoken with are "human" in nature as in "formed in a human brain". > Hard to measure internal thought and understanding. Hard for you to even prove I have it. Indeed, but brain scans exist to show mental activity in response to all sorts of things, including unprompted internal thought. You've heard of the EEG test for ADHD, right? You place someone in a room all by themselves, put some electrodes on their head, and leave them there for around 25 minutes. Then you measure brain activity to see if it looks... uhh, ADHD-y. I don't know what exactly they check for, but I'm pretty sure it's some certain type of uncontrollable mind wandering that people with ADHD suffer from. With a language model, you are computing the mathematical transforms that result in its output. And there's no evidence to show that there is any sort of internal thought that occurs there, especially since the entire thing is re-run from scratch for every token. Any "thought" or "understanding" can at best be encoded by mathematical weights in the layers, or something, but it's definitely not the same sort of understanding that biological humans have, or even most animals. The way they run is just fundamentally different. And obviously a mere product of consciousness such as an internet comment can be difficult to tell apart from the output of a hallucinating language model. :) -Emily
- botro 4y ago
  But airplanes don't fly by flapping their wings.
  - LoganDark 4y ago
    Some of them do actually: https://www.youtube.com/watch?v=d1z8YUs2FLc https://www.youtube.com/watch?v=d1z8YUs2FLc What is this an analogy for? -Emily
    - dougmwne 4y ago
      Commenter is saying that it could be possible to get to the same outcome but through a different mechanism. So maybe if we do invent an AGI, it will be able to do all the things a human can do, but the underlying mechanism will not resemble or imitate what our brains are doing. See also: https://en.m.wikipedia.org/wiki/Blindsight_(Watts_novel) https://en.m.wikipedia.org/wiki/Blindsight_(Watts_novel)
      - LoganDark 4y ago
        Ah, the black box theory - since AGIs are defined as being able to perform any task that a human can do, the only thing that matters is that they can perform that task just as good as (or better than) a human can, not that they are implemented the exact same way. Even in jobs requiring empathy and understanding, like (the ideal version of) customer service. When I think of AGI, I think of some sort of agent that can converse and interact in the same way that a human can, that has the same sort of curiosity, imagination, and internal thought that a human does. But, as it turns out, that's not actually required or even implied by the term "AGI" itself. -Emily
        dougmwne 4y ago
        Sure, what’s the difference between empathy and the behavior of empathy. We would say it’s how a person “feels”, a qualia. How to make highly processed sand experience qualia is an ongoing area of research. Maybe the sand already experiences qualia. Some people like to play music for their plants just in case.
- dougmwne 4y ago
  Would a generative model that can output both text and images of thinking brain scans be more convincing? I’m not saying you are incorrect, just pointing out that we’re not exactly sure what we are looking at in those brain scans anyway. Is a thought a mathematical calculation being run on a wet computer? What does the scan of an inferencing GPU reveal about the model architecture it is running? And then again, maybe math is not all you need. Unknown unknowns, possibly unknowable.
  - LoganDark 4y ago
    > Would a generative model that can output both text and images of thinking brain scans be more convincing? No, for the same reason why asking ChatGPT what it is thinking or feeling does not indicate that it actually thinks or feels that way. ChatGPT is a language model designed to generate outputs that mimic its training data, therefore it will learn to mimic the expression and communication of understanding that it does not itself feel. This is why language models are so convincing when they hallucinate, because hallucinating is actually their default state, and the hallucinations just tend to line up with some of the facts contained in the training data, but not always. Therefore they can state falsehoods just as convincingly as they can state facts. > I’m not saying you are incorrect, just pointing out that we’re not exactly sure what we are looking at in those brain scans anyway. Is a thought a mathematical calculation being run on a wet computer? Thought is sort of an ongoing electrochemical chain reaction, though beyond that I indeed don't have much I can say for sure. It's certainly not a mathematical calculation, though it could possibly be simulated by one. -Emily

LoganDark 4y ago

> At least you know I am human. ...we technically don't always assume that people we speak to on the internet identify human, since we don't ourselves, but we at least know that all host bodies (that we know of) are physically human. And we also know that all consciousnesses we've spoken with are "human" in nature as in "formed in a human brain". > Hard to measure internal thought and understanding. Hard for you to even prove I have it. Indeed, but brain scans exist to show mental activity in response to all sorts of things, including unprompted internal thought. You've heard of the EEG test for ADHD, right? You place someone in a room all by themselves, put some electrodes on their head, and leave them there for around 25 minutes. Then you measure brain activity to see if it looks... uhh, ADHD-y. I don't know what exactly they check for, but I'm pretty sure it's some certain type of uncontrollable mind wandering that people with ADHD suffer from. With a language model, you are computing the mathematical transforms that result in its output. And there's no evidence to show that there is any sort of internal thought that occurs there, especially since the entire thing is re-run from scratch for every token. Any "thought" or "understanding" can at best be encoded by mathematical weights in the layers, or something, but it's definitely not the same sort of understanding that biological humans have, or even most animals. The way they run is just fundamentally different. And obviously a mere product of consciousness such as an internet comment can be difficult to tell apart from the output of a hallucinating language model. :) -Emily

botro 4y ago
But airplanes don't fly by flapping their wings.
- LoganDark 4y ago
  Some of them do actually: https://www.youtube.com/watch?v=d1z8YUs2FLc https://www.youtube.com/watch?v=d1z8YUs2FLc What is this an analogy for? -Emily
  - dougmwne 4y ago
    Commenter is saying that it could be possible to get to the same outcome but through a different mechanism. So maybe if we do invent an AGI, it will be able to do all the things a human can do, but the underlying mechanism will not resemble or imitate what our brains are doing. See also: https://en.m.wikipedia.org/wiki/Blindsight_(Watts_novel) https://en.m.wikipedia.org/wiki/Blindsight_(Watts_novel)
    - LoganDark 4y ago
      Ah, the black box theory - since AGIs are defined as being able to perform any task that a human can do, the only thing that matters is that they can perform that task just as good as (or better than) a human can, not that they are implemented the exact same way. Even in jobs requiring empathy and understanding, like (the ideal version of) customer service. When I think of AGI, I think of some sort of agent that can converse and interact in the same way that a human can, that has the same sort of curiosity, imagination, and internal thought that a human does. But, as it turns out, that's not actually required or even implied by the term "AGI" itself. -Emily
      - dougmwne 4y ago
        Sure, what’s the difference between empathy and the behavior of empathy. We would say it’s how a person “feels”, a qualia. How to make highly processed sand experience qualia is an ongoing area of research. Maybe the sand already experiences qualia. Some people like to play music for their plants just in case.
dougmwne 4y ago
Would a generative model that can output both text and images of thinking brain scans be more convincing? I’m not saying you are incorrect, just pointing out that we’re not exactly sure what we are looking at in those brain scans anyway. Is a thought a mathematical calculation being run on a wet computer? What does the scan of an inferencing GPU reveal about the model architecture it is running? And then again, maybe math is not all you need. Unknown unknowns, possibly unknowable.
- LoganDark 4y ago
  > Would a generative model that can output both text and images of thinking brain scans be more convincing? No, for the same reason why asking ChatGPT what it is thinking or feeling does not indicate that it actually thinks or feels that way. ChatGPT is a language model designed to generate outputs that mimic its training data, therefore it will learn to mimic the expression and communication of understanding that it does not itself feel. This is why language models are so convincing when they hallucinate, because hallucinating is actually their default state, and the hallucinations just tend to line up with some of the facts contained in the training data, but not always. Therefore they can state falsehoods just as convincingly as they can state facts. > I’m not saying you are incorrect, just pointing out that we’re not exactly sure what we are looking at in those brain scans anyway. Is a thought a mathematical calculation being run on a wet computer? Thought is sort of an ongoing electrochemical chain reaction, though beyond that I indeed don't have much I can say for sure. It's certainly not a mathematical calculation, though it could possibly be simulated by one. -Emily