4 ms·

But that all boils down to are we having a scientific conversation or a philosophical conversation? In my opinion the only useful conversation is a scientific o

But that all boils down to are we having a scientific conversation or a philosophical conversation? In my opinion the only useful conversation is a scientific on. A philosophical conversation will and can never be resolved so if of no importance to this discussion. We can use philosophy to help guide our scientific conversation, but in the end only a scientific conversation can be helpful in reaching a meaningful/practical conclusion. So back to the questions of "What is knowing?" "Are talking like someone with theory of mind and having a theory of mind the same thing?" If your argument is that the only way to answer this it to have a first person experience of that consciousness then that's not a scientific question. No one will ever have one for an LLM or any other AI. It's like asking "What's happening right now outside of the observable universe?". If it can't impact us, it's irrelevant to science. If that ever changes it will become relevant, but until then it's not a scientific question. Similarly no person can ever have a first person experience of the consciousness of an LLM, so anything that requires being the LLM isn't relevant. So that means the only relevant question is what distinction can outside observers make between an agent talking like a theory of mind and having a theory of mind. And given a high enough accuracy / fidelity of responses I think we're only forced to conclude one of two things: 1. Something that is able to simulate having a theory of mind sufficiently well does actually have a theory of mind. OR 2. I am the only person on the planet with a theory of mind, and all of you are all just simulating having but don't actually have one. It's all "Searle's Chinese room" and "What consciousness is" discussions all over again. And from a scientific point of you either you get into the "it must be implemented identically as me to count" (which is as wrong as saying an object must flap its wings to fly), or you have to conclude the room plus the person combined are knowledgeable and conscious.

abeppu 4y ago
I think you're making a strawman to argue against. Nowhere above have I claimed that "knowing" requires "consciousness", or "it must be implemented identically to me to count", and in fact I believe neither. But: - In this context, following on the whole 2nd half of the 20th century where cognitive science and psychology moved past behaviorism and sought explanations of the _mechanisms_ underlying mental phenomena, a scientific discussion doesn't have to restrict itself to only considering what the LLM says. Neither we, nor the LLM are black boxes. Evidence of _how_ we do what we do is part of scientific inquiry. - But the LLM does _not_ reproduce all the behaviors of an agent with a theory of mind. A two year-old with a developing theory of mind may try to hide food they don't want to eat. A 4-year-old playing hide-and-seek picks locations where they think their play-partner won't look. They take _actions_ which are appropriate for their goals and context which require consideration of the goals of others. The LLM shows elaborate behaviors in one dimension, in which it has been extensively trained. It has no capacity to do anything else, or even receive exposure to non-linguistic contexts. I am in no way arguing that only meat-based minds can "know". I'm saying that the data, training regime and model structure used for LLMs specifically is extremely impoverished, in that we show it language but no other representation of the things language refers to. Similarly, image-generating AIs know what images look like, but they don't know how bodies or physical objects interact, because they have never been exposed to them. Of _course_ we get LLMs that hallucinate and image-generators that produce messed up bodies. On the other hand, there are some pretty cool reinforcement-learning results where agents show what looks like cooperation, develop adversarial strategies, etc. There's experiments where software agents collaboratively invent a language to refer to objects in their (virtual) environment to accomplish simple tasks. I think there are a lot of near and medium-term possibilities coming from multi-modal models (i.e. can models trained on related text, images, audio, video) and RL which could yield knowledge of a kind that LLMs simply do not have.
- BoiledCabbage 4y ago
  Yes valid points you make, but I feel they are still skipping something. To me it seems like you are asking "Does it know the same things we know?"> With the obvious answer is no because it doesn't have all of the senses we have. Someone who is blind, doesn't have a lesser concept of knowing even though they are blind. They might not "know" things in the same way a someone who is seeing, but doesn't mean their version of knowing is any less, they just know fewer facts about the world. Specifically the visual facts of what things look like. Their "knowing" functionality is equal to someone who sees. Similarly, someone who is blind, and deaf also has full ability for "knowing" even if they'll never know things in the visual or auditory spaces. So my argument is that your premise is wrong, the fact that someone or something has fewer senses doesn't mean it's ability to know is any less. So back to your LLM the fact it doesn't exists in the real world is not an exclusion from its ability to know. It does not need to have all of those experiences "to know". It will never know the physical meaning of concepts like we do. Just like I'll never know the details of a city block in Jakarta (as I've never been). But not having that experience (or any experiences of multiple senses) doesn't mean I don't know. LLMs don't need multiple cross connected sensory experiences, nor extensive history with a physical or virtual world to know things. For an entity "to know" it means it has a model it can use to make predictions.
  - abeppu 4y ago
    I think your argument goes off the rails when it jumps from "you don't need any particular sense modality to know" to "you don't need any percepts, or experience of reality or simulated unreality to know". That's a big leap, and I can't disagree more. > For an entity "to know" it means it has a model it can use to make predictions. Great, every PID controller, every jupyter notebook or excel spreadsheet with a linear regression model, every count-down timer can make predictions and therefore "know" under this definition. But perhaps there's a broader class of things that "make predictions". Down this path lies panpsychism. When I throw a rock, its velocity in the x direction at time t is a great "predictor" of its velocity in the x direction at time t+delta, etc, etc. And maybe there's nothing inconsistent or fundamentally wrong with saying that every part of the physical universe "knows" at least something insofar as it participates in predicting or computing the future. But I think by so over-broadening the concept of knowing, it becomes useless, and impossible to make distinctions that matter.
markmaglana 4y ago
> Something that is able to simulate having a theory of mind sufficiently well does actually have a theory of mind. That presupposes that our existing tools for detecting the presence of ToM are 100% accurate. Might it be possible that they are imprecise and it’s only now that their critical flaws have been exposed?
- int_19h 4y ago
  But if our understanding of ToM is so flawed in practice, what does it say about all the confident proclamations that AIs "aren't real" because they don't have it?
  - markmaglana 4y ago
    Your question aligns with the argument I'm trying to make which is: If it turns out that our understanding of ToM is wrong, should we be making proclamations about--whether for or against--the real-ness of our current AI implementations?

abeppu 4y ago

I think you're making a strawman to argue against. Nowhere above have I claimed that "knowing" requires "consciousness", or "it must be implemented identically to me to count", and in fact I believe neither. But: - In this context, following on the whole 2nd half of the 20th century where cognitive science and psychology moved past behaviorism and sought explanations of the _mechanisms_ underlying mental phenomena, a scientific discussion doesn't have to restrict itself to only considering what the LLM says. Neither we, nor the LLM are black boxes. Evidence of _how_ we do what we do is part of scientific inquiry. - But the LLM does _not_ reproduce all the behaviors of an agent with a theory of mind. A two year-old with a developing theory of mind may try to hide food they don't want to eat. A 4-year-old playing hide-and-seek picks locations where they think their play-partner won't look. They take _actions_ which are appropriate for their goals and context which require consideration of the goals of others. The LLM shows elaborate behaviors in one dimension, in which it has been extensively trained. It has no capacity to do anything else, or even receive exposure to non-linguistic contexts. I am in no way arguing that only meat-based minds can "know". I'm saying that the data, training regime and model structure used for LLMs specifically is extremely impoverished, in that we show it language but no other representation of the things language refers to. Similarly, image-generating AIs know what images look like, but they don't know how bodies or physical objects interact, because they have never been exposed to them. Of _course_ we get LLMs that hallucinate and image-generators that produce messed up bodies. On the other hand, there are some pretty cool reinforcement-learning results where agents show what looks like cooperation, develop adversarial strategies, etc. There's experiments where software agents collaboratively invent a language to refer to objects in their (virtual) environment to accomplish simple tasks. I think there are a lot of near and medium-term possibilities coming from multi-modal models (i.e. can models trained on related text, images, audio, video) and RL which could yield knowledge of a kind that LLMs simply do not have.

BoiledCabbage 4y ago
Yes valid points you make, but I feel they are still skipping something. To me it seems like you are asking "Does it know the same things we know?"> With the obvious answer is no because it doesn't have all of the senses we have. Someone who is blind, doesn't have a lesser concept of knowing even though they are blind. They might not "know" things in the same way a someone who is seeing, but doesn't mean their version of knowing is any less, they just know fewer facts about the world. Specifically the visual facts of what things look like. Their "knowing" functionality is equal to someone who sees. Similarly, someone who is blind, and deaf also has full ability for "knowing" even if they'll never know things in the visual or auditory spaces. So my argument is that your premise is wrong, the fact that someone or something has fewer senses doesn't mean it's ability to know is any less. So back to your LLM the fact it doesn't exists in the real world is not an exclusion from its ability to know. It does not need to have all of those experiences "to know". It will never know the physical meaning of concepts like we do. Just like I'll never know the details of a city block in Jakarta (as I've never been). But not having that experience (or any experiences of multiple senses) doesn't mean I don't know. LLMs don't need multiple cross connected sensory experiences, nor extensive history with a physical or virtual world to know things. For an entity "to know" it means it has a model it can use to make predictions.
- abeppu 4y ago
  I think your argument goes off the rails when it jumps from "you don't need any particular sense modality to know" to "you don't need any percepts, or experience of reality or simulated unreality to know". That's a big leap, and I can't disagree more. > For an entity "to know" it means it has a model it can use to make predictions. Great, every PID controller, every jupyter notebook or excel spreadsheet with a linear regression model, every count-down timer can make predictions and therefore "know" under this definition. But perhaps there's a broader class of things that "make predictions". Down this path lies panpsychism. When I throw a rock, its velocity in the x direction at time t is a great "predictor" of its velocity in the x direction at time t+delta, etc, etc. And maybe there's nothing inconsistent or fundamentally wrong with saying that every part of the physical universe "knows" at least something insofar as it participates in predicting or computing the future. But I think by so over-broadening the concept of knowing, it becomes useless, and impossible to make distinctions that matter.

markmaglana 4y ago

> Something that is able to simulate having a theory of mind sufficiently well does actually have a theory of mind. That presupposes that our existing tools for detecting the presence of ToM are 100% accurate. Might it be possible that they are imprecise and it’s only now that their critical flaws have been exposed?

int_19h 4y ago
But if our understanding of ToM is so flawed in practice, what does it say about all the confident proclamations that AIs "aren't real" because they don't have it?
- markmaglana 4y ago
  Your question aligns with the argument I'm trying to make which is: If it turns out that our understanding of ToM is wrong, should we be making proclamations about--whether for or against--the real-ness of our current AI implementations?