18 ms·

I wonder every time I see this take what it would mean under this definition of knowing things for a machine learning algorithm to ever know something. I find t

I wonder every time I see this take what it would mean under this definition of knowing things for a machine learning algorithm to ever know something. I find that especially important because to every appearance we are a machine learning algorithm. I don’t know how different the sort of knowing this algorithm has to the sort of knowing a human has, but you’re far more confident than I am that it’s a difference of kind rather than degree. Some interesting facts that point to it being a difference of degree. LLM are actually are more accurate when asked to explain their thinking. They make similar mistakes to humans intuitive reasoning. It might help to define what we even mean by knowing things. To me being able to make novel predictions that require the knowledge is the only definition one could use that doesn’t run into the possibility of deciding humans don’t actually know anything

ves 4y ago
> I find that especially important because to every appearance we are a machine learning algorithm. Speak for yourself.
robertlagrant 4y ago
> It might help to define what we even mean by knowing things This is it, I think. It's interesting that we now have a practical example to point at when asking formerly-abstruse philosophical questions.
colechristensen 4y ago
The kinds of mistakes something makes are a strong indicator of whether answers are a product of understanding or memorization.
avgcorrection 4y ago
The problem with this facile view of things is that it seems to be a dead end for scientific theories. What if we just limited the science of birds to explaining how limb-flapping could produce levitation? Hmm yes. Birds are kind of like helicopters, it seems. Who’s to say that they are not basically one and the same? Moving on. If you are only interested in the most superficial tests and theories—like the Turing Test—then consider psychology conquered once you’ve tricked a human with your chat bot. Game Over. And what did you learn...?
- HTTP418 4y ago
  > Birds are kind of like helicopters, it seems. We could easily argue that birds are not a type of helicopter because for helicopter's we have a very specific set of flying properties required. It must have a main propeller for lift and a tail propeller to counter balance the main propeller from spinning the helicopter. If a bird flew with similar mechanism I would argue it was a helicopter. We don't have a 100% accurate gauge for ToM as far as we know. This paper simply uses some of the best known tests for ToM and then states that either LLM can lead to emergent properties or that the current tests for ToM need to be re-thought.
- curiousllama 4y ago
  > What if we just limited the science of birds to explaining how limb-flapping could produce levitation? Hmm yes. Birds are kind of like helicopters, it seems. Who’s to say that they are not basically one and the same? Moving on. Well said. I'm gonna steal this explanation. Also reminds me of the famous Carbonara quote: "if my grandmother had wheels, then she would be a bike" [1] [1] https://www.youtube.com/watch?v=A-RfHC91Ewc https://www.youtube.com/watch?v=A-RfHC91Ewc
  - HTTP418 4y ago
    > "if my grandmother had wheels, then she would be a bike" Well it could be argued that she would be a bike. Its possible to be multiple things at once. If she had 2 wheels and could be ridden by other humans to a destination she might qualify has a bike. She would also continue to be your grandmother.
- s0ulphire 4y ago
  The question being asked was "what it would mean under this definition of knowing things for a machine learning algorithm to ever know something". Aside from your answer being rude, it's also unhelpful in that it doens't address the question asked and instead relies on reductio ad absurdum to pretend to make a point. If you'd like to take a crack at a helpful answer, perhaps educate us all on what it WOULD take for you to consider a NN to actually "know" something in the same way that we say a human or other sentient animal does.
abeppu 4y ago
Defining what "knowing" is would be useful, yes, and analytic philosophers in epistemology do argue about this. One attribute that's classically part of the definition of "knowing" is that the thing which is known must be true. LLMs are pretty bad at this, but perhaps that can be fixed. But I would challenge you to imagine the situation the LLM is actually in. Do you understand Thai? If so, in the following, feel free to imagine some other language which you don't know and is not closely related to any languages you do know. Suppose I gather reams and reams of Thai text, without images, without context. Books without their covers, or anything which would indicate genre. There's no Thai-English dictionary available, or any Thai speakers. You aren't taught which symbols map to which sounds. You're on your own with a giant pile of text, and asked to learn to predict symbols. If you had sufficient opportunity to study this pile of text, you'd begin to pick out patterns of which words appear together, and what order words often appear in. Suppose you study this giant stack of Thai text for years in isolation. After all this study, you're good enough that given a few written Thai words, you can write sequences of words that are likely to follow, given what you know of these patterns. You can fill in blanks. But should anyone guess that you "know" what you're saying? Nothing has ever indicated to you what any of these words _mean_. If you give back a sequence of words, which a Thai speakers understands to be expressing an opinion about monetary policy, because you read several similar sequences in the pile, is that even your opinion? I think algorithms can 'know' something, given sufficient grounding. LLMs 'know' what text looks like. They can 'know' what tokens belong where, even if they don't know anything about the things referred to. That's all, because that's what they have to learn from. I think an game-playing RL-trained agent can 'know' the likely state-change that a given action will cause. An image segmentation model can 'know' which value-differences in adjacent pixels are segment boundaries. But if we want AIs that 'know' the same things we know, then we have to build them to perceive in a multi-modal way, and interact with stuff in the world, rather than just self-supervising on piles of internet data.
- skjoldr 4y ago
  >But if we want AIs that 'know' the same things we know, then we have to build them to perceive in a multi-modal way, and interact with stuff in the world, rather than just self-supervising on piles of internet data. In other words, a LLM that is tied to a GAN that generates images, produces an system that can both describe to you what is a cat verbally and show you a picture of a cat. Does it, then, know what "a cat" is? Edit: Furthermore, if you then tie this AI to a CV model with a camera which you can point at a cat and it will tell you that it is, indeed, a cat, and then it will also be able to produce a verbal description of a cat as well as show you an abstract picture of a cat or pick cats out of a random set of images, does this whole system know what "a cat" is? If you, then, make a robot with a camera and hands, attach to the system a more complex CV model that can see in 3D, ask the LLM to produce you a set of code instructions that can be parametrized to produce a motion that would pet the cat, input those instructions into the robot to make it pet a specific cat that has the specific 3D point cloud (I guess that's currently difficult but solveable), and the system will then indeed pet the cat, would it then know what "a cat" is?.. The underlying LLM is still the same in all these scenarios. Where is the boundary?
NikolaNovak 4y ago
I am not in the field so I cannot speak very eloquently what it would mean for machine learning algorithm to "ever know something". But I feel that e.g. Simulations and perhaps expert systems of yore, were qualitatively closer to getting there. Their error modes were radically different. They started inductively with rules, rather than arriving at them statistically almost by accident.
- pixl97 4y ago
  Did we end up with human intelligence statistically pretty much by accident?
simonh 4y ago
We can probe issues like what do language models know and what they understand in several ways. One is through an understanding of the process it’s following. Another is through seeing how that leads to its responses. Then thirdly by looking at the kinds of errors it makes. Using multiple axes of approach like this we can triangulate in on what it’s doing and what it understands. In terms of how it works, that’s well known and hardly worth repeating in depth, but to summarise it calculates a probability for the next word in a sequence based on a massive training set of human language word sequences. So what kind of output do they produce? If you ask what it likes to do on the weekend, GPT3 will say generally something about how it likes to spend time with family and friends, because that’s what it has in it’s training set. GPT3 doesn’t have a family, or friends, it doesn’t hang out. It talks about itself because its training set includes people talking about themselves, but it has no concept of self or what it is. It’s a text generator function. It can write a poem about the warm sun on its face, but it doesn't have a face or feel the sun. It’s just regurgitating stuff people wrote about that. Newer systems like ChatGPT have guard rail functions that catch things like this and say it’s a language model, but the guard rails don’t change the nature of what it is, they’re just overrides. So what kind of errors do they make? They can be trivially tricked into talking utter nonsense, or say sensible things in absurd contexts. Here’s an example where someone asked ChatGPT if it spoke Danish, and it replied that no it can’t speak Danish, it’s an English language model , etc. except here’s the kicker, it gave the reply in perfect Danish. https://www.reddit.com/r/GPT3/comments/zb4msc/speaking_to_chatgpt_in_perfect_danish_while_it/ https://www.reddit.com/r/GPT3/comments/zb4msc/speaking_to_ch... Again they’ve now added guard rails for this failure mode as well. Nevertheless the basic problem persists in the architecture. It’s doesn’t have a clue what anything means, beyond calculating word probabilities. This means if you know how they work, you can craft text prompts that expose how ludicrously unaware they are. This ability to expose their weaknesses demonstrates that we do genuinely understand how they function and what their limitations are. So I agree yours is a very reasonable question and it’s not trivial to answer satisfactorily, but we can triangulate in using multiple lines of approach on what these things are or are not. As the guard rails become more complete the failure modes will get harder up find, but they’re still there in the core implementation, they’re just being papered over. There’s not going to be a simple answer. We need to look deeper at the mechanisms and functions of these things. The same goes for human brains of course, we’re just scratching the surface of those too. But while I agree we are neural systems and share some characteristics with LLMs and Alphazero and such, Alphazero isn’t an LLM, and we aren’t either of them. One day we will create something as sophisticated and maybe even as genuinely conscious as ourselves and the questions you ask will be important guides, but these things are a long, long way from that.
TheOtherHobbes 4y ago
Stick a pin in your finger. That pain is what knowing something means. Philosophically we're talking about embodied qualia, which is how humans experience objects and more basic sensations. Language happens later - much later. The defining property of a bag isn't that you can put things in it. Like language that comes later. The defining properties are how it feels when you hold it, when you open it, the differences in sensation between empty/partially empty/full. And so on. An LLM has no embodied experience, so it has no idea what a bag feels like as a set of physical sensations and directly perceived relationships. Failure to understand embodiment has done more to hold back AI than any other philosophical error. Researchers have assumed - wrongly - that you can define an object by its visual properties and its linguistic associations. That's simply not how it works for humans. We get there after a while, but we start from something far more visceral - so much so that many fundamental linguistic abstractions are metaphors based on the simplest and most common qualia.
- frogperson 4y ago
  I agree with this line of reasoning. I suspect true A.I. will leverage some form of embodied qualia and some level of self-preservation. I will start to worry when the Boston Dynamic robots start refusing orders to save battery power.
  - goatlover 4y ago
    If we can make machines conscious, it's quite possible some will have completely alien experiences to our own, particularly as they're designed for different purposes than how our bodies function. Reminds me of the Battlestar Galactica model complaining that he was made too human and wanted to taste stars and what not.
- seanhunter 4y ago
  Isn't that reasoning a "philosophical error" in itself though? If you make embodied experience be a prerequisite, then things which can't embody stuff can't meet that prerequisite. That doesn't seem a very interesting insight. An AI literally cannot embody pain - it has no nervous system and no pain receptors. So AI is excluded from understanding it in that way by definition. It has no sensory perception of any kind so cannot have the kind of embodied experience. Heck it doesn't even have a body with which to embody anything. This is obviously unsatisfactory because it seems just a logical/rhetorical trick. It's also no different from the concept of a person with no visual apparatus (mentioned in another comment thread) and whether they have thought about light and colour and so on. The fact that they are physically unable to have the same kind of experience of these things as someone else doesn't preclude them from having thoughts and experiences that are within the domain of their perception. An LLM is even more limited than an AI generally because it is literally a model of language. I don't personally think that any LLM could conceivably have a theory of mind, but arguing that it cannot have a theory of mind simply because of things that are exogenous to language by definition seems arbitrary.

ves 4y ago

> I find that especially important because to every appearance we are a machine learning algorithm. Speak for yourself.

robertlagrant 4y ago

> It might help to define what we even mean by knowing things This is it, I think. It's interesting that we now have a practical example to point at when asking formerly-abstruse philosophical questions.

colechristensen 4y ago

The kinds of mistakes something makes are a strong indicator of whether answers are a product of understanding or memorization.

avgcorrection 4y ago

The problem with this facile view of things is that it seems to be a dead end for scientific theories. What if we just limited the science of birds to explaining how limb-flapping could produce levitation? Hmm yes. Birds are kind of like helicopters, it seems. Who’s to say that they are not basically one and the same? Moving on. If you are only interested in the most superficial tests and theories—like the Turing Test—then consider psychology conquered once you’ve tricked a human with your chat bot. Game Over. And what did you learn...?

HTTP418 4y ago
> Birds are kind of like helicopters, it seems. We could easily argue that birds are not a type of helicopter because for helicopter's we have a very specific set of flying properties required. It must have a main propeller for lift and a tail propeller to counter balance the main propeller from spinning the helicopter. If a bird flew with similar mechanism I would argue it was a helicopter. We don't have a 100% accurate gauge for ToM as far as we know. This paper simply uses some of the best known tests for ToM and then states that either LLM can lead to emergent properties or that the current tests for ToM need to be re-thought.
curiousllama 4y ago
> What if we just limited the science of birds to explaining how limb-flapping could produce levitation? Hmm yes. Birds are kind of like helicopters, it seems. Who’s to say that they are not basically one and the same? Moving on. Well said. I'm gonna steal this explanation. Also reminds me of the famous Carbonara quote: "if my grandmother had wheels, then she would be a bike" [1] [1] https://www.youtube.com/watch?v=A-RfHC91Ewc https://www.youtube.com/watch?v=A-RfHC91Ewc
- HTTP418 4y ago
  > "if my grandmother had wheels, then she would be a bike" Well it could be argued that she would be a bike. Its possible to be multiple things at once. If she had 2 wheels and could be ridden by other humans to a destination she might qualify has a bike. She would also continue to be your grandmother.
s0ulphire 4y ago
The question being asked was "what it would mean under this definition of knowing things for a machine learning algorithm to ever know something". Aside from your answer being rude, it's also unhelpful in that it doens't address the question asked and instead relies on reductio ad absurdum to pretend to make a point. If you'd like to take a crack at a helpful answer, perhaps educate us all on what it WOULD take for you to consider a NN to actually "know" something in the same way that we say a human or other sentient animal does.

abeppu 4y ago

Defining what "knowing" is would be useful, yes, and analytic philosophers in epistemology do argue about this. One attribute that's classically part of the definition of "knowing" is that the thing which is known must be true. LLMs are pretty bad at this, but perhaps that can be fixed. But I would challenge you to imagine the situation the LLM is actually in. Do you understand Thai? If so, in the following, feel free to imagine some other language which you don't know and is not closely related to any languages you do know. Suppose I gather reams and reams of Thai text, without images, without context. Books without their covers, or anything which would indicate genre. There's no Thai-English dictionary available, or any Thai speakers. You aren't taught which symbols map to which sounds. You're on your own with a giant pile of text, and asked to learn to predict symbols. If you had sufficient opportunity to study this pile of text, you'd begin to pick out patterns of which words appear together, and what order words often appear in. Suppose you study this giant stack of Thai text for years in isolation. After all this study, you're good enough that given a few written Thai words, you can write sequences of words that are likely to follow, given what you know of these patterns. You can fill in blanks. But should anyone guess that you "know" what you're saying? Nothing has ever indicated to you what any of these words _mean_. If you give back a sequence of words, which a Thai speakers understands to be expressing an opinion about monetary policy, because you read several similar sequences in the pile, is that even your opinion? I think algorithms can 'know' something, given sufficient grounding. LLMs 'know' what text looks like. They can 'know' what tokens belong where, even if they don't know anything about the things referred to. That's all, because that's what they have to learn from. I think an game-playing RL-trained agent can 'know' the likely state-change that a given action will cause. An image segmentation model can 'know' which value-differences in adjacent pixels are segment boundaries. But if we want AIs that 'know' the same things we know, then we have to build them to perceive in a multi-modal way, and interact with stuff in the world, rather than just self-supervising on piles of internet data.

skjoldr 4y ago
>But if we want AIs that 'know' the same things we know, then we have to build them to perceive in a multi-modal way, and interact with stuff in the world, rather than just self-supervising on piles of internet data. In other words, a LLM that is tied to a GAN that generates images, produces an system that can both describe to you what is a cat verbally and show you a picture of a cat. Does it, then, know what "a cat" is? Edit: Furthermore, if you then tie this AI to a CV model with a camera which you can point at a cat and it will tell you that it is, indeed, a cat, and then it will also be able to produce a verbal description of a cat as well as show you an abstract picture of a cat or pick cats out of a random set of images, does this whole system know what "a cat" is? If you, then, make a robot with a camera and hands, attach to the system a more complex CV model that can see in 3D, ask the LLM to produce you a set of code instructions that can be parametrized to produce a motion that would pet the cat, input those instructions into the robot to make it pet a specific cat that has the specific 3D point cloud (I guess that's currently difficult but solveable), and the system will then indeed pet the cat, would it then know what "a cat" is?.. The underlying LLM is still the same in all these scenarios. Where is the boundary?

NikolaNovak 4y ago

I am not in the field so I cannot speak very eloquently what it would mean for machine learning algorithm to "ever know something". But I feel that e.g. Simulations and perhaps expert systems of yore, were qualitatively closer to getting there. Their error modes were radically different. They started inductively with rules, rather than arriving at them statistically almost by accident.

pixl97 4y ago
Did we end up with human intelligence statistically pretty much by accident?

simonh 4y ago

We can probe issues like what do language models know and what they understand in several ways. One is through an understanding of the process it’s following. Another is through seeing how that leads to its responses. Then thirdly by looking at the kinds of errors it makes. Using multiple axes of approach like this we can triangulate in on what it’s doing and what it understands. In terms of how it works, that’s well known and hardly worth repeating in depth, but to summarise it calculates a probability for the next word in a sequence based on a massive training set of human language word sequences. So what kind of output do they produce? If you ask what it likes to do on the weekend, GPT3 will say generally something about how it likes to spend time with family and friends, because that’s what it has in it’s training set. GPT3 doesn’t have a family, or friends, it doesn’t hang out. It talks about itself because its training set includes people talking about themselves, but it has no concept of self or what it is. It’s a text generator function. It can write a poem about the warm sun on its face, but it doesn't have a face or feel the sun. It’s just regurgitating stuff people wrote about that. Newer systems like ChatGPT have guard rail functions that catch things like this and say it’s a language model, but the guard rails don’t change the nature of what it is, they’re just overrides. So what kind of errors do they make? They can be trivially tricked into talking utter nonsense, or say sensible things in absurd contexts. Here’s an example where someone asked ChatGPT if it spoke Danish, and it replied that no it can’t speak Danish, it’s an English language model , etc. except here’s the kicker, it gave the reply in perfect Danish. https://www.reddit.com/r/GPT3/comments/zb4msc/speaking_to_chatgpt_in_perfect_danish_while_it/ https://www.reddit.com/r/GPT3/comments/zb4msc/speaking_to_ch... Again they’ve now added guard rails for this failure mode as well. Nevertheless the basic problem persists in the architecture. It’s doesn’t have a clue what anything means, beyond calculating word probabilities. This means if you know how they work, you can craft text prompts that expose how ludicrously unaware they are. This ability to expose their weaknesses demonstrates that we do genuinely understand how they function and what their limitations are. So I agree yours is a very reasonable question and it’s not trivial to answer satisfactorily, but we can triangulate in using multiple lines of approach on what these things are or are not. As the guard rails become more complete the failure modes will get harder up find, but they’re still there in the core implementation, they’re just being papered over. There’s not going to be a simple answer. We need to look deeper at the mechanisms and functions of these things. The same goes for human brains of course, we’re just scratching the surface of those too. But while I agree we are neural systems and share some characteristics with LLMs and Alphazero and such, Alphazero isn’t an LLM, and we aren’t either of them. One day we will create something as sophisticated and maybe even as genuinely conscious as ourselves and the questions you ask will be important guides, but these things are a long, long way from that.

TheOtherHobbes 4y ago

Stick a pin in your finger. That pain is what knowing something means. Philosophically we're talking about embodied qualia, which is how humans experience objects and more basic sensations. Language happens later - much later. The defining property of a bag isn't that you can put things in it. Like language that comes later. The defining properties are how it feels when you hold it, when you open it, the differences in sensation between empty/partially empty/full. And so on. An LLM has no embodied experience, so it has no idea what a bag feels like as a set of physical sensations and directly perceived relationships. Failure to understand embodiment has done more to hold back AI than any other philosophical error. Researchers have assumed - wrongly - that you can define an object by its visual properties and its linguistic associations. That's simply not how it works for humans. We get there after a while, but we start from something far more visceral - so much so that many fundamental linguistic abstractions are metaphors based on the simplest and most common qualia.

frogperson 4y ago
I agree with this line of reasoning. I suspect true A.I. will leverage some form of embodied qualia and some level of self-preservation. I will start to worry when the Boston Dynamic robots start refusing orders to save battery power.
- goatlover 4y ago
  If we can make machines conscious, it's quite possible some will have completely alien experiences to our own, particularly as they're designed for different purposes than how our bodies function. Reminds me of the Battlestar Galactica model complaining that he was made too human and wanted to taste stars and what not.
seanhunter 4y ago
Isn't that reasoning a "philosophical error" in itself though? If you make embodied experience be a prerequisite, then things which can't embody stuff can't meet that prerequisite. That doesn't seem a very interesting insight. An AI literally cannot embody pain - it has no nervous system and no pain receptors. So AI is excluded from understanding it in that way by definition. It has no sensory perception of any kind so cannot have the kind of embodied experience. Heck it doesn't even have a body with which to embody anything. This is obviously unsatisfactory because it seems just a logical/rhetorical trick. It's also no different from the concept of a person with no visual apparatus (mentioned in another comment thread) and whether they have thought about light and colour and so on. The fact that they are physically unable to have the same kind of experience of these things as someone else doesn't preclude them from having thoughts and experiences that are within the domain of their perception. An LLM is even more limited than an AI generally because it is literally a model of language. I don't personally think that any LLM could conceivably have a theory of mind, but arguing that it cannot have a theory of mind simply because of things that are exogenous to language by definition seems arbitrary.