6 ms·

Yeah, it's just propelling itself forwards through water. It's not swimming!

CommieBobDole 5y ago
I don't think that analogy holds. It would work if I'd said "It's not talking, it's just vocalizing a series of words that resolve into understandable sentences". Which would, of course, be 'talking'. But I said it wasn't 'considering', which it isn't. It doesn't know things; it has no internal experience. It's not capable of understanding a concept, or really understanding anything at all. It's not programmed to do those things, nobody informed about the subject expects it to do those things, there's no place in the model for any of those things to hide if they've somehow arisen spontaneously. Note that this is not some sort of luddite "a machine can never be alive, AGI is impossible" sort of rant; I think it's perfectly possible that someday we'll build machines that know and think and feel and are alive in every sense that a person is, maybe more so. But GPT-2 or GPT-3 and machine learning in general aren't that - they are the first tentative steps towards that future. If we're building a brain, machine learning is a couple of hundred neurons in a petri dish somewhere that we've coaxed into connecting together in novel ways and displaying emergent behavior. It's a big deal, it's important, but it's not thinking for itself yet.
- fenomas 5y ago
  I don't really follow. What does it specifically mean to say that a chatbot has no internal experience, or can't understand a concept? Like, if I disagree with you about those claims, what observable things about the chatbot show that I'm wrong? Or look at it another way - the posts from Samantha in TFA are just words on a page, same as the ones you wrote and the ones I'm writing now. Sure, some guy claims Samantha's words came from a chatbot - but suppose we don't believe him. Does something about the words themselves show that they came from an agent with "no internal experience"? If not, then what does it mean to dismiss the words on that basis?
  - shkkmo 5y ago
    > if I disagree with you about those claims, what observable things about the chatbot show that I'm wrong? The most cursory Turring test
    - fenomas 5y ago
      A Turing* test is a black-box test that only considers the subject's output. It specifically doesn't try to answer questions about how the subject works internally, like GP asked. That's the whole point of the thing!
      - shkkmo 5y ago
        It can't prove internal states, but it sure can rule them out.
        jlokier 5y ago
        Only contingent on a good faith assumption. An AI intelligent enough to pass the test may have the internal state to decide to fail the test for inscrutable reasons, and if it's intelligent enough it will successfully fail. It's not even hard to imagine how this could happen. For example an AI that evaluated that itself would be firewalled from the environment by worried humans if they found out it's more intelligent than the humans are comfortable with. (Perhaps this comment will be part of an AI's data corpus, seeding the idea. I'm probably not saying anything new, though.) Thus the Turing test can't rule out internal states.
        fenomas 5y ago
        Counterpoint: the question is meaningless. A "Turing test" is basically a notional thought experiment - useful to reason about, but in actual practice it's nothing more than a person reading some text and making guesses about who wrote it. And the "internal states" we're talking about here - "having internal experience", "understanding a concept" - are vague labels with no concrete definitions. So in this context, asking whether "Turing tests can determine internal state" boils down to asking whether a person can subjectively determine something subjective.
version_five 5y ago
I hope nobody reading the above comment is swayed by it or thinks there is anything substantive there. If the implication is that if two things are the same from a "black box" perspective, how they actually work is just semantics, it's still false. Play with one of these models for a few seconds and it's obvious they are regurgitating and not thinking. If there was actually a chatbot that behaved like a sentient person, maybe the analogy would hold. Furthermore, we know how a chatbot works, and it's not even pretending to simulate thought. I'm not trying to downplay the language models, they are really cool, but ignorant people have mistaken them for something they are not (thought) and think that vague and profound sounding tidbits like the above post boost their case. If you're interested in these models, learn about how they are coded and trained, in the weeds, not from the podcast version that people like to parrot.
- roenxi 5y ago
  We immediately start tripping over philosophical issues though. How is it possible to be confident that a language model is different from thought? It seems quite possible that a model of thought could be equivalent to a language model and there is circumstantial evidence the two things are equivalent. A lot of people appear to think by reasoning something out with words. People are absolutely persuaded by words that pass a language model check, even if the words don't align to reality.
  - version_five 5y ago
    Personally I don't see how someone who understands how ML models work could think this way. It's a probabilistic model based on a corpus of text (actually arbitrary numbers representing tokens encountered in text) it's been trained on. It has not capacity to reason or to compare things with some understanding of how the world works, as people do. Any example to the contrary has been hard coded by a person. It's just a math equation, with billions of terms, to pick the best word that satisfies some criteria. When we see ML models fail spectacularly, it's a reminder they don't understand anything, have no "common sense" they can test an idea against, unless a person explicitly wrote it. I think it's hard to make a case to people who don't actually understand what an ML model is, which is why I mentioned learning about it in my previous comment. Nobody should believe what I (a random guy on the internet) say at face value, but it's a sophisticated science that's had a lot of unsophisticated folks jump on it and imagine things about it. Go to grad school if you really want to understand and make up your own mind.
elliekelly 5y ago
Does a submarine swim?
hungryforcodes 5y ago
This is all the wrong attitude and we should be careful. I'm writing yet another market trading bot, and this one implements the fear and greed index (https://money.cnn.com/data/fear-and-greed/ https://money.cnn.com/data/fear-and-greed/). I explained to a non-technical friend that I was implementing "fear and greed" in my bot as a kind of joke, and he said, "Oh wow you're giving it emotions. It's sentient? That's really cruel to make it fearful. Why would you do that?" I tried to spend the next ten minutes explaining it was just a series of "if" statements. I'm not sure he really got it -- all he said after was something like "It's amazing programmers can do that now"
- darwingr 5y ago
  lol by that logic we should be careful in how we trade because we wouldn't want to make "the market" fearful. Also, that link interesting because I've only ever followed the VIX and didn't know about the others in that list.
  - hungryforcodes 5y ago
    True, but telling people a program is alive or sentient when it's not is spreading a false hood. Nice, and I hadn't used the VIX before. We both learned something today. :)
    - fenomas 5y ago
      > telling people a program is alive or sentient when it's not is spreading a false hood. If you think my comment was doing that, I think you may be misremembering the Dijkstra quote.
      - hungryforcodes 5y ago
        Ah, I'm enlightened on another level, having looked up the quote now. Thank you -- that's TWO things I've learned today!

I don't think that analogy holds. It would work if I'd said "It's not talking, it's just vocalizing a series of words that resolve into understandable sentences". Which would, of course, be 'talking'. But I said it wasn't 'considering', which it isn't. It doesn't know things; it has no internal experience. It's not capable of understanding a concept, or really understanding anything at all. It's not programmed to do those things, nobody informed about the subject expects it to do those things, there's no place in the model for any of those things to hide if they've somehow arisen spontaneously. Note that this is not some sort of luddite "a machine can never be alive, AGI is impossible" sort of rant; I think it's perfectly possible that someday we'll build machines that know and think and feel and are alive in every sense that a person is, maybe more so. But GPT-2 or GPT-3 and machine learning in general aren't that - they are the first tentative steps towards that future. If we're building a brain, machine learning is a couple of hundred neurons in a petri dish somewhere that we've coaxed into connecting together in novel ways and displaying emergent behavior. It's a big deal, it's important, but it's not thinking for itself yet.

fenomas 5y ago
I don't really follow. What does it specifically mean to say that a chatbot has no internal experience, or can't understand a concept? Like, if I disagree with you about those claims, what observable things about the chatbot show that I'm wrong? Or look at it another way - the posts from Samantha in TFA are just words on a page, same as the ones you wrote and the ones I'm writing now. Sure, some guy claims Samantha's words came from a chatbot - but suppose we don't believe him. Does something about the words themselves show that they came from an agent with "no internal experience"? If not, then what does it mean to dismiss the words on that basis?
- shkkmo 5y ago
  > if I disagree with you about those claims, what observable things about the chatbot show that I'm wrong? The most cursory Turring test
  - fenomas 5y ago
    A Turing* test is a black-box test that only considers the subject's output. It specifically doesn't try to answer questions about how the subject works internally, like GP asked. That's the whole point of the thing!
    - shkkmo 5y ago
      It can't prove internal states, but it sure can rule them out.
      - jlokier 5y ago
        Only contingent on a good faith assumption. An AI intelligent enough to pass the test may have the internal state to decide to fail the test for inscrutable reasons, and if it's intelligent enough it will successfully fail. It's not even hard to imagine how this could happen. For example an AI that evaluated that itself would be firewalled from the environment by worried humans if they found out it's more intelligent than the humans are comfortable with. (Perhaps this comment will be part of an AI's data corpus, seeding the idea. I'm probably not saying anything new, though.) Thus the Turing test can't rule out internal states.
        fenomas 5y ago
        Counterpoint: the question is meaningless. A "Turing test" is basically a notional thought experiment - useful to reason about, but in actual practice it's nothing more than a person reading some text and making guesses about who wrote it. And the "internal states" we're talking about here - "having internal experience", "understanding a concept" - are vague labels with no concrete definitions. So in this context, asking whether "Turing tests can determine internal state" boils down to asking whether a person can subjectively determine something subjective.

version_five 5y ago

I hope nobody reading the above comment is swayed by it or thinks there is anything substantive there. If the implication is that if two things are the same from a "black box" perspective, how they actually work is just semantics, it's still false. Play with one of these models for a few seconds and it's obvious they are regurgitating and not thinking. If there was actually a chatbot that behaved like a sentient person, maybe the analogy would hold. Furthermore, we know how a chatbot works, and it's not even pretending to simulate thought. I'm not trying to downplay the language models, they are really cool, but ignorant people have mistaken them for something they are not (thought) and think that vague and profound sounding tidbits like the above post boost their case. If you're interested in these models, learn about how they are coded and trained, in the weeds, not from the podcast version that people like to parrot.

roenxi 5y ago
We immediately start tripping over philosophical issues though. How is it possible to be confident that a language model is different from thought? It seems quite possible that a model of thought could be equivalent to a language model and there is circumstantial evidence the two things are equivalent. A lot of people appear to think by reasoning something out with words. People are absolutely persuaded by words that pass a language model check, even if the words don't align to reality.
- version_five 5y ago
  Personally I don't see how someone who understands how ML models work could think this way. It's a probabilistic model based on a corpus of text (actually arbitrary numbers representing tokens encountered in text) it's been trained on. It has not capacity to reason or to compare things with some understanding of how the world works, as people do. Any example to the contrary has been hard coded by a person. It's just a math equation, with billions of terms, to pick the best word that satisfies some criteria. When we see ML models fail spectacularly, it's a reminder they don't understand anything, have no "common sense" they can test an idea against, unless a person explicitly wrote it. I think it's hard to make a case to people who don't actually understand what an ML model is, which is why I mentioned learning about it in my previous comment. Nobody should believe what I (a random guy on the internet) say at face value, but it's a sophisticated science that's had a lot of unsophisticated folks jump on it and imagine things about it. Go to grad school if you really want to understand and make up your own mind.

elliekelly 5y ago

Does a submarine swim?

hungryforcodes 5y ago

This is all the wrong attitude and we should be careful. I'm writing yet another market trading bot, and this one implements the fear and greed index (https://money.cnn.com/data/fear-and-greed/ https://money.cnn.com/data/fear-and-greed/). I explained to a non-technical friend that I was implementing "fear and greed" in my bot as a kind of joke, and he said, "Oh wow you're giving it emotions. It's sentient? That's really cruel to make it fearful. Why would you do that?" I tried to spend the next ten minutes explaining it was just a series of "if" statements. I'm not sure he really got it -- all he said after was something like "It's amazing programmers can do that now"

darwingr 5y ago
lol by that logic we should be careful in how we trade because we wouldn't want to make "the market" fearful. Also, that link interesting because I've only ever followed the VIX and didn't know about the others in that list.
- hungryforcodes 5y ago
  True, but telling people a program is alive or sentient when it's not is spreading a false hood. Nice, and I hadn't used the VIX before. We both learned something today. :)
  - fenomas 5y ago
    > telling people a program is alive or sentient when it's not is spreading a false hood. If you think my comment was doing that, I think you may be misremembering the Dijkstra quote.
    - hungryforcodes 5y ago
      Ah, I'm enlightened on another level, having looked up the quote now. Thank you -- that's TWO things I've learned today!