4 ms·

I hope nobody reading the above comment is swayed by it or thinks there is anything substantive there. If the implication is that if two things are the same fro

I hope nobody reading the above comment is swayed by it or thinks there is anything substantive there. If the implication is that if two things are the same from a "black box" perspective, how they actually work is just semantics, it's still false. Play with one of these models for a few seconds and it's obvious they are regurgitating and not thinking. If there was actually a chatbot that behaved like a sentient person, maybe the analogy would hold. Furthermore, we know how a chatbot works, and it's not even pretending to simulate thought. I'm not trying to downplay the language models, they are really cool, but ignorant people have mistaken them for something they are not (thought) and think that vague and profound sounding tidbits like the above post boost their case. If you're interested in these models, learn about how they are coded and trained, in the weeds, not from the podcast version that people like to parrot.

roenxi 5y ago
We immediately start tripping over philosophical issues though. How is it possible to be confident that a language model is different from thought? It seems quite possible that a model of thought could be equivalent to a language model and there is circumstantial evidence the two things are equivalent. A lot of people appear to think by reasoning something out with words. People are absolutely persuaded by words that pass a language model check, even if the words don't align to reality.
- version_five 5y ago
  Personally I don't see how someone who understands how ML models work could think this way. It's a probabilistic model based on a corpus of text (actually arbitrary numbers representing tokens encountered in text) it's been trained on. It has not capacity to reason or to compare things with some understanding of how the world works, as people do. Any example to the contrary has been hard coded by a person. It's just a math equation, with billions of terms, to pick the best word that satisfies some criteria. When we see ML models fail spectacularly, it's a reminder they don't understand anything, have no "common sense" they can test an idea against, unless a person explicitly wrote it. I think it's hard to make a case to people who don't actually understand what an ML model is, which is why I mentioned learning about it in my previous comment. Nobody should believe what I (a random guy on the internet) say at face value, but it's a sophisticated science that's had a lot of unsophisticated folks jump on it and imagine things about it. Go to grad school if you really want to understand and make up your own mind.
  - roenxi 5y ago
    > Personally I don't see how someone who understands how ML models work could think this way. Then you've misunderstood the problem; because we all know how they work. What we don't know is how human reason works. We know from how lawyers and courts work that it is possible to reason about things purely using language from a corpus of text with no appreciation for how the details actually function.
    - snowwrestler 5y ago
      A ML system reflects back the input it is given. If that input is human language, a very good ML system will produce output that looks like human language. It is conceptually like a mirror. A good mirror, if you put a human in front of it, will produce output that looks exactly like a human. But we know it is not a human. Not because of some flaw in its representation, but because we understand how mirrors work.
      - roenxi 5y ago
        But humans are also conceptually mirrors. For example, none of the words you are using are ones you thought up; they mostly date back to pre-Shakespearian times. Nor is the grammar you're using, which you've copied off other people. And the arguments you are using have concepts like intelligence or the mathematics of neural nets which aren't things that you came up with either; they are being mirrored off others. And yet we'd all agree you are thinking, despite just being a mirror with very little room to add anything new. So if you want to argue that a neural net is a mirror, that does not in any way disqualify it from considering something. The part of human reasoning that isn't a mirror of other ideas is small to the point where it might not actually exist.
        verygoodname 5y ago
        Humans may even "be conceptually mirrors" (whatever that may mean): this still doesn't make mirrors be humans (conceptually or otherwise). Current language models are much more closer to "Dissociated Press" (i.e. an equation that tells you which which words are more likely, given the context) than to "human thought", and the fact that humans learn by copying does not really change this.
wly_cdgr 5y ago
A lot of people have also mistaken humans writing plausible-sounding paragraphs for thought
fenomas 5y ago
This seems like a very simplistic position on a rather subtle topic. I don't know of any good reasons to doubt that an arbitrarily large chatbot could pass an arbitrarily hard Turing test. If that's so, where is the bright line between Samantha and such a chatbot? The fact that current chatbots aren't very convincing shows that there's still a long way to go, but it doesn't demonstrate any categorical difference between chatbots and creatures that think - or that chatbots are categorically not "simulating thought" (whatever that means). I mean, I'm certainly not here to argue that Samantha is definitely "thinking", for any definition of that term. But you seem to be suggesting that anyone who considers the idea must be ignorant or not understand how ML models are trained, and I don't follow that at all.

roenxi 5y ago

We immediately start tripping over philosophical issues though. How is it possible to be confident that a language model is different from thought? It seems quite possible that a model of thought could be equivalent to a language model and there is circumstantial evidence the two things are equivalent. A lot of people appear to think by reasoning something out with words. People are absolutely persuaded by words that pass a language model check, even if the words don't align to reality.

version_five 5y ago
Personally I don't see how someone who understands how ML models work could think this way. It's a probabilistic model based on a corpus of text (actually arbitrary numbers representing tokens encountered in text) it's been trained on. It has not capacity to reason or to compare things with some understanding of how the world works, as people do. Any example to the contrary has been hard coded by a person. It's just a math equation, with billions of terms, to pick the best word that satisfies some criteria. When we see ML models fail spectacularly, it's a reminder they don't understand anything, have no "common sense" they can test an idea against, unless a person explicitly wrote it. I think it's hard to make a case to people who don't actually understand what an ML model is, which is why I mentioned learning about it in my previous comment. Nobody should believe what I (a random guy on the internet) say at face value, but it's a sophisticated science that's had a lot of unsophisticated folks jump on it and imagine things about it. Go to grad school if you really want to understand and make up your own mind.
- roenxi 5y ago
  > Personally I don't see how someone who understands how ML models work could think this way. Then you've misunderstood the problem; because we all know how they work. What we don't know is how human reason works. We know from how lawyers and courts work that it is possible to reason about things purely using language from a corpus of text with no appreciation for how the details actually function.
  - snowwrestler 5y ago
    A ML system reflects back the input it is given. If that input is human language, a very good ML system will produce output that looks like human language. It is conceptually like a mirror. A good mirror, if you put a human in front of it, will produce output that looks exactly like a human. But we know it is not a human. Not because of some flaw in its representation, but because we understand how mirrors work.
    - roenxi 5y ago
      But humans are also conceptually mirrors. For example, none of the words you are using are ones you thought up; they mostly date back to pre-Shakespearian times. Nor is the grammar you're using, which you've copied off other people. And the arguments you are using have concepts like intelligence or the mathematics of neural nets which aren't things that you came up with either; they are being mirrored off others. And yet we'd all agree you are thinking, despite just being a mirror with very little room to add anything new. So if you want to argue that a neural net is a mirror, that does not in any way disqualify it from considering something. The part of human reasoning that isn't a mirror of other ideas is small to the point where it might not actually exist.
      - verygoodname 5y ago
        Humans may even "be conceptually mirrors" (whatever that may mean): this still doesn't make mirrors be humans (conceptually or otherwise). Current language models are much more closer to "Dissociated Press" (i.e. an equation that tells you which which words are more likely, given the context) than to "human thought", and the fact that humans learn by copying does not really change this.

wly_cdgr 5y ago

A lot of people have also mistaken humans writing plausible-sounding paragraphs for thought

fenomas 5y ago

This seems like a very simplistic position on a rather subtle topic. I don't know of any good reasons to doubt that an arbitrarily large chatbot could pass an arbitrarily hard Turing test. If that's so, where is the bright line between Samantha and such a chatbot? The fact that current chatbots aren't very convincing shows that there's still a long way to go, but it doesn't demonstrate any categorical difference between chatbots and creatures that think - or that chatbots are categorically not "simulating thought" (whatever that means). I mean, I'm certainly not here to argue that Samantha is definitely "thinking", for any definition of that term. But you seem to be suggesting that anyone who considers the idea must be ignorant or not understand how ML models are trained, and I don't follow that at all.