4 ms·

Personally I don't see how someone who understands how ML models work could think this way. It's a probabilistic model based on a corpus of text (actually arbit

Personally I don't see how someone who understands how ML models work could think this way. It's a probabilistic model based on a corpus of text (actually arbitrary numbers representing tokens encountered in text) it's been trained on. It has not capacity to reason or to compare things with some understanding of how the world works, as people do. Any example to the contrary has been hard coded by a person. It's just a math equation, with billions of terms, to pick the best word that satisfies some criteria. When we see ML models fail spectacularly, it's a reminder they don't understand anything, have no "common sense" they can test an idea against, unless a person explicitly wrote it. I think it's hard to make a case to people who don't actually understand what an ML model is, which is why I mentioned learning about it in my previous comment. Nobody should believe what I (a random guy on the internet) say at face value, but it's a sophisticated science that's had a lot of unsophisticated folks jump on it and imagine things about it. Go to grad school if you really want to understand and make up your own mind.

roenxi 5y ago
> Personally I don't see how someone who understands how ML models work could think this way. Then you've misunderstood the problem; because we all know how they work. What we don't know is how human reason works. We know from how lawyers and courts work that it is possible to reason about things purely using language from a corpus of text with no appreciation for how the details actually function.
- snowwrestler 5y ago
  A ML system reflects back the input it is given. If that input is human language, a very good ML system will produce output that looks like human language. It is conceptually like a mirror. A good mirror, if you put a human in front of it, will produce output that looks exactly like a human. But we know it is not a human. Not because of some flaw in its representation, but because we understand how mirrors work.
  - roenxi 5y ago
    But humans are also conceptually mirrors. For example, none of the words you are using are ones you thought up; they mostly date back to pre-Shakespearian times. Nor is the grammar you're using, which you've copied off other people. And the arguments you are using have concepts like intelligence or the mathematics of neural nets which aren't things that you came up with either; they are being mirrored off others. And yet we'd all agree you are thinking, despite just being a mirror with very little room to add anything new. So if you want to argue that a neural net is a mirror, that does not in any way disqualify it from considering something. The part of human reasoning that isn't a mirror of other ideas is small to the point where it might not actually exist.
    - verygoodname 5y ago
      Humans may even "be conceptually mirrors" (whatever that may mean): this still doesn't make mirrors be humans (conceptually or otherwise). Current language models are much more closer to "Dissociated Press" (i.e. an equation that tells you which which words are more likely, given the context) than to "human thought", and the fact that humans learn by copying does not really change this.
      - vokep 5y ago
        But it does mean mirrors might be humans/human-like, if humans are actually reducable to (self-aware) mirrors.
        verygoodname 5y ago
        Would you say that a "Markov chain"-type (e.g. Dissociated Press) language model is "self-aware" in any way? If yes, then... how, exactly? It is basically an N x N matrix of values. Current language models (GPT et al.), are qualitatively nothing fancier than this: probabilistic models which encode regularity in language and from which you can sample to get "plausible content". If a bunch of values in a matrix is "self-aware", then I guess GPT can be seen as "self-aware"; if not, then it can't. My problem is trying to imagine an N x N matrix being self-aware (like... what does that even mean in this context?). Is GPT human-like? Sure... if you stretch the meaning of "human-like" enough (it produces content that is similar to content produced by a human). Is a human GPT-like? That's harder to argue (and I don't see how your argument would support it).

roenxi 5y ago

> Personally I don't see how someone who understands how ML models work could think this way. Then you've misunderstood the problem; because we all know how they work. What we don't know is how human reason works. We know from how lawyers and courts work that it is possible to reason about things purely using language from a corpus of text with no appreciation for how the details actually function.

snowwrestler 5y ago
A ML system reflects back the input it is given. If that input is human language, a very good ML system will produce output that looks like human language. It is conceptually like a mirror. A good mirror, if you put a human in front of it, will produce output that looks exactly like a human. But we know it is not a human. Not because of some flaw in its representation, but because we understand how mirrors work.
- roenxi 5y ago
  But humans are also conceptually mirrors. For example, none of the words you are using are ones you thought up; they mostly date back to pre-Shakespearian times. Nor is the grammar you're using, which you've copied off other people. And the arguments you are using have concepts like intelligence or the mathematics of neural nets which aren't things that you came up with either; they are being mirrored off others. And yet we'd all agree you are thinking, despite just being a mirror with very little room to add anything new. So if you want to argue that a neural net is a mirror, that does not in any way disqualify it from considering something. The part of human reasoning that isn't a mirror of other ideas is small to the point where it might not actually exist.
  - verygoodname 5y ago
    Humans may even "be conceptually mirrors" (whatever that may mean): this still doesn't make mirrors be humans (conceptually or otherwise). Current language models are much more closer to "Dissociated Press" (i.e. an equation that tells you which which words are more likely, given the context) than to "human thought", and the fact that humans learn by copying does not really change this.
    - vokep 5y ago
      But it does mean mirrors might be humans/human-like, if humans are actually reducable to (self-aware) mirrors.
      - verygoodname 5y ago
        Would you say that a "Markov chain"-type (e.g. Dissociated Press) language model is "self-aware" in any way? If yes, then... how, exactly? It is basically an N x N matrix of values. Current language models (GPT et al.), are qualitatively nothing fancier than this: probabilistic models which encode regularity in language and from which you can sample to get "plausible content". If a bunch of values in a matrix is "self-aware", then I guess GPT can be seen as "self-aware"; if not, then it can't. My problem is trying to imagine an N x N matrix being self-aware (like... what does that even mean in this context?). Is GPT human-like? Sure... if you stretch the meaning of "human-like" enough (it produces content that is similar to content produced by a human). Is a human GPT-like? That's harder to argue (and I don't see how your argument would support it).