6 ms·

> knowledge workers Aka Thinkers. I don't think the author considered the full extent of automating thinking. They are underestimating what it is. This technol

by seydor 4y ago

> knowledge workers Aka Thinkers. I don't think the author considered the full extent of automating thinking. They are underestimating what it is. This technology can only get better and is probably already superhuman. This time it is different indeed and makes lot of knowledge work not just be obsolete, but also inferior.

AnimalMuppet 4y ago
It's probably already superhuman? That's... one interpretation of GPT. I interpret it quite differently. Unless you're going to say that computers were already superhuman? They could do arithmetic far faster than any human ever could. Also pure logic. Chess and go, even. Were they superhuman then? Because as I look at GPT, I don't see superhuman. I see a babbling idiot that babbles in correctly-constructed English sentences, but has no idea what it's talking about, and that manages to be factually correct... maybe 70% of the time? (That's hard to measure, because the universe of possible discourse is really large.) And, worst of all, GPT has no idea of when it doesn't know something, but will irresponsibly blurt out... something. Something random that its training set made it think is the least-implausible thing to say right there. You know, if they could just fix that - if they could add some kind of weight so that it knows when it's getting into an area that it doesn't have adequate training data for, and program it to express uncertainty rather than certainty in something wrong - that one change would make it, still not superhuman, but at least much more useful. And, if we could do that, maybe we could do one more step. If it has more than one strong possibility, but they are contradictory, then there's contradictory data in it's training set, and it's likely dealing with something controversial rather than certain. I know, I know. Everything is easy to the one who doesn't have to do it. It's probably much harder than I said. But I think that's what GPT needs going forward.
- deleted 4y ago
  [deleted]
- ctoth 4y ago
  > and that manages to be factually correct... maybe 70% of the time? Do you update on this when considering that GPT-3 got in the bottom 10% in several exams, whereas GPT-4 is now in the top 10%? Do you not see this clear improvement as evidence that it's just a matter of time?
  - AnimalMuppet 4y ago
    Clear evidence of improvement? Sure. Evidence that it's just a matter of time? Time will tell. Evidence that it's already superhuman (the original claim)? Very much no. Only in the top 10% isn't superhuman.
    - ctoth 4y ago
      Take a look at the GPT-4 paper, and the list of exams and accompanying scores. Then tell me if you know any humans that did similarly? Looks superhuman to me. Especially considering it took all of these in less than a year.
- thomastjeffery 4y ago
  > Something random that its training set made it think That's the mistake right there. It didn't think. The text did. Specifically, humans encoded thought into language, and encoded language into text. GPT modeled the patterns from that text, and used those patterns to restructure it. Because language is the most dominant pattern, blindly following an inference model is very likely to result in correct language transformations. Because people don't write nonsense, the result of a correct language transformation is very likely to make sense.
  - AnimalMuppet 4y ago
    You missed the point there. When the information isn't in the training corpus, then GPT still generates... something. It's not in the text, but GPT's weights show something as the best correlation. Sometimes it's mining connections that the authors didn't even realize were there. And sometimes, it's following random correlations that don't mean anything, and as a result it's generating nonsense.
    - thomastjeffery 4y ago
      It is in the text. It's not the same shape, location, or order because GPT followed a complex pattern to "generate" the continuation. None of the tokens are new. > Sometimes it's mining connections that the authors didn't even realize were there. Exactly. GPT is not limited to modeling patterns that align to language. It models every pattern it can find. Text contains so much more than language: it contains powerful implicit patterns of behavior; and GPT proves that by modeling them.
      - meh8881 4y ago
        I asked it to create a portmanteau insult for GPT doubter on hn and got GLTard or GPTSlother when asking for an animal one. I suspect you are not right. It also advised against name calling Edit: wrong in regards to no new tokens
      - AnimalMuppet 4y ago
        But it will also confidently tell me who won the 2023 NCAA tournament. It did not find that in "implicit patterns of behavior". It made it up from patterns in the text, but the answer was not there. At best what was in the text was projections, opinions, guesses. GPT takes all that in, but it's not actual information to answer the question, because there is no answer - not yet. So it finds a pattern, but a pattern that is not the answer. But it gives it as the answer, with full confidence.
thomastjeffery 4y ago
On the contrary, I think you are overestimating it; and I don't blame you for doing so. The overwhelming majority of narratives surrounding LLMs have shared the same character flaw: personification. Almost no one writes explicitly about what an LLM itself does. An LLM models patterns from text. That's it. Not a single thing more. Personification is a great tool for explaining what you already know. The mistake is to draw conclusions about what you don't know from the personified character instead of the thing itself. Yet, that is exactly what is being done for LLMs. The first mistake is in the name: "Language Learning Model". That's the intended purpose, yes, but it's a misleading description for what the tool actually does. I propose we instead call them, "Text Learning Models". An LLM models all of the patterns in can find in the text it is given. It doesn't categorize those patterns, or have any notion of their importance to humans. An LLM doesn't do a single explicit thing. Everything an LLM does is implicit. An LLM models the patterns in the text, and exhibits them. The patterns in the text do all the exciting stuff. Everything interesting about LLMs is human generated. Without human authored text, the model is worthless. Without human authored prompts, the model is a black box. Every interesting thing an LLM does, one or more humans told it to do.
- seydor 4y ago
  That's very misleading. The current large LLMs learn tons and tons of "reference maps" that map how words (tokens) are referring (attending) to other tokens within a certain context. In the process they learn a map that may be the equivalent of what we humans call "syntax", and they apply it to completely different contexts and inputs. It's not memorization, nor simple pattern matching, but something that allows them to be creative
  - thomastjeffery 4y ago
    The LLM does not know what a word or a context are. It doesn't have that kind of categorization in its model or anywhere else. Tokens don't even match word boundaries: they are intentionally misaligned to better accommodate contractions. > In the process they learn a map that may be the equivalent of what we humans call "syntax", and they apply it to completely different contexts and inputs. Exactly. Alongside those patterns, they model all of the other patterns that are present in the text. There is no categorization between these patterns: the word "syntax" has no bearing on the model whatsoever, except where it literally exists in the text being modeled. LLMs do not parse. They tokenize, but not to match any predefined grammar. It's not the LLM itself "being creative". It's the content of the text: the patterns that are modeled. Those patterns exist independent of the LLM. They come from the complex human behavior that is, "writing".

by seydor 4y ago

AnimalMuppet 4y ago

It's probably already superhuman? That's... one interpretation of GPT. I interpret it quite differently. Unless you're going to say that computers were already superhuman? They could do arithmetic far faster than any human ever could. Also pure logic. Chess and go, even. Were they superhuman then? Because as I look at GPT, I don't see superhuman. I see a babbling idiot that babbles in correctly-constructed English sentences, but has no idea what it's talking about, and that manages to be factually correct... maybe 70% of the time? (That's hard to measure, because the universe of possible discourse is really large.) And, worst of all, GPT has no idea of when it doesn't know something, but will irresponsibly blurt out... something. Something random that its training set made it think is the least-implausible thing to say right there. You know, if they could just fix that - if they could add some kind of weight so that it knows when it's getting into an area that it doesn't have adequate training data for, and program it to express uncertainty rather than certainty in something wrong - that one change would make it, still not superhuman, but at least much more useful. And, if we could do that, maybe we could do one more step. If it has more than one strong possibility, but they are contradictory, then there's contradictory data in it's training set, and it's likely dealing with something controversial rather than certain. I know, I know. Everything is easy to the one who doesn't have to do it. It's probably much harder than I said. But I think that's what GPT needs going forward.

deleted 4y ago
[deleted]
ctoth 4y ago
> and that manages to be factually correct... maybe 70% of the time? Do you update on this when considering that GPT-3 got in the bottom 10% in several exams, whereas GPT-4 is now in the top 10%? Do you not see this clear improvement as evidence that it's just a matter of time?
- AnimalMuppet 4y ago
  Clear evidence of improvement? Sure. Evidence that it's just a matter of time? Time will tell. Evidence that it's already superhuman (the original claim)? Very much no. Only in the top 10% isn't superhuman.
  - ctoth 4y ago
    Take a look at the GPT-4 paper, and the list of exams and accompanying scores. Then tell me if you know any humans that did similarly? Looks superhuman to me. Especially considering it took all of these in less than a year.
thomastjeffery 4y ago
> Something random that its training set made it think That's the mistake right there. It didn't think. The text did. Specifically, humans encoded thought into language, and encoded language into text. GPT modeled the patterns from that text, and used those patterns to restructure it. Because language is the most dominant pattern, blindly following an inference model is very likely to result in correct language transformations. Because people don't write nonsense, the result of a correct language transformation is very likely to make sense.
- AnimalMuppet 4y ago
  You missed the point there. When the information isn't in the training corpus, then GPT still generates... something. It's not in the text, but GPT's weights show something as the best correlation. Sometimes it's mining connections that the authors didn't even realize were there. And sometimes, it's following random correlations that don't mean anything, and as a result it's generating nonsense.
  - thomastjeffery 4y ago
    It is in the text. It's not the same shape, location, or order because GPT followed a complex pattern to "generate" the continuation. None of the tokens are new. > Sometimes it's mining connections that the authors didn't even realize were there. Exactly. GPT is not limited to modeling patterns that align to language. It models every pattern it can find. Text contains so much more than language: it contains powerful implicit patterns of behavior; and GPT proves that by modeling them.
    - meh8881 4y ago
      I asked it to create a portmanteau insult for GPT doubter on hn and got GLTard or GPTSlother when asking for an animal one. I suspect you are not right. It also advised against name calling Edit: wrong in regards to no new tokens
    - AnimalMuppet 4y ago
      But it will also confidently tell me who won the 2023 NCAA tournament. It did not find that in "implicit patterns of behavior". It made it up from patterns in the text, but the answer was not there. At best what was in the text was projections, opinions, guesses. GPT takes all that in, but it's not actual information to answer the question, because there is no answer - not yet. So it finds a pattern, but a pattern that is not the answer. But it gives it as the answer, with full confidence.

thomastjeffery 4y ago

On the contrary, I think you are overestimating it; and I don't blame you for doing so. The overwhelming majority of narratives surrounding LLMs have shared the same character flaw: personification. Almost no one writes explicitly about what an LLM itself does. An LLM models patterns from text. That's it. Not a single thing more. Personification is a great tool for explaining what you already know. The mistake is to draw conclusions about what you don't know from the personified character instead of the thing itself. Yet, that is exactly what is being done for LLMs. The first mistake is in the name: "Language Learning Model". That's the intended purpose, yes, but it's a misleading description for what the tool actually does. I propose we instead call them, "Text Learning Models". An LLM models all of the patterns in can find in the text it is given. It doesn't categorize those patterns, or have any notion of their importance to humans. An LLM doesn't do a single explicit thing. Everything an LLM does is implicit. An LLM models the patterns in the text, and exhibits them. The patterns in the text do all the exciting stuff. Everything interesting about LLMs is human generated. Without human authored text, the model is worthless. Without human authored prompts, the model is a black box. Every interesting thing an LLM does, one or more humans told it to do.

seydor 4y ago
That's very misleading. The current large LLMs learn tons and tons of "reference maps" that map how words (tokens) are referring (attending) to other tokens within a certain context. In the process they learn a map that may be the equivalent of what we humans call "syntax", and they apply it to completely different contexts and inputs. It's not memorization, nor simple pattern matching, but something that allows them to be creative
- thomastjeffery 4y ago
  The LLM does not know what a word or a context are. It doesn't have that kind of categorization in its model or anywhere else. Tokens don't even match word boundaries: they are intentionally misaligned to better accommodate contractions. > In the process they learn a map that may be the equivalent of what we humans call "syntax", and they apply it to completely different contexts and inputs. Exactly. Alongside those patterns, they model all of the other patterns that are present in the text. There is no categorization between these patterns: the word "syntax" has no bearing on the model whatsoever, except where it literally exists in the text being modeled. LLMs do not parse. They tokenize, but not to match any predefined grammar. It's not the LLM itself "being creative". It's the content of the text: the patterns that are modeled. Those patterns exist independent of the LLM. They come from the complex human behavior that is, "writing".