4 ms·

It is in the text. It's not the same shape, location, or order because GPT followed a complex pattern to "generate" the continuation. None of the tokens are new

It is in the text. It's not the same shape, location, or order because GPT followed a complex pattern to "generate" the continuation. None of the tokens are new. > Sometimes it's mining connections that the authors didn't even realize were there. Exactly. GPT is not limited to modeling patterns that align to language. It models every pattern it can find. Text contains so much more than language: it contains powerful implicit patterns of behavior; and GPT proves that by modeling them.

meh8881 4y ago
I asked it to create a portmanteau insult for GPT doubter on hn and got GLTard or GPTSlother when asking for an animal one. I suspect you are not right. It also advised against name calling Edit: wrong in regards to no new tokens
AnimalMuppet 4y ago
But it will also confidently tell me who won the 2023 NCAA tournament. It did not find that in "implicit patterns of behavior". It made it up from patterns in the text, but the answer was not there. At best what was in the text was projections, opinions, guesses. GPT takes all that in, but it's not actual information to answer the question, because there is no answer - not yet. So it finds a pattern, but a pattern that is not the answer. But it gives it as the answer, with full confidence.
- thomastjeffery 4y ago
  > It did not find that in "implicit patterns of behavior". It made it up from patterns in the text There is literally no difference between the two. > At best what was in the text was projections, opinions, guesses. And a pattern of response to the structure of your direct question,. It didn't subjectively decide what the right answer is: it followed the pattern closest to your prompt, and printed the tokens that pattern was made of.
  - AnimalMuppet 4y ago
    I think that you are agreeing with the main point I'm trying to make, and you're even going further than me. You're just disagreeing with my wording. What you're describing is, it doesn't think, it doesn't understand, it just follows whatever patterns in its training corpus kind of correlate with patterns in the prompt. And I agree. Where I may differ is how to regard the training corpus. True, it contains a massive amount of distilled thought, encoded in words. But it is also true that there are subjects on which the training corpus contains no distilled thought - and GPT will respond to questions on those subjects as well, finding some spurious correlation to respond with.
    - thomastjeffery 4y ago
      > I think that you are agreeing with the main point I'm trying to make, and you're even going further than me. Pretty much. > You're just disagreeing with my wording. Yeah, that's the "going even further" part. Words are important. The importance of words is really foundational for a lot of what we are talking about. > What you're describing is, it doesn't think, it doesn't understand, it just follows whatever patterns in its training corpus kind of correlate with patterns in the prompt. And I agree. Cool. It's nice to be on the same page. Let's read the next one: > Where I may differ is how to regard the training corpus. True, it contains a massive amount of distilled thought, encoded in words. That's the biggest point I'm trying to make by "disagreeing with your wording". Words have significant implications. In this case, the implication is "problem domain": language vs. text. Language is a subdomain of written text. Written text is a subdomain of all possible text. GPT is dealing with written text. GPT does not only model the language patterns that humans intentionally put into text. GPT models all of the patterns that exist in the text. Humans, when writing, do "distill" thought into language; but the result is messy: we end up encoding more patterns than we intended to. Our behavior, our choices, our reasons, and the surrounding situation we exist in, all inform our writing. Those are the patterns that get encoded into the text, right alongside language. GPT doesn't see any pattern as different from another; so it's misleading to say that it models "language". GPT doesn't categorize that specific set of patterns: it models all patterns equally. We humans have a good idea of what to expect from only language patterns. If that were all that was modeled, GPT's output wouldn't be nearly as interesting to us. GPT's output is interesting to us. If we look objectively at the model it generates, we can see why: there are patterns there we had no intention of writing; and GPT modeled them just like it modeled the language-specific patterns we expected it to.
- meh8881 4y ago
  > As an artificial intelligence, I do not have the ability to predict future events or access real-time information. My training data only goes up until 2021, so I don't have any information on the 2023 NCAA championship or any future events. I suggest checking with reliable news sources or following the NCAA updates for the latest information. Not for me

meh8881 4y ago

I asked it to create a portmanteau insult for GPT doubter on hn and got GLTard or GPTSlother when asking for an animal one. I suspect you are not right. It also advised against name calling Edit: wrong in regards to no new tokens

AnimalMuppet 4y ago

But it will also confidently tell me who won the 2023 NCAA tournament. It did not find that in "implicit patterns of behavior". It made it up from patterns in the text, but the answer was not there. At best what was in the text was projections, opinions, guesses. GPT takes all that in, but it's not actual information to answer the question, because there is no answer - not yet. So it finds a pattern, but a pattern that is not the answer. But it gives it as the answer, with full confidence.

thomastjeffery 4y ago
> It did not find that in "implicit patterns of behavior". It made it up from patterns in the text There is literally no difference between the two. > At best what was in the text was projections, opinions, guesses. And a pattern of response to the structure of your direct question,. It didn't subjectively decide what the right answer is: it followed the pattern closest to your prompt, and printed the tokens that pattern was made of.
- AnimalMuppet 4y ago
  I think that you are agreeing with the main point I'm trying to make, and you're even going further than me. You're just disagreeing with my wording. What you're describing is, it doesn't think, it doesn't understand, it just follows whatever patterns in its training corpus kind of correlate with patterns in the prompt. And I agree. Where I may differ is how to regard the training corpus. True, it contains a massive amount of distilled thought, encoded in words. But it is also true that there are subjects on which the training corpus contains no distilled thought - and GPT will respond to questions on those subjects as well, finding some spurious correlation to respond with.
  - thomastjeffery 4y ago
    > I think that you are agreeing with the main point I'm trying to make, and you're even going further than me. Pretty much. > You're just disagreeing with my wording. Yeah, that's the "going even further" part. Words are important. The importance of words is really foundational for a lot of what we are talking about. > What you're describing is, it doesn't think, it doesn't understand, it just follows whatever patterns in its training corpus kind of correlate with patterns in the prompt. And I agree. Cool. It's nice to be on the same page. Let's read the next one: > Where I may differ is how to regard the training corpus. True, it contains a massive amount of distilled thought, encoded in words. That's the biggest point I'm trying to make by "disagreeing with your wording". Words have significant implications. In this case, the implication is "problem domain": language vs. text. Language is a subdomain of written text. Written text is a subdomain of all possible text. GPT is dealing with written text. GPT does not only model the language patterns that humans intentionally put into text. GPT models all of the patterns that exist in the text. Humans, when writing, do "distill" thought into language; but the result is messy: we end up encoding more patterns than we intended to. Our behavior, our choices, our reasons, and the surrounding situation we exist in, all inform our writing. Those are the patterns that get encoded into the text, right alongside language. GPT doesn't see any pattern as different from another; so it's misleading to say that it models "language". GPT doesn't categorize that specific set of patterns: it models all patterns equally. We humans have a good idea of what to expect from only language patterns. If that were all that was modeled, GPT's output wouldn't be nearly as interesting to us. GPT's output is interesting to us. If we look objectively at the model it generates, we can see why: there are patterns there we had no intention of writing; and GPT modeled them just like it modeled the language-specific patterns we expected it to.
meh8881 4y ago
> As an artificial intelligence, I do not have the ability to predict future events or access real-time information. My training data only goes up until 2021, so I don't have any information on the 2023 NCAA championship or any future events. I suggest checking with reliable news sources or following the NCAA updates for the latest information. Not for me