4 ms·

>it's just predicting what the next sentence >There is no plan behind this What is the difference between predicting the next sentence vs. having a plan. Perh

>it's just predicting what the next sentence >There is no plan behind this What is the difference between predicting the next sentence vs. having a plan. Perhaps the only real difference between us and GPT3 is the number of steps ahead it anticipates.

edgyquant 5y ago
The number of steps is exactly the difference between predicting the next sentence and having a plan.
- visarga 5y ago
  There's no reason a planning system like AlphaZero could not be used. It's just necessary to generate multiple rounds of conversation a few steps ahead and pick the most promising one. "just" being an exaggeration, of course.
bee_rider 5y ago
I believe that is the difference they are highlighting. Another way to look at it is, humans generally have some broad-picture idea of where they'd like a conversation to go (anticipating many steps ahead, but more vaguely the farther out you go). It seems like a big difference -- often the end gives the intermediary steps much more meaning.
9dev 5y ago
The real difference between GPT-3 and us is that we have an agenda when we converse, while the model only ever generates text. It's a piece of code, it's about as "intelligent" as the source code of the gnu utils, that is: not at all.
- visarga 5y ago
  The model assumes a persona from the training set, we just don't know which one unless we tell it upfront.
- IIAOPSW 5y ago
  Ok but what does that mean. What makes for having an agenda that isn't already encapsulated within "the things that are n steps ahead in your speech planning?". If intelligence works in a way that is fundamentally different from GPT-3, what specific thing can't be accomplished by simply increasing the steps planned ahead in the next GPT? Suppose we do more of the same to build GPT-4, and now it can stay focused for whole paragraphs at a time. Is it intelligent yet? How about when GPT-5 starts writing whole books. If the approach of GPT starts generating text that stays on topic long enough to pass the Turing test, is it time to accept that there is nothing deeper to human intelligence than a hidden Markov model? What if we're all deluded about how intricate and special human intelligence really is?
visarga 5y ago
I think one important difference is that GPT-3 is a generator without a discriminator, and decides the output in one pass. Humans have both and use multiple rounds to decide. Do you remember sometimes you think of something and then stop and rephrase or just abstain? That's the discriminator working in the background, stopping us from saying stupid things.

edgyquant 5y ago

The number of steps is exactly the difference between predicting the next sentence and having a plan.

visarga 5y ago
There's no reason a planning system like AlphaZero could not be used. It's just necessary to generate multiple rounds of conversation a few steps ahead and pick the most promising one. "just" being an exaggeration, of course.

bee_rider 5y ago

I believe that is the difference they are highlighting. Another way to look at it is, humans generally have some broad-picture idea of where they'd like a conversation to go (anticipating many steps ahead, but more vaguely the farther out you go). It seems like a big difference -- often the end gives the intermediary steps much more meaning.

9dev 5y ago

The real difference between GPT-3 and us is that we have an agenda when we converse, while the model only ever generates text. It's a piece of code, it's about as "intelligent" as the source code of the gnu utils, that is: not at all.

visarga 5y ago
The model assumes a persona from the training set, we just don't know which one unless we tell it upfront.
IIAOPSW 5y ago
Ok but what does that mean. What makes for having an agenda that isn't already encapsulated within "the things that are n steps ahead in your speech planning?". If intelligence works in a way that is fundamentally different from GPT-3, what specific thing can't be accomplished by simply increasing the steps planned ahead in the next GPT? Suppose we do more of the same to build GPT-4, and now it can stay focused for whole paragraphs at a time. Is it intelligent yet? How about when GPT-5 starts writing whole books. If the approach of GPT starts generating text that stays on topic long enough to pass the Turing test, is it time to accept that there is nothing deeper to human intelligence than a hidden Markov model? What if we're all deluded about how intricate and special human intelligence really is?

visarga 5y ago

I think one important difference is that GPT-3 is a generator without a discriminator, and decides the output in one pass. Humans have both and use multiple rounds to decide. Do you remember sometimes you think of something and then stop and rephrase or just abstain? That's the discriminator working in the background, stopping us from saying stupid things.