3 ms·

The funny thing is that it is neither lying, nor inventing something new. What OpenAI did pretty well was collect data. And wouldn't you know it, the folks who

The funny thing is that it is neither lying, nor inventing something new. What OpenAI did pretty well was collect data. And wouldn't you know it, the folks who developed that new puzzle describe it as what it is---a new kind of puzzle. So now in the training data you have a combination of puzzle, sudoku, and new/novel. And wouldn't you know it, by asking for a new puzzle, based on sudoku, you make ChatGPT dig for that kind of text. If ChatGPT really had a novel idea, I would not expect it to be this coherent---after all, logic and coherence are not a constrain on how language models work, just what words are likely to occur next. That is why it is being compared to entry level college writing, because that is how an excited student writes who hops from topic to topic.

djmips 4y ago
But how is it different from humans? I can't tell you how many times now that I've come up with what I thought was a really cool idea but upon web searching found it was already invented/discovered etc. In fact before the Internet I had come up with my own algorithms and only upon the Internet existing did I find they were already discovered years earlier. There's no way that I was regurgitating something I had read in that case.
- travisjungroth 4y ago
  There’s a difference between coming up with a puzzle then finding out it already exists versus finding a puzzle and saying you came up with it. If I told you “We need a brand new, never-never-before-seen puzzle for our next game release.” and you searched Google for “brand new, never-before-seen puzzle”, found a puzzle game with those words in its marketing copy and pitched it to me, that would be some combination of unintelligent and dishonest behavior. Like, surprisingly so. It’s different from forgetting some puzzle you played with as a little kid and thinking you made it up, or creating a puzzle you’d never seen but has been made before.
  - deleted 4y ago
    [deleted]
  - not2b 4y ago
    But ChatGPT is not a person, it is a text generator. By asking it to generate a new puzzle, you are prompting it to find text in its training data showing someone describing a new puzzle, and it is going to speak in their voice. It's going to emit sentences that were influenced by what the puzzle developer originally wrote, and that person correctly said that it was new.
    - urbandw311er 4y ago
      This. people who can ground themselves in what ChatGPT is (an auto completion text predictor) are able to best understand the origins of its output.
    - int_19h 4y ago
      You're not prompting it to "find text". Comparing the size of the model to the size of the training data is sufficient to conclusively establish that it's an impossibility. We train it to predict the next word based on the training data, that is true. But we still have no idea what kind of internal structures said training actually produces inside of neural net. It sure as hell isn't just a "stochastic parrot", though, which is rather obvious if you ever tried giving it a complicated multi-step task and solve it while "thinking out loud".
    - travisjungroth 4y ago
      I agree with everything you’ve written here, so I’m not sure what the “But” that’s starts your comment is contrasting. I was answering the question “But how is this different from a person?”. Being asked for something new and finding something that already exists with the word “new” in front of it isn’t normal human behavior. That’s how it’s different from a person. Zooming out a bit, I think there’s some confusion in this whole chain. There’s a common topic about ChatGPT you could call Question of Creativity. If you ask for a new poem, it just smashes together its patterns around poems. You can debate if this is creativity, and if not, how are humans different. A few comments up, someone brought in a different idea you could call New Matching. If you ask for a new poem it will just grab you a poem that had the words “new poem” in front of it. New Matching is a different idea than Question of Creativity. The person I replied to seemed to be mistaking one idea for the other.
- juujian 4y ago
  It is different to what you do. If I tell you that this is already a thing, you might go back to the drawing board, and do something from scratch. Maybe do some abstract drawing with numbers for brainstorming. A language model is not able to do this, the starting point for a language model is always the training data. That is why there is so many instances where you see some wrong (or correct) response from ChatGPT and when the other person corrects this, the model just agrees to whatever the user says. That is the right thing to do according to language etiquette, but it has nothing to do with what is true and right. (It invokes the image of a sociopath manager trying to sell you a product---they will find a way to agree with you to close the deal.) I don't know what introspective is, but I know it when I see it. People around me genuinely come up with new concepts---some of what they came up decades ago with is now ubiquitous---and the sources is often not language. It comes from observing the world with your eyes, from physical or natural mechanisms. If you want to put it into the language of models: we just have so much more data to draw on. And we have a good feedback mechanism. If you invent a toy, you can build it and test it. Language models only get second hand feedback from users. They cannot prototype stuff if the data isn't out there already.
  - jacobr1 4y ago
    One implication though, is that LLMs can currently come up with novel mixes of existing ideas. It might be a good blender, integrating different pieces into a new whole.
    - juujian 4y ago
      Yes, but the language model does not have the feedback mechanism we have. We can test ideas against reality. Language models can make up all kinds of crap until there is data somewhere mentioning that it's not going to work. You could come up with an idea and workshop it, e.g., seeing if it's physically feasible to make something, before sharing it with others, language models cannot.
  - coldtea 4y ago
    >It is different to what you do. If I tell you that this is already a thing, you might go back to the drawing board, and do something from scratch. Wouldn't your "something from scratch" idea, be based on your "training set" (knowledge you've learned in your life), and ways of re-arranging it inside your brain, using neuron stuctures created, shaped, and reiforced in certain ways by exposure to said training set and various kinds of re-inforcement?
- SoftTalker 4y ago
  There are very few new ideas, but many different people have the same ideas.

djmips 4y ago

But how is it different from humans? I can't tell you how many times now that I've come up with what I thought was a really cool idea but upon web searching found it was already invented/discovered etc. In fact before the Internet I had come up with my own algorithms and only upon the Internet existing did I find they were already discovered years earlier. There's no way that I was regurgitating something I had read in that case.

travisjungroth 4y ago
There’s a difference between coming up with a puzzle then finding out it already exists versus finding a puzzle and saying you came up with it. If I told you “We need a brand new, never-never-before-seen puzzle for our next game release.” and you searched Google for “brand new, never-before-seen puzzle”, found a puzzle game with those words in its marketing copy and pitched it to me, that would be some combination of unintelligent and dishonest behavior. Like, surprisingly so. It’s different from forgetting some puzzle you played with as a little kid and thinking you made it up, or creating a puzzle you’d never seen but has been made before.
- deleted 4y ago
  [deleted]
- not2b 4y ago
  But ChatGPT is not a person, it is a text generator. By asking it to generate a new puzzle, you are prompting it to find text in its training data showing someone describing a new puzzle, and it is going to speak in their voice. It's going to emit sentences that were influenced by what the puzzle developer originally wrote, and that person correctly said that it was new.
  - urbandw311er 4y ago
    This. people who can ground themselves in what ChatGPT is (an auto completion text predictor) are able to best understand the origins of its output.
  - int_19h 4y ago
    You're not prompting it to "find text". Comparing the size of the model to the size of the training data is sufficient to conclusively establish that it's an impossibility. We train it to predict the next word based on the training data, that is true. But we still have no idea what kind of internal structures said training actually produces inside of neural net. It sure as hell isn't just a "stochastic parrot", though, which is rather obvious if you ever tried giving it a complicated multi-step task and solve it while "thinking out loud".
  - travisjungroth 4y ago
    I agree with everything you’ve written here, so I’m not sure what the “But” that’s starts your comment is contrasting. I was answering the question “But how is this different from a person?”. Being asked for something new and finding something that already exists with the word “new” in front of it isn’t normal human behavior. That’s how it’s different from a person. Zooming out a bit, I think there’s some confusion in this whole chain. There’s a common topic about ChatGPT you could call Question of Creativity. If you ask for a new poem, it just smashes together its patterns around poems. You can debate if this is creativity, and if not, how are humans different. A few comments up, someone brought in a different idea you could call New Matching. If you ask for a new poem it will just grab you a poem that had the words “new poem” in front of it. New Matching is a different idea than Question of Creativity. The person I replied to seemed to be mistaking one idea for the other.
juujian 4y ago
It is different to what you do. If I tell you that this is already a thing, you might go back to the drawing board, and do something from scratch. Maybe do some abstract drawing with numbers for brainstorming. A language model is not able to do this, the starting point for a language model is always the training data. That is why there is so many instances where you see some wrong (or correct) response from ChatGPT and when the other person corrects this, the model just agrees to whatever the user says. That is the right thing to do according to language etiquette, but it has nothing to do with what is true and right. (It invokes the image of a sociopath manager trying to sell you a product---they will find a way to agree with you to close the deal.) I don't know what introspective is, but I know it when I see it. People around me genuinely come up with new concepts---some of what they came up decades ago with is now ubiquitous---and the sources is often not language. It comes from observing the world with your eyes, from physical or natural mechanisms. If you want to put it into the language of models: we just have so much more data to draw on. And we have a good feedback mechanism. If you invent a toy, you can build it and test it. Language models only get second hand feedback from users. They cannot prototype stuff if the data isn't out there already.
- jacobr1 4y ago
  One implication though, is that LLMs can currently come up with novel mixes of existing ideas. It might be a good blender, integrating different pieces into a new whole.
  - juujian 4y ago
    Yes, but the language model does not have the feedback mechanism we have. We can test ideas against reality. Language models can make up all kinds of crap until there is data somewhere mentioning that it's not going to work. You could come up with an idea and workshop it, e.g., seeing if it's physically feasible to make something, before sharing it with others, language models cannot.
- coldtea 4y ago
  >It is different to what you do. If I tell you that this is already a thing, you might go back to the drawing board, and do something from scratch. Wouldn't your "something from scratch" idea, be based on your "training set" (knowledge you've learned in your life), and ways of re-arranging it inside your brain, using neuron stuctures created, shaped, and reiforced in certain ways by exposure to said training set and various kinds of re-inforcement?
SoftTalker 4y ago
There are very few new ideas, but many different people have the same ideas.