3 ms·

I don't want to suggest you shot off a random comment without actually reading the linked survey. But it spends a whole section discussing that, and why it does

by polm23 6y ago

I don't want to suggest you shot off a random comment without actually reading the linked survey. But it spends a whole section discussing that, and why it doesn't see commercial use. Here's a particularly relevant bit: > Sometimes the results produced by GPT-2 and its ilk are quite startling in their apparent authenticity. More often than not they are just a bit off. And sometimes they are just gibberish. As is widely acknowledged, neural text generation as it stands today has a significant problem: driven as it is by information that is ultimately about language use, rather than directly about the real world, it roams untethered to the truth. While the output of such a process might be good enough for the presidential teleprompter, it would not cut it if you want the hard facts about how your pension fund is performing. So, at least for the time being, nobody who develops commercial applications of NLG technology is going to rely on this particular form of that technology.

picodguyo 6y ago
I agree with the author on GPT-2. But GPT-3, which became available shortly after this was published, is quite a bit more powerful and there are many commercial applications being built on it now.
- leereeves 6y ago
  Even GPT-3 knows nothing about the real world; it's merely trained to repeat the words that most often followed the prompt in its training data. That's obviously not useful for news...if a fact is in the training data, it's not news. It's not useful for "hard facts about how your pension fund is performing" unless you want to know how it performed a long time ago. But I agree there are some applications it is useful for, like education.
  - FeepingCreature 6y ago
    > Even GPT-3 knows nothing about the real world; it's merely trained to repeat the words that most often followed the prompt in its training data. I don't know why that would imply that it knows nothing about the real world, unless the data corpus it is trained on likewise bears no relation to reality...
    - nmfisher 6y ago
      > unless the data corpus it is trained on likewise bears no relation to reality It’s trained on Reddit, so I wouldn’t rule that out.
- probably_wrong 6y ago
  I do not see how GPT-3 could solve the basic architectural problem that the parent comment quotes, namely, that "driven as it is by information that is ultimately about language use, rather than directly about the real world, it roams untethered to the truth". As an experiment I used a GPT-3-powered website [1] to see what GPT-3 has to say about bears, and the first answer was: > "Weird that every day, there are so many cute/funny/entertaining bears to enjoy online but hardly any on the ground." When asked about beards, the first answer has no relation with beards at all: > "If a person doesn’t constantly outwit, outplay, outlast, others, the strong eat the weak." And then there's that time when GPT-3 told someone to kill themselves [2]. While funny and (mostly) grammatically correct, these "thoughts" are nonsense and no amount of extra parameters is going to solve the disconnection between GPT-3 and reality. I imagine you could condition GPT-3 to generate text for a specific piece of data in such a way that guarantees the correctness of its output, but at that point you might as well throw GPT-3 away and write a rule-based system. [1] https://thoughts.sushant-kumar.com/bears https://thoughts.sushant-kumar.com/bears [2] https://www.nabla.com/blog/gpt-3/ https://www.nabla.com/blog/gpt-3/
  - picodguyo 6y ago
    Is language use not inherently shaped by the real world? The site you tried is a tweet generator, not a question answering site. I prompted GPT-3 with "Bears and beards are different because" and got... "Bears and beards are different because they are not the same thing. Bears are animals. Beards are facial hair. Bears are dangerous. Beards are not. Bears live in the woods. Beards live on your face. Bears eat people. Beards do not." But my original point was mainly that this field is moving fast and the the old school NLG companies (I created one back in the day!) are toast.
    - hannasanarion 6y ago
      There is more to "the real world" than the definition of words, which is the most that you can expect a language model to learn. Yes it is true that bears are animals. No it is not true that, as GPT-3 said, "There aren't any on the ground"

by polm23 6y ago

picodguyo 6y ago

I agree with the author on GPT-2. But GPT-3, which became available shortly after this was published, is quite a bit more powerful and there are many commercial applications being built on it now.

leereeves 6y ago
Even GPT-3 knows nothing about the real world; it's merely trained to repeat the words that most often followed the prompt in its training data. That's obviously not useful for news...if a fact is in the training data, it's not news. It's not useful for "hard facts about how your pension fund is performing" unless you want to know how it performed a long time ago. But I agree there are some applications it is useful for, like education.
- FeepingCreature 6y ago
  > Even GPT-3 knows nothing about the real world; it's merely trained to repeat the words that most often followed the prompt in its training data. I don't know why that would imply that it knows nothing about the real world, unless the data corpus it is trained on likewise bears no relation to reality...
  - nmfisher 6y ago
    > unless the data corpus it is trained on likewise bears no relation to reality It’s trained on Reddit, so I wouldn’t rule that out.
probably_wrong 6y ago
I do not see how GPT-3 could solve the basic architectural problem that the parent comment quotes, namely, that "driven as it is by information that is ultimately about language use, rather than directly about the real world, it roams untethered to the truth". As an experiment I used a GPT-3-powered website [1] to see what GPT-3 has to say about bears, and the first answer was: > "Weird that every day, there are so many cute/funny/entertaining bears to enjoy online but hardly any on the ground." When asked about beards, the first answer has no relation with beards at all: > "If a person doesn’t constantly outwit, outplay, outlast, others, the strong eat the weak." And then there's that time when GPT-3 told someone to kill themselves [2]. While funny and (mostly) grammatically correct, these "thoughts" are nonsense and no amount of extra parameters is going to solve the disconnection between GPT-3 and reality. I imagine you could condition GPT-3 to generate text for a specific piece of data in such a way that guarantees the correctness of its output, but at that point you might as well throw GPT-3 away and write a rule-based system. [1] https://thoughts.sushant-kumar.com/bears https://thoughts.sushant-kumar.com/bears [2] https://www.nabla.com/blog/gpt-3/ https://www.nabla.com/blog/gpt-3/
- picodguyo 6y ago
  Is language use not inherently shaped by the real world? The site you tried is a tweet generator, not a question answering site. I prompted GPT-3 with "Bears and beards are different because" and got... "Bears and beards are different because they are not the same thing. Bears are animals. Beards are facial hair. Bears are dangerous. Beards are not. Bears live in the woods. Beards live on your face. Bears eat people. Beards do not." But my original point was mainly that this field is moving fast and the the old school NLG companies (I created one back in the day!) are toast.
  - hannasanarion 6y ago
    There is more to "the real world" than the definition of words, which is the most that you can expect a language model to learn. Yes it is true that bears are animals. No it is not true that, as GPT-3 said, "There aren't any on the ground"