3 ms·
If you pay just a little attention, you absolutely can: GPT-3 is not saying anything. Even the 'lowest' humans are usually trying to communicate something when
by tsimionescu 6y ago
If you pay just a little attention, you absolutely can: GPT-3 is not saying anything. Even the 'lowest' humans are usually trying to communicate something when they are telling a story, or teaching you how to do a basic skill, or giving you directions.
GPT-3 can't do any of that. It can pick up clues from the text to produce incredibly realistic sentences that are related to the broad topics of some text, but it's all smoke and mirrors in the end - there is no model of the world getting expressed in communication, it is just mindless aping of similar speech.
And yes, this basic skill that GPT-3 has is enough to replace some human tasks, like inventing plausible sounding stories for a recipe or perhaps even taking news from one site and writing them on another with slight alterations. Perhaps it will even be able to take some facts and weave them into a speech about that topic.
But it is not even close to doing something like real journalism, even at the level of a car mechanic telling you what happened down at the mall.
- JadeNB 6y ago> GPT-3 can't do any of that. It can pick up clues from the text to produce incredibly realistic sentences that are related to the broad topics of some text, but it's all smoke and mirrors in the end - there is no model of the world getting expressed in communication, it is just mindless aping of similar speech. You know this inevitably tempts the cynical question of how different what you describe is from (to put it optimistically) clickbait generators and (to put it still more cynically) much of the content generated today ….
- I-M-S 6y agoGPT's text as quoted in the article has a clear thesis stated upfront, later expands on it with examples, summarizes its arguments at the end, and does all this with gusto. You cap your comment by a non-sequitur that GPT is not going to replace journalism. IMO the piece of text generated by GPT offers more insight and is wittier than yours.
- tsimionescu 6y agoMy comment was not an essay, it was a response to someone else's comment. The GP was explicitly saying that GPT-3 could probably produce many of the news content they read, and I was replying to that. Please also note that it's not very clear to what extent the text of the article is edited - the non-bold text is written by GPT-3, but I don't think it was produced as a single block of text. Instead different parts (sentences? Paragraphs?) were produced individually, selected by a human from many other responses, and assembled together in the shape we are shown. The train of thought among the paragraphs is most likely entirely human work, not AI work, and only the best sounding paragraphs out of a lot of gibberish were likely selected. It would also be interesting to see how close those paragraphs are to something in GOT-3's training corpus, in terms of structure if not explicit language.
- deleted 6y ago[deleted]
- BoiledCabbage 6y ago> And yes, this basic skill that GPT-3 has is enough to replace some human tasks, like inventing plausible sounding stories for a recipe or perhaps even taking news from one site and writing them on another with slight alterations. Perhaps it will even be able to take some facts and weave them into a speech about that topic. > But it is not even close to doing something like real journalism, even at the level of a car mechanic telling you what happened down at the mall. You're just talking about layers of abstraction. GPT-3 works with chunks of 3 letters. It can now. Combine chunks to form valid words. Combine words to form syntactic sentences. Combine words to form semantic sentences. Combine sentences to form consistent paragraphs. Combine paragraphs to form trains of thought. It can't combine trains of thought to produce a consistent point. But it's already working successfully at 5 or 6 levels of abstraction up. I don't think it's that much harder to get one to two levels higher in abstraction. It doesn't know level 7 is any different from level 6. It just needs the data and compute to start modeling at that level. In the past you could've convinced me a different architecture is needed to accomplish that. But I also wouldve doubted it could get this far - why would something stop it now? It just needs to study more long form work and how to reach a conclusion in an essay starting from paragraph 1.
- TheOtherHobbes 6y agoDefine "semantic" and "trains of thought." Personally I'd label this human satire, not raw GPT-3 output. I'll change my mind if I see evidence that proves that's incorrect.
- jasonv 6y agoI feel like I can understand it (GPT-3 generated text) better than meme-oriented Reddit callback threads. Which may have more to do with burying meaning more than explicating it.
- edouard-harris 6y ago> If you pay just a little attention, you absolutely can: GPT-3 is not saying anything. Check out Figure 3.13 in the GPT-3 paper, https://arxiv.org/pdf/2005.14165.pdf https://arxiv.org/pdf/2005.14165.pdf. The authors experimented with 200-word news articles, to see whether 80 human judges could tell the difference between human-generated and GPT-3-generated ones. It turns out they could not: the human judges correctly identified GPT-3-generated content only 52% of the time, essentially as good as random guessing. (And no, the machine-generated articles were not cherry-picked for the experiment.)
- tsimionescu 6y agoThis is a good counterpoint. I do wonder though how close the articles that GPT-3 produced were to the articles it had been trained on. For example, the Methodist Church Split article that it produced has a lot of very specific facts about the Methodist Church and about the split, which shows that it had texts about that spefic event in the training set. It also has a sentence which contains a pretty obvious non-sequitur, but it's easy to miss it or assume that it's a mistake that a human made. So overall, I'm guessing GPT-3 may actually be pretty decent at re-telling a story with different words, which sometimes is very hard hard to distinguish from a human doing the same thing. They also don't describe the way they programmatically selected the output, though I am willing to believe that they more or less randomly sampled the output from each model.
- YeGoblynQueenne 6y ago>> The authors experimented with 200-word news articles, to see whether 80 human judges could tell the difference between human-generated and GPT-3-generated ones. I disagree with tsimionescu that this is a good counterpoint. The comment you reply to says that "GPT-3 is not saying anything". The figure you refer to shows that human judges could not tell the difference between human-generated and GPT-3 generated text. That's apples and oranges. That some humans weren't able to detect autogenerated text doesn't say anything about whether the autogenerated text said anything.
- FeepingCreature 6y ago
- leetcrew 6y ago> If you pay just a little attention, you absolutely can: GPT-3 is not saying anything. Even the 'lowest' humans are usually trying to communicate something when they are telling a story, or teaching you how to do a basic skill, or giving you directions. you're right; this AI is no substitute for a journalist. however, I do think the "essay" compares favorably with some papers I peer-reviewed for my college writing seminar. sometimes humans really aren't trying to communicate anything; they are just trying to hit the minimum word count and get a passing grade.
- tsimionescu 6y agoSure, as I said, there are human tasks that may appear creative but aren't. Also, please note that the 'essay' is assembled by human selection from GPT-3 produced output (whole paragraphs?).
- quicklime 6y agoIt may or may not be smoke and mirrors, but there's a clear structure to the essay, as well as a logical structure to the arguments in the essay. For example, "humans think they're intelligent", "humans are wrong about everything", and therefore "intelligence isn't about being right". It even concludes with reasonably good advice: to pass a Turing test, AIs should say things that are true, and tap into human emotions. To me, this disproves the idea that it's "not saying anything". It's definitely saying something, and that something is both true and not commonly understood by the general public. It is therefore capable of "teaching a basic skill". I find that very impressive, and I'm surprised that so many others here don't.
- tsimionescu 6y agoAs I stated elsewhere, the essay is almost certainly constructed by the human writing the article, out of cherry-picked output by GPT-3 stitched together (I'm guessing at the paragraph level). Also, again almost certainly, the information about what it takes to pass the Turing test is taken from some text in the corpus it was trained on. What GPT-3 did do is recognize that that is relevant to the topic of AI vs human intelligence, but not much more.