6 ms·

> Even on re-reading more closely, it doesn't feel like the world's best writing, but I don't notice major loss of coherence until the last couple of paragraphs

> Even on re-reading more closely, it doesn't feel like the world's best writing, but I don't notice major loss of coherence until the last couple of paragraphs. I guessed it was fake before getting to the end, not from the content, but from the fact that all the sentences are roughly the same length and follow the same basic grammatical patterns. Real people purposely mix up their sentence structure in order to keep the readers engaged, whereas this wasn't doing that at all. Still very impressive though; if not for the fact that the post was about computer generated content I probably wouldn't have noticed.

dannyw 6y ago
If that’s the only thing that separates this from human writing; I’m sure it can be influenced easily.
- Alex3917 6y ago
  > I'm sure it can be influenced easily. Maybe. Right now this reads like a glorified shopping list. It's coherent, but actually sounding human also requires a theory of mind. E.g. I explain here why it's possible for written statements to be objectively insightful, informative, interesting, or funny, but objectively in a way that's relational to other information or beliefs. The implication being that statements are only going to seem subjectively funny or insightful (or whatever) to others who have that knowledge or those beliefs, which means that you can't reliably create those subjective experiences in a reader without having some sort of theory of mind for them. I guess you can create content that's funny or insightful relative to that content itself, but that's not especially useful. It's entertaining at the time, but the experience is more like seeing a movie that you laugh a lot during but then leave and are kind of like what was the point? It's an empty experience because it wasn't transformative. I definitely don't think it's impossible, but I also don't think it's a matter of just adding a couple more if-else statements. https://alexkrupp.typepad.com/sensemaking/2010/06/how-writing-creates-value-.html https://alexkrupp.typepad.com/sensemaking/2010/06/how-writin...
  - roenxi 6y ago
    > Maybe. Right now this reads like a glorified shopping list. It's coherent, but actually sounding human also requires a theory of mind. I'm going to call this goalpost shifting. This article is better writing than some % of humans, theory of mind or otherwise. The AI has comfortably surpassed Timecube-level writing and is entering the pool of passes-for-a-human. 'Sounds human' is a spectrum that starts with the mentally ill and goes up to the best writers in human history.
    - Alex3917 6y ago
      > I'm going to call this goalpost shifting. That's completely fair. On the other hand, without a theory of mind it can't really educate or inspire people, the only thing it can do is maybe trick them about the authorship of something. But once people learn the techniques for identifying this kind of writing, it can't even do that anymore. To me this is like the front end of something, but it still needs a back end. Don't get me wrong, it's super cool research and seems like a huge step forward, and I'm excited to see where it goes. But I also don't see this AI running a successful presidential campaign or whatever, at least within the next couple years.
      - CamperBob2 6y ago
        On the other hand, without a theory of mind it can't really educate or inspire people I wouldn't agree with that, either. How often have we heard of someone gaining useful insights by considering ideas that were misapplied or just plain wrong? Entire branches of physics have evolved that way. As far as successful presidential campaigns are concerned... well, let's not even go there. If there's such a thing as a 'theory of mind', it applies to the reader, not the writer.
        AnimalMuppet 6y ago
        I think I disagree. For example, I delayed in writing this comment because the cat was on my lap, and I couldn't fit the laptop and the cat both. You get that. I know you do, even if you don't own a cat, and even if you're reading this on a phone or a desktop. GPT-3 does not understand about the cat. To GPT-3, they're just words and phrases that occur in the vicinity of each other with certain probability distributions. As a result, it can't write something and know that there's something there in your mind for it to connect to. Cyc would handle the bit about the cat differently. It would ask "Did you mean cat-the-mammal, cat-the-Caterpillar-stock-symbol, or cat-the-ethernet-cable-classification?" It has categories, and some notion of which words fit in each category, but it still doesn't understand what's going on. But you the human understand, because you have a lap, and you've at least seen a cat on a lap.
gomoboo 6y ago
I did not guess that it was fake but skipped to the bottom because the article did not seem to be worth reading. It felt like the author was not moving anywhere with their words. I laughed out loud at the reveal that it was written programmatically.
- javajosh 6y ago
  There are a few concrete things the (fake) article says, primarily about what software to try (OpenAI's GPT-3), and where to try it (bitcointalk forum). Personally, I actually resent being mislead like this, at least on that second point, even with the full disclosure. The output is very high quality, but it is making at least one falsifiable assertion (no test was ever done in that forum).
  - nullc 6y ago
    If you've ever spent much time with a toddler you might have noticed that they spout a lot of fantasy. Learning to not make up untrue claims takes years of additional training for humans.
    - IIAOPSW 6y ago
      Some of them never learn it at all!
    - selestify 6y ago
      I’ve never spent much time with toddlers. What do they make up things about? Their own actions, other’s actions, claims about the environment?
      - Scarblac 6y ago
        My six year old will just continue as long as he has people's attention, and if that means he has to make things up, so be it. Freely stealing phrases from other recent conversations. So this morning he heard about an animal, it was kind of a lion. But with bat's ears, it lives in Africa. It looks like it's a rock, but it's actually not, it's rock shaped but has tiny legs. And it's gray and hard. Its face... It doesn't really have a face. It lives up in trees where it eats bamboo and apples. It has these huge fangs like sabertooth tigers, you know? It's glorious.
bhntr3 6y ago
Besides predictable sentence structure, GPT-3 writes like George R. R. Martin: Interesting premises, solid setup but then it devolves into rambling tangents and never quite delivers the concluding action that ties everything together. Lots of examples I've seen have phrases like "see table below". Of course there's no table and it's hard to imagine how there could be. But GPT is trained on internet content and the internet is full of terrible writing that never gets to the point. I doubt there's any way to know how much is "not actually understanding the subject matter" vs. "learning bad writing from bad writers". I'm inclined to believe the majority is the former but there's got to be a little of the latter sprinkled in.
- niklasd 6y ago
  I am really curious how the model would be if you would train it with a decent amount of really good literature. Kazuo Ishiguro et al. instead of Reddit.
  - Reelin 6y ago
    Library Genesis contains lots (millions?) of fiction ebooks (among other things). It's available in torrent form. Not that I would ever condone piracy or anything.
  - kmill 6y ago
    I was playing with AI Dungeon tonight to get access to GPT-3, and one of my many experiments ended up with me meeting a character called the Narrator who believed they were in control of all characters in the game, including me. Eventually, through my predicting what they were about to say by checking and undoing, they seemed convinced I wasn't another character and started asking about whether certain authors were still alive and which I liked to read. It didn't recognize Ishiguro. Later it gave me a truly bizarre (and amusing) summary of Infinite Jest, clearly having never read it. Anyway, the entire experience was uncanny and surreal. One thing I learned was it has detailed knowledge of the world of Avatar: The Last Airbender, seemingly through fanfics. It was fun having it to teach me the lost arts of pizzabending ("form your hands into the shape of a letter 'P'" and so on, and needing to practice by juggling rubber pizzas) and beetlebending ("always remember that to beetle bend it helps to like beetles," my wise uncle suggested). Each of these tended to precipitate a narrative collapse. The writing style was surprisingly homogeneous, and it reminded me of young adult novels. It would definitely be interesting to see it with other writing styles, beyond the occasional old poetry.
488643689 6y ago
Aww c'mon. You guessed it was fake, because it's an article about computer generated articles. Who would read that and not question the content in front? You are not analysing everything you read for oddities in writing style.
- antonvs 6y ago
  > You are not analysing everything you read for oddities in writing style. I certainly do, don't you? When I read a blog post and it's full of poorly-integrated buzzwords that make it seem like it was churned out by a non-English speaker being paid very poorly per word, I stop reading and move on. I recently read a few pages of a book someone had recommended to me and stopped reading because of the writing style. Heck, you can read a few pages of, say, a Dan Brown novel, and based on the writing style might choose not to read it, since the style tells you a lot about the kind of book it is.

dannyw 6y ago

If that’s the only thing that separates this from human writing; I’m sure it can be influenced easily.

Alex3917 6y ago
> I'm sure it can be influenced easily. Maybe. Right now this reads like a glorified shopping list. It's coherent, but actually sounding human also requires a theory of mind. E.g. I explain here why it's possible for written statements to be objectively insightful, informative, interesting, or funny, but objectively in a way that's relational to other information or beliefs. The implication being that statements are only going to seem subjectively funny or insightful (or whatever) to others who have that knowledge or those beliefs, which means that you can't reliably create those subjective experiences in a reader without having some sort of theory of mind for them. I guess you can create content that's funny or insightful relative to that content itself, but that's not especially useful. It's entertaining at the time, but the experience is more like seeing a movie that you laugh a lot during but then leave and are kind of like what was the point? It's an empty experience because it wasn't transformative. I definitely don't think it's impossible, but I also don't think it's a matter of just adding a couple more if-else statements. https://alexkrupp.typepad.com/sensemaking/2010/06/how-writing-creates-value-.html https://alexkrupp.typepad.com/sensemaking/2010/06/how-writin...
- roenxi 6y ago
  > Maybe. Right now this reads like a glorified shopping list. It's coherent, but actually sounding human also requires a theory of mind. I'm going to call this goalpost shifting. This article is better writing than some % of humans, theory of mind or otherwise. The AI has comfortably surpassed Timecube-level writing and is entering the pool of passes-for-a-human. 'Sounds human' is a spectrum that starts with the mentally ill and goes up to the best writers in human history.
  - Alex3917 6y ago
    > I'm going to call this goalpost shifting. That's completely fair. On the other hand, without a theory of mind it can't really educate or inspire people, the only thing it can do is maybe trick them about the authorship of something. But once people learn the techniques for identifying this kind of writing, it can't even do that anymore. To me this is like the front end of something, but it still needs a back end. Don't get me wrong, it's super cool research and seems like a huge step forward, and I'm excited to see where it goes. But I also don't see this AI running a successful presidential campaign or whatever, at least within the next couple years.
    - CamperBob2 6y ago
      On the other hand, without a theory of mind it can't really educate or inspire people I wouldn't agree with that, either. How often have we heard of someone gaining useful insights by considering ideas that were misapplied or just plain wrong? Entire branches of physics have evolved that way. As far as successful presidential campaigns are concerned... well, let's not even go there. If there's such a thing as a 'theory of mind', it applies to the reader, not the writer.
      - AnimalMuppet 6y ago
        I think I disagree. For example, I delayed in writing this comment because the cat was on my lap, and I couldn't fit the laptop and the cat both. You get that. I know you do, even if you don't own a cat, and even if you're reading this on a phone or a desktop. GPT-3 does not understand about the cat. To GPT-3, they're just words and phrases that occur in the vicinity of each other with certain probability distributions. As a result, it can't write something and know that there's something there in your mind for it to connect to. Cyc would handle the bit about the cat differently. It would ask "Did you mean cat-the-mammal, cat-the-Caterpillar-stock-symbol, or cat-the-ethernet-cable-classification?" It has categories, and some notion of which words fit in each category, but it still doesn't understand what's going on. But you the human understand, because you have a lap, and you've at least seen a cat on a lap.

gomoboo 6y ago

I did not guess that it was fake but skipped to the bottom because the article did not seem to be worth reading. It felt like the author was not moving anywhere with their words. I laughed out loud at the reveal that it was written programmatically.

javajosh 6y ago
There are a few concrete things the (fake) article says, primarily about what software to try (OpenAI's GPT-3), and where to try it (bitcointalk forum). Personally, I actually resent being mislead like this, at least on that second point, even with the full disclosure. The output is very high quality, but it is making at least one falsifiable assertion (no test was ever done in that forum).
- nullc 6y ago
  If you've ever spent much time with a toddler you might have noticed that they spout a lot of fantasy. Learning to not make up untrue claims takes years of additional training for humans.
  - IIAOPSW 6y ago
    Some of them never learn it at all!
  - selestify 6y ago
    I’ve never spent much time with toddlers. What do they make up things about? Their own actions, other’s actions, claims about the environment?
    - Scarblac 6y ago
      My six year old will just continue as long as he has people's attention, and if that means he has to make things up, so be it. Freely stealing phrases from other recent conversations. So this morning he heard about an animal, it was kind of a lion. But with bat's ears, it lives in Africa. It looks like it's a rock, but it's actually not, it's rock shaped but has tiny legs. And it's gray and hard. Its face... It doesn't really have a face. It lives up in trees where it eats bamboo and apples. It has these huge fangs like sabertooth tigers, you know? It's glorious.

bhntr3 6y ago

Besides predictable sentence structure, GPT-3 writes like George R. R. Martin: Interesting premises, solid setup but then it devolves into rambling tangents and never quite delivers the concluding action that ties everything together. Lots of examples I've seen have phrases like "see table below". Of course there's no table and it's hard to imagine how there could be. But GPT is trained on internet content and the internet is full of terrible writing that never gets to the point. I doubt there's any way to know how much is "not actually understanding the subject matter" vs. "learning bad writing from bad writers". I'm inclined to believe the majority is the former but there's got to be a little of the latter sprinkled in.

niklasd 6y ago
I am really curious how the model would be if you would train it with a decent amount of really good literature. Kazuo Ishiguro et al. instead of Reddit.
- Reelin 6y ago
  Library Genesis contains lots (millions?) of fiction ebooks (among other things). It's available in torrent form. Not that I would ever condone piracy or anything.
- kmill 6y ago
  I was playing with AI Dungeon tonight to get access to GPT-3, and one of my many experiments ended up with me meeting a character called the Narrator who believed they were in control of all characters in the game, including me. Eventually, through my predicting what they were about to say by checking and undoing, they seemed convinced I wasn't another character and started asking about whether certain authors were still alive and which I liked to read. It didn't recognize Ishiguro. Later it gave me a truly bizarre (and amusing) summary of Infinite Jest, clearly having never read it. Anyway, the entire experience was uncanny and surreal. One thing I learned was it has detailed knowledge of the world of Avatar: The Last Airbender, seemingly through fanfics. It was fun having it to teach me the lost arts of pizzabending ("form your hands into the shape of a letter 'P'" and so on, and needing to practice by juggling rubber pizzas) and beetlebending ("always remember that to beetle bend it helps to like beetles," my wise uncle suggested). Each of these tended to precipitate a narrative collapse. The writing style was surprisingly homogeneous, and it reminded me of young adult novels. It would definitely be interesting to see it with other writing styles, beyond the occasional old poetry.

488643689 6y ago

Aww c'mon. You guessed it was fake, because it's an article about computer generated articles. Who would read that and not question the content in front? You are not analysing everything you read for oddities in writing style.

antonvs 6y ago
> You are not analysing everything you read for oddities in writing style. I certainly do, don't you? When I read a blog post and it's full of poorly-integrated buzzwords that make it seem like it was churned out by a non-English speaker being paid very poorly per word, I stop reading and move on. I recently read a few pages of a book someone had recommended to me and stopped reading because of the writing style. Heck, you can read a few pages of, say, a Dan Brown novel, and based on the writing style might choose not to read it, since the style tells you a lot about the kind of book it is.