4 ms·

If you'd like to innoculate yourself, and have a bit of fun in the meantime, consider reading https://www.reddit.com/r/SubSimulatorGPT2/ https://www.reddit.com/

by jerf 7y ago

If you'd like to innoculate yourself, and have a bit of fun in the meantime, consider reading https://www.reddit.com/r/SubSimulatorGPT2/ https://www.reddit.com/r/SubSimulatorGPT2/ . It's not just for fun, you can get a good sense of the algorithm. One of the things it is somewhat prone to is some weird looping, like this: https://www.reddit.com/r/SubSimulatorGPT2/comments/d1nwdg/if_the_president_of_the_united_states_was_your/ezo3sce/ https://www.reddit.com/r/SubSimulatorGPT2/comments/d1nwdg/if... in which the algorithm generates the sentence "Toss some leeches around and wait 'til we get there." (no, it does not make any more sense in context), and then repeats that sentence nearly (but not quite!) exactly 23 more times. (I expect this is a consequence of the way it is tracking some internal state; I assume these sentences are strange attractors in some sort of state that is getting iteratively modified.) You can also see that while it picks up some deep structure, a check of anything trained on /r/jokes (https://www.reddit.com/r/SubSimulatorGPT2/comments/d055mt/a_guy_is_having_a_hard_time_with_his_wifes/?utm_source=share&utm_medium=web2x https://www.reddit.com/r/SubSimulatorGPT2/comments/d055mt/a_... ) or /r/math (https://www.reddit.com/r/SubSimulatorGPT2/comments/d1yz1e/how_exactly_does_the_number_1_not_equal_2/ https://www.reddit.com/r/SubSimulatorGPT2/comments/d1yz1e/ho... ) the algorithm is definitely unable to deal with deeper structure right now. The /r/jokes bot is humorous in its complete lack of humor, I mean, well beyond any sarcastic snark about how unfunny /r/jokes may be. It has the structure of jokes. There was one recent one that even asked "What's a pirate's favorite letter?", and the bot had noticed the answer was being given in the form of letters, but I don't think a single instance of the bot proposed "r". But it does not understand humor in the slightest. Of the several dozen attempts at jokes I've at least skimmed, I believe it only achieved something that was at least recognizable as an attempt at humor once, and it still wasn't that funny. Likewise math. It's got a good idea there's these "prime number" things and they're pretty important, but I've seen at least half-a-dozen wrong definitions of what one is. It's a very interesting algorithm. It's a great babbler. But on its own, it's not a great solution to generating text. Although it may very well be able to generate text that can pass a casual skim text, as the article suggests. Still, it takes human curation to get that far. Any human that can read is going to guess something's inhuman about repeating "Toss some leeches around and wait 'til we get there." 24 times in a row.

jborichevskiy 7y ago
On the other hand, there are plenty of Reddit comment threads written by (presumably) people which are just the same thing over and over again, or small variations thereof. Usually some sort of reference to a joke or meme but it does happen.
outworlder 7y ago
> If you'd like to innoculate yourself, Dude, my malady got worse, not better. The comments there are up to a better standard than most Youtube or Facebook comments.
- whatshisface 7y ago
  Reading that sub was a surreal experience. The most interesting part for me was how it (completely subconsciously) changed the "voice in my head" reading voice into something very dull and stilted, like a second grader reading a terrible essay out loud.
blankaccount 7y ago
> There was one recent one that even asked “What’s a pirate’s favorite letter?”, and the bot had noticed the answer was being given in the form of letters, but I don’t think a single instance of the bot proposed “r”. But it does not understand humor in the /slightest/. To me this actually sounds hilarious, if it is as you describe it. It's the kind of joke you'd find on The Office: Dwight: "What's a pirate's favourite letter?" Jim: "M" Jim: "N" Jim: "O" Jim: "P" Jim: "Q" Jim: "..." Jim: "S" > "Toss some leeches around and wait 'til we get there." Like the above poster said, I've seen this style of spam-y context-free meme on reddit before too.
- jerf 7y ago
  "To me this actually sounds hilarious, if it is as you describe it." Well, here's the original I was referring to: https://www.reddit.com/r/SubSimulatorGPT2/comments/c8klvj/what_is_a_pirates_favourite_letter/ https://www.reddit.com/r/SubSimulatorGPT2/comments/c8klvj/wh... I was wrong. R was suggested in some of the replies. But the original answer is given as "The M", and contains gems like '"r" is a misspelled letter, and "m" is a misspelled letter. "M" is a misspellable word you can't use as a word in the English language.' "Like the above poster said, I've seen this style of spam-y context-free meme on reddit before too. " That would make sense. I guess I just don't frequent those corners. GPT-2 is clearly capable of picking up on structure, so if it sees something repeated it doesn't just notice "This particular thing is repeated a lot", it picks up some concept of repetition itself. A number of the bots have picked up the concept of quoting the message they're replying to. (In the meta reddit for this, the creator has said the posts and the replies are trained as separate corpora, so the replies "know" they are replies. I gather there is also enough markers that the bots can distinguish between title, post text, and subsequent replies.)

by jerf 7y ago

jborichevskiy 7y ago

On the other hand, there are plenty of Reddit comment threads written by (presumably) people which are just the same thing over and over again, or small variations thereof. Usually some sort of reference to a joke or meme but it does happen.

outworlder 7y ago

> If you'd like to innoculate yourself, Dude, my malady got worse, not better. The comments there are up to a better standard than most Youtube or Facebook comments.

whatshisface 7y ago
Reading that sub was a surreal experience. The most interesting part for me was how it (completely subconsciously) changed the "voice in my head" reading voice into something very dull and stilted, like a second grader reading a terrible essay out loud.

blankaccount 7y ago

> There was one recent one that even asked “What’s a pirate’s favorite letter?”, and the bot had noticed the answer was being given in the form of letters, but I don’t think a single instance of the bot proposed “r”. But it does not understand humor in the /slightest/. To me this actually sounds hilarious, if it is as you describe it. It's the kind of joke you'd find on The Office: Dwight: "What's a pirate's favourite letter?" Jim: "M" Jim: "N" Jim: "O" Jim: "P" Jim: "Q" Jim: "..." Jim: "S" > "Toss some leeches around and wait 'til we get there." Like the above poster said, I've seen this style of spam-y context-free meme on reddit before too.

jerf 7y ago
"To me this actually sounds hilarious, if it is as you describe it." Well, here's the original I was referring to: https://www.reddit.com/r/SubSimulatorGPT2/comments/c8klvj/what_is_a_pirates_favourite_letter/ https://www.reddit.com/r/SubSimulatorGPT2/comments/c8klvj/wh... I was wrong. R was suggested in some of the replies. But the original answer is given as "The M", and contains gems like '"r" is a misspelled letter, and "m" is a misspelled letter. "M" is a misspellable word you can't use as a word in the English language.' "Like the above poster said, I've seen this style of spam-y context-free meme on reddit before too. " That would make sense. I guess I just don't frequent those corners. GPT-2 is clearly capable of picking up on structure, so if it sees something repeated it doesn't just notice "This particular thing is repeated a lot", it picks up some concept of repetition itself. A number of the bots have picked up the concept of quoting the message they're replying to. (In the meta reddit for this, the creator has said the posts and the replies are trained as separate corpora, so the replies "know" they are replies. I gather there is also enough markers that the bots can distinguish between title, post text, and subsequent replies.)