5 ms·

Author of the stories here! Happy to answer questions about the process. I'm a big believer that tech like GPT-3 can provocate creative writers similar to tools

Author of the stories here! Happy to answer questions about the process. I'm a big believer that tech like GPT-3 can provocate creative writers similar to tools like oblique strategies and story dice. But it can also go further like what I've shown here: it can embody a character, allowing authors to converse and play with it to spark new ideas.

castlecrasher2 6y ago
If I'm interested in tinkering with this, how do I get started?
2bitencryption 6y ago
fun tip - apparently there's enough Harry Potter fanfiction out there that GPT-2/3 is quite good at understanding that universe. So if you go into AI Dungeon and pick "custom story", and give it a prompt that indicates you are at Hogwarts, it will pretty much adapt all the rules of the HP universe. The downside is, when you do this, GPT's intelligence drops to about the level of a Harry Potter fanfic writer.
- jamesjyu 6y ago
  Yeah, GPT-3 contains essentially all the historical internet up to October 2019 and many books, you can easily trigger it to enter a "known" world. But what's more interesting to me as a fiction writer is to be able to prime it to enter a new world that I create.
- geoelectric 6y ago
  So if I prompt it with sexy Spock, do I get TOS slash stories? There are a few genres like that which have significant internet (and only internet) presence. How about creepypasta?
  - cosmojg 6y ago
    Yes and yes. GPT-2 is pretty capable of this as well. You can try it yourself here: https://bellard.org/textsynth/ https://bellard.org/textsynth/
  - TeMPOraL 6y ago
    > So if I prompt it with sexy Spock, do I get TOS slash stories? Most likely, yes. See my comment elsewhere in the thread: I recently prompted AIDungeon with a mention of a holodeck, and among other things, the game introduced - by itself - a PADD, turbolifts, Andorians, and was able to correctly guess what happens if you shoot yourself in your face with a phaser on stun, with holodeck safeties enabled ("Nothing happens.") and disabled ("You are knocked unconscious by the blast."). It's honestly the first time for me that I'm actually impressed with deep neural networks.
- smusamashah 6y ago
  I tried this. Took role of Harry Potter. Mentioned Voldemort, Hagrid, his pet bird. It didn't take me into the story I was desperately hoping for.
  - zzbzq 6y ago
    AI dungeon is like a bad dream the way it goes around doesn't get anywhere. Similar to all the conversations in the OP's story. It makes you realize even with an AI this good, we've got a way to go before it can really do the things we want.
- notahacker 6y ago
  It's amazing just the sort of Harry Potter fanfic that's out there... Prompt: You look for Hermione. Response: You see her snogging a non-magical boy. His head is buried in her cleavage, which are being covered by her loose gown. You shake your head and walk away. This is actually a better indication of how derivative GPT-3 is than some of my experiments with higher quality source material...
- the_af 6y ago
  Probably because I'm not a native English speaker, I failed miserably at this. I tried giving AI Dungeon a prompt that mentioned Hogwarts, Voldemort, the Sorting Hat, dark wizards, etc, and it went nowhere (it seemed obsessed with telling me about a "voice inside your mind", go figure). Can you share a prompt that will make it answer with something Harry Potter-like? (it can also be Star Trek-like, or anything from pop culture. I just want to see why my prompt didn't work)
  - notahacker 6y ago
    I started off with 'You are Harry Potter. You return to Hogwarts'. It's designed so everything is written in second person like that, and for the most part you can just hit enter, giving it another action involving a character or place if it gets stuck or goes miles off topic Sometimes the responses are vaguely on topic, like 'The memories of your parents' death haunt you every day. They were killed by Lord Voldemort. Your mother was his first victim. Your father followed shortly after' and sometimes it's a lot of rubbish or gets the rules of the world completely wrong. Mine had a long conversation about movies and detentions involving various assorted Harry Potter characters, a dream sequence involving a giant girl and a wizard called Bruno explaining that last time he saw Harry's lowly hedge-witch mother she was terrified Harry would send her to the gallows. But then, I'm willing to believe actual Harry Potter fiction is nearly as inconsistent
    - the_af 6y ago
      Haha, it reads like deranged fanfiction! I tried your prompt and it got surreal and angsty fast (everything but the prompt is AI Dungeon): > You are Harry Potter. You return to Hogwarts. The year is 1997. The world has changed, and you have not been able to adapt. Your life as it once was is gone forever. "I don't want to die…" you whisper to yourself.
  - TeMPOraL 6y ago
    > it can also be Star Trek-like Ok, here we go. I put this prompt in AIDungeon: $ You are Ted, an ensign in Starfleet. You work as a holodeck technician. You test this AI-generated world in order to determine its limits. You start the simulation with a program of a beautiful beach. As you enter the holodeck, I kept letting AI to write freely (except I had to press a Retry button a couple times to stop it from spinning a murder mystery or lewd story). The following is the response of the model: > you see that it is indeed a beautiful place. There are palm trees and sand dunes stretching out into infinity. The sky is blue, the ocean green and there are people everywhere. > "This is paradise," you think to yourself. "I could spend my life here." > Then something happens... > The simulation ends abruptly, and you find yourself back on the bridge of your ship. A voice suddenly speaks up from behind you: "Captain, we have detected a subspace disturbance coming from the direction of planet Earth." It is the voice of your helmsman, Ensign Thompson. > "Earth?" you ask. "That would be a week away at warp speed!" ... That sounds Star Trek enough to me.
andreyk 6y ago
Quite cool! The explanation is a bit vague, did your prompts to it include the description of the ideas in the next paragraph (eg superintelligence coming about, creating sadness, etc.) or was some of it invented by GPT-3? Cool demo for sudowrite, looking forward to trying it out myself sometime, as someone with way too many story ideas and not enough time...
- jamesjyu 6y ago
  Yes, I typically (but not always) included a high level summary of the story in the prompt. And sometimes, when I knew what kind of story turn I wanted (ominous, funny, extraordinary) I would prompt it with something like “and then something ominous happened:”
Imnimo 6y ago
What is the prompt that's used for the side chats? Is it the entire story up to that point, plus "Author: <question> GPT-3:"?
- jamesjyu 6y ago
  I gave it a summary of the story up to that point, as well as the passage verbatim, then something like: “This is a conversation between the author and the superintelligent AI in the story. The AI is [description based on context]:“
jessaustin 6y ago
In this context, that's an interesting use of a fairly rare verb, "provocate".
M277 6y ago
This is really, really interesting. Great job! As an amateur writer who is interesting in playing with this -- any tips on how to get started? I tried the AI Playground website, and fed it a premise, but it constantly decides to "go off the rails" so as to speak. I suspect that I am using it wrong, but I am not sure. It has really great, great potential. I see word processors for writers coming with some sort of AI integrated in the future, sort of like some of Photoshop's tools. This may be a good idea for a project, actually haha.
- jamesjyu 6y ago
  You should come into it having a premise for the story laid out. GPT-3 is great at suggesting possibilities at the paragraph level, but beyond that, it'll lose the thread. The simplest thing is to go to it when you get stuck. You sort of know what the scene should be doing, but maybe you're having trouble connecting to the end. Give it a whirl in playground and see if it sparks an idea—give it the last few paragraphs and see what it comes up with. At the macro level, I've been able to get GPT-3 to generate plot possibilities by giving it example scenes or endings. It's all about prompt engineering, and you'll need to spend a lot of time in the playground or writing your own app to take full advantage.
  - M277 6y ago
    I see, thanks a lot! Will definitely experiment with it more.
m463 6y ago
author or... co-author ? maybe there's room for another name here. or maybe not. Do you say "co-authored by word spellcheck? grammatical checker?"
ianhorn 6y ago
This is cool! I've been playing around with GPT-3 a bunch lately with the same goal in mind. I spent too much of the afternoon feeding it a few 7-point plot examples like [0] and then brainstorming my own with it. It was incredibly fun, and GPT-3 is surprisingly good at casting a story. Feed it "Main characters:" or "Eamon's character arc:" and comes up with interesting examples. It takes a bunch of tries, but it's 1000% enough to spark creativity. I haven't tried getting it to do prose, since the 2048 token limit is so small. I'm excited for the new O(n) transformers to remove that limitation. I'd love to hear more about the kinds of prompts you're using. [0] https://www.helpingwritersbecomeauthors.com/book-storystructure/warbreaker/ https://www.helpingwritersbecomeauthors.com/book-storystruct...
pistachiopro 6y ago
Really enjoyed the stories (the third one, especially), thanks for posting them. It's exciting to see a writer embrace this technology and use it in such an effective way. Are there any other GPT-3 "collaborations" you'd recommend?
- jamesjyu 6y ago
  Gwern has a huge rundown: https://www.gwern.net/GPT-3 https://www.gwern.net/GPT-3 And I like what Arram has been producing: https://arr.am/2020/07/14/elon-musk-by-dr-seuss-gpt-3/ https://arr.am/2020/07/14/elon-musk-by-dr-seuss-gpt-3/
renjimen 6y ago
Thanks for the great read! I enjoy your style and I can see you’re also a big fan of Ted Chiang and Ursula Le Guin. Are there any other authors you’d recommend who’ve been big influences for you?
- jamesjyu 6y ago
  Thanks! Here are some of my fav reads: https://www.jamesyu.org/reading/ https://www.jamesyu.org/reading/
  - renjimen 6y ago
    Ah great! Not sure how I missed that. I've added a few to my reading list, thanks.
msaltz 6y ago
This is fascinating! For the side dialogues, did you do run the generator several times until it came up with something that sounded relevant/reasonable, or were you able to really chat in that manner and what you have here is the first thing it generated?

castlecrasher2 6y ago

If I'm interested in tinkering with this, how do I get started?

2bitencryption 6y ago

fun tip - apparently there's enough Harry Potter fanfiction out there that GPT-2/3 is quite good at understanding that universe. So if you go into AI Dungeon and pick "custom story", and give it a prompt that indicates you are at Hogwarts, it will pretty much adapt all the rules of the HP universe. The downside is, when you do this, GPT's intelligence drops to about the level of a Harry Potter fanfic writer.

jamesjyu 6y ago
Yeah, GPT-3 contains essentially all the historical internet up to October 2019 and many books, you can easily trigger it to enter a "known" world. But what's more interesting to me as a fiction writer is to be able to prime it to enter a new world that I create.
geoelectric 6y ago
So if I prompt it with sexy Spock, do I get TOS slash stories? There are a few genres like that which have significant internet (and only internet) presence. How about creepypasta?
- cosmojg 6y ago
  Yes and yes. GPT-2 is pretty capable of this as well. You can try it yourself here: https://bellard.org/textsynth/ https://bellard.org/textsynth/
- TeMPOraL 6y ago
  > So if I prompt it with sexy Spock, do I get TOS slash stories? Most likely, yes. See my comment elsewhere in the thread: I recently prompted AIDungeon with a mention of a holodeck, and among other things, the game introduced - by itself - a PADD, turbolifts, Andorians, and was able to correctly guess what happens if you shoot yourself in your face with a phaser on stun, with holodeck safeties enabled ("Nothing happens.") and disabled ("You are knocked unconscious by the blast."). It's honestly the first time for me that I'm actually impressed with deep neural networks.
smusamashah 6y ago
I tried this. Took role of Harry Potter. Mentioned Voldemort, Hagrid, his pet bird. It didn't take me into the story I was desperately hoping for.
- zzbzq 6y ago
  AI dungeon is like a bad dream the way it goes around doesn't get anywhere. Similar to all the conversations in the OP's story. It makes you realize even with an AI this good, we've got a way to go before it can really do the things we want.
notahacker 6y ago
It's amazing just the sort of Harry Potter fanfic that's out there... Prompt: You look for Hermione. Response: You see her snogging a non-magical boy. His head is buried in her cleavage, which are being covered by her loose gown. You shake your head and walk away. This is actually a better indication of how derivative GPT-3 is than some of my experiments with higher quality source material...
the_af 6y ago
Probably because I'm not a native English speaker, I failed miserably at this. I tried giving AI Dungeon a prompt that mentioned Hogwarts, Voldemort, the Sorting Hat, dark wizards, etc, and it went nowhere (it seemed obsessed with telling me about a "voice inside your mind", go figure). Can you share a prompt that will make it answer with something Harry Potter-like? (it can also be Star Trek-like, or anything from pop culture. I just want to see why my prompt didn't work)
- notahacker 6y ago
  I started off with 'You are Harry Potter. You return to Hogwarts'. It's designed so everything is written in second person like that, and for the most part you can just hit enter, giving it another action involving a character or place if it gets stuck or goes miles off topic Sometimes the responses are vaguely on topic, like 'The memories of your parents' death haunt you every day. They were killed by Lord Voldemort. Your mother was his first victim. Your father followed shortly after' and sometimes it's a lot of rubbish or gets the rules of the world completely wrong. Mine had a long conversation about movies and detentions involving various assorted Harry Potter characters, a dream sequence involving a giant girl and a wizard called Bruno explaining that last time he saw Harry's lowly hedge-witch mother she was terrified Harry would send her to the gallows. But then, I'm willing to believe actual Harry Potter fiction is nearly as inconsistent
  - the_af 6y ago
    Haha, it reads like deranged fanfiction! I tried your prompt and it got surreal and angsty fast (everything but the prompt is AI Dungeon): > You are Harry Potter. You return to Hogwarts. The year is 1997. The world has changed, and you have not been able to adapt. Your life as it once was is gone forever. "I don't want to die…" you whisper to yourself.
- TeMPOraL 6y ago
  > it can also be Star Trek-like Ok, here we go. I put this prompt in AIDungeon: $ You are Ted, an ensign in Starfleet. You work as a holodeck technician. You test this AI-generated world in order to determine its limits. You start the simulation with a program of a beautiful beach. As you enter the holodeck, I kept letting AI to write freely (except I had to press a Retry button a couple times to stop it from spinning a murder mystery or lewd story). The following is the response of the model: > you see that it is indeed a beautiful place. There are palm trees and sand dunes stretching out into infinity. The sky is blue, the ocean green and there are people everywhere. > "This is paradise," you think to yourself. "I could spend my life here." > Then something happens... > The simulation ends abruptly, and you find yourself back on the bridge of your ship. A voice suddenly speaks up from behind you: "Captain, we have detected a subspace disturbance coming from the direction of planet Earth." It is the voice of your helmsman, Ensign Thompson. > "Earth?" you ask. "That would be a week away at warp speed!" ... That sounds Star Trek enough to me.

andreyk 6y ago

Quite cool! The explanation is a bit vague, did your prompts to it include the description of the ideas in the next paragraph (eg superintelligence coming about, creating sadness, etc.) or was some of it invented by GPT-3? Cool demo for sudowrite, looking forward to trying it out myself sometime, as someone with way too many story ideas and not enough time...

jamesjyu 6y ago
Yes, I typically (but not always) included a high level summary of the story in the prompt. And sometimes, when I knew what kind of story turn I wanted (ominous, funny, extraordinary) I would prompt it with something like “and then something ominous happened:”

Imnimo 6y ago

What is the prompt that's used for the side chats? Is it the entire story up to that point, plus "Author: <question> GPT-3:"?

jamesjyu 6y ago
I gave it a summary of the story up to that point, as well as the passage verbatim, then something like: “This is a conversation between the author and the superintelligent AI in the story. The AI is [description based on context]:“

jessaustin 6y ago

In this context, that's an interesting use of a fairly rare verb, "provocate".

M277 6y ago

This is really, really interesting. Great job! As an amateur writer who is interesting in playing with this -- any tips on how to get started? I tried the AI Playground website, and fed it a premise, but it constantly decides to "go off the rails" so as to speak. I suspect that I am using it wrong, but I am not sure. It has really great, great potential. I see word processors for writers coming with some sort of AI integrated in the future, sort of like some of Photoshop's tools. This may be a good idea for a project, actually haha.

jamesjyu 6y ago
You should come into it having a premise for the story laid out. GPT-3 is great at suggesting possibilities at the paragraph level, but beyond that, it'll lose the thread. The simplest thing is to go to it when you get stuck. You sort of know what the scene should be doing, but maybe you're having trouble connecting to the end. Give it a whirl in playground and see if it sparks an idea—give it the last few paragraphs and see what it comes up with. At the macro level, I've been able to get GPT-3 to generate plot possibilities by giving it example scenes or endings. It's all about prompt engineering, and you'll need to spend a lot of time in the playground or writing your own app to take full advantage.
- M277 6y ago
  I see, thanks a lot! Will definitely experiment with it more.

m463 6y ago

author or... co-author ? maybe there's room for another name here. or maybe not. Do you say "co-authored by word spellcheck? grammatical checker?"

ianhorn 6y ago

This is cool! I've been playing around with GPT-3 a bunch lately with the same goal in mind. I spent too much of the afternoon feeding it a few 7-point plot examples like [0] and then brainstorming my own with it. It was incredibly fun, and GPT-3 is surprisingly good at casting a story. Feed it "Main characters:" or "Eamon's character arc:" and comes up with interesting examples. It takes a bunch of tries, but it's 1000% enough to spark creativity. I haven't tried getting it to do prose, since the 2048 token limit is so small. I'm excited for the new O(n) transformers to remove that limitation. I'd love to hear more about the kinds of prompts you're using. [0] https://www.helpingwritersbecomeauthors.com/book-storystructure/warbreaker/ https://www.helpingwritersbecomeauthors.com/book-storystruct...

pistachiopro 6y ago

Really enjoyed the stories (the third one, especially), thanks for posting them. It's exciting to see a writer embrace this technology and use it in such an effective way. Are there any other GPT-3 "collaborations" you'd recommend?

jamesjyu 6y ago
Gwern has a huge rundown: https://www.gwern.net/GPT-3 https://www.gwern.net/GPT-3 And I like what Arram has been producing: https://arr.am/2020/07/14/elon-musk-by-dr-seuss-gpt-3/ https://arr.am/2020/07/14/elon-musk-by-dr-seuss-gpt-3/

renjimen 6y ago

Thanks for the great read! I enjoy your style and I can see you’re also a big fan of Ted Chiang and Ursula Le Guin. Are there any other authors you’d recommend who’ve been big influences for you?

jamesjyu 6y ago
Thanks! Here are some of my fav reads: https://www.jamesyu.org/reading/ https://www.jamesyu.org/reading/
- renjimen 6y ago
  Ah great! Not sure how I missed that. I've added a few to my reading list, thanks.

msaltz 6y ago

This is fascinating! For the side dialogues, did you do run the generator several times until it came up with something that sounded relevant/reasonable, or were you able to really chat in that manner and what you have here is the first thing it generated?