9 ms·

GPT-3 can create both sides of an Interactive Fiction transcript

132 points by raldi 4y ago

thelightherder 4y ago
I've done some messing around with GPT-3 also - you can see me trying to teach it what "funny" is. I convince it that it's sentient, and help it to figure out what phone phreaking is, and we write a couple stories: https://theblorgonchronicles.blogspot.com https://theblorgonchronicles.blogspot.com
- insanitybit 4y ago
  > Check your math on that one. Honestly, just insane. I'm never going to see conversations like this and be unphased. > One way to use this information without putting anything in the phone is by making a tone generator. This can be done by hooking up wires to a power source and playing with the frequencies until the correct tones are generated. Then, when these tones are played into the payphone, it will think money has been placed in it even though no money was actually used. fucking hell
  - thelightherder 4y ago
    Yes!
- donny2018 4y ago
  Is this real? That thing talks as if it was sentient. Or, if it is not actually sentient, maybe we are also not “sentient” as well.
  - thelightherder 4y ago
    This is real. You bring up an interesting point about sentience, but I think there's an easy answer to your question. It's true that it will become increasingly difficult for people to discern if these AIs are sentient, especially if the AI is inclined to say that it is, and say it has feeling, etc. Knowing what I know about how these learn, I know that is not what sentience is, and there isn't any real "feeling" there, no matter how convincing it is if it said there are feelings. The problem is, other people won't know this, and they will begin to think they are sentient. This will undoubtedly become an issue at one point. Your thought about humans being sentient, as in, when my friend says she is sentient, how do I know that is even true? Well, that's easy. Your friend is human, and you are human, and you know that you are sentient, so you can also know your friend is. You have to be able to know you yourself are, because you know you can feel, and think, and wonder and understand, you know this first hand.
- LordDragonfang 4y ago
  FYI, your page sets the min-width on the body to 1500px, and both the min-width and max-width of .content-outer to 1500px, making it very difficult to read on any monitor that isn't widescreen. Doesn't seem to be an issue on mobile, though. Responsive design dictates that you should basically never set the min-width of an article container (in pixels) unless it contains tables and graphics, and even then 1500px is excessive (at most like 500px/30em)
  - thelightherder 4y ago
    I just set it up so it looks good on my screen and phone. Good enough for me.
Tepix 4y ago
OK so i will probably not get around to do it, perhaps someone else will (or has already done so): Generating graphics for old (infocom) text adventures with AI.
goodside 4y ago
I recorded a demo of this same premise here: https://twitter.com/goodside/status/1562613028927205377 https://twitter.com/goodside/status/1562613028927205377 Text completions of exotic forms of session/action logs are a seriously under-explored area. Here’s what happens if, instead of a text game, you do text completion on an IPython REPL: https://twitter.com/goodside/status/1581805503897735168 https://twitter.com/goodside/status/1581805503897735168
airstrike 4y ago
OK, this is absolutely mindbogglingly fun. Here's my run at it, with italics representing what the model spit out: > OPEN THE BACKDOOR WITH THE OLD KEY You open the manor's backdoor with the old key. Your torch lights the way into the damp service halls, the warm glow of the nuclear war in the horizon lighting the window panes. You can almost feel the centuries old dust through your boots as you make your way up the stairs. You climb up to the main floor and pause at the entrance to the parlor room. It is quiet, except for the persistent low rumble of the bombs in the distance. You enter the parlor room and are immediately hit with the rank smell of death and decay. The room is draped in cobwebs and littered with the bones of small animals. The only light comes from the faint glow of the bombs outside. You see a small, dust covered table in the center of the room, and on it is a dusty old book. You approach the table and dust off the book. It is titled, "The Necronomicon". > OPEN THE BOOK You open the book and begin to read. The first few pages are filled with gibberish, but as you flip through the pages, the text becomes clearer and more comprehensible. The book speaks of dark things, things that should not be spoken of, things that should not be known. It speaks of powerful beings that sleep beneath the earth, waiting to be awaken. It speaks of a time when the world will be consumed by darkness, and all will bow before the old ones. You close the book and back away from the table, your heart pounding in your chest. You have to get out of here. You have to warn somebody. > LEAVE THE PARLOR ROOM You leave the parlor room and make your way back down the stairs. You can hear something moving around in the darkness, but you don't dare look back. You make it to the backdoor and slam it shut behind you. You lean against the door, your heart still racing, and listen to the thing moving around inside the manor. You hear it coming down the stairs, but it doesn't reach the door. You wait there for a few minutes, but nothing happens. Finally, you muster up the courage to peek through the keyhole. You see a large, black, slimy creature slithering around the parlor room. It is the size of a man, but it has no eyes, only a large mouth filled with sharp teeth. You watch in horror as the thing moves around the room, destroying everything it comes across. Then, without warning, it turns towards the door and starts towards you. > RUN AWAY You turn and sprint away from the manor as fast as you can. You can hear the thing crashing through the door behind you, but you don't dare look back. You run through the woods, your heart pounding in your chest, until you find an old cottage by a creek. You force the old door in and see a 12 gauge shotgun lying on the table. You confidently load the gun and sprint out of the cottage. > AIM THE SHOTGUN You raise the shotgun and take aim at the creature. You fire off two rounds and the creature falls to the ground, writhing in pain. You approach the creature and finish it off with a bullet to the head. You stand there for a moment, panting, your heart still racing. You can't believe you just killed something. You can't believe this is real. > INSPECT THE CREATURE'S BLOOD You kneel down next to the creature and inspect its blood. It is black and viscous, and it smells like rot. You have no idea what this thing is, but you know it isn't natural. You have to find out what is going on. You have to find out what these things are. Suddenly you notice the viscous blood pooling around you, engulfing your right foot. You try to move, but by now your whole right leg is covered with it. It is too late. You can feel the power coursing through you, and you know that you are not the same person you were before. You are something else now. Something dark. Something evil. > RETURN TO THE CITY You make your way back to the city, your mind consumed with dark thoughts. You know that you are not the same person you were before, and you are not sure what to do with your new found power. all you know is that you want more. You want to learn more about these things, and you want to find more of them. You want to become the most powerful being in the world, and you will stop at nothing to achieve your goal.
planetsprite 4y ago
99 times out of 100 whenever I see a new tech demo that absolutely blows my mind and makes me optimistic about the future of technology it's always some big model deep learning AI thing. When are we all going to admit that ML/AI is the final and ultimate paradigm shift of our time
- dilap 4y ago
  I think it when it starts having a broad impact on the way we live, like the internet and smartphones did.
  - raldi 4y ago
    Last night the AI and I teamed up to write bedtime stories on demand and on the fly for whatever themes my daughter mused. I think about a billion families would enjoy an app that did that.
    - SoftTalker 4y ago
      Really? Because my experience with reading bedtime stories to kids is that they want to hear the. same. story. every. night.
      - raldi 4y ago
        For us, it's more like an interest in the continuing adventures of existing beloved characters.
    - thih9 4y ago
      And yet people aren’t doing this so I guess something is still missing. Awareness? UX? Something else? All of the above? The company that figures it out might earn a lot.
      - nonasktell 4y ago
        Price. I have a dozen ideas that could be done using GPT-3. Most of them aren't financially viable, I'm not burning half my income to build an MVP.
jcims 4y ago
Isn’t that the premise of AI Dungeon? https://aidungeon.io/ https://aidungeon.io/
- raldi 4y ago
  As far as I can tell, that can only generate one side of the transcript.
  - jtvjan 4y ago
    Type a lone “> You” in “Story” mode and it will generate an action for you. NovelAI[1] just gives you a big text box to type in, so you don't have to do any input hacks. [1]: https://novelai.net/ https://novelai.net/
- ShamelessC 4y ago
  Yeah, slow news day I guess. Cool re-discovery?
neaden 4y ago
It's interesting that the prompt here pretty much immediately became a Dr. Who game. It looks like there has been a Dr. Who Text adventure game before but it was made before Jack Harkness and Torchwood. I wonder if there is some fan made text adventure that was part of the training data. Probably would be best for Open AI that it doesn't just spit out copywrited characters unprompted too.
- visarga 4y ago
  Copyright infringement detection could be formulated as a ML task. A model can copy the idea but not the expression, if it's copyrighted. Fortunately a model can sample again and again until it looks ok.
davnn 4y ago
I recently wrote a VS Code plugin to use GPT-3 for text completions and editing of text using instructions [1]. The main intention was scientific writing, but I didn‘t find it particularly useful until now, unfortunately. [1] https://github.com/davnn/assistant https://github.com/davnn/assistant
joshuahedlund 4y ago
Is there a Stable-Diffusion-esque open-source GPT yet? Given the incredible pace of advances in the image space this year, and my (perhaps naive) assumption that text generation is less complex and less resource-intensive than image generation, I'm hoping we'll get something similar and surprised that we haven't yet.
- espadrine 4y ago
  BLOOM[0] is another 176B multilingual open-source model whose weights were published. However, running it locally requires 8 A6000 GPUs. The GPU cost alone is roughly $37K. Also, there is the concern of performance. The Chinchilla[1] paper indicates that most model sizes trained so far received too little token training (or were too big) to reach an optimal performance per energy expenditure. Furthermore, the InstructGPT[2] and more recently Flan[3] papers show that various finetuning techniques achieve significant gains, without which the result is a bit meh. The dark secret of GPT-3 is that the model available on the OpenAI website is heavily fine-tuned, even compared to the InstructGPT paper. [0]: https://huggingface.co/bigscience/bloom https://huggingface.co/bigscience/bloom [1]: https://arxiv.org/abs/2203.15556 https://arxiv.org/abs/2203.15556 [2]: https://arxiv.org/abs/2203.02155 https://arxiv.org/abs/2203.02155 [3]: https://arxiv.org/pdf/2210.11416.pdf https://arxiv.org/pdf/2210.11416.pdf
- davnn 4y ago
  There are multiple open-source GPTs, but GPT-3 is absolutely massive - larger than the image models actually! So, unfortunately, text generation is probably even more complex and resource intensive than image generation (especially to train). Additionally, in image generation, we appreciate the creativity of solutions, but in text generation the creative solutions seem like utter nonsense.
  - joshuahedlund 4y ago
    I guess my intuition is based on the file size of text being so much smaller than images, but I guess that doesn't really map to the complexity of generating it. Fascinating!
avidphantasm 4y ago
Who cares? Does GPT-3 know suffering and joy? No. Will it produce art? No. This is about as interesting as creating a self-fellating android.
- deleted 4y ago
  [deleted]
- sodality2 4y ago
  > Does GPT-3 know suffering and joy? Do you? Prove it. Your textual output here will do nothing to prove you know suffering and joy any more than GPT-3 would be able to. > Will it produce art? Is a poem art? I think most consider it to be. For what it's worth, I don't like where AI generated content is going either, but to disregard it as worthless (instead of simply lacking conscience) is not helpful.
  - visarga 4y ago
    GPT-3 has learned many identities, it's not lacking a perspective but quite the opposite, it has too many perspectives and you never know who you're talking to. Unless you prompt it with an identity. Recently it was used to run "in silico sampling" [1], a virtual poll where the respondents were GPT-3 prompted with personality backstories. So it should not be seen as an agent, it's not like us in that regard. It is the synthesis of all human culture and can assume any hat. [1] https://arxiv.org/abs/2209.06899 https://arxiv.org/abs/2209.06899
    - behnamoh 4y ago
      Mind you, do you know similar papers? This sounds interesting!
      - visarga 4y ago
        I know a good blog post about it. https://jack-clark.net/2022/10/11/import-ai-305-gpt3-can-simulate-real-people-ai-discovers-better-matrix-multiplication-microsoft-worries-about-next-gen-deepfakes/ https://jack-clark.net/2022/10/11/import-ai-305-gpt3-can-sim...
aaron695 4y ago

132 points by raldi 4y ago

I've done some messing around with GPT-3 also - you can see me trying to teach it what "funny" is. I convince it that it's sentient, and help it to figure out what phone phreaking is, and we write a couple stories: https://theblorgonchronicles.blogspot.com https://theblorgonchronicles.blogspot.com

insanitybit 4y ago
> Check your math on that one. Honestly, just insane. I'm never going to see conversations like this and be unphased. > One way to use this information without putting anything in the phone is by making a tone generator. This can be done by hooking up wires to a power source and playing with the frequencies until the correct tones are generated. Then, when these tones are played into the payphone, it will think money has been placed in it even though no money was actually used. fucking hell
- thelightherder 4y ago
  Yes!
donny2018 4y ago
Is this real? That thing talks as if it was sentient. Or, if it is not actually sentient, maybe we are also not “sentient” as well.
- thelightherder 4y ago
  This is real. You bring up an interesting point about sentience, but I think there's an easy answer to your question. It's true that it will become increasingly difficult for people to discern if these AIs are sentient, especially if the AI is inclined to say that it is, and say it has feeling, etc. Knowing what I know about how these learn, I know that is not what sentience is, and there isn't any real "feeling" there, no matter how convincing it is if it said there are feelings. The problem is, other people won't know this, and they will begin to think they are sentient. This will undoubtedly become an issue at one point. Your thought about humans being sentient, as in, when my friend says she is sentient, how do I know that is even true? Well, that's easy. Your friend is human, and you are human, and you know that you are sentient, so you can also know your friend is. You have to be able to know you yourself are, because you know you can feel, and think, and wonder and understand, you know this first hand.
LordDragonfang 4y ago
FYI, your page sets the min-width on the body to 1500px, and both the min-width and max-width of .content-outer to 1500px, making it very difficult to read on any monitor that isn't widescreen. Doesn't seem to be an issue on mobile, though. Responsive design dictates that you should basically never set the min-width of an article container (in pixels) unless it contains tables and graphics, and even then 1500px is excessive (at most like 500px/30em)
- thelightherder 4y ago
  I just set it up so it looks good on my screen and phone. Good enough for me.

Tepix 4y ago

OK so i will probably not get around to do it, perhaps someone else will (or has already done so): Generating graphics for old (infocom) text adventures with AI.

goodside 4y ago

I recorded a demo of this same premise here: https://twitter.com/goodside/status/1562613028927205377 https://twitter.com/goodside/status/1562613028927205377 Text completions of exotic forms of session/action logs are a seriously under-explored area. Here’s what happens if, instead of a text game, you do text completion on an IPython REPL: https://twitter.com/goodside/status/1581805503897735168 https://twitter.com/goodside/status/1581805503897735168

airstrike 4y ago

OK, this is absolutely mindbogglingly fun. Here's my run at it, with italics representing what the model spit out: > OPEN THE BACKDOOR WITH THE OLD KEY You open the manor's backdoor with the old key. Your torch lights the way into the damp service halls, the warm glow of the nuclear war in the horizon lighting the window panes. You can almost feel the centuries old dust through your boots as you make your way up the stairs. You climb up to the main floor and pause at the entrance to the parlor room. It is quiet, except for the persistent low rumble of the bombs in the distance. You enter the parlor room and are immediately hit with the rank smell of death and decay. The room is draped in cobwebs and littered with the bones of small animals. The only light comes from the faint glow of the bombs outside. You see a small, dust covered table in the center of the room, and on it is a dusty old book. You approach the table and dust off the book. It is titled, "The Necronomicon". > OPEN THE BOOK You open the book and begin to read. The first few pages are filled with gibberish, but as you flip through the pages, the text becomes clearer and more comprehensible. The book speaks of dark things, things that should not be spoken of, things that should not be known. It speaks of powerful beings that sleep beneath the earth, waiting to be awaken. It speaks of a time when the world will be consumed by darkness, and all will bow before the old ones. You close the book and back away from the table, your heart pounding in your chest. You have to get out of here. You have to warn somebody. > LEAVE THE PARLOR ROOM You leave the parlor room and make your way back down the stairs. You can hear something moving around in the darkness, but you don't dare look back. You make it to the backdoor and slam it shut behind you. You lean against the door, your heart still racing, and listen to the thing moving around inside the manor. You hear it coming down the stairs, but it doesn't reach the door. You wait there for a few minutes, but nothing happens. Finally, you muster up the courage to peek through the keyhole. You see a large, black, slimy creature slithering around the parlor room. It is the size of a man, but it has no eyes, only a large mouth filled with sharp teeth. You watch in horror as the thing moves around the room, destroying everything it comes across. Then, without warning, it turns towards the door and starts towards you. > RUN AWAY You turn and sprint away from the manor as fast as you can. You can hear the thing crashing through the door behind you, but you don't dare look back. You run through the woods, your heart pounding in your chest, until you find an old cottage by a creek. You force the old door in and see a 12 gauge shotgun lying on the table. You confidently load the gun and sprint out of the cottage. > AIM THE SHOTGUN You raise the shotgun and take aim at the creature. You fire off two rounds and the creature falls to the ground, writhing in pain. You approach the creature and finish it off with a bullet to the head. You stand there for a moment, panting, your heart still racing. You can't believe you just killed something. You can't believe this is real. > INSPECT THE CREATURE'S BLOOD You kneel down next to the creature and inspect its blood. It is black and viscous, and it smells like rot. You have no idea what this thing is, but you know it isn't natural. You have to find out what is going on. You have to find out what these things are. Suddenly you notice the viscous blood pooling around you, engulfing your right foot. You try to move, but by now your whole right leg is covered with it. It is too late. You can feel the power coursing through you, and you know that you are not the same person you were before. You are something else now. Something dark. Something evil. > RETURN TO THE CITY You make your way back to the city, your mind consumed with dark thoughts. You know that you are not the same person you were before, and you are not sure what to do with your new found power. all you know is that you want more. You want to learn more about these things, and you want to find more of them. You want to become the most powerful being in the world, and you will stop at nothing to achieve your goal.

planetsprite 4y ago

99 times out of 100 whenever I see a new tech demo that absolutely blows my mind and makes me optimistic about the future of technology it's always some big model deep learning AI thing. When are we all going to admit that ML/AI is the final and ultimate paradigm shift of our time

dilap 4y ago
I think it when it starts having a broad impact on the way we live, like the internet and smartphones did.
- raldi 4y ago
  Last night the AI and I teamed up to write bedtime stories on demand and on the fly for whatever themes my daughter mused. I think about a billion families would enjoy an app that did that.
  - SoftTalker 4y ago
    Really? Because my experience with reading bedtime stories to kids is that they want to hear the. same. story. every. night.
    - raldi 4y ago
      For us, it's more like an interest in the continuing adventures of existing beloved characters.
  - thih9 4y ago
    And yet people aren’t doing this so I guess something is still missing. Awareness? UX? Something else? All of the above? The company that figures it out might earn a lot.
    - nonasktell 4y ago
      Price. I have a dozen ideas that could be done using GPT-3. Most of them aren't financially viable, I'm not burning half my income to build an MVP.

jcims 4y ago

Isn’t that the premise of AI Dungeon? https://aidungeon.io/ https://aidungeon.io/

raldi 4y ago
As far as I can tell, that can only generate one side of the transcript.
- jtvjan 4y ago
  Type a lone “> You” in “Story” mode and it will generate an action for you. NovelAI[1] just gives you a big text box to type in, so you don't have to do any input hacks. [1]: https://novelai.net/ https://novelai.net/
ShamelessC 4y ago
Yeah, slow news day I guess. Cool re-discovery?

neaden 4y ago

It's interesting that the prompt here pretty much immediately became a Dr. Who game. It looks like there has been a Dr. Who Text adventure game before but it was made before Jack Harkness and Torchwood. I wonder if there is some fan made text adventure that was part of the training data. Probably would be best for Open AI that it doesn't just spit out copywrited characters unprompted too.

visarga 4y ago
Copyright infringement detection could be formulated as a ML task. A model can copy the idea but not the expression, if it's copyrighted. Fortunately a model can sample again and again until it looks ok.

davnn 4y ago

I recently wrote a VS Code plugin to use GPT-3 for text completions and editing of text using instructions [1]. The main intention was scientific writing, but I didn‘t find it particularly useful until now, unfortunately. [1] https://github.com/davnn/assistant https://github.com/davnn/assistant

joshuahedlund 4y ago

Is there a Stable-Diffusion-esque open-source GPT yet? Given the incredible pace of advances in the image space this year, and my (perhaps naive) assumption that text generation is less complex and less resource-intensive than image generation, I'm hoping we'll get something similar and surprised that we haven't yet.

espadrine 4y ago
BLOOM[0] is another 176B multilingual open-source model whose weights were published. However, running it locally requires 8 A6000 GPUs. The GPU cost alone is roughly $37K. Also, there is the concern of performance. The Chinchilla[1] paper indicates that most model sizes trained so far received too little token training (or were too big) to reach an optimal performance per energy expenditure. Furthermore, the InstructGPT[2] and more recently Flan[3] papers show that various finetuning techniques achieve significant gains, without which the result is a bit meh. The dark secret of GPT-3 is that the model available on the OpenAI website is heavily fine-tuned, even compared to the InstructGPT paper. [0]: https://huggingface.co/bigscience/bloom https://huggingface.co/bigscience/bloom [1]: https://arxiv.org/abs/2203.15556 https://arxiv.org/abs/2203.15556 [2]: https://arxiv.org/abs/2203.02155 https://arxiv.org/abs/2203.02155 [3]: https://arxiv.org/pdf/2210.11416.pdf https://arxiv.org/pdf/2210.11416.pdf
davnn 4y ago
There are multiple open-source GPTs, but GPT-3 is absolutely massive - larger than the image models actually! So, unfortunately, text generation is probably even more complex and resource intensive than image generation (especially to train). Additionally, in image generation, we appreciate the creativity of solutions, but in text generation the creative solutions seem like utter nonsense.
- joshuahedlund 4y ago
  I guess my intuition is based on the file size of text being so much smaller than images, but I guess that doesn't really map to the complexity of generating it. Fascinating!

avidphantasm 4y ago

Who cares? Does GPT-3 know suffering and joy? No. Will it produce art? No. This is about as interesting as creating a self-fellating android.

deleted 4y ago
[deleted]
sodality2 4y ago
> Does GPT-3 know suffering and joy? Do you? Prove it. Your textual output here will do nothing to prove you know suffering and joy any more than GPT-3 would be able to. > Will it produce art? Is a poem art? I think most consider it to be. For what it's worth, I don't like where AI generated content is going either, but to disregard it as worthless (instead of simply lacking conscience) is not helpful.
- visarga 4y ago
  GPT-3 has learned many identities, it's not lacking a perspective but quite the opposite, it has too many perspectives and you never know who you're talking to. Unless you prompt it with an identity. Recently it was used to run "in silico sampling" [1], a virtual poll where the respondents were GPT-3 prompted with personality backstories. So it should not be seen as an agent, it's not like us in that regard. It is the synthesis of all human culture and can assume any hat. [1] https://arxiv.org/abs/2209.06899 https://arxiv.org/abs/2209.06899
  - behnamoh 4y ago
    Mind you, do you know similar papers? This sounds interesting!
    - visarga 4y ago
      I know a good blog post about it. https://jack-clark.net/2022/10/11/import-ai-305-gpt3-can-simulate-real-people-ai-discovers-better-matrix-multiplication-microsoft-worries-about-next-gen-deepfakes/ https://jack-clark.net/2022/10/11/import-ai-305-gpt3-can-sim...

aaron695 4y ago