4 ms·
Do you mean the "internal monolog" in parentheses are added by you, and only the text after the parentheses are generated by GPT-3? Either way, it's kind of fun
by fredliu 6y ago
Do you mean the "internal monolog" in parentheses are added by you, and only the text after the parentheses are generated by GPT-3? Either way, it's kind of fun to think of it as kind of an experiment on the bicameral mind theory (or at least Westworld's interpretation it), with the "monolog" being provided by you, and GPT-3 responding to the "internal voice from "God". Maybe that's how GPT-4 can gain consciousness, or at least the appearance of gaining it :)
- thomasahle 6y agoI read it as the internal monolog (as well as the answers) are generated by GPT-3, but the prompts (list of questions + answers used to seed the conversation) had internal monologs written by OP. It is indeed pretty interesting to read.
- qwertox 6y agoI'm wondering this as well. If the content in the parentheses is given by a human to aid learning, I think it's a really interesting approach. If not, then what he has archived there is just scary.
- fredliu 6y agoLooks like it's the latter, which makes it even more impressive!
- blixt 6y agoI only wrote the "Q: ..." part – the answer, including monolog, is output generated by GPT-3. The way it works is you first give it a few sample lines, and those do include both the Q and A parts, and then GPT-3 generates consistent output based on the pattern of your samples. But all the answers/monologs in my comment were 100% GPT-3!
- fredliu 6y agoOMG, then it is even more impressive than I thought! There's no way GPT-3 knows the part in parentheses is supposed to be "the monolog", or even what a monolog is supposed to be. And yet based on the pattern between Q, A and the text inside and outside the parentheses, it can mimic the effect of answering the question with "monolog". It shows GPT-3's capability to capture something that's super "meta". I wonder what can it do, if we feed GPT-3 with the philosophical discussions between Socrates and his students. Could we essentially teach GPT-3 the dialectical discussion method, and creating a real Sage that we can converse with?!
- blixt 6y agoYeah it's very capable to follow patterns. Here are some more fun examples that I posted on my Twitter: https://twitter.com/blixt/status/1285274259170955265 https://twitter.com/blixt/status/1285274259170955265 The fact that it can take vague English and turn it into fully functional code or shell commands is quite amazing. It's not just that it finds the correct result from its training data, it can also tweak parts of the output to correctly match your free text query.
- fredliu 6y agoWhat amazes me is that all those super meta "patterns" are already embedded in GPT-3's giant model. As compared to GPT-2, where you'd need to have your own large dataset to fine-tune for a specific use case/ the pattern you want to support. For instance, we fine-tuned GPT-2 against all episodes of South Park's scripts, and it took a long time to fine-tune. It kinda succeeded generating a model that "speaks south park" with pretty decent character alignment. (https://www.soulreplica.com/brodown https://www.soulreplica.com/brodown). I'm wondering if GPT-3 can achieve the same effect with just a few lines of South Park script? Does GPT-3 already know how Cartman usually talks?
- JoeSmithson 6y agoWow. What did the training prompts look like?
- nothis 6y agoYo, GPT-3 is freaky!
- justinpombrio 6y ago> But all the answers/monologs in my comment were 100% GPT-3! Were they the first output, or did you run it multiple times? (And if so, about how many?)