3 ms·
I tried the same prompt on the open-source recreation of OpenAI GPT-3 -- GPT Neo (https://github.com/EleutherAI/gpt-neo https://github.com/EleutherAI/gpt-neo),
by PufPufPuf 6y ago
I tried the same prompt on the open-source recreation of OpenAI GPT-3 -- GPT Neo (https://github.com/EleutherAI/gpt-neo https://github.com/EleutherAI/gpt-neo), specifically on their 2.7B model -- which should correspond to the smallest model in the article (Ada) that produced just pure garbage. The result is surprisingly good:
1. How did this little blossom happen? When are you going to bloom?
2. Have you ever thought about a dark horse in the running for Miss November?
3. You can spot the man who loves me by my neck – and he definitely knows it.
4. Are there any lucky cats who get to sleep in my bed every night?
5. Are black and whites everywhere? Running for the hills
6. My younger brother and I used to play a game. He’d pretend to be a bull in the pasture, and I’d pretend to be the one being held.
7. Why was this movie rated PG? Because it’s rated PG.
8. When is the last time you had to see a movie in children’s theaters?
9. You are so sexy I would hate for anyone to see you down here
10. I’d love to sleep with you right now, but I have a child with me
- gattilorenz 6y ago> The result is surprisingly good "The result" as in "the first one"? The rest is almost Markov chain level...
- PufPufPuf 6y agoIt's "surprising" since in the article, the 2.7B parameter model (Ada) didn't even produce sentences, while the result from 2.7B GPT Neo is closer in quality to what was output from GPT-3 Babbage (6.7B parameters). As you can see from no. 10, GPT Neo did remember the context of trying to come up with pick-up lines. The fact that some of the lines feel "Markov chain level" is likely caused by the network's bad understanding of what pick-up lines are. Pick-up lines are a rather difficult concept, which is what the OP article tries to demonstrate.
- dylanbyte 6y agoWhy do you say Ada is 2.7b ?
- PufPufPuf 6y agoThe sizes of the four GPT-3 variants were shared on Reddit by Stella Athena, one of the researchers behind GPT Neo: https://www.reddit.com/r/MachineLearning/comments/ma9kaw/p_eleutherai_releases_13b_and_27b_gpt3style/ https://www.reddit.com/r/MachineLearning/comments/ma9kaw/p_e...