3 ms·

It sounds like we're using very similar setups for writing. :) I've primarily been using GPT-3 (and burning through millions of tokens) so I've been experiment

It sounds like we're using very similar setups for writing. :) I've primarily been using GPT-3 (and burning through millions of tokens) so I've been experimenting with GPT-J more lately and I've found it makes significantly more basic logic errors (e.g. mixing up pronouns, "forgetting" characters, using new characters), which makes me lean more towards constantly regenerating text versus revising it, and I feel like I'm backed into a box of having to babysit ~100-200 tokens at a time instead of generating significantly more at a time like I do with GPT-3. I also built a quick tool that lets me adjust how much context I'm using in the prompt and generate 2-3 side-by-side completions to pick and choose from (just to speed up the flow of `click best suggestion` --> `keep generating from there`), but I haven't integrated GPT-J yet since it just feels... lower quality (it feels similar to ~Babbage, IMO). But being comparatively* free, I'm still excited about GPT-J. Do you have any tips or processes you've found to make it spit out higher-quality text? 20k tokens at a time is quite a lot -- do you also have problems with winding paths / staying on a general "plot"? Would love to hear any suggestions you have, because I'd sure love to move off GPT-3 to something comparable in quality!

deleted 5y ago
[deleted]
zitterbewegung 5y ago
I didn't experiment that much to get the difference between Babbage and GPTJ but that sounds similar than what I experienced. I am not sure what your novel is going to look like but due to the limitations between GPT-3 / GPTJ etc.. I plan on it being something like a stream of consciousness. I thought of babysitting lower amounts of tokens but I was getting okay quality with a few thousand but when I looked into it I saw that in the middle usually there was some weird change. I know that there is a GPTJ setting (and I believe GPT-3 has this) where you can see the best of X tries. Clicking best suggestion sounds like a good idea of a tool but it might be better just to recursively store all outputs. I am interested in the tool you are using are you able to share it? The writing process I am using is that there was a YouTube reviewer that used a dataset of Light Novels and utilized a refined version of GPT-2 on that to create a title of some light novels and then basically cherry picked them / did analysis / had his friends do a Turing test. I would assume that using GPT-3 / GPTJ could do the same thing but doing a zero shot with a 10-20 token limitation. (EDIT GPT-3 using Davinci and the Two-sentence Horror Story would allow you to generate interacting story titles and then from there you can use GPT-3 / GPTJ to do the rest.) (also if anyone is interested in the novel I am writing email me at zitterbewegung at gmail dot com).
- drusepth 5y ago
  Thanks for the answer! Best of N is super useful. With GPT-3 (and probably GPT-J) you can actually pass an n parameter to match best_of and get back all of the N texts generated, which is how my little tool works (displaying them all to pick from, since the "best" one by GPT-3's standards isn't always the "best" one from my author POV). It's also nice to not feel like i'm throwing away tokens, since best_of=3 uses 3x the tokens whether you look at all the generations or not! Thanks for the details on the Light Novels process. I'll have to look into it and see if something like that works for me! The tool I wrote is at https://young-savannah-97958.herokuapp.com/ https://young-savannah-97958.herokuapp.com/ (source code at https://github.com/indentlabs/gpt-3-writer https://github.com/indentlabs/gpt-3-writer), but only works with GPT-3 right now. You just stick in your API key (which doesn't get stored/saved) and customize completion settings if you want, then trigger completions while writing with the button in the bottom-right. It's very rough around the edges since I just built it for my own use, have a ton on the to-do list, and haven't publicized it, but it could be helpful. :)
  - zitterbewegung 5y ago
    What I do is I take the first title then I make derivative titles based on the starting title. Like “write a story about cookies” then “write a story about chocolate chip titles”.

deleted 5y ago

[deleted]

zitterbewegung 5y ago

I didn't experiment that much to get the difference between Babbage and GPTJ but that sounds similar than what I experienced. I am not sure what your novel is going to look like but due to the limitations between GPT-3 / GPTJ etc.. I plan on it being something like a stream of consciousness. I thought of babysitting lower amounts of tokens but I was getting okay quality with a few thousand but when I looked into it I saw that in the middle usually there was some weird change. I know that there is a GPTJ setting (and I believe GPT-3 has this) where you can see the best of X tries. Clicking best suggestion sounds like a good idea of a tool but it might be better just to recursively store all outputs. I am interested in the tool you are using are you able to share it? The writing process I am using is that there was a YouTube reviewer that used a dataset of Light Novels and utilized a refined version of GPT-2 on that to create a title of some light novels and then basically cherry picked them / did analysis / had his friends do a Turing test. I would assume that using GPT-3 / GPTJ could do the same thing but doing a zero shot with a 10-20 token limitation. (EDIT GPT-3 using Davinci and the Two-sentence Horror Story would allow you to generate interacting story titles and then from there you can use GPT-3 / GPTJ to do the rest.) (also if anyone is interested in the novel I am writing email me at zitterbewegung at gmail dot com).

drusepth 5y ago
Thanks for the answer! Best of N is super useful. With GPT-3 (and probably GPT-J) you can actually pass an n parameter to match best_of and get back all of the N texts generated, which is how my little tool works (displaying them all to pick from, since the "best" one by GPT-3's standards isn't always the "best" one from my author POV). It's also nice to not feel like i'm throwing away tokens, since best_of=3 uses 3x the tokens whether you look at all the generations or not! Thanks for the details on the Light Novels process. I'll have to look into it and see if something like that works for me! The tool I wrote is at https://young-savannah-97958.herokuapp.com/ https://young-savannah-97958.herokuapp.com/ (source code at https://github.com/indentlabs/gpt-3-writer https://github.com/indentlabs/gpt-3-writer), but only works with GPT-3 right now. You just stick in your API key (which doesn't get stored/saved) and customize completion settings if you want, then trigger completions while writing with the button in the bottom-right. It's very rough around the edges since I just built it for my own use, have a ton on the to-do list, and haven't publicized it, but it could be helpful. :)
- zitterbewegung 5y ago
  What I do is I take the first title then I make derivative titles based on the starting title. Like “write a story about cookies” then “write a story about chocolate chip titles”.