I'm also working a lot in gpt3/j story generation. One approach I've found that addresses some long-term coherence problems is to generate high-level, bulleted

I'm also working a lot in gpt3/j story generation. One approach I've found that addresses some long-term coherence problems is to generate high-level, bulleted outlines upfront, then use those bullets (plus some metadata from previous/next bullets) as prompts to "expand" into longer, more-standalone scenes. Of course, there's a lot of problems with this approach also, but it does help specifically with consistency across larger generated works.