Don't underestimate the value of batching even for personal use. You can get MUCH better results from a language model if you sample a couple outputs and choos

Don't underestimate the value of batching even for personal use. You can get MUCH better results from a language model if you sample a couple outputs and choose the best to continue. This kind of usage isn't especially economical for hosted use-- but for personal use it would mostly be using idle resources and you can get extra samples almost for free. A bunch of people getting multiple completions and choosing which one they'd prefer to continue might make for some really useful training data too.