Yeah, if you only want coherent language, 1B parameters is more or less good enough. If you also want general context awareness and some basic world knowledge,

Yeah, if you only want coherent language, 1B parameters is more or less good enough. If you also want general context awareness and some basic world knowledge, you will need to up it another order of magnitude. Either way, this is nothing that you'd run on dated hardware. Smaller transformers can be useful for very specific task, but autoregressive LMs are just insane when it comes to hardware requirements.