Most AI runners just ignore licensing and (illegally?) run LLaMA finetunes. There are tons of good 33B/65B instruct models you can run on any decent PC. But if

Most AI runners just ignore licensing and (illegally?) run LLaMA finetunes. There are tons of good 33B/65B instruct models you can run on any decent PC. But if you want to avoid the non commercial LLaMA license, you have 3 good options for a base model. - OpenLlama 13B - MPT 30B - Falcon 40B Of these, Falcon 40B is very difficult to run (slow in 4 bit, basically requires a professional GPU, no good cpu offloading yet). OpenLLaMA 13B only supports a context size of 2048 as of today... But that could change soon. So you probably want MPT instruct 30B, specifically this one: https://huggingface.co/TheBloke/mpt-30B-instruct-GGML https://huggingface.co/TheBloke/mpt-30B-instruct-GGML As the page says, you can try it out on a decent PC of your own with the OpenCL build of KoboldCPP. Change it to "instruct" mode, use the template on the page, offload as many layers as you can to your PC's dGPU, and run it in instruct mode. It may already work for your summarization needs. If not, you can finetune it with MPT's code and summarization data. https://github.com/mosaicml/llm-foundry https://github.com/mosaicml/llm-foundry Or train OpenLLaMA 13B with SuperHOT + summarization data using QLORA.