Additionally, there here some long context 30B models on huggigface now. For instance: https://huggingface.co/TheBloke/GPlatty-30B-SuperHOT-8K-GGML https://hug

Additionally, there here some long context 30B models on huggigface now. For instance: https://huggingface.co/TheBloke/GPlatty-30B-SuperHOT-8K-GGML https://huggingface.co/TheBloke/GPlatty-30B-SuperHOT-8K-GGML https://huggingface.co/TheBloke/Vicuna-33B-1-3-SuperHOT-8K-GGML https://huggingface.co/TheBloke/Vicuna-33B-1-3-SuperHOT-8K-G... These will work with 8K context on even a modest system, thanks to Koboldcpp. 16K context is more exotic atm.