4 ms·
Are there models available that have been fine tuned in this way? I don't have much interest training on my own data but (like many I'm sure) would love a Vicun
by version_five 3y ago
Are there models available that have been fine tuned in this way? I don't have much interest training on my own data but (like many I'm sure) would love a Vicuna-like model that had an expanded context. Has anyone found or hand any success with such a thing?
- jchook 3y agoIt seems that the Vicuna folks have trained some LongChat weights, e.g. https://huggingface.co/lmsys/longchat-7b-16k https://huggingface.co/lmsys/longchat-7b-16k
- brucethemoose2 3y agoAdditionally, there here some long context 30B models on huggigface now. For instance: https://huggingface.co/TheBloke/GPlatty-30B-SuperHOT-8K-GGML https://huggingface.co/TheBloke/GPlatty-30B-SuperHOT-8K-GGML https://huggingface.co/TheBloke/Vicuna-33B-1-3-SuperHOT-8K-GGML https://huggingface.co/TheBloke/Vicuna-33B-1-3-SuperHOT-8K-G... These will work with 8K context on even a modest system, thanks to Koboldcpp. 16K context is more exotic atm.