3 ms·
Somewhat related; how to run an uncensored model locally? I run llamafile (llamafile-server-0.1-llava-v1.5-7b-q4 and mistral-7b-instruct-v0.1-Q4_K_M-server) one
by anonzzzies 3y ago
Somewhat related; how to run an uncensored model locally? I run llamafile (llamafile-server-0.1-llava-v1.5-7b-q4 and mistral-7b-instruct-v0.1-Q4_K_M-server) ones on my macbook m1 and they run file (fast enough for playing), but they both seem neutered quite a bit. It's hard to get them off the rails and mistral (the above one) actually barfs really quickly just repeating the same letter (fffffff usually) where it should've said fuck. Now i'm not looking for something that writes porn or whatnot, but the online models are so pc, it's getting on my nerves.
- dvfjsdhgfv 3y agoYes, I wonder the same. I'm not so much about porn but pretty much every conversation with a model gets abruptly cut off when it's just getting interesting. Every model I tried - mistral, codellama etc. - is terribly maimed in this respect.
- MPSimmons 3y agoWhen you're running inference, it's super important to make sure that you're using the right prompt format (if you're using the Oobabooga web text ui, make sure you have 'chat', 'chat-instruct', or 'instruct' properly selected). The model card on Huggingface will usually tell you.