3 ms·
Zephyr 7B – Mistral Finetune that responds like ChatGPT
- Flux159 3y agoOnline chat interface here: https://huggingface.co/spaces/HuggingFaceH4/zephyr-chat https://huggingface.co/spaces/HuggingFaceH4/zephyr-chat You should be able to run this locally on CPU or GPU as long as you have 16GB RAM. Less if using a quantized model like https://huggingface.co/TheBloke/zephyr-7B-alpha-GGUF https://huggingface.co/TheBloke/zephyr-7B-alpha-GGUF with llama.cpp
- brucethemoose2 3y agoAnd this will fit in 6GB of VRAM with very little quality loss: https://huggingface.co/LoneStriker/zephyr-7b-alpha-4.0bpw-h6-exl2 https://huggingface.co/LoneStriker/zephyr-7b-alpha-4.0bpw-h6...
- 7734128 3y agoI tried it out and I am super impressed by something so small. It hallucinated a lot, but formulated its incorrect answers wonderfully. It's certainly in the realm of being able to voice a NPC in an RPG game by now, as occasional hallucinations doesn't really matter there.
- brucethemoose2 3y agoThese are not even the best 7B models for an RPG. The RP Llama/Mistral finetunes should do much better.
- MuffinFlavored 3y ago> It hallucinated a lot How do we (the "LLM community") solve this/overcome this?
- behnamoh 3y agomillion dollar startup idea.
- 7734128 3y agoI suggest that we make a universal set of truths of everything which the LLM can compare its answer to. Perhaps a relational database? Or just one huge JSON?
- brucethemoose2 3y agoThis is kind of an inherent property of the llama architecture. Maybe new architectures will be better, but companies will likely be making the best foundational models. In the meantime, you can hook it up to a vectordb and ask the llm to check itself with zero temperature, but that only helps so much.
- brucethemoose2 3y agoIMO the title is a bit misleading: > Zephyr-7B-α has not been aligned to human preferences with techniques like RLHF or deployed with in-the-loop filtering of responses like ChatGPT, so the model can produce problematic outputs (especially when prompted to do so). It is also unknown what the size and composition of the corpus was used to train the base model (mistralai/Mistral-7B-v0.1), however it is likely to have included a mix of Web data and technical sources like books and code. Its very deliberately unlike ChatGPT, which is what makes its performance interesting.
- nullc 3y agoA machine that follows instructions, refreshing. So sad that this is where we are and that some companies and the doomsday cultist partners are pushing to enshrine the sad state into law.
- matteoraso 3y agolooks at collection of local LLMs Eh, one more couldn't hurt.
- behnamoh 3y agoI wonder if I should delete the old ones. I bought a 4T external SSD exactly for LLMs.