3 ms·
Online chat interface here: https://huggingface.co/spaces/HuggingFaceH4/zephyr-chat https://huggingface.co/spaces/HuggingFaceH4/zephyr-chat You should be able
by Flux159 3y ago
Online chat interface here: https://huggingface.co/spaces/HuggingFaceH4/zephyr-chat https://huggingface.co/spaces/HuggingFaceH4/zephyr-chat
You should be able to run this locally on CPU or GPU as long as you have 16GB RAM. Less if using a quantized model like https://huggingface.co/TheBloke/zephyr-7B-alpha-GGUF https://huggingface.co/TheBloke/zephyr-7B-alpha-GGUF with llama.cpp
- brucethemoose2 3y agoAnd this will fit in 6GB of VRAM with very little quality loss: https://huggingface.co/LoneStriker/zephyr-7b-alpha-4.0bpw-h6-exl2 https://huggingface.co/LoneStriker/zephyr-7b-alpha-4.0bpw-h6...
- 7734128 3y agoI tried it out and I am super impressed by something so small. It hallucinated a lot, but formulated its incorrect answers wonderfully. It's certainly in the realm of being able to voice a NPC in an RPG game by now, as occasional hallucinations doesn't really matter there.
- brucethemoose2 3y agoThese are not even the best 7B models for an RPG. The RP Llama/Mistral finetunes should do much better.
- MuffinFlavored 3y ago> It hallucinated a lot How do we (the "LLM community") solve this/overcome this?
- behnamoh 3y agomillion dollar startup idea.
- 7734128 3y agoI suggest that we make a universal set of truths of everything which the LLM can compare its answer to. Perhaps a relational database? Or just one huge JSON?
- brucethemoose2 3y agoThis is kind of an inherent property of the llama architecture. Maybe new architectures will be better, but companies will likely be making the best foundational models. In the meantime, you can hook it up to a vectordb and ask the llm to check itself with zero temperature, but that only helps so much.