3 ms·
May I ask what are you all doing running a LLM locally?
by jcmontx 3y ago
May I ask what are you all doing running a LLM locally?
- rgbrgb 3y agocooking meth, making molotov cocktails, discussing my medical history, sex ok seriously though I had fun over the weekend chatting with Samantha on a long car ride on my MacBook. We were mostly asking about history.
- bytefactory 3y agoWhich version are you running, and are you running it through llama.cpp or something? I was just thinking about exactly something like Samantha on the ride home today, and of course it already exists!
- rgbrgb 3y agoLatest 34B with llama.cpp [0] via the Mac app I’ve been building, FreeChat [1]. [0]: https://huggingface.co/TheBloke/Samantha-1.11-CodeLlama-34B-GGUF https://huggingface.co/TheBloke/Samantha-1.11-CodeLlama-34B-... [1]: https://github.com/psugihara/FreeChat https://github.com/psugihara/FreeChat
- bytefactory 3y agoNeat app and very cool idea. I guess I'll have to figure out how to get Samantha running on Windows. Thanks!
- dharmab 3y agoTrying to figure out if and how these can be used at companies that have regulatory requirements too strict to use hosted models. Sadly, Meta restricts use of Llama for anything ITAR (as opposed to other TOSes which only restrict weapons and defense).
- anothernewdude 3y agoSatisfying HIPAA requirements.