3 ms·
New FOSS All-Local AI with Ollama, Gemma 2, and Fabric
Seriously smart all-local AI with excellant documentation and clear well-commented code: https://github.com/MikeyBeez/RAGAgent.
- wkat4242 2y agoThis was not submitted as a link so you can't click on it. That doesn't really follow the HN convention. PS: Personally I prefer ollama + openwebui <3 For its extensibility. Though I like the playful approach here.
- MikeBee 2y agoThanks. I think I figured out how to do this for new posts. The main differences between RAGAGENT and openwebui is that chat history is vector searched for context assemble and fabric is used for system and user messages. It's not just a frontend for the default REPLs. It's a real agent. Moreover, my documentation is designed to allow you to add whatever features you want. It's designed to be an engineer's best friend.
- ndr_ 2y agoIf you're on macOS, check out the MLX community: https://huggingface.co/mlx-community https://huggingface.co/mlx-community - Ollama not needed, potentially more efficient(?). The Gemma 2 27B model was broken as of a few days ago, but a fix is already committed and underway to the pip repo. Gemma 2 9B works directly.
- MikeBee 2y agoThanks! I've been using gemma 2 9B and Llamma3 8B interchangeably. They're both really good models.
- Tepix 2y agoGemma2 isn't FOSS, it has a rather peculiar license. See also: https://x.com/justinetunney/status/1808165898743878108 https://x.com/justinetunney/status/1808165898743878108
- MikeBee 2y agoThat's cool. You can switch the model to whatever you like. type /help and find the command to switch models.