4 ms·
Question without judgement: why would I want to run LLM locally? Say I'm building a SaaS app and connecting to Anthropic using the `ai` package. Would I want to
by mrcwinn 2y ago
Question without judgement: why would I want to run LLM locally? Say I'm building a SaaS app and connecting to Anthropic using the `ai` package. Would I want to cut over to ollama+something for local dev?
- andrewmunsell 2y agoData privacy-- some stuff, like all my personal notes I use with a RAG system, just don't need to be sent to some cloud provider to be data mined and/or have AI trained on them
- gonesurfing 2y agoI’ve been thinking about the local notes use case, but know zero about doing it. Do you have a good setup you could point me at?
- jwitthuhn 2y agoFor me it is consistency. I control the model and the software so I know a local LLM will remain exactly the same until I want to change it. It also avoids the trouble of using a hosted LLM that decides to double their price overnight, costs are very predictable.
- satvikpendem 2y agoLack of censorship.