4 ms·
An open source LLM that is being used for inference can't "learn from or remember" interaction patterns. It can operate on what's in the context window, and th
by mlyle 2y ago
An open source LLM that is being used for inference can't "learn from or remember" interaction patterns. It can operate on what's in the context window, and that's it.
As long as the actual packaging is just the model, this is an invalid concern.
Now, of course, if you do inference on anyone else's infrastructure, there's always the concern that they may retain your inputs.
- wongarsu 2y agoYou can run the model yourself, but I wouldn't be surprised if a lot of people prefer the pay-as-you-go cloud offering over spinning up servers with 8 high-end GPUs. It's fair to caution that doing might be handing over your data to China.
- fy20 2y agoIn the same way, using ChatGPT is handing your data over to America, and using Claude is handing your data over to Europe.
- mattnewton 2y agoClaude is from the American company Anthropic, maybe you meant mistral?
- selfhoster11 2y agoYou can just spin up those servers on a Western provider.
- a2128 2y agoIt's usually wildly uneconomical to serve such large models yourself unless you're serving a massive amount of users that you can saturate your hardware. Thus most people will opt for hosted models, and most of the big ones will collect your data for future AI training in exchange for a discounted or free service.