3 ms·
One thing I realized is just how much offline local models can hurt mass data collection. For example, I needed to write an invitation letter for immigration c
by DanielHB 2mo ago
One thing I realized is just how much offline local models can hurt mass data collection.
For example, I needed to write an invitation letter for immigration control for a relative visiting me. Previously I would have used a search engine for a template. Today I fire up my local qwen 3.5-9b for this kind of stuff and feed it all the private data I need.
Unfortunately it is unlikely the average user will known how to avoid this data collection. Even if the LLM is local you are likely feeding the prompts to remote servers if you harness/chat-interface is not properly vetted.
- epihelix 2mo agoI hope for local model chat inexperienced users are just using llama.cpp's built-in web server interface, which gives you everything you need. No need for a harness or any other chat client.
- DanielHB 2mo agoI have tried running llamma.cpp on my PC and I found it hard getting it to run at decent speed. On Qwen 3.5-9b I get at most 10tk/s. I eventually switched to LM studio and the same model runs much better, like 70tk/s. Not sure if it was because I was running llama.cpp inside podman or badly tuned LLM arguments. But LM studio is unfortunately much more practical. Although I agree with you. I do not really know what kind of telemetry LM studio is running and I would rather not be using it.