3 ms·
Ask HN: How do you use LLMs for private discussions?
I sometimes have things to discuss with LLMs, which are more private than usual. I was thinking how to do it in a way that doesn't reveal my identity.
1. Use Tor to access the provider?
2. Create a random account?
3. Use some form of untraceable payment (which one?)
4. Scrub all information provided to the LLM from personally identifiable information?
It seems like a lot of effort. So is running a local LLM, for which I don't even have the hardware. How do you do it?
- blinded 3mo agoI spin up a gpu instance in a cloud, run my model via vllm, connect to it via an ssh tunnel. done.
- stuxnet79 3mo agoCan you elaborate on the first step? Which cloud and which service? What's the cost outlay if you are just having a convo and not doing anything 'agentic'?
- blinded 3mo agoHey sure. It depends, but usually spin up an h100 on lambda.ai or coreweave. They have capacity and their UIs/APIs are nice. I spin it up for an hour or two, believe it was 6~ dollars an hour. Once the gpu instance is up, you need to run vllm and a model, ie https://docs.lambda.ai/education/large-language-models/deploying-nemotron-3-nano/ https://docs.lambda.ai/education/large-language-models/deplo.... Then you can connect your pi.dev, openwebui, etc etc to vllm and interact with it like normal.
- downbad_ 3mo agoCan't you do it logged out?
- hbcondo714 3mo agoMaybe try Ollama Cloud, Prompt or response data is never logged or trained on: https://ollama.com/pricing https://ollama.com/pricing
- Havoc 3mo agoWith a local one
- 7402 3mo agoWith llama.cpp, Intel i9-13900KS CPU, 96 GB RAM, RTX 4070 running locally. The models I'm using right now with that are: gpt-oss-120b-F16.gguf Qwen_Qwen3.5-27B-Q4_K_M.gguf Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf gemma-4-31B-it-UD-Q6_K_XL.gguf
- ahdgs 3mo agoi've been using OpenAI's os PII-masking model, works decently and lightweight enough to run virtually anywhere https://huggingface.co/openai/privacy-filter https://huggingface.co/openai/privacy-filter
- RobotCaleb 3mo agoI don't
- PaiDxng 3mo agoThe hardest part isn’t payment; the prompt itself may identify you. For truly sensitive topics, I’d abstract the details first, then use a local model—or avoid an LLM entirely.
- tomsop 3mo ago[flagged]
- noman-land 3mo agoBest is to use local LLMs obviously. I haven't used it but nano-gpt accepts crypto payments. https://cake.nano-gpt.com https://cake.nano-gpt.com
- makeyouragent 3mo ago[flagged]
- MyMemoryfails 3mo agoRunning Local LLM isn't really a lot effort anymore, since its one command away to get the familar interface of chat AI. This setup requires 24GB ish VRAM and 32GB ram, so if you have capable gaming PC which can do AAA games, you can run it. - llama-server -hf ggml-org/gemma-4-26b-a4b-it-GGUF:Q4_K_M After that simply open browser and enter: http://localhost:8080 What this does: This will download Gemma4 AI with 26B param & start a http server for chat Its shockingly capable for its size. Does it beat the top end models? No, but as long your don't fall into the hallucations. Its just fine. Edit: the software is llama.cpp you can download it from "releases" which u can find at github right side. No need to know how to build it Edit2: Pro tip is, only use chat per context you want to use. Lots users want "dynamically" change the context, but that doesnt really work from my experience.
- HNYN 3mo ago[flagged]
- simquat 3mo agoNot sure what kind of model you're interested in, but you may give Duck.ai[0] a try. It doesn't require any signup and let you use gpt-5.4 nano/mini, Haiku 4.5, Mistral Small 4 and Gemma 4 31B. [0] https://duck.ai/ https://duck.ai/
- deleted 3mo ago[deleted]
- deleted 3mo ago[deleted]
- andrew_ocs 3mo agohi, you should try https://ollama.com/ https://ollama.com/ which is imo the most convenient way to run local LLMs
- tomsop 3mo ago[flagged]
- deleted 3mo ago[deleted]
- tomsop 3mo ago[flagged]
- tomsop 3mo ago[flagged]
- tomsop 3mo ago[flagged]
- tomsop 3mo ago[flagged]
- tomsop 3mo ago[flagged]
- tomsop 3mo ago[flagged]
- tomsop 3mo ago[dead]
- 3Sophons 2mo ago[flagged]
- tomsop 2mo ago[flagged]
- tomsop 2mo ago[dead]