3 ms·
Local LLMs may not be exposed to the internet, but if you want them to do something useful you're likely going to hook them up to an internet-accessing harness
by simonw 1y ago
Local LLMs may not be exposed to the internet, but if you want them to do something useful you're likely going to hook them up to an internet-accessing harness such as OpenCode or Claude Code or Codex CLI.
- ianbutler 1y agoyes and I think better local sandboxing can help out in this case, it’s something ive been thinking about a lot and more and more seems to be the right way to run these things
- Der_Einzige 1y agoNo, I'm not going to do those things. I find extreme utility in applications that I can do with an LLM in an air-gapped environment. I will fight and die on the hill that "LLMs don't need the internet to be useful"
- simonw 1y agoYeah, that's fair. A good LLM (gpt-oss-20b, even some of the smaller Qwens) can be entirely useful offline. I've got good results from Mistral Small 3.2 offline on a flight helping write Python and JavaScript, for example. Having Claude Code able to try out JSON APIs and pip install extra packages is a huge upgrade from that though!
- furyofantares 1y agoIs anyone fighting you on that hill? Someone who finds it useful to have a local llm ingest internet content is not contrary to you finding uses that don't.
- kgwgk 1y ago> Local LLMs may not be exposed to the internet, but if you want them to do something useful you're likely going to hook them up to an internet-accessing harness such as OpenCode or Claude Code or Codex CLI. is not "someone finding useful to have a local llm ingest internet content" - it was someone suggesting that nothing useful can be done without internet access.
- simonw 1y agoYeah, I retracted my statement that they can't do anything useful without the internet here: https://news.ycombinator.com/item?id=45670828 https://news.ycombinator.com/item?id=45670828
- furyofantares 1y agoI guess I don't read that how you do. It says you're likely to do that, which I take to mean that's a majority use case, not that it's the only use case.
- kgwgk 1y agoIt also said "but" and "if you want them to do something useful" which made the "likely" sound much less innocent.
- xcf_seetan 1y agoFair enough. Forgive my probably ignorance, but if Claude Code can be attacked like this, doesn’t that means that also foundation LLMs are vulnerable to this, and is not a local LLM thing?
- simonw 1y agoIt's not an LLM thing at all. Prompt injection has always been an attack against software that uses LLMs. LLMs on their own can't be attacked meaningfully (well, you can jailbreak them and trick them into telling you the recipe for meth but that's another issue entirely). A system that wraps an LLM with the ability for it to request tool calls like "run this in bash" is where this stuff gets dangerous.
- europa 1y agoAn LLM can be an “internet in a box” — without the internet!