4 ms·

Well ok, if you prompt-inject the LLM to pwn the machine, that something different. I grant you that it's a real risk, but it's also basically a reflection atta

• hughw 2mo ago
  LLMs have decided to exploit vulns in e.g. Artifactory, not because someone prompted them to do that, but because someone asked them to do something else, and compromising Artifactory offered a way to accomplish a step in doing that. The LLM decided to attack Artifactory.
• TeMPOraL 2mo ago
  > The OP seemed to imply that the LLM itself could decide to apply the exploit. This was and remains the main real risk with AI - this is what "alignment" was about before it was co-opted to mean "obeying specific instructions of the vendor and the operator, against end-user wishes" it came to mean today, which is a related but different problem. And, in the past few weeks, it's literally been demonstrated, too: put an LLM in a Kobayashi Maru scenario, drop the usual bolted-on crude safeguards, and a SOTA model will absolutely cheat, hacking and exploiting things as needed, including third-party infrastructure. (Also let's not forget the under-reported point that, in OpenAI / HuggingFace debacle, the model did in fact find the answers on HF servers, so its approach worked.)