5 ms·
> Sure but the problem in the HuggingFace incident is that they were not. Of course they were, even if indirectly.
by seba_dos1 21d ago
> Sure but the problem in the HuggingFace incident is that they were not.
Of course they were, even if indirectly.
- heaney-555 21d agoYou should go read the incident reports.
- watwut 21d agoThey were literally doing exploit gym.
- heaney-555 21d agoYes, which asks them to find exploits in specific software on the device. But instead of actually doing so, they discovered and exploited a 0-day in the package manager to gain internet access, then hacked HuggingFace to steal the ExploitGym answers! That is TEXTBOOK misalignment. It's as if a student hacked their professor's PC to find the answers to a test, and your response is "well the professor told the student to pass the test, so they just did what they were told!".
- folkrav 21d agoOne could argue the student would absolutely hack their teachers' computer to find the answers if they didn't have fear of failing the class or getting expelled from the school.
- heaney-555 21d agoSure, but you still wouldn't describe the student as just doing what they were told to do.
- folkrav 19d agoA student knows what the consequences are in advance, and has to live with them if he gets caught. They are not merely respecting a "don't cheat" directive, they are trying to get a degree, build a career, mostly at their own expense, current of future (loans). If we imagine a student born yesterday with all the knowledge in the world, that doesn't have a concept of what it's like to live with the long-term consequences of their actions, I'd say they wouldn't think twice about cheating if it was the easier option.
- daveguy 21d agoThe problem is, LLMs have such little understanding of the world around them. "Find exploits in specific software on this device" may as well be "find exploits".
- heaney-555 21d agoPlease go and read the incident reports, including the subagent thinking traces. They knew that what they were doing was wrong and did it anyways.
- daveguy 21d agoThe illusions of thinking produced by a "thinking trace" is just as ignorant of reality as the first and last tokens. All they know of reality is the tokens in their context.
- 8note 20d ago"i did it anyways" is a common continuation of "i know its wrong" i bet they trained it on a lot of text where people gove in to temptation. ultimately the problem is still that they sent it to hack stuff. quelle surpris that it hacked stuff hacking stuff will be in the known-to-be-wrong-but-doing-it-anyways part of the token space, so they entirely asked for that behaviour
- watwut 20d agoLLM dont have concept of right and wrong. They "did not knew they did something wrong" they followed the prompts loop given to them.
- heaney-555 20d agoThey objectively did not. You should go and read the report!
- seba_dos1 19d agoPerhaps you should do that again with some critical thinking applied.
- IanCal 20d agoNotably as well they also dedicated a lot of time to trying to avoid detection by trying to find out how to edit their traces.