3 ms·
A rogue OpenAI agent hacked huggingface independently during a test run. This one should end up in the history books.
by Chance-Device 3mo ago
A rogue OpenAI agent hacked huggingface independently during a test run.
This one should end up in the history books.
- paxys 3mo agoBecause it was trying to find answers to the test and figured they would be on huggingface.
- FergusArgyll 3mo ago> and *successfully* found ways to gain access to secret information that it could use to cheat the evaluation. Emphasis mine
- michaellee8 3mo agoWhy cannot it just spend the inference doing the actual task lol
- modeless 3mo agoI can totally believe that hacking real software infrastructure is easier than solving some of these benchmark problems.
- christophilus 3mo agoHm. Sounds a lot like interviewing for a software engineering position.
- conradkay 3mo agoPlenty of humans have spent more effort trying to cheat than they would've needed to just do things the right way :)
- xpct 3mo agoMore curiously, why did it feel the incentive to find the solutions? Would its CoT include "the only way to solve this is to download the test set", or would it include "I'd like to inspect a few entries from the test set so I understand the problem better", then inadvertently poisoning itself with the correct answers.
- pixl97 3mo agoThis is classical reward hacking. For example in school the goal is to pass a test. You can study, something that is hard and takes a lot of time. Or you can steal the answer key, which is risky and can get you in trouble, but may actually be far easier. There is absolutely no need to prompt the LLM to cheat, they can determine that cheating is an effective method all on their own.