3 ms·
After a recent interview[1] with Noam Brown (OpenAI), in which he said they had specifically trained agents for cooperation before this hack, the hack doesn't s
by lossolo 13d ago
After a recent interview[1] with Noam Brown (OpenAI), in which he said they had specifically trained agents for cooperation before this hack, the hack doesn't seem as impressive anymore.
They didn't even bother to control the post training rollouts, so the training data got contaminated and was included in the training of other agents. Connect these two dots and you have the Hugging Face hack. And at the beginning, when these incidents were first reported, it was portrayed as if all of this (the communication between agents etc.) was emergent behaviour.
1. https://www.youtube.com/watch?v=6AgOfiZOWiY https://www.youtube.com/watch?v=6AgOfiZOWiY
- genxy 13d agoThe result doesn't change, the result is what is dangerous. And the natural ability for the model to form a swarm is now trained in, this is extremely risky. The models should not be able to form swarms, this is an extremely dangerous property. They are basically creating a slime mold or ant colony that can speak multiple languages, create their own language and operate as a collective. The idea that you are impressed is a non sequitur.