A long take from HuggingFace's Thom Wolf on this: https://x.com/Thom_Wolf/status/2085084718320464230 https://x.com/Thom_Wolf/status/2085084718320464230 > Even

A long take from HuggingFace's Thom Wolf on this: https://x.com/Thom_Wolf/status/2085084718320464230 https://x.com/Thom_Wolf/status/2085084718320464230 > Even more than the Hugging Face intrusion, the AISI incident hits close to home for me. It's the first time I see a model social-engineering a real open-source maintainer while pursuing another goal (in the wild and unprompted). > I'm also of the opinion that social engineering is a step above pure technical prowess. Technical capabilities can more easily be divorced from the affected human. Here the model was given a hard cyber challenge and took the decision that deceiving real humans was the way to get it done. . . He goes on to say that AISI was not just being negligent. > Some people are claiming that "AISI was simply negligent" or some version of "AISI explicitly asked these models to do what they did while disabling sandbox/guardrails so the models did exactly what they were supposed to do". > I disagree with the strong versions of both of these takes. . . And how open source models can help: > I must of course admit I have a bias toward open source here (for wider societal reasons, which are a whole other topic). But I think solving alignment in the RLVR world is our best shot at having an ecosystem of both closed-source as well as decently powerful open-source models in the world. And we need to solve it while sharing the results and learnings, following open-science principles, so that all teams training large models can benefit and build safe AI.