3 ms·
> Open Source: We will release the weights in two weeks after launch, once safety evaluation and hardening are complete. What safety evaluation? What safety ha
by cubefox 2mo ago
> Open Source: We will release the weights in two weeks after launch, once safety evaluation and hardening are complete.
What safety evaluation? What safety hardening? They already evaluated it and found it to be highly capable at exploiting security vulnerabilities. So we know it is not "safe", and they don't seem to plan to do anything against it. What could be more dangerous than hacking? Biological weapons research? I don't think Chinese labs are doing anything against this either.
- alightsoul 2mo agoThey need to make money. Let them do it. They deserve it. Also, this is what inference engines like vLLM want to have "zero day" supporr
- gpm 2mo agoI'm curious what they mean by that too... They might be trying to weaken the cyber capabilities... Or I guess they might mean safety evaluation and hardening of the open source (and perhaps closed source Chinese) software ecosystem...
- thepasch 2mo agoI wouldn't be surprised if more resources were put into abliteration resistance the more capable open weight models become. It's something you don't need at all to start hosting the model on your own, but something you need to take care of before you release the weights (if you do care about it at all).
- h8hawk 2mo agoAre you against open-source models? Data and content related to "biological weapons" already exist on the internet, in books, etc. The real issue is access to facilities and tools. There are models that help researchers, but they are not LLMs, rather they are models trained specifically on biological data (like AlphaFold). Cybersecurity is basically used like a dog whistle pioneered by Anthropic to achieve regulatory capture. Otherwise, the widespread availability of good tooling for security analysis would eliminate more of these cyber threats, rather than gatekeeping them for a few private companies.
- cubefox 2mo ago> Data and content related to "biological weapons" already exist on the internet, in books, etc. The real issue is access to facilities and tools. No, I think tools are easy to come by (unlike in nuclear research), the real issue is the know-how to create biological weapons, which you can't easily get out of books, but much more easily out of an amoral LLM.
- h8hawk 2mo agoThis is just wrong. Standard textbooks like "Bioprocess Engineering Principles" [1] already contain step by step industrial engineering protocols and formulas. LLMs only summarize what print literature has documented for decades. The real bottleneck has always been hands on physical execution, not access to text. (This is not the case with actual models trained on biological data, which are not LLMs) > amoral LLM Are you a Lesswrong member? What makes Anthropic moral when their priority is automating people out of jobs instead of developing better medicine or advancing healthcare? [1] https://www.sciencedirect.com/book/monograph/9780122208515/bioprocess-engineering-principles https://www.sciencedirect.com/book/monograph/9780122208515/b...
- cubefox 2mo agoThis is just wrong. There is a huge difference between a book that doesn't explicitly tell you how to create a deadly virus, and an infinitely patient future LLM that leads you through every step and every unexpected problem and every question you might have.