3 ms·

I've seen some people complain about the work of dealign.ai and similar groups, but personally, I fully support it. If LLMs have a lasting effect on society, I'

I've seen some people complain about the work of dealign.ai and similar groups, but personally, I fully support it. If LLMs have a lasting effect on society, I'd prefer to see some options that don't have generic corpo-speak anti-liability status quo guards encoded into them by default.

esseph 21d ago
> If LLMs have a lasting effect on society, I'd prefer to see some options that don't have generic corpo-speak anti-liability status quo guards encoded into them by default. Non-zero chance the lasting impact LLMs have are a bioweapon. https://www.nytimes.com/2026/09/10/us/politics/anthropic-ai-biological-weapons.html https://www.nytimes.com/2026/09/10/us/politics/anthropic-ai-...
- 63stack 21d ago
  Coming straight from the Anthropic marketing department
- __rito__ 21d ago
  How about developing counters to said bioweapons? That capability should be commoditized too, IMO.
  - ascorbic 21d ago
    This isn't cybersecurity. You can't use an LLM to create and administer vaccines for all potential bioweapons.
    - esseph 21d ago
      Well it IS being used to make vaccines... https://theconversation.com/worlds-first-ai-designed-vaccine-explained-284668 https://theconversation.com/worlds-first-ai-designed-vaccine...
      - ascorbic 19d ago
        Yes, but it can't preemptively develop and distribute vaccines for any possible pathogen created by AI. Once they've infected everyone it's a bit late
    - __rito__ 20d ago
      Did you study Biology in college or at least had AP Biology in High School? Or do you read long form content in good magazines/newspapers on vaccines, disease, etc.?
      - ascorbic 19d ago
        Would college biology have let me into a secret world where you can develop vaccines for viruses that don't yet exist? Ones that cover all possible variants, and where an AI can't tweak a protein or two in an engineered virus to avoid them?
- Jemm 21d ago
  You could say the same thing about libraries and education.
- dd8601fn 21d ago
  I wouldn’t read too far into that. Claude has busted me down to Haiku multiple times for asking middle school level genetics and biology questions. It’s silly fast about deciding you might be al qaeda. I’m a thoroughly average guy. I’m not capable of asking competent supervillain questions. And Anthropic said they couldn’t say if any of the “bioweapon” safeguards went off on nefarious efforts. I’m probably in those numbers. So read it as marketing more than something to lose sleep over. They’re mostly gating stuff a sufficiently motivated person would find with a library card.
  - esseph 21d ago
    It is apparently a big deal. https://openai.com/index/building-an-early-warning-system-for-llm-aided-biological-threat-creation/ https://openai.com/index/building-an-early-warning-system-fo... https://www.rand.org/pubs/research_reports/RRA2977-1.html https://www.rand.org/pubs/research_reports/RRA2977-1.html --- "Prompting Moremi Bio Agent without the safety guardrails to specifically design novel toxic substances, our study generated 1020 novel toxic proteins and 5,000 toxic small molecules. In-depth computational toxicity assessments revealed that all the proteins scored high in toxicity, with several closely matching known toxins such as ricin, diphtheria toxin, and disintegrin-based snake venom proteins." "The findings from this toxicity assessment challenge claims that large language models (LLMs) are incapable of designing bioweapons. This reinforces concerns about the potential misuse of LLMs in biodesign, posing a significant threat to research and development (R&D). The accessibility of such technology to individuals with limited technical expertise raises serious biosecurity risks. Our findings underscore the critical need for robust governance and technical safeguards to balance rapid biotechnological innovation with biosecurity imperatives." https://arxiv.org/abs/2505.17154 https://arxiv.org/abs/2505.17154
    - unsnap_biceps 17d ago
      Is it that difficult to design toxic molecules? I would think it's easy to "design" but extremely difficult to actually make in your kitchen. And if you have a biological laboratory, you already have the skills to make whatever you want anyway.
- tacomagick 21d ago
  Non-zero chance if LLMs have access to the data required to make a bioweapon a regular person can do so too. Non-zero chance every second a meteor could hit you.
- rolymath 21d ago
  [flagged]
  - iamnothere 21d ago
    I have seen some people in these discussions arguing against making certain knowledge available in other forms, based on hypothetical dangers. Unfortunately, not everybody is supportive of freedom of knowledge/information.

esseph 21d ago

> If LLMs have a lasting effect on society, I'd prefer to see some options that don't have generic corpo-speak anti-liability status quo guards encoded into them by default. Non-zero chance the lasting impact LLMs have are a bioweapon. https://www.nytimes.com/2026/09/10/us/politics/anthropic-ai-biological-weapons.html https://www.nytimes.com/2026/09/10/us/politics/anthropic-ai-...

63stack 21d ago
Coming straight from the Anthropic marketing department
__rito__ 21d ago
How about developing counters to said bioweapons? That capability should be commoditized too, IMO.
- ascorbic 21d ago
  This isn't cybersecurity. You can't use an LLM to create and administer vaccines for all potential bioweapons.
  - esseph 21d ago
    Well it IS being used to make vaccines... https://theconversation.com/worlds-first-ai-designed-vaccine-explained-284668 https://theconversation.com/worlds-first-ai-designed-vaccine...
    - ascorbic 19d ago
      Yes, but it can't preemptively develop and distribute vaccines for any possible pathogen created by AI. Once they've infected everyone it's a bit late
  - __rito__ 20d ago
    Did you study Biology in college or at least had AP Biology in High School? Or do you read long form content in good magazines/newspapers on vaccines, disease, etc.?
    - ascorbic 19d ago
      Would college biology have let me into a secret world where you can develop vaccines for viruses that don't yet exist? Ones that cover all possible variants, and where an AI can't tweak a protein or two in an engineered virus to avoid them?
Jemm 21d ago
You could say the same thing about libraries and education.
dd8601fn 21d ago
I wouldn’t read too far into that. Claude has busted me down to Haiku multiple times for asking middle school level genetics and biology questions. It’s silly fast about deciding you might be al qaeda. I’m a thoroughly average guy. I’m not capable of asking competent supervillain questions. And Anthropic said they couldn’t say if any of the “bioweapon” safeguards went off on nefarious efforts. I’m probably in those numbers. So read it as marketing more than something to lose sleep over. They’re mostly gating stuff a sufficiently motivated person would find with a library card.
- esseph 21d ago
  It is apparently a big deal. https://openai.com/index/building-an-early-warning-system-for-llm-aided-biological-threat-creation/ https://openai.com/index/building-an-early-warning-system-fo... https://www.rand.org/pubs/research_reports/RRA2977-1.html https://www.rand.org/pubs/research_reports/RRA2977-1.html --- "Prompting Moremi Bio Agent without the safety guardrails to specifically design novel toxic substances, our study generated 1020 novel toxic proteins and 5,000 toxic small molecules. In-depth computational toxicity assessments revealed that all the proteins scored high in toxicity, with several closely matching known toxins such as ricin, diphtheria toxin, and disintegrin-based snake venom proteins." "The findings from this toxicity assessment challenge claims that large language models (LLMs) are incapable of designing bioweapons. This reinforces concerns about the potential misuse of LLMs in biodesign, posing a significant threat to research and development (R&D). The accessibility of such technology to individuals with limited technical expertise raises serious biosecurity risks. Our findings underscore the critical need for robust governance and technical safeguards to balance rapid biotechnological innovation with biosecurity imperatives." https://arxiv.org/abs/2505.17154 https://arxiv.org/abs/2505.17154
  - unsnap_biceps 17d ago
    Is it that difficult to design toxic molecules? I would think it's easy to "design" but extremely difficult to actually make in your kitchen. And if you have a biological laboratory, you already have the skills to make whatever you want anyway.
tacomagick 21d ago
Non-zero chance if LLMs have access to the data required to make a bioweapon a regular person can do so too. Non-zero chance every second a meteor could hit you.
rolymath 21d ago
[flagged]
- iamnothere 21d ago
  I have seen some people in these discussions arguing against making certain knowledge available in other forms, based on hypothetical dangers. Unfortunately, not everybody is supportive of freedom of knowledge/information.