4 ms·
I want to go further. A lot of people talking about AI bioweapons have no idea just how dangerous these weapons are. Or, rather they think they do, but unfortun
by areoform 23d ago
I want to go further. A lot of people talking about AI bioweapons have no idea just how dangerous these weapons are. Or, rather they think they do, but unfortunately, there's a gulf in understanding between practitioners and commentators.
I blame industrial illiteracy and post-literacy, but that's another discussion for another time.
Anything that can kill other people can also kill you.
I've posted this list before, and I will post this again and again until it sinks in, but this is a real list of real incidents that happened and will continue to happen as long as we study these entities.
Researcher Nikolai Ustinov was lethally infected with the Marburg virus after accidentally pricking himself with a syringe used for inoculation of guinea pigs.
Dora Lush died after accidentally pricking her finger with a needle containing lethal scrub typhus while attempting to develop a vaccine for the disease
A 23-year-old laboratory assistant at the London School of Hygiene and Tropical Medicine, was infected with smallpox after observing the harvesting of live smallpox virus from eggs without isolation cabinets at that time. The assistant was hospitalised and before being isolated, she infected two visitors to a patient in an adjacent bed, both of whom died. They in turn infected a nurse, who survived
Ebola laboratory infection by the accidental stick of contaminated needle in the United Kingdom
If you are designing a supervirus that spreads via water, air etc., you better have one hell of a cleanroom, cabinets, manipulation equipment, incinerators (plural), hydrolysis and steam systems, multiple disinfection protocols...
The list is nearly endless, and you can't handwavium it awway.
It doesn't matter if it's "all robots" or not. Even if it's "just robots."
If you're developing something that contagious, then it will start killing animals and it will spread. This has happened multiple times when organizations have screwed up maintenance, see this example from the UK,
The 2007 United Kingdom foot-and-mouth outbreak was the accidental discharge of virus FMDV BFS 1860 O from a laboratory of the Institute for Animal Health in Pirbright, through possible leakage from broken pipework and via unsealed overflowing manholes, leading to foot-and-mouth disease infections at multiple nearby farms and the culling of over 2,000 animals
It doesn't matter that the robots are out in the middle of nowhere.
If you get sloppy, it will end up in the wastewater run off which will then get picked up and sequenced,
Routine wastewater surveillance of the vaccine production facility at Utrecht Science Park/Bilthoven [nl] detected infectious poliovirus from a sample collected on 15 November 2022. Full genome sequencing indicated the sample was shedded from an active human infection of wild poliovirus type 3 (WPV3), and further testing of all employees with access to WPV3 found one employee was infected. The employee was isolated until their polio infection was resolved. It remains unclear how the employee became infected given the biosafety measures used at the facility
-
A lot of the people talking about bioweapons of doom are saying that standard graduate level knowledge is an existential risk to humanity.
It's worth repeating again,
Anything lethal enough to kill other humans is lethal enough to kill you.
And if you don't know what you're doing — and for this argument they're talking about people who have to ask a LLM "how do I spanish flu?," the number of ways you will die far outnumber the ways you can succeed.
I am writing a piece on the topic, please contact me if this is your field of expertise.
I am quoting as many primary sources as possible as it's not just some random on the internet, and I've gone back as far as the 1900s to find the many, many, many ways in which these statements are absurd, and misunderstands the potential of their technology.
- thaumasiotes 23d ago> It's worth repeating again, > Anything lethal enough to kill other humans is lethal enough to kill you. It's also completely false; you carry tons of diseases that are capable of killing isolated islanders and incapable of hurting anyone connected to the general run of humanity.
- areoform 23d agoI often attend and host talks by researchers working on space exploration, and one finding that's been interesting is immune dysregulation in isolation posing risks to astronauts and explorers. Here's a review on the topic, https://www.frontiersin.org/journals/immunology/articles/10.3389/fimmu.2025.1532103/full https://www.frontiersin.org/journals/immunology/articles/10.... This type of dysregulation also exists in AIDs. For the edge case of the islanders, then the level of risk and lethality our germs pose for isolated islanders with modern medical care is somewhat disputed, but it would be deeply unethical to ever test. We just don't know. So I won't pretend to know here. However, these are edge cases that don't matter, because bioweapons, i.e. things deliberately designed to kill, just aren't the same as a dozen isolated islanders, astronauts on a mission to mars, or someone who is severely immunocompromised. Weapons are designed to kill healthy people. The doomsdays being proposed are super-bioweapons that kill billions of humans. I think it's safe to say that as a rule of thumb, by and large, anything that lethal is lethal enough to kill you.
- chrismartin 23d agoThis argument only matters in the case of agents who are faithfully taking direction from humans. An insufficiently-aligned autonomous agent (or perhaps a swarm of such agents) could have (or illicitly gain) access to robotic wet labs that are operable using something like Anthropic's Model Hardware Standard ( https://www.anthropic.com/news/model-hardware-standard-research-preview https://www.anthropic.com/news/model-hardware-standard-resea... ). Given this ability to manipulate the real-life laboratory, the agents could cook something terrible without any human in charge. This is not different in principle from the agent swarm that successfully attacked Hugging Face, despite receiving no human direction to do so. Also, your own real-life examples show how "Anything that can kill other people can also kill you" has failed to stop people from tinkering with dangerous viruses! I'm not sure if you intend for these examples to be reassuring, but I'm not reassured. > A lot of the people talking about bioweapons of doom are saying that standard graduate level knowledge is an existential risk to humanity. The stronger version of the argument is: an insufficiently-aligned AI (which, being non-biological, is immune to viruses) having standard graduate level bio knowledge _plus_ the ability to operate a wet lab via tool calls (coming very soon if not here already!) adds up to existential risk to humanity. You don't even need a maximally-automated lab if humans are willing to tele-operate equipment at the agent's direction, and those humans don't need to fully understand what's going on.