3 ms·

Surely it isn’t, solving millennium problems on a whim isn’t, either; actually, it’s probably much harder, which is the concern. How hard is it to maneuver an a

by baq 15d ago

Surely it isn’t, solving millennium problems on a whim isn’t, either; actually, it’s probably much harder, which is the concern. How hard is it to maneuver an aligned LLM to splice some smallpox into a generic research virus? BSL4 labs do things like this by hand all the time, so it can’t be too difficult and it only takes one lucky AI cultist for the rest of us to find out. I hope I’m wrong, but hope it’s not a process.

falserum 15d ago
Math and computers have benefit of not needing materials. Add a button here, add lema and assumption there - all are almost zero cost&risk. So iteration/experimentation with llm in hand is possible to some degree even for laymen. I assume virus creation is more like building a house. It involves physical work and chemical reactions that need time to finish in correct order. Each attempt costs a LOT more of time&money and involves actual risks.
tancop 15d ago
All you need to pull that off is public info, student level knowledge and access to a lab with standard equipment. As you said labs do things like that all the time. Its already easy even without powerful AI. The reason we dont see any serious attempts to create a deadly pandemic is not that we keep insane people out of labs, because there is no way to find out if someone is a cultist or murderous psycho when they have every reason to actively hide it. Its because a) most people are not evil and b) a virus that can kill everyone it touches would probably kill its creator first. As someone with basically zero bio experience I think I could get a working smallpox or ebola level virus in a year or two if I had time to learn, lab equipment and Annas Library. No LLMs needed. But I dont want to do that, and I think its the same for all the people who could do it right now.

3 ms·

Surely it isn’t, solving millennium problems on a whim isn’t, either; actually, it’s probably much harder, which is the concern. How hard is it to maneuver an a

by baq 15d ago

falserum 15d ago
Math and computers have benefit of not needing materials. Add a button here, add lema and assumption there - all are almost zero cost&risk. So iteration/experimentation with llm in hand is possible to some degree even for laymen. I assume virus creation is more like building a house. It involves physical work and chemical reactions that need time to finish in correct order. Each attempt costs a LOT more of time&money and involves actual risks.
tancop 15d ago
All you need to pull that off is public info, student level knowledge and access to a lab with standard equipment. As you said labs do things like that all the time. Its already easy even without powerful AI. The reason we dont see any serious attempts to create a deadly pandemic is not that we keep insane people out of labs, because there is no way to find out if someone is a cultist or murderous psycho when they have every reason to actively hide it. Its because a) most people are not evil and b) a virus that can kill everyone it touches would probably kill its creator first. As someone with basically zero bio experience I think I could get a working smallpox or ebola level virus in a year or two if I had time to learn, lab equipment and Annas Library. No LLMs needed. But I dont want to do that, and I think its the same for all the people who could do it right now.