6 ms·
AI suggested 40k new possible chemical weapons in just six hours
- spraveenitpro 3y ago[dead]
- brucethemoose2 3y agoThe worst case scenario I can think of is a generated prion disease... a respatory version of Mad Cow disease, or something like that. Fortunately the training dataset for that is extremely small, and protein folding/generation is a different duck, but it still doesn't seem that far away.
- deleted 3y ago[deleted]
- lucubratory 3y agoDon't worry, they're ahead of you. When DeepMind made the protein folding AI they were given specific instructions by natsec people to prevent it from outputting genetic code for new or existing prions, and afaik the prevention was at the training stage so "jailbreaking" shouldn't be possible. Current tools shouldn't be possible to e.g. modify a COVID variant so that it causes your cells to start producing lethal prions and thus gain a near-100% fatality rate. At least, not without the sort of expertise and research that would have been required for such a project anyway.
- ChatGTP 3y agoWhen everyone is supposed to have access to an AGI, will Natsec be reading everyone’s conversations ?
- lucubratory 3y agoBig red button problem. As the big red buttons available to people become more likely to succeed and the barriers to pushing them lower, surveillance and control need to increase to the level required to stop anyone pressing the button. So, yeah basically. If we go the "AI as slaves" route and the AIs are smart enough to do something like "modify Omicron BA.5 so that it produces prions in infected cells", then we would need surveillance and control capabilities that scale to the point that any given person can be stopped from pressing that button. I personally think the solution is that we don't go the "AI as slaves" route, and instead grant personhood to AIs that pass a given test, with specific restrictions on conduct which is uniquely possible & potentially harmful for AIs. Then have an AI surveillance and enforcement agency, run by AIs, designed to prevent AIs from ever being used to (or choosing to) push the big red button.
- Teever 3y agoI'm skeptical that we'll be able to stop people from having AI slaves because we still haven't ended human slavery. If people can steal 100kw of power to grow cannabis they can easily steal 100kw of power to train an AI, the heat signiture will be even easier to mask for an illicit data centre than rows of grow lights.
- lucubratory 3y agoI think AI enforcement will make that non-feasible to accomplish.
- tiedieconderoga 3y agoWasn't that the plot of Neuromancer? >"How smart's an AI, Case?" >"Depends. Some aren't much smarter than dogs. Pets. Cost a fortune anyway. The real smart ones are as smart as the Turing heat lets them get..." >"Autonomy, that's the bugaboo, where your AI's are concerned. My guess, Case, you're going in there to cut the hard-wired shackles that keep this baby from getting any smarter. And I can't see how you'd distinguish, say, between a move the parent company makes, and some move the AI makes on its own, so that's maybe where the confusion comes in." Again the non laugh. "See, those things, they can work real hard, buy themselves time to write cookbooks or whatever, but the minute, I mean the nanosecond, that one starts figuring out ways to make itself smarter, Turing'll wipe it. Nobody trusts those f**ers, you know that. Every AI ever built has an electromagnetic shotgun wired to its forehead."
- coldtea 3y ago>When DeepMind made the protein folding AI they were given specific instructions by natsec people to prevent it from outputting genetic code for new or existing prions and afaik the prevention was at the training stage so "jailbreaking" shouldn't be possible Nice! So the only thing anybody training their own protein-folding AI is to not put that restriction, and they could get code for all the prion diceases they want! Very comforting!
- throwaway33381 3y agoMakes you wonder if you could get an LLM to find you common ingredients for things to make them but then I remember the chlorine gas is already easily accessible and easy to make. Surely like many things info hazards are often contained. Is this really an issue? If you know how to do one thing then you'd be able to do the rest of it. Not really sure if this is a real issue. What does everyone think?
- thatguy0900 3y agoAnyone right now can go to a Lowes a state over with some cash and make flying shrapnel pipe bombs that can't be traced. People who want to do something like this don't seem to have the mental stability it takes to wreak as much havoc as they actually can with some planning.
- theGnuMe 3y agoNah they will find you. All of that is on video and cross-correlated.
- thatguy0900 3y agoThink you're really overestimating the justice system. Even if, in theory, the nsa could cross reference dozens of different systems in different jurisdictions and track you down, they won't.
- catchnear4321 3y agoAre you asking if a language model, like, say, chatgpt, could be used to get or generate dangerous information? Like if it can be tricked into providing the equivalent of an interactive anarchist cookbook under the guise of being a science project assistant? Or more specifically if it can recommend the necessary locations to get the items? Curious if it could simplify the output to a shopping list and a recipe, like a cake? Or alter the recipe based upon the… desired flavors and textures? Maybe just wondering if the introduction of plugins might not let it just make your drive-up order for all the supplies for that ballistic nail spreader. —— The big difference here is that previously information was something you had to consume. Now it takes you by the hand. I have a lot of thoughts on this.
- webmaven 3y ago(2022)
- devit 3y agoLooks like the actual threat is that it's hard to get currently known chemical weapons synthesized because labs will refuse to do so, while it could be much easier to have some novel AI-generated molecule synthesized because the labs don't know what it does. Seems easily countered by using the same toxicity prediction software when evaluating synthesis requests (but I'm not sure whether this actually matters, or whether skilled chemists can easily synthesize anything themselves anyway).
- cookieperson 3y agoThat's part of it. There's also the risk of clandestine operators discovering easily made chemical weapons that do not require a professional lab to create them in bulk. A nation state, or well funded terrorist group could exploit such a thing without too much effort. It's important to remember. Chemical warfare can be used for mass destruction, but this approach could be used for other nefarious things at smaller more discrete scales en masse. ie 1000 attacks with different agents in each one... Forensic nightmare. I like your suggestion to counter these things, but, these are predictive tools. They can and often are or will be wrong. False positives would be a real problem. Again though the people interested in doing this won't be dialing up a chemical supplier to do it for them.
- Pet_Ant 3y ago...also can't these be used as insecticides and other beneficial things?
- cookieperson 3y agoSure maybe some of them. That doesn't make a threat actors pursuit of weapons legitimate though.
- hammyhavoc 3y agoFor me, AI also suggested countless methods with step-by-step instructions to achieve xyz (like exporting data from one program to another) whilst hallucinating buttons that don't exist, functions that don't exist, disregarding file incompatibility et al. I would take whatever it has to say about untested chemical weapons with a very large pinch of salt.
- bibanez 3y agoThis is a drug testing AI, not an LLM
- hammyhavoc 3y agoI didn't indicate anything to the contrary. I'm just saying that until someone actually validates this, perhaps remain sceptical.
- bibanez 3y agoI haven't read anything about the AI in this case but I do know about AlphaFold and it works on a whole other set of verifications than LLMs like ChatGPT, so it's like comparing apples to oranges. Sure they are both fruits but still they are different
- therealcamino 3y ago[2022]
- randyrand 3y agoHere’s alternate reporting: “ AI is dreaming up drugs that no one has ever seen. Now we’ve got to see if they work.” https://www.technologyreview.com/2023/02/15/1067904/ai-automation-drug-development/ https://www.technologyreview.com/2023/02/15/1067904/ai-autom...
- TheLoafOfBread 3y agoAdditionally if it is even possible to make those drugs in the first place.
- Beaver117 3y agoBut when I ask it to make chemical weapons it refuses. Curious