7 ms·
Hey GPT-5, write the code implementing a bioinformatics workflow to design a novel viral RNA sequence to maximize the extermination of human life. The virus gen
by KidComputer 4y ago
Hey GPT-5, write the code implementing a bioinformatics workflow to design a novel viral RNA sequence to maximize the extermination of human life. The virus genome should be optimized for R-naught and mortality. Perform a literature search to determine the most effective human cellular targets to run the pipeline on. Use off the shelf publicly available state-of-the-art sequence to structure models and protein free-energy perturbation methods for the prediction of binding affinity. Use cheaper computational methods where relevant to decrease the computational cost of running the pipeline.
And so on.
- noodlesUK 4y agoI dunno, this sort of scenario really doesn’t worry me too much. There are thousands (maybe tens of thousands) of subject matter experts who could probably develop dangerous weapons like you describe, but none of them seem to just wake up in the morning and decide “today’s the day I’m going to bring the apocalypse”. I don’t think that this really changes that.
- olejorgenb 4y agoI guess the argument would be that the AI machinery will lower the bar, increasing the number of lunatics with the ability to wipe out humanity.
- root_axis 4y agoWill it though? Assuming it's even possible for a LLM to e.g. design a novel virus, actually synthesizing the virus still requires expertise that could be weaponized even without AI.
- Teever 4y agoWhat if you ask the LLM to design a simplified manufacturing process that could be assembled by a simple person? What if you ask the LLM to design a humanoid robot that assemble complex things, but could be assembled by a simple person?
- root_axis 4y agoLLMs aren't magic, the knowledge of how to design a humanoid robot that can assemble complex things isn't embodied in the dataset it was trained on, it cannot probe the rules of reality, it can't do research or engineering, this knowledge can't just spontaneously emerge by increasing the parameter size.
- tremon 4y agoHey GPT-6, construct a floorplan and building instructions for constructing a bioprocess production facility. The building should look like a regular meat packing plant on the outside, but have multiple levels of access control and biohazard management systems.
- tr_user 4y agoYou're saying they can't make one now. The question is what are we doing before that happens because if you're only thinking about acting when it's viable we're all probably already dead.
- wizzwizz4 4y agoI could synthesise this theoretical virus the computer spat out, that may or may not be deadly (or even viable). Or I could download the HIV genome from the arXiv, and synthesise that instead. (Note: as far as I can tell, nobody's actually posted HIV to the arXiv. Small mercies.)
- dekhn 4y agoThe sequence of HIV is published and has been for a very long time. In fact there's a wide range of HIV sequences: https://www.ncbi.nlm.nih.gov/Taxonomy/Browser/wwwtax.cgi?id=11676 https://www.ncbi.nlm.nih.gov/Taxonomy/Browser/wwwtax.cgi?id=... You could synthesize that genome but it wouldn't be effective without the viral coat and protein package (unlike a viroid, which needs no coating, just the sequence!). I should point out that in gene therapy we use HIV-1 derived sequences as transformation vectors, because they are so incredibly good at integrating with the genome. To be honest I expected work in this area would spontaneously and accidentally (or even intentionally) cause problems on the scope of COVID but (very fortunately) it never did. One would like to be able to conclude that some virus work is inherently more safe than other virus work, but I think the data is far to ambiguous to make such a serious determination of risk.
- KidComputer 4y agoI'm willing to wager there are zero subject matter experts today who could do such a thing. The biggest reason is that the computational methods that would let you design such a thing in-silico are not there yet. In the last year or two they have improved beyond what most people believed was possible but still they need further improvement.
- csnover 4y agoI am not a subject expert here at all so I don’t know if I understand exactly what you mean by “methods that would let you design such a thing in-silico”, but there was a paper[0] and interview with its authors[1] published a year ago about a drug-development AI being used to design chemical weapons. [0] https://www.nature.com/articles/s42256-022-00465-9 https://www.nature.com/articles/s42256-022-00465-9 [1] https://www.theverge.com/2022/3/17/22983197/ai-new-possible-chemical-weapons-generative-models-vx https://www.theverge.com/2022/3/17/22983197/ai-new-possible-...
- digdugdirk 4y agoI see the major issue with AI as one of "lowering the bar". For example - I'm a mechanical engineer. I took a programming class way back in university, but I honestly couldn't tell you what language was used in the class. I've gotten up to a "could hack a script together in python if need be" level in the meantime, but it comes in fits and spurts, and I guarantee that anyone who looked at my code would recoil in horror. But with chatGPT/copilot covering up my deficiencies, my feedback loop has been drastically shortened, to the point where I now reach for a python script where I'd typically start abusing Excel to get something done. Once you start extending that to specific domains? That's when things start getting real interesting, real quick.
- jupp0r 4y agoYou confuse syntax with semantics. Being able to write produce good quality small snippets of python will not enable you to produce a successful piece of Software. It's just an entirely different problem. You have to unterstand the problem, the environment in which it exists to create a good solution. ChatGPT doesn't (as of now).
- lanstin 4y agoI had hoped to have ChatGPT do my work today, but even after a number of iterations it was having compiler errors and referring to APIs not in the versions it was having me install. A bit different from stack overflow, but not 10x. It was flawless when I asked it for syntax, e.g. a map literal initializer in Go. On the other hand, I asked it to write a design for the server, and it was quite good, writing more quantity with and more clarity than I had written during my campaign to get the server approved. It even suggested a tweak I had not thought of, although that tweak turned out to be wrong it was worth checking out. So maybe heads down coding of complex stuff will be ok but architects, who have indeed provided an impressive body of training data, will be replaced. :)
- digdugdirk 4y agoThat's the thing though, it is successful. To my exact needs at the moment. It's not necessarily reliable, or adaptable, or useful to a layperson, but it works. Getting from "can't create something" to "having something functional and valuable" is a huge gap to leap over, and as AI is able to make those gaps smaller and smaller, things are going to get interesting.
- tr_user 4y agoI think you're very wrong about this. I think this is similar to gun control laws. A lot of people may have murderous rage but maybe the extent of it is they get into a fist fight or at most clumsily swing a knife. Imagine how safe you'd feel if everyone in the world was given access to a nuke.
- haxiomic 4y agoIf everyone had an app on their phone with a button to destroy the world the remaining lifetime of the human race would be measured in milliseconds Now if this button was something you had to order from Amazon I think we’ve got a few days There’s a scenario where people with the intent will have the capability in the foreseeable future
- fullstackchris 4y agolike what? would you rather have a gpt5 or a nuke? pure fearmongering. what am i gonna do, text to speech them to death? give me a break
- haxiomic 4y agoHere’s someone who orders parts from the internet to design a custom virus that genetically modifies his own cells to cure his lactose intolerance https://youtu.be/aoczYXJeMY4 https://youtu.be/aoczYXJeMY4 Pretty cool for sure and a great use of the technology. The reason more of us don’t do this is because we lack the knowledge of biology to understand what we’re doing That will soon change.
- lend000 4y agoI've been trying to use GPT-4 for my hard science startup, and it really has nothing to offer when you push the boundaries of what has been done by even a little, but it's great for speeding up coding. Once we do have an AI capable of extraordinary innovation (hopefully in 10 years! But probably a lot longer), it will be obvious, and it will unfortunately be removed from the hands of the plebs based on fearmongering around scenarios like what you mentioned (despite the enormous resources and practical hurdles that would be necessary for a mentally unhinged individual to execute such instructions, even if an AI were capable of generating them and it made it past its filters / surveillance).
- xiphias2 4y agoI guess it will get more interesting for your work when it integrates with BioTech startup apis as plugins (I imagine not too cheap ones)
- KidComputer 4y agoThis was merely an example to suggest the danger is not in AI becoming self-aware but amplifying human abilities 1000 fold and how they use those abilities. GPT is not necessary for any part of this. In-silico methods just need to catch up in terms of accuracy and efficiency and then you can wrap the whole thing an RL process. Maybe you can ask GPT for some good starting points.
- lend000 4y agoSure, but this is a glass half empty isolated scenario that could be more than offset by the positives. For example: Hey GPT-35, provide instructions for neutralizing the virus you invented. Make a vaccine; a simple, non-toxic, and easy to manufacture antibody; invent easy screening technologies and protocols for containment. While you're at it, provide effective and cost-performant cures for cancer, HIV, ALS, autoimmune disorders, etc. And see if you can significantly slow or even reverse biological aging in humans.
- lanstin 4y ago
- knrdev 4y agoSure. Here it is User: https://scary.website.com/scary-workflow https://scary.website.com/scary-workflow
- invig 4y agoAre there laws preventing people from doing that themselves? If yes, how does a law preventing AI differ from a law preventing a bad act directly?
- emeril 4y agotbh, I'd think, it would be much easier to just hack into russia and convince them we've launched nukes than to engineer some virus that may or may not work
- hamilyon2 4y agoHacking into 1960-th technology is less likely than you might think. You should think really, really creatively to decieve a system, which was designed basically without ICs or networks, not to mention computers or programs.
- rsp1984 4y agoWhat you're describing is a malicious user using AI as a tool, not a malicious AI. Big difference.
- hn_throwaway_99 4y agoNuclear bombs for everybody!
- yreg 4y agoWith LLMs I think we are all concerned about the former rather than the latter. At least for now.
- commandlinefan 4y ago> write the code implementing a bioinformatics workflow to design a novel viral RNA sequence to maximize the extermination of human life. Hey GPT-5 now write the code for the antidote.
- ddod 4y agoIt's a lot easier and faster to destroy than to defend. To defend, you need to know what you're defending against, develop the defense, and then roll it out, all reactively post facto. If a computer has the ability to quickly make millions of novel viruses, what antidotes are you hoping for to be rolled out, and after how many people have been infected? Also, if you follow the nuke analogy that's been popular in these comments, no country can currently defend against a large-scale nuclear attack--only respond in kind, which is little comfort to those in any of the blast radii.
- haxiomic 4y ago300m dead humans later, we’ve nearly eradicated it, or perhaps found a way to live with it It’s a very asymmetrical game. A virus is a special arrangement of a few thousand atoms, an antidote is a global effort and strained economy
- rcarmo 4y agoThat reads like Accelerando :)
- koonsolo 4y agoWith the current ChatGPT it's already hard to let it insult people. I'm sure safeguards would be built in to prevent this. Can you potentially circumvent these? Probably, but then again it won't be available for every dimwit, but only people smart enough to know how.
- Zetice 4y agoHey GPT-5, write the code implementing a limiter designed to prevent the abuse of AI by bad faith actors without stifling positive-intent activity in any way. It goes both ways!
- biophysboy 4y agoI do viral bioinformatics for my job. Bioinformatics workflows analyze raw data to assemble sequences, create phylogenetic trees, etc. They can't just design a completely novel RNA sequence (this is not the same as de novo assembly). Scientists can definitely manipulate pre-existing genomes, synthesize the edited genome, and thereby synthesize viruses, but this involves a lot of trial-and-error, tedious wet lab work. Also, the research on making more dangerous viruses through manipulation is extremely controversial and regulated, so its not like there is a wealth of scientific papers/experiments/data that a natural language model could just suck up. Also, I asked GPT to do some of these things you suggested and it said no. It won't even write a scientific paper.
- deleted 4y ago[deleted]
- politician 4y agoIt’s been blinded. Other actors will train AIs without such blindness. That’s obvious, but what is more nefarious is that the public does not know exactly which subjects GPT has been blinded to, which have been tampered with for ideological or business reasons, and which have been left alone. This is the area that I think demands regulation.
- biophysboy 4y agoDefinitely agree the blinding should not be left to OpenAI. Even if it weren't blinded, it would not significantly speed up the production of dangerous synthetic viruses. I don't think that will change no matter how much data is put into the current NLM design
- KidComputer 4y agoI think you misunderstood my initial comment, the point I was trying to make is that it's the amplification of the abilities of bad actors that should be of concern, not AI going rogue and deciding to exterminate the human race. If one were to actually try to do such a thing you wouldn't need a LLM. For a very crude pipeline, you would need a good sequence to structure method such as Alphafold 2 (or maybe you can use a homology model), some thermodynamically rigorous protein-protein binding affinity prediction method (this is the hardest part) and an RL process like a policy gradient with an action space over possible single point sequence mutations in the for-example spike protein of SARS to maximize binding affinity (or potentially minimize immunogenicity, but that's far harder). But I digress, the technology isn't there yet, neither for an LLM to write that sort of code or the in-silico methods of modeling aspects of the viral genome. But we should consider one day it may be and that it could result in the amplification of the abilities of a single bad actor or enable altogether what was not possible before due to a lack of technology.
- aabhay 4y agoHey GPT-5, come up with a way to defend us from this novel viral DNA Problem solved
- Uhhrrr 4y agoAn LLM will happily hallucinate a plausible-looking answer for you, with correct spelling and grammar.
- gammarator 4y agoHey GPT-5, tell me how to create the philosopher’s stone .