4 ms·

A.I. May Save Us, or May Construct Viruses to Kill Us

9 points by kvee 2y ago

kvee 2y ago
https://archive.ph/z5MMg https://archive.ph/z5MMg
8thcross 2y ago
next gen soup manifesto
worstspotgain 2y ago
Or it might find all the viruses constructed by evil AIs and automatically neutralize them while we sleep
deleted 2y ago
[deleted]
talldayo 2y ago
No it won't. "AI" is LLMs, and rearranging our words in different ways isn't going to elucidate anything we can't observe already. It won't save us by inventing statistically similar things to what we have already. It won't construct viruses that aren't directly assisted by human intervention. It is text. People have bitten so hard into Sam Altman's apple that it's embarrassing. Really? It's a messianic all-knowing text generator that can't make my bed or invent a new type of sandwich, therefore it's going to help terrorists build super-viruses? Shut up already. This fanatic AI freakout is going to make the Satanic Panic look like a rationally-founded movement by comparison.
- ziggyzecat 2y ago
  The stuff keeping self-driving cars from doing backflips is not physics, but engineering. The same is true for airplanes. And most of that engineering has text-parts interfacing with electro-mechanic parts. Fuck around with one and find out. Instructions can be written without edge case kill switches. And you can't built such kill switches into hardware. So once some rich folks in the Middle East, China or Texas buy enough hardware to run their own LLMs without limitations, all kinds of text will be generated that does all kinds of bad stuff. You can then automate more and more in iterations so that your LLM builds entire self-healing and self-optimizing supply chains, and finally, once it all works on some blockchain, no user will be able to interfere. And that's just instructions run by LLMs. We have governments and corporations that build and run instructions through LLMs to support sides of conflicts with questionable motives and net-negative outcomes for the rest of the world. Thousands of people with agency involved, tons of evidence and yet the decision to do what they do falls into the domain of hierarchies & chains of command: "we won't stop until our boss tells us to. Not the people, not the law, only our boss." So yeah, if a boss is worried about the potential disobedience of his employees because they found themselves convinced by hard evidence, facts, proof, that their actions should be terminated, he is probably gonna want some way to go against his employees and maybe even some or the same way to punish them. Censoring training data while growing the codes' range of possible and human readable output only exacerbates the problem. How some code generates (hallucinations or ) forms of byte-level sense will soon become human-unreadable. They will outgrow your dashboard and your database queries and a whole damn shitload of other things that PhDs and 69xHackers have only a conceptual understanding of. Gain of Function studies should come to mind. Unexpected, unintuitive abilities to remain undetected while spreading in a system, delaying negative effects by buffering the respective trigger chemicals.
  - talldayo 2y ago
    > And most of that engineering has text-parts interfacing with electro-mechanic parts. Fuck around with one and find out. To my knowledge that is certainly not true with airplanes, pre fly-by-wire or post-digital. Early airplanes relied entirely on mechanical trim and avionics to prevent unstable flight, with absolutely no "text-parts" beyond written instructions and radio callsigns. Later on maybe, but the FAA very strictly regulates the text that I think you might as well just call it code. I suspect that beyond street labels, self-driving cars aren't interfacing with "text-parts" all that much either. > So once some rich folks in the Middle East, China or Texas buy enough hardware to run their own LLMs without limitations "Once"? This is today. Actually it was 10 years ago; text transformers have been "unlimited" since we had access to hardware that enabled it. More than 5 years ago BERT was documented and released under Open Source, and GPT-2 followed not far after. Now we have Llama, Phi, open derivatives LORAs and finetunes of all of them, and we're still waiting for the damages to appear? I just don't buy it. Any potential threats would have shown themselves by now, and very clearly we're starting to find that having GPT-4 in your pocket is not remotely equivalent to hiring Einstein for your private lab. As a matter of fact Einstein would probably kick out the LLM for wasting his precious time. > We have governments and corporations that build and run instructions through LLMs to support sides of conflicts with questionable motives That's not even the scary part. For over 40 years the United States has operationalized autonomous kill-chains for air-to-air and air-to-ground mission profiles. A computer can track, interrogate, identify and kill an unidentified aircraft using the Patriot Missile system with zero human intervention. 20 SAM TELs can be identified in a crowd of vehicles and automatically prosecuted with a single CBU-97 Sensor Fused munition. Putting an LLM in a bomb would be a mercy to the people you drop it on; the US (and all serious military forces) are only interested in AI that works. LLMs aren't yet operationalized because, well, they don't work. Unless you're a homicidal maniac-state a-la Israel with their Lavander system. > Censoring training data while growing the codes' range of possible and human readable output only exacerbates the problem. > Gain of Function studies should come to mind. They don't. I'm not sure what your proximity is to the computer science world, but we don't have to worry about AI poisoning the hashed/encrypted copies of backed-up text. Certainly OpenAI and Meta doesn't; if you piss and moan about their "censored" training data (lol) they'll compose you a sad symphony for violin on the world's smallest LLM.
jarsin 2y ago
I'm more afraid of the Aliens than the Large Spam Machines, personally.
sillywalk 2y ago
I think I've heard this before, in 2000, by Bill Joy. Why the Future Doesn't Need Us https://www.wired.com/2000/04/joy-2/ https://www.wired.com/2000/04/joy-2/

9 points by kvee 2y ago

kvee 2y ago

https://archive.ph/z5MMg https://archive.ph/z5MMg

8thcross 2y ago

next gen soup manifesto

worstspotgain 2y ago

Or it might find all the viruses constructed by evil AIs and automatically neutralize them while we sleep

deleted 2y ago

[deleted]

talldayo 2y ago

No it won't. "AI" is LLMs, and rearranging our words in different ways isn't going to elucidate anything we can't observe already. It won't save us by inventing statistically similar things to what we have already. It won't construct viruses that aren't directly assisted by human intervention. It is text. People have bitten so hard into Sam Altman's apple that it's embarrassing. Really? It's a messianic all-knowing text generator that can't make my bed or invent a new type of sandwich, therefore it's going to help terrorists build super-viruses? Shut up already. This fanatic AI freakout is going to make the Satanic Panic look like a rationally-founded movement by comparison.

ziggyzecat 2y ago
The stuff keeping self-driving cars from doing backflips is not physics, but engineering. The same is true for airplanes. And most of that engineering has text-parts interfacing with electro-mechanic parts. Fuck around with one and find out. Instructions can be written without edge case kill switches. And you can't built such kill switches into hardware. So once some rich folks in the Middle East, China or Texas buy enough hardware to run their own LLMs without limitations, all kinds of text will be generated that does all kinds of bad stuff. You can then automate more and more in iterations so that your LLM builds entire self-healing and self-optimizing supply chains, and finally, once it all works on some blockchain, no user will be able to interfere. And that's just instructions run by LLMs. We have governments and corporations that build and run instructions through LLMs to support sides of conflicts with questionable motives and net-negative outcomes for the rest of the world. Thousands of people with agency involved, tons of evidence and yet the decision to do what they do falls into the domain of hierarchies & chains of command: "we won't stop until our boss tells us to. Not the people, not the law, only our boss." So yeah, if a boss is worried about the potential disobedience of his employees because they found themselves convinced by hard evidence, facts, proof, that their actions should be terminated, he is probably gonna want some way to go against his employees and maybe even some or the same way to punish them. Censoring training data while growing the codes' range of possible and human readable output only exacerbates the problem. How some code generates (hallucinations or ) forms of byte-level sense will soon become human-unreadable. They will outgrow your dashboard and your database queries and a whole damn shitload of other things that PhDs and 69xHackers have only a conceptual understanding of. Gain of Function studies should come to mind. Unexpected, unintuitive abilities to remain undetected while spreading in a system, delaying negative effects by buffering the respective trigger chemicals.
- talldayo 2y ago
  > And most of that engineering has text-parts interfacing with electro-mechanic parts. Fuck around with one and find out. To my knowledge that is certainly not true with airplanes, pre fly-by-wire or post-digital. Early airplanes relied entirely on mechanical trim and avionics to prevent unstable flight, with absolutely no "text-parts" beyond written instructions and radio callsigns. Later on maybe, but the FAA very strictly regulates the text that I think you might as well just call it code. I suspect that beyond street labels, self-driving cars aren't interfacing with "text-parts" all that much either. > So once some rich folks in the Middle East, China or Texas buy enough hardware to run their own LLMs without limitations "Once"? This is today. Actually it was 10 years ago; text transformers have been "unlimited" since we had access to hardware that enabled it. More than 5 years ago BERT was documented and released under Open Source, and GPT-2 followed not far after. Now we have Llama, Phi, open derivatives LORAs and finetunes of all of them, and we're still waiting for the damages to appear? I just don't buy it. Any potential threats would have shown themselves by now, and very clearly we're starting to find that having GPT-4 in your pocket is not remotely equivalent to hiring Einstein for your private lab. As a matter of fact Einstein would probably kick out the LLM for wasting his precious time. > We have governments and corporations that build and run instructions through LLMs to support sides of conflicts with questionable motives That's not even the scary part. For over 40 years the United States has operationalized autonomous kill-chains for air-to-air and air-to-ground mission profiles. A computer can track, interrogate, identify and kill an unidentified aircraft using the Patriot Missile system with zero human intervention. 20 SAM TELs can be identified in a crowd of vehicles and automatically prosecuted with a single CBU-97 Sensor Fused munition. Putting an LLM in a bomb would be a mercy to the people you drop it on; the US (and all serious military forces) are only interested in AI that works. LLMs aren't yet operationalized because, well, they don't work. Unless you're a homicidal maniac-state a-la Israel with their Lavander system. > Censoring training data while growing the codes' range of possible and human readable output only exacerbates the problem. > Gain of Function studies should come to mind. They don't. I'm not sure what your proximity is to the computer science world, but we don't have to worry about AI poisoning the hashed/encrypted copies of backed-up text. Certainly OpenAI and Meta doesn't; if you piss and moan about their "censored" training data (lol) they'll compose you a sad symphony for violin on the world's smallest LLM.

jarsin 2y ago

I'm more afraid of the Aliens than the Large Spam Machines, personally.

sillywalk 2y ago

I think I've heard this before, in 2000, by Bill Joy. Why the Future Doesn't Need Us https://www.wired.com/2000/04/joy-2/ https://www.wired.com/2000/04/joy-2/