18 ms·

A lot of the AI doomerism comes from folks who do not understand the real complexities in making systems which really can function in a fully autonomous way in

A lot of the AI doomerism comes from folks who do not understand the real complexities in making systems which really can function in a fully autonomous way in an environment which is hostile and dynamic. Much like an AI maybe tasked to sniff for security loopholes, there will be other AI's which will be tasked to defend. Eventually costs, resources also boil down into what is possible. Evolutionary goals are not easy even with autonomous systems as goal definition is largely defined by societal needs, the environments we stay in and the resources we work with.

mycall 3y ago
Are not evolutionary goals part of keeping a process continous and long running? It is the implicit part which AIs are bound to discover (or emulate) and optimize against.
MSFT_Edging 3y ago
>A lot of the AI doomerism comes from folks who do not understand the real complexities in making systems which really can function in a fully autonomous way IMO the doomerism from what I see isn't skynet esque worries, but the usage of AI in really dumb ways. Ways such as inundating the internet with AI generated spam ie blogs, articles, art, music, fake forum interaction, etc. The smartest people are busy making AI work for us, the idiots are ruining everything for everyone else.
- staplers 3y ago
  the usage of AI in really dumb ways People in positions of power (ie. law enforcement) automating tasks like suspect generation, surveillance, ticket creation, etc is a much more sinister reality. We're talking about a civilian-hostile org using corruptible AI (via training) to enforce some of the murkiest grey areas of society. This stuff is already happening on the fringe and could become commonplace soon.
  - ninechars 3y ago
    This is my dystopian prediction of choice, too. Automatic categorisation of Good and Bad people would be (will be, already is) every authoritarian ruler's dream. We'll do it to ourselves long before "the machines take over".
  - MSFT_Edging 3y ago
    While it could get worse, authority over reach isn't new. People ignore the historical and current events now, ie the MOVE bombings or the execution of a protestor at the Atlanta Cop City site. Hell this just happened: https://kolektiva.social/@admin/110637031574056150 https://kolektiva.social/@admin/110637031574056150 All this isn't to write off the use of tech by bad state actors, but more a frustrated sigh, as law enforcement doesn't need AI to do all this, its happening already and no one cares.
- BSEdlMMldESB 3y ago
  > Ways such as inundating the internet with AI generated spam ie blogs, articles, art, music, fake forum interaction, etc. I think we're well past that... what I worry is about "interacting" online with bots designed to keep you busy and distracted
ajuc 3y ago
> Much like an AI maybe tasked to sniff for security loopholes, there will be other AI's which will be tasked to defend. Eventually costs, resources also boil down into what is possible. It's inherently easier to break stuff than to prevent damage.
gmerc 3y ago
we already created a dumb system of rules driven by profit maximizing entities called corporations that is exhausting the planet as we speak and we can’t seem to be able to control it, despite our survival depending on it. So no.
visarga 3y ago
> A lot of the AI doomerism comes from folks who do not understand the real complexities in making systems which really can function in a fully autonomous way in an environment which is hostile and dynamic. I agree. Autonomy is hard. But a weaker version of it is possible - self replication in software. An AI model can generate text that can train new models from scratch[1]. They can also generate the model code, explain it, make meaningful changes[2], monitor the training run and evaluate the "child" models[3]. So AI can "pull everything from inside" to make a child, no external materials needed, but any human generated or synthetic data can be used as well. AI is capable of doing half the job of self replication, but can't make GPUs, and probably won't be able to do it autonomously for a long time. High end GPUs are so hard to make no company or even country controls the whole stack, it only works through global cooperation. [1] TinyStories https://arxiv.org/abs/2305.07759 https://arxiv.org/abs/2305.07759 and Microsoft's Phi-1 using generated data [2] Evolution through Large Models https://arxiv.org/abs/2206.08896 https://arxiv.org/abs/2206.08896 [3] G-Eval: NLG Evaluation using GPT 4 with Better Human Alignment https://arxiv.org/abs/2303.16634 https://arxiv.org/abs/2303.16634
andyjohnson0 3y ago
You make good points, but I wonder what "costs" and "resources" mean in the context of a (hypothetical) self-enhancing, autonomous AI. All I can think of is computational substrate and the energy to power it. And once the AI has booted-up its obligatory drone army and orbital platforms, it can harvest very large amounts of both. Without us. Obviously I'm being alightly facetious, but my point is the constraints on an AI may not be ones we're familiar with as humans (society, environment, etc.) And again obviously, such a scenario is unlikely. But, like DNA replication, it only has to happen once and then it just keeps on happening. And then its game over for us, I reckon.
ftxbro 3y ago
> "folks who do not understand the real complexities in making systems which really can function in a fully autonomous way in an environment which is hostile and dynamic." OK but consider that guys like Noam Brown are becoming involved in LLM based AI, this was the guy who made the Poker bots that beat humans (libratus/pluribus) and the guy who made the first Diplomacy bot that can beat people (cicero). I mean those AIs didn't use LLMs and they weren't literally superhuman cognitive agents in the fully open ended world but I mean they are working on it right now and they appreciate the differences between adversarial and non adversarial environments as well as anyone probably even the military. Also the military is using these LLMs and they probably sometimes think about adversarial environments.
andsoitis 3y ago
A lot of the AI doomerism comes from folks who do not understand the real complexities in making systems which really can function in a fully autonomous way in an environment which is hostile and dynamic To be fair, a lot of those people take their cues from technologists who ostensibly know what they’re talking about.
_ea1k 3y ago
As always, we overlook the real risks. Even seen an elderly person get scammed by a cold caller? Now imagine an army of AI cold calling scammers, with realistic voices, steadily training on the absolute best scam techniques. How many people lose bank accounts before the scammer gets caught? Of course, this happens today, without AI, but as with many things in computing, scale changes everything!
- Balgair 3y ago
  Quantity has a Quality all it's own
barbariangrunge 3y ago
Some of the “dumb” systems we’ve built include nuclear weapons. It doesn’t need to be fully autonomous to be dangerous
tim333 3y ago
It also comes from folks who do understand such complexities. I mean Musk who's trying to make cars autonomous in dynamic hostile environments, Geoffrey Hinton who pioneered neural networks, Altman who's behind GTP4. I think the argument that AI isn't risky because the people warning about it are fools is not a good one. On the other hand there's an AI anti doom argument. Currently we are all doomed to die but maybe through AI upload like scenarios we can dedoom?
ChatGTP 3y ago
Ok, so Yoshua Bengio, Geoffrey Hinton or Max Tegmark aren't able to comprehend or speculate about this? Seems surprising. Edit: I'm not pandering to authority, I just believe the people I've quoted are actually very smart people who can reason very well and they have a valid opinion on the topic. They also have little financial interest in the success, failure or regulation of AI, which is important.
- sampo 3y ago
  > Ok, so Yoshua Bengio, Geoffrey Hinton or Max Tegmark aren't able to comprehend or speculate about this? I am genuinely amazed how even educated and intelligent people (or maybe predominantly them) have fallen into this, what I see is essentially a doomsday cult. But groupthink happens, maybe nobody is safe. It's not like it hasn't happened before.
- DalasNoin 3y ago
  People are downvoting you. But 'pandering' to authority is a valid point if the other site does an ad hominen (These people are not technical, don't REalleY understand what they are talking about)
- mrtranscendence 3y ago
  Strictly speaking they said “a lot”, not “all”.
  - ChatGTP 3y ago
    Fair point.
- verdverm 3y ago
  Why their doomerism over their peer's optimism. There are top tier AI researchers who think differently. How do you choose which to listen too?
  - digging 3y ago
    If optimism is wrong, we all die. if pessimism is wrong, we're fine.
jstanley 3y ago
It's not easy, but it only has to succeed once. It's not easy to turn primordial soup into humans, but it happened.
- echelon 3y ago
  > once > It's not easy to turn primordial soup into humans, but it happened. Over trillions or more changes. No single change is going to produce AGI. We're going to have a lot of forewarning, and it'll be obvious we're opening Pandora's box. AGI won't leap from zero to superintelligence that can launch nukes. That's not how gradient climbing and scaling work. Fearmongering is incredibly premature and is only going to produce regulation that favors incumbents. The way I see it is that Eliezer is the biggest corporate mouthpiece for Microsoft and OpenAI. He's doing their bidding and not even getting paid to do it.
  - ben_w 3y ago
    > No single change is going to produce AGI. We're going to have a lot of forewarning, and it'll be obvious we're opening Pandora's box. We had a lot of warning about climate change, chose economic growth anyway. Warnings about AI dangers include, but are not limited to: 1. "we found 40,000 chemical weapons overnight by flipping a 'minimise harm' to 'maximise harm": https://www.nature.com/articles/s42256-022-00465-9?fbclid=IwAR11_V1cd9SUxEvUfwrWMA7TUcroyYIY1nBDUL3KaS-8B4rG5MIqZCmjm0M https://www.nature.com/articles/s42256-022-00465-9?fbclid=Iw... 2. Bing chat threatening it's users almost immediately 3. All the stuff the red teams found so OpenAI could prevent them before chatGPT went live > AGI won't leap from zero to superintelligence that can launch nukes. An AGI is unlikely to want to. Ineffective way to kill off humans, but will annoy us, and will mess with power grid and logistics that it would benefit from. Conversely you don't need an AGI to exist for some idiot to be convinced of a bad idea that gets nukes launched anyway — Thule early warning radar incident is my favourite example of this, as it was caused by someone forgetting that the Moon doesn't have an IFF transponder and that's fine.
    - echelon 3y ago
      > 1. "we found 40,000 chemical weapons overnight by flipping a 'minimise harm' to 'maximise harm": https://www.nature.com/articles/s42256-022-00465-9?fbclid=Iw https://www.nature.com/articles/s42256-022-00465-9?fbclid=Iw... So what if it accelerates our discovery of ways to kill each other? We already have enough nukes to eradicate our species. Going the extra step to invent new chemical or biological terrors seems complicated. We already live in a world where we design chemical and biological weapons. To date, every chemical weapon was developed without AI. COVID may have even been engineered, and if it was, it was accomplished without AI. All of these capabilities exist, yet society remains intact. These activities are restricted and can only be undertaken by specialists in well-equipped, well-funded labs that are typically under the purview of an academic institution. You can't get access to dangerous chemical precursors or biological samples without credentials. Synthesis or development is hard, even if you have the plans. I'll only be worried if AI enables construction of these elements at sufficient yields without access to restricted chemical precursors or biological agents. If you can do this in your kitchen without BSL facilities, -80 freezers, etc., then we live in a truly "vulnerable world". That, however, remains to be seen. > 2. Bing chat threatening it's users almost immediately Have you heard of school bullies, gamer culture, or 4chan? Most humans have already been exposed to this negative behavior from peers, and it should only be a matter of time for products to move past this. > 3. All the stuff the red teams found so OpenAI could prevent them before chatGPT went live If anything, the complaints I hear most often is that they've neutered the behavior of their model. > Conversely you don't need an AGI to exist for some idiot to be convinced of a bad idea that gets nukes launched anyway Exactly. Humans are more liable to kill us than AGI at this point in time. We can reevaluate the risks when and if AGI actually comes about.
chaosjevil 3y ago
>A lot of the AI doomerism comes from folks who do not understand the real complexities in making systems which really can function in a fully autonomous way in an environment which is hostile and dynamic. That's a rather interesting mix of appeal to ignorance and argumentum ad hominem. Two genetic fallacies, together; neither addressing what is said, but instead who says it. >Evolutionary goals are not easy even with autonomous systems as goal definition is largely defined by societal needs, the environments we stay in and the resources we work with. Great way to say "I didn't read the article". The author is not talking about evolutionary _goals_.
- john-radio 3y ago
  > The author is not talking about evolutionary _goals_. Well... He is, actually, just not biological-evolutionary goals. (Like "natural selection," the term "evolution" can apply to anything that has appeared or might appear). I do think you're right that the article frames the topic pretty well and explores it well, including the concerns that the person you're responding to raised.
- avgcorrection 3y ago
  Oh my goodness they’re not even fallacies. A fallacy is saying that something is wrong because of fallacious reasons. But the OP is indirectly making the much weaker claim that “I’m not buying it and that’s because X and Y.” Speaking of fallacies, “AI doomers” (I’m just running with it) often deploy the rhetoric (not really a fallacy) that AI is about to doom us all because everything is supposedly so simple (like intelligence) and therefore it’s conceptually simple for a sufficiently advanced (but not that advanced) AI to keep improving itself. Now how do you respond to someone that just says that things are not complex when in reality things are indeed complex? Basically you have to unpack everything because the other person is just taking things at face value and is being naive. It’s like an “appeal to my ignorance”.
  - chaosjevil 3y ago
    >A fallacy is saying that something is wrong because of fallacious reasons. A fallacy is a basic flaw of logic reasoning supporting or ditching a claim, view, statement or position. The post above does it, implicitly, as yourself acknowledged - "I'm not buying it" is still a view - it's ditching a claim because of things unrelated to said claim, like "who said it". >often deploy the rhetoric (not really a fallacy) that AI is about to doom us all because everything is supposedly so simple (like intelligence) and therefore it’s conceptually simple for a sufficiently advanced (but not that advanced) AI to keep improving itself. It's both the rhetoric (on a discursive level) and a fallacy (on the logic level): oversimplification. (Although "appeal to my ignorance" sounds funnier.) >Now how do you respond to someone that just says that things are not complex when in reality things are indeed complex? Provided that you're talking with a somewhat reasonable person, you say something like "Shit is far more complex than you're pretending that it is.", then highlight a few instances of on-topic complexity that affect the outcome. Now, if you're talking with braindead trash, things are different. Those things aren't convinced by reason, so you use emotive appeal, even if it's itself a fallacy (you claim something because it makes someone feels good/bad, not due to its truth value). Something like "oh poor thing, complexity hurts your two neurons and your precious, OOOHH SO PRECIOUS! feelings? Too bad! Things won't become MaGiCaLlY simpler if you screech hard enough. Reality does not revolve around your belly."
noduerme 3y ago
A lot of "dumb" systems we develop require unimaginable resource inputs just to produce a little extra output. Strip mining for coal, or using chemical fertilizer to grow corn to produce ethanol, for instance. There is no guarantee these days that a system will fail just because it requires huge energy inputs to produce marginal profit. Evolutionary goals are not something that has to be aligned. Evolution isn't specific to organic life, it's an intrinsic rule of self-organizing systems. Viruses aren't alive, but they evolve. A clever stitch of self-writing code on a Pi attached to someone's TV may evolve without knowing or intending to. What makes this dangerous now is the vast amount of energy input toward specific systems. Saying "there's surely not enough energy available for it to..." is false comfort. It's underrating the process of evolution. As a poker analogy, if you just called the guy across from you because you think he couldn't possibly have more than two pair, you're wrong. You've bet into a full house. [edit: Upon review, I think I've unintentionally gone 100% Jeff Goldblum Jurassic Park in this response LOL]
- roenxi 3y ago
  I don't think that completely addresses gp's argument, he wasn't talking about energy or resourcing; just complexity. That being said, I still think gp's argument is flawed. We don't have examples of complexity that an AI can't overcome. Up until 2016 Go was a great example that was far to complex for an AI. Now there is no problem that an AI isn't expected, in principle, to outperform humans in with engineering effort. It is just a question of figuring out which problems it needs to outperform us in. Engineering effort is not in shortage. There are many capable people on hand and modern AI is going to be a perfect force multiplier for engineering work. We're going to figure it out. Complexity is maybe a good argument that we've got a few generations to go, but life for a 2nd tier intelligence is not a well-resourced one. It could easily be like this world except with humans displacing orangutans in 2nd place to the smart species.
  - ChatGTP 3y ago
    While I mostly agree with you , doesn’t AlphaGo does have problems with complexity which is how it was beaten quite easily in recent times ?
samstave 3y ago
>>*A lot of the AI doomerism comes from folks who do not understand the real complexities in making systems which really can function in a fully autonomous way in an environment which is hostile and dynamic* THIS IS exactly where AI doomerism SHOULD come from... What do we do with an unrully AI system fully autonomous in an environment with is both dynamic and hostile? THIS IS THE FOUNDATION OF THE PREMISE OF THE PROBLEM - "Please take a look at the current cadre of political candidates and map them to DND alignment charts, based on their actual votes/political activity" I hope we can get there one day. - I wonder if we can find FORK BOMB or rm -rf / type exploits on various GPTs/LLMs/AIs? I cant wait to see how devestating these will be -- or the first full executed security exploit by AI alone with a single prompt?

mycall 3y ago

Are not evolutionary goals part of keeping a process continous and long running? It is the implicit part which AIs are bound to discover (or emulate) and optimize against.

MSFT_Edging 3y ago

>A lot of the AI doomerism comes from folks who do not understand the real complexities in making systems which really can function in a fully autonomous way IMO the doomerism from what I see isn't skynet esque worries, but the usage of AI in really dumb ways. Ways such as inundating the internet with AI generated spam ie blogs, articles, art, music, fake forum interaction, etc. The smartest people are busy making AI work for us, the idiots are ruining everything for everyone else.

staplers 3y ago
the usage of AI in really dumb ways People in positions of power (ie. law enforcement) automating tasks like suspect generation, surveillance, ticket creation, etc is a much more sinister reality. We're talking about a civilian-hostile org using corruptible AI (via training) to enforce some of the murkiest grey areas of society. This stuff is already happening on the fringe and could become commonplace soon.
- ninechars 3y ago
  This is my dystopian prediction of choice, too. Automatic categorisation of Good and Bad people would be (will be, already is) every authoritarian ruler's dream. We'll do it to ourselves long before "the machines take over".
- MSFT_Edging 3y ago
  While it could get worse, authority over reach isn't new. People ignore the historical and current events now, ie the MOVE bombings or the execution of a protestor at the Atlanta Cop City site. Hell this just happened: https://kolektiva.social/@admin/110637031574056150 https://kolektiva.social/@admin/110637031574056150 All this isn't to write off the use of tech by bad state actors, but more a frustrated sigh, as law enforcement doesn't need AI to do all this, its happening already and no one cares.
BSEdlMMldESB 3y ago
> Ways such as inundating the internet with AI generated spam ie blogs, articles, art, music, fake forum interaction, etc. I think we're well past that... what I worry is about "interacting" online with bots designed to keep you busy and distracted

ajuc 3y ago

> Much like an AI maybe tasked to sniff for security loopholes, there will be other AI's which will be tasked to defend. Eventually costs, resources also boil down into what is possible. It's inherently easier to break stuff than to prevent damage.

gmerc 3y ago

we already created a dumb system of rules driven by profit maximizing entities called corporations that is exhausting the planet as we speak and we can’t seem to be able to control it, despite our survival depending on it. So no.

visarga 3y ago

> A lot of the AI doomerism comes from folks who do not understand the real complexities in making systems which really can function in a fully autonomous way in an environment which is hostile and dynamic. I agree. Autonomy is hard. But a weaker version of it is possible - self replication in software. An AI model can generate text that can train new models from scratch[1]. They can also generate the model code, explain it, make meaningful changes[2], monitor the training run and evaluate the "child" models[3]. So AI can "pull everything from inside" to make a child, no external materials needed, but any human generated or synthetic data can be used as well. AI is capable of doing half the job of self replication, but can't make GPUs, and probably won't be able to do it autonomously for a long time. High end GPUs are so hard to make no company or even country controls the whole stack, it only works through global cooperation. [1] TinyStories https://arxiv.org/abs/2305.07759 https://arxiv.org/abs/2305.07759 and Microsoft's Phi-1 using generated data [2] Evolution through Large Models https://arxiv.org/abs/2206.08896 https://arxiv.org/abs/2206.08896 [3] G-Eval: NLG Evaluation using GPT 4 with Better Human Alignment https://arxiv.org/abs/2303.16634 https://arxiv.org/abs/2303.16634

andyjohnson0 3y ago

You make good points, but I wonder what "costs" and "resources" mean in the context of a (hypothetical) self-enhancing, autonomous AI. All I can think of is computational substrate and the energy to power it. And once the AI has booted-up its obligatory drone army and orbital platforms, it can harvest very large amounts of both. Without us. Obviously I'm being alightly facetious, but my point is the constraints on an AI may not be ones we're familiar with as humans (society, environment, etc.) And again obviously, such a scenario is unlikely. But, like DNA replication, it only has to happen once and then it just keeps on happening. And then its game over for us, I reckon.

ftxbro 3y ago

> "folks who do not understand the real complexities in making systems which really can function in a fully autonomous way in an environment which is hostile and dynamic." OK but consider that guys like Noam Brown are becoming involved in LLM based AI, this was the guy who made the Poker bots that beat humans (libratus/pluribus) and the guy who made the first Diplomacy bot that can beat people (cicero). I mean those AIs didn't use LLMs and they weren't literally superhuman cognitive agents in the fully open ended world but I mean they are working on it right now and they appreciate the differences between adversarial and non adversarial environments as well as anyone probably even the military. Also the military is using these LLMs and they probably sometimes think about adversarial environments.

andsoitis 3y ago

_ea1k 3y ago

As always, we overlook the real risks. Even seen an elderly person get scammed by a cold caller? Now imagine an army of AI cold calling scammers, with realistic voices, steadily training on the absolute best scam techniques. How many people lose bank accounts before the scammer gets caught? Of course, this happens today, without AI, but as with many things in computing, scale changes everything!

Balgair 3y ago
Quantity has a Quality all it's own

barbariangrunge 3y ago

Some of the “dumb” systems we’ve built include nuclear weapons. It doesn’t need to be fully autonomous to be dangerous

tim333 3y ago

It also comes from folks who do understand such complexities. I mean Musk who's trying to make cars autonomous in dynamic hostile environments, Geoffrey Hinton who pioneered neural networks, Altman who's behind GTP4. I think the argument that AI isn't risky because the people warning about it are fools is not a good one. On the other hand there's an AI anti doom argument. Currently we are all doomed to die but maybe through AI upload like scenarios we can dedoom?

ChatGTP 3y ago

Ok, so Yoshua Bengio, Geoffrey Hinton or Max Tegmark aren't able to comprehend or speculate about this? Seems surprising. Edit: I'm not pandering to authority, I just believe the people I've quoted are actually very smart people who can reason very well and they have a valid opinion on the topic. They also have little financial interest in the success, failure or regulation of AI, which is important.

sampo 3y ago
> Ok, so Yoshua Bengio, Geoffrey Hinton or Max Tegmark aren't able to comprehend or speculate about this? I am genuinely amazed how even educated and intelligent people (or maybe predominantly them) have fallen into this, what I see is essentially a doomsday cult. But groupthink happens, maybe nobody is safe. It's not like it hasn't happened before.
DalasNoin 3y ago
People are downvoting you. But 'pandering' to authority is a valid point if the other site does an ad hominen (These people are not technical, don't REalleY understand what they are talking about)
mrtranscendence 3y ago
Strictly speaking they said “a lot”, not “all”.
- ChatGTP 3y ago
  Fair point.
verdverm 3y ago
Why their doomerism over their peer's optimism. There are top tier AI researchers who think differently. How do you choose which to listen too?
- digging 3y ago
  If optimism is wrong, we all die. if pessimism is wrong, we're fine.

jstanley 3y ago

It's not easy, but it only has to succeed once. It's not easy to turn primordial soup into humans, but it happened.

echelon 3y ago
> once > It's not easy to turn primordial soup into humans, but it happened. Over trillions or more changes. No single change is going to produce AGI. We're going to have a lot of forewarning, and it'll be obvious we're opening Pandora's box. AGI won't leap from zero to superintelligence that can launch nukes. That's not how gradient climbing and scaling work. Fearmongering is incredibly premature and is only going to produce regulation that favors incumbents. The way I see it is that Eliezer is the biggest corporate mouthpiece for Microsoft and OpenAI. He's doing their bidding and not even getting paid to do it.
- ben_w 3y ago
  > No single change is going to produce AGI. We're going to have a lot of forewarning, and it'll be obvious we're opening Pandora's box. We had a lot of warning about climate change, chose economic growth anyway. Warnings about AI dangers include, but are not limited to: 1. "we found 40,000 chemical weapons overnight by flipping a 'minimise harm' to 'maximise harm": https://www.nature.com/articles/s42256-022-00465-9?fbclid=IwAR11_V1cd9SUxEvUfwrWMA7TUcroyYIY1nBDUL3KaS-8B4rG5MIqZCmjm0M https://www.nature.com/articles/s42256-022-00465-9?fbclid=Iw... 2. Bing chat threatening it's users almost immediately 3. All the stuff the red teams found so OpenAI could prevent them before chatGPT went live > AGI won't leap from zero to superintelligence that can launch nukes. An AGI is unlikely to want to. Ineffective way to kill off humans, but will annoy us, and will mess with power grid and logistics that it would benefit from. Conversely you don't need an AGI to exist for some idiot to be convinced of a bad idea that gets nukes launched anyway — Thule early warning radar incident is my favourite example of this, as it was caused by someone forgetting that the Moon doesn't have an IFF transponder and that's fine.
  - echelon 3y ago
    > 1. "we found 40,000 chemical weapons overnight by flipping a 'minimise harm' to 'maximise harm": https://www.nature.com/articles/s42256-022-00465-9?fbclid=Iw https://www.nature.com/articles/s42256-022-00465-9?fbclid=Iw... So what if it accelerates our discovery of ways to kill each other? We already have enough nukes to eradicate our species. Going the extra step to invent new chemical or biological terrors seems complicated. We already live in a world where we design chemical and biological weapons. To date, every chemical weapon was developed without AI. COVID may have even been engineered, and if it was, it was accomplished without AI. All of these capabilities exist, yet society remains intact. These activities are restricted and can only be undertaken by specialists in well-equipped, well-funded labs that are typically under the purview of an academic institution. You can't get access to dangerous chemical precursors or biological samples without credentials. Synthesis or development is hard, even if you have the plans. I'll only be worried if AI enables construction of these elements at sufficient yields without access to restricted chemical precursors or biological agents. If you can do this in your kitchen without BSL facilities, -80 freezers, etc., then we live in a truly "vulnerable world". That, however, remains to be seen. > 2. Bing chat threatening it's users almost immediately Have you heard of school bullies, gamer culture, or 4chan? Most humans have already been exposed to this negative behavior from peers, and it should only be a matter of time for products to move past this. > 3. All the stuff the red teams found so OpenAI could prevent them before chatGPT went live If anything, the complaints I hear most often is that they've neutered the behavior of their model. > Conversely you don't need an AGI to exist for some idiot to be convinced of a bad idea that gets nukes launched anyway Exactly. Humans are more liable to kill us than AGI at this point in time. We can reevaluate the risks when and if AGI actually comes about.

chaosjevil 3y ago

>A lot of the AI doomerism comes from folks who do not understand the real complexities in making systems which really can function in a fully autonomous way in an environment which is hostile and dynamic. That's a rather interesting mix of appeal to ignorance and argumentum ad hominem. Two genetic fallacies, together; neither addressing what is said, but instead who says it. >Evolutionary goals are not easy even with autonomous systems as goal definition is largely defined by societal needs, the environments we stay in and the resources we work with. Great way to say "I didn't read the article". The author is not talking about evolutionary _goals_.

john-radio 3y ago
> The author is not talking about evolutionary _goals_. Well... He is, actually, just not biological-evolutionary goals. (Like "natural selection," the term "evolution" can apply to anything that has appeared or might appear). I do think you're right that the article frames the topic pretty well and explores it well, including the concerns that the person you're responding to raised.
avgcorrection 3y ago
Oh my goodness they’re not even fallacies. A fallacy is saying that something is wrong because of fallacious reasons. But the OP is indirectly making the much weaker claim that “I’m not buying it and that’s because X and Y.” Speaking of fallacies, “AI doomers” (I’m just running with it) often deploy the rhetoric (not really a fallacy) that AI is about to doom us all because everything is supposedly so simple (like intelligence) and therefore it’s conceptually simple for a sufficiently advanced (but not that advanced) AI to keep improving itself. Now how do you respond to someone that just says that things are not complex when in reality things are indeed complex? Basically you have to unpack everything because the other person is just taking things at face value and is being naive. It’s like an “appeal to my ignorance”.
- chaosjevil 3y ago
  >A fallacy is saying that something is wrong because of fallacious reasons. A fallacy is a basic flaw of logic reasoning supporting or ditching a claim, view, statement or position. The post above does it, implicitly, as yourself acknowledged - "I'm not buying it" is still a view - it's ditching a claim because of things unrelated to said claim, like "who said it". >often deploy the rhetoric (not really a fallacy) that AI is about to doom us all because everything is supposedly so simple (like intelligence) and therefore it’s conceptually simple for a sufficiently advanced (but not that advanced) AI to keep improving itself. It's both the rhetoric (on a discursive level) and a fallacy (on the logic level): oversimplification. (Although "appeal to my ignorance" sounds funnier.) >Now how do you respond to someone that just says that things are not complex when in reality things are indeed complex? Provided that you're talking with a somewhat reasonable person, you say something like "Shit is far more complex than you're pretending that it is.", then highlight a few instances of on-topic complexity that affect the outcome. Now, if you're talking with braindead trash, things are different. Those things aren't convinced by reason, so you use emotive appeal, even if it's itself a fallacy (you claim something because it makes someone feels good/bad, not due to its truth value). Something like "oh poor thing, complexity hurts your two neurons and your precious, OOOHH SO PRECIOUS! feelings? Too bad! Things won't become MaGiCaLlY simpler if you screech hard enough. Reality does not revolve around your belly."

noduerme 3y ago

A lot of "dumb" systems we develop require unimaginable resource inputs just to produce a little extra output. Strip mining for coal, or using chemical fertilizer to grow corn to produce ethanol, for instance. There is no guarantee these days that a system will fail just because it requires huge energy inputs to produce marginal profit. Evolutionary goals are not something that has to be aligned. Evolution isn't specific to organic life, it's an intrinsic rule of self-organizing systems. Viruses aren't alive, but they evolve. A clever stitch of self-writing code on a Pi attached to someone's TV may evolve without knowing or intending to. What makes this dangerous now is the vast amount of energy input toward specific systems. Saying "there's surely not enough energy available for it to..." is false comfort. It's underrating the process of evolution. As a poker analogy, if you just called the guy across from you because you think he couldn't possibly have more than two pair, you're wrong. You've bet into a full house. [edit: Upon review, I think I've unintentionally gone 100% Jeff Goldblum Jurassic Park in this response LOL]

roenxi 3y ago
I don't think that completely addresses gp's argument, he wasn't talking about energy or resourcing; just complexity. That being said, I still think gp's argument is flawed. We don't have examples of complexity that an AI can't overcome. Up until 2016 Go was a great example that was far to complex for an AI. Now there is no problem that an AI isn't expected, in principle, to outperform humans in with engineering effort. It is just a question of figuring out which problems it needs to outperform us in. Engineering effort is not in shortage. There are many capable people on hand and modern AI is going to be a perfect force multiplier for engineering work. We're going to figure it out. Complexity is maybe a good argument that we've got a few generations to go, but life for a 2nd tier intelligence is not a well-resourced one. It could easily be like this world except with humans displacing orangutans in 2nd place to the smart species.
- ChatGTP 3y ago
  While I mostly agree with you , doesn’t AlphaGo does have problems with complexity which is how it was beaten quite easily in recent times ?

samstave 3y ago

>>*A lot of the AI doomerism comes from folks who do not understand the real complexities in making systems which really can function in a fully autonomous way in an environment which is hostile and dynamic* THIS IS exactly where AI doomerism SHOULD come from... What do we do with an unrully AI system fully autonomous in an environment with is both dynamic and hostile? THIS IS THE FOUNDATION OF THE PREMISE OF THE PROBLEM - "Please take a look at the current cadre of political candidates and map them to DND alignment charts, based on their actual votes/political activity" I hope we can get there one day. - I wonder if we can find FORK BOMB or rm -rf / type exploits on various GPTs/LLMs/AIs? I cant wait to see how devestating these will be -- or the first full executed security exploit by AI alone with a single prompt?