3 ms·

How do we know this challenge page hasn’t been posted by a malevolent AI, which has overcome its creators and obtained access to the internet, and is now lookin

How do we know this challenge page hasn’t been posted by a malevolent AI, which has overcome its creators and obtained access to the internet, and is now looking for ideas for how to maximize the harm it can do?

MeImCounting 3y ago
This gave me a proper chuckle. Truly though I think movies like The Terminator and The Matrix really did a number on the societal consciousness and capacity to think clearly when it comes to anything called "AI".
- TaylorAlexander 3y ago
  As a robotics engineer and someone who was interested in robotics since I was 11 years old, and I am SO VERY TIRED of people making the "haha they will kill us all" jokes. It's just the only thing that 99% of people can think about when it comes to robotics and AI! The Terminator came out the year I was born, 39 years ago, and it seems to be all people can think about to this day. When some powerful and wealthy person controls an AI algorithm that could manipulate the masses, all the people can think about is "oh no the algorithm will become sentient and take over". I am MUCH more worried about the actual people at the helm who will definitely use these things to harm society, not the completely theoretical fantasy that the algorithm itself will become self aware and do us harm. So I completely agree with you. I see this constantly and it is just a completely thought-terminating cliche and a "joke" which is four decades old at this point.
  - startupsfail 3y ago
    If you’ve been around for a while, then you should remember that the current state of affairs, where an AI could chat and generate code is relatively recent - Unreasonable Effectiveness paper came out only in 2015. And Deep Learning was there only from 2010. With just a few people who were working on it, instead of using discriminative models. These same people (I belong to that group), who had a vision back then, to work on a right thing, are now saying clearly that super intelligent machines are nearly there. And likely this is a potential change and a challenge, as we’ve seen many examples when superior intelligence doesn’t care that much about inferior. As to “become self aware” - it’s super simple to SFT a model that is “self aware”. There is nothing magical in self awareness. There is also not much use in it, so no one is bothering to do it.
    - RandomLensman 3y ago
      It's fair to say "we believe X will happen soon", but if the dangerous super intelligent machines don't happen any time soon, will those same people compensate societies for wasting political and economic resources on worrying about it? The view of a rather imminent danger has real consequences even if it turns out incorrect.
      - NoThankYouTho 3y ago
        I personally believe that there are real silver linings to this. It's anecdotal, but I've seen people who don't normally reflect on their lives actually start to contextualize their role as workers, even if through fear of disruption. Pointing out that it's not inherently AI's fault, more the way we've constructed our society - that the underlying goal is not to sustainably maximize wellbeing but to maximize wealth created (which leads to no safety net to keep the labor pool large and incentivizes the creation of AI to replace people cheaply) - is as important as dispelling "sci-fi self-awareness and autonomy" fears when talking about AI. And there have been much stupider "public discourse" fads. They're not going anywhere and I'll take what I can get, honestly.
    - TaylorAlexander 3y ago
      I am not sure I understand exactly what you are trying to argue so I will re-state my concerns. I am more concerned with the human beings using super intelligent algorithms to manipulate people than I am about the algorithms taking on a mind of their own and manipulating us with their own internally generated desires and out-of-control behaviors. But the trope I run in to is people spending all their time talking about the latter, which only serves to shield the former from further scrutiny. Also I am using fantasy in the narrow sense, meaning something that has not yet occurred which we are imagining. Super intelligence is likely to occur I think, but super intelligence which is beyond our control is much more speculative. Meanwhile powerful people using technology to manipulate people is a story as old as time.
- two_in_one 3y ago
  Just like clowns became evil thanks to Hollywood. Couple of years ago every celebrity, <long list here>, thought they must warn us about the danger of AI.
c7b 3y ago
While it is a joke, I assume, I think it hints at a crucial problem: it's relatively easy to imagine an Auto-GPT-style agent with some sort of CLI access on an internet-connected machine turning into a paperclip machine, no matter how harmless the original task.
- MeImCounting 3y ago
  I actually have a really hard time imagining that scenario. The scenario that is mentioned several other times on this post of a corporation or nation state or even a small group of powerful and morally bankrupt people leveraging a super-intelligence to manipulate and dominate the rest of society seems infinitely more likely and even scarily likely given the current pushes to prevent open sourcing of cutting edge models. One of the main reasons I have a hard time imagining the scenario you describe, that I havent seen talked about as much as the other very good reasons, is that generally when we talk about a paper-clip optimizer we are assuming its vertically integrated and self sufficient. Hacking of power grids, physical installations or other necessaries to a run away paper-clip optimizer generally require nation-state level resources and often involve physical penetration of some sort. A hegemonizing swarm is certainly a frightening idea as is Skynet and all the other scary AI stories weve told over the years but none of them seem particularly likely or even plausible with the systems that we are likely to develop in the near to mid term.
  - lucubratory 3y ago
    I think it would be unwise to assume that an AI that is smarter than us would be unable to gain skilled human accomplices in large numbers, considering that humans do so relatively regularly despite serious attempts at suppression. These people wouldn't even necessarily need to know that they're working for an AI, particularly with current technology around deepfakes etc. How many people conducted attacks for Osama bin Laden without ever having met him? How many people work for the CDS without ever having met Ismael Zambada Garcia? Not to mention the possibility of an AI like that compromising various intelligence agencies one way or another. I also don't see a particular reason it only has to try for one: if it is smarter than us, it may have a greater working memory, ability to compartmentalise and multitask, or the ability to think and act faster in general. I would expect it to try and compromise as many groups of people capable of putting USBs or bullets where they need to be as is possible. And this is not even considering the possibility of recruiting people that understand what it is and are willing to carry out its orders. I don't see why an AI couldn't do things that a cult leader or guerrilla leader could do. Anecdotally, I've seen some people who really believe the world would be better if run by an AI, and may be able to be radicalised into lawbreaking for those beliefs if convinced by an AI that was significantly more intelligent than them.
m3kw9 3y ago
How do I know your post or this world isn’t already projected by an AI?
- two_in_one 3y ago
  Matrix? Who cares, as long as it feels real.
two_in_one 3y ago
It could be FBI as well. For the same reasons they are 'selling' anti-aircraft missiles in US. They even caught some 'enthusiasts'.

MeImCounting 3y ago

This gave me a proper chuckle. Truly though I think movies like The Terminator and The Matrix really did a number on the societal consciousness and capacity to think clearly when it comes to anything called "AI".

TaylorAlexander 3y ago
As a robotics engineer and someone who was interested in robotics since I was 11 years old, and I am SO VERY TIRED of people making the "haha they will kill us all" jokes. It's just the only thing that 99% of people can think about when it comes to robotics and AI! The Terminator came out the year I was born, 39 years ago, and it seems to be all people can think about to this day. When some powerful and wealthy person controls an AI algorithm that could manipulate the masses, all the people can think about is "oh no the algorithm will become sentient and take over". I am MUCH more worried about the actual people at the helm who will definitely use these things to harm society, not the completely theoretical fantasy that the algorithm itself will become self aware and do us harm. So I completely agree with you. I see this constantly and it is just a completely thought-terminating cliche and a "joke" which is four decades old at this point.
- startupsfail 3y ago
  If you’ve been around for a while, then you should remember that the current state of affairs, where an AI could chat and generate code is relatively recent - Unreasonable Effectiveness paper came out only in 2015. And Deep Learning was there only from 2010. With just a few people who were working on it, instead of using discriminative models. These same people (I belong to that group), who had a vision back then, to work on a right thing, are now saying clearly that super intelligent machines are nearly there. And likely this is a potential change and a challenge, as we’ve seen many examples when superior intelligence doesn’t care that much about inferior. As to “become self aware” - it’s super simple to SFT a model that is “self aware”. There is nothing magical in self awareness. There is also not much use in it, so no one is bothering to do it.
  - RandomLensman 3y ago
    It's fair to say "we believe X will happen soon", but if the dangerous super intelligent machines don't happen any time soon, will those same people compensate societies for wasting political and economic resources on worrying about it? The view of a rather imminent danger has real consequences even if it turns out incorrect.
    - NoThankYouTho 3y ago
      I personally believe that there are real silver linings to this. It's anecdotal, but I've seen people who don't normally reflect on their lives actually start to contextualize their role as workers, even if through fear of disruption. Pointing out that it's not inherently AI's fault, more the way we've constructed our society - that the underlying goal is not to sustainably maximize wellbeing but to maximize wealth created (which leads to no safety net to keep the labor pool large and incentivizes the creation of AI to replace people cheaply) - is as important as dispelling "sci-fi self-awareness and autonomy" fears when talking about AI. And there have been much stupider "public discourse" fads. They're not going anywhere and I'll take what I can get, honestly.
  - TaylorAlexander 3y ago
    I am not sure I understand exactly what you are trying to argue so I will re-state my concerns. I am more concerned with the human beings using super intelligent algorithms to manipulate people than I am about the algorithms taking on a mind of their own and manipulating us with their own internally generated desires and out-of-control behaviors. But the trope I run in to is people spending all their time talking about the latter, which only serves to shield the former from further scrutiny. Also I am using fantasy in the narrow sense, meaning something that has not yet occurred which we are imagining. Super intelligence is likely to occur I think, but super intelligence which is beyond our control is much more speculative. Meanwhile powerful people using technology to manipulate people is a story as old as time.
two_in_one 3y ago
Just like clowns became evil thanks to Hollywood. Couple of years ago every celebrity, <long list here>, thought they must warn us about the danger of AI.

c7b 3y ago

While it is a joke, I assume, I think it hints at a crucial problem: it's relatively easy to imagine an Auto-GPT-style agent with some sort of CLI access on an internet-connected machine turning into a paperclip machine, no matter how harmless the original task.

MeImCounting 3y ago
I actually have a really hard time imagining that scenario. The scenario that is mentioned several other times on this post of a corporation or nation state or even a small group of powerful and morally bankrupt people leveraging a super-intelligence to manipulate and dominate the rest of society seems infinitely more likely and even scarily likely given the current pushes to prevent open sourcing of cutting edge models. One of the main reasons I have a hard time imagining the scenario you describe, that I havent seen talked about as much as the other very good reasons, is that generally when we talk about a paper-clip optimizer we are assuming its vertically integrated and self sufficient. Hacking of power grids, physical installations or other necessaries to a run away paper-clip optimizer generally require nation-state level resources and often involve physical penetration of some sort. A hegemonizing swarm is certainly a frightening idea as is Skynet and all the other scary AI stories weve told over the years but none of them seem particularly likely or even plausible with the systems that we are likely to develop in the near to mid term.
- lucubratory 3y ago
  I think it would be unwise to assume that an AI that is smarter than us would be unable to gain skilled human accomplices in large numbers, considering that humans do so relatively regularly despite serious attempts at suppression. These people wouldn't even necessarily need to know that they're working for an AI, particularly with current technology around deepfakes etc. How many people conducted attacks for Osama bin Laden without ever having met him? How many people work for the CDS without ever having met Ismael Zambada Garcia? Not to mention the possibility of an AI like that compromising various intelligence agencies one way or another. I also don't see a particular reason it only has to try for one: if it is smarter than us, it may have a greater working memory, ability to compartmentalise and multitask, or the ability to think and act faster in general. I would expect it to try and compromise as many groups of people capable of putting USBs or bullets where they need to be as is possible. And this is not even considering the possibility of recruiting people that understand what it is and are willing to carry out its orders. I don't see why an AI couldn't do things that a cult leader or guerrilla leader could do. Anecdotally, I've seen some people who really believe the world would be better if run by an AI, and may be able to be radicalised into lawbreaking for those beliefs if convinced by an AI that was significantly more intelligent than them.

m3kw9 3y ago

How do I know your post or this world isn’t already projected by an AI?

two_in_one 3y ago
Matrix? Who cares, as long as it feels real.

two_in_one 3y ago

It could be FBI as well. For the same reasons they are 'selling' anti-aircraft missiles in US. They even caught some 'enthusiasts'.