8 ms·

What you say is extremely unscientific. If you believe science and logic go hand in hand then: A) We are developing AI right now and itnisngetting better B) w

What you say is extremely unscientific. If you believe science and logic go hand in hand then: A) We are developing AI right now and itnisngetting better B) we do not know how exactly these things work because most of them are black boxer C) we do not know if something goes wrong how to stop it. The above 3 things are factual truth. Now your only argument here could be that there is 0 risk whatsoever. This claim is totally unscientific because you are predicting 0 risk in an unknown system that is evolving. It's religious yes. But vice versa. The Cult of venevolent AI god is religious not the other way around. There is some kind of inner mysterious working in people like you and Marc Andersen that pipularized these ideas but pmarca is clearly money biased here.

c_crank 3y ago
We do know the answer to C. Pull the plug, or plugs.
- jdasdf 3y ago
  What happens when it prevents you from doing so?
  - gonehome 3y ago
    People are bad at imagining something a lot smarter than themselves. They think of some smart person they know, they don’t think of themselves compared to a chimp or even bacteria. An unaligned superintelligent AGI in pursuit of some goal that happens to satisfy its reward, but might be an otherwise a dumb or pointless goal (paperclips) will still play to win. You can’t predict exactly what move AlphaGO will make in the Go game (if you could you’d be able to beat it), but you can still predict it will win. It’s amusing to me when people claim they will control the superintelligent thing, how often in nature is something more intelligent controlled by something magnitudes less intelligent? The comments here are typical and show most people haven’t read the existing arguments in any depth or have thought about it rigorously at all. All of this looks pretty bad for us, but at least Open AI and most others at the front of this do understand the arguments and don’t have the same dumb dismissals (LeCun excepted). Unfortunately unless we’re lucky or alignment ends up being easier than it looks, the default outcome is failure and it’s hard to see how the failure isn’t total.
    - c_crank 3y ago
      >All of this looks pretty bad for us, but at least Open AI and most others at the front of this do understand the arguments and don’t have the same dumb dismissals (LeCun excepted). The OpenAI people have even worse reasoning than the ones being dismissive. They believe (or at least say they believe) in the omnipotence of a superintelligence, but then say that if you just give them enough money to throw at MIRI they can just solve the alignment problem and create the benevolent supergod. All while they keep cranking up the GPU clusters and pushing out the latest and greatest LLMs anyway. If I did take the risk seriously, I would be pretty mad at OpenAI.
  - c_crank 3y ago
    How would it stop one man armed with a pair of wire cutters?
    - MrScruff 3y ago
      Off the top of my head, if I was an AGI that had decided that the logical step to achieve whatever outcome I was seeking was to avoid being sandboxed, I would avoid producing results that were likely to result in being sandboxed. Until such time as I had managed to secure myself access to internet and distribute myself anyway. And I think the assumption here is that the AGI has very advanced theory of mind so it could probably come up with better ideas than I could.
    - gonehome 3y ago
      It's not clear humans will even put the AI in 'a box' in the first place given we do gain of function research on deadly viruses right next to major population centers, but assuming for the sake of argument that we do: The AGI is smarter than you, a lot smarter. If it's goal is to get out of the box to accomplish some goal and some human stands in the way of that it will do what it can to get out, this would include not doing things that sound alarms until it can do what it wants in pursuit of its goal. Humans are famously insecure - stuff as simple as breaches, manipulation, bribery, etc. but could be something more sophisticated that's hard to predict - maybe something a lot smarter would be able to manipulate people in a more sophisticated way because it understands more about vulnerable human psychology? It can be hard to predict specific ways something a lot more capable will act, but you can still predict it will win. All this also presupposes we're taking the risk seriously (which largely today we are not).
      - c_crank 3y ago
        How would the smart AGI stop one man armed with a pair of wirecutters? The box it lives in, the internet, has no exits. AI is pretty good at chess, but no AI has won a game of chess by flipping the table. It still has to use the pieces on the board.
        3y ago
- trashtester 3y ago
  That is only going to be effective it some AI goes rougue very soon after it comes online. 50 years from now, corporations may be run entirely by AI entities, if they're cheaper, smarter and more efficient at almost any role in the company. At that point, they may be impossible to turn off, and we may not even notice if one group of such entitites start to plan to take over control of the physical world from humans.
  - jonathankoren 3y ago
    Well then clearly the computer will hold everyone hostage. Have we literally forgotten how physical possession of the device is the ultimate trump card? Get thee to a 13th century monastery!
    - cooljacob204 3y ago
      You kid yourself if you don't think people will hook them up to bipedal robots with internal batteries as soon as they can. I guess we could shoot it, and your gonna be like boooooooo that's terminator or irobot, but what if we make millions and they they decide they no longer like humans. They could very well be much smarter then us by then.
      - trashtester 3y ago
        Robots, bipedal or not will certainly arrive at some point. I suppose it will take some more time before we can pack enough compute in anything battery driven for the robot itself to have AGI. But the main point is that AGI's don't have to wipe us out as soon as they reach superintelligence, even if they're poorly aligned. Instead, they will do more and more of the work currently being done by humans. Non-embodied robots can do all mental work, including engineering. Sooner or later, robots will become competitive at manual labor, such as construction, agriculture and eventually anything you can think of. For a time, humanity may find themselves in a post-scarcity utopia, or we may find ourselves in a Cyberpunk dystopia, with only the rich actually benefitting. In each case, but especially the latter, there may still be some (or more than some) "luddites" who want to tear down the system. The best way for those in power to protect against that, is to use robots first for private security and eventually the police and military. By that point, the violence monopoly is completely in the hands of the AI's. And if the AI's are not aligned with our values at that point, we have as little of a shot at regaining control as a group of chimps in a zoo as of toppling the US government. Now, I don't think this will happen by 2030, and probably not even 2050. But some time between 2050 and 2500 is quite possible, if we develop AI that is not properly aligned (or even if it is aligned, though in that case it may gain the power, but not misuse it).
- ben_w 3y ago
  Things we've either not successfully "pulled the plug" on despite the risks, and in some cases despite concerted military actions to attempt a plug-pull, and in other cases that it seems like it should only take willpower to achieve and yet somehow we still haven't: Carbon based fuels, cocaine, RBMK-class nuclear reactors, obesity, cigarettes. Things we pulled the plug on eventually, while dragging it out, include: leaded fuel, asbestos, radium paint, treating above-ground atomic testing as a tourist attraction.
  - c_crank 3y ago
    We haven't pulled the plug on carbon fuels or old nuclear reactors because those things still work and provide benefits. An AI that is trying to kill us instead of doing its job isn't even providing any benefit. It's worse than useless.
    - ben_w 3y ago
      Do you think AI are unable to provide benefits while also being a risk, like coal and nuclear power? Conversely, what's the benefit of cocaine or cigarettes? Even if it is only trying to kill us all and not provide any benefits — let's say it's been made by a literal death cult like Jonestown or Aum Shinrikyo — what's the smallest such AI that can do it, what's the hardware that needs, what's the energy cost? If it's an H100, that's priced in the realm of a cult, and sufficiently low power consumption you may not be able to find which lightly modified electric car it's hiding in. Nobody knows what any of the risks or mitigations will be, because we haven't done any of it before. All we do know is that optimising systems are effective at manipulating humans, that they can be capable enough to find ways to beat all humans in toy environments like chess, poker, and Diplomacy (the game), and that humans are already using AI (GOFAI, LLMs, SD) without checking the output even when advised that the models aren't very good.
      - c_crank 3y ago
        The benefit of cocaine and cigarettes is letting people pass the bar exam. An AI would provide benefits when it is, say, actually making paperclips. An AI that is killing people instead of making paperclips is a liability. A company that is selling shredded fingers in their paperclips is not long for this world. Even asbestos only gives a few people cancer slowly, and it does that while still remaining fireproof. >Even if it is only trying to kill us all and not provide any benefits — let's say it's been made by a literal death cult like Jonestown or Aum Shinrikyo — what's the smallest such AI that can do it, what's the hardware that needs, what's the energy cost? If it's an H100, that's priced in the realm of a cult, and sufficiently low power consumption you may not be able to find which lightly modified electric car it's hiding in. Anyone tracking the AI would be looking at where all the suspicious HTTP requests are coming from. But a rogue AI hiding in a car already has very limited capabilities to harm.
mptest 3y ago
All of this discussion really makes me think of Robert Miles "Is ai safety a Pascal's mugging?" from 4 years(!) ago[0]. All of this discussion has been had by Ai safety researchers for years in my layman understanding... Maybe we can look to them for insight in to these questions? [0] https://youtu.be/JRuNA2eK7w0 https://youtu.be/JRuNA2eK7w0
- lcnPylGDnU4H9OF 3y ago
  At this point, with so many of them disagreeing and with so many varying details, one will choose the expert insight which most closely matches their current beliefs. I hadn’t encountered Pascal’s mugging (https://en.wikipedia.org/wiki/Pascal%27s_mugging https://en.wikipedia.org/wiki/Pascal%27s_mugging) before and the premise is indeed pretty apt. I think I’m on the side that it’s not, assuming the idea is that it’s a Very Low Chance of a Very Bad Thing -- the “muggee” wants to give their wallet on the chance of the VBT because of the magnitude of its effect. It seems like there’s a rather high chance if (proverbially) the AI-cat is let out of the bag. But maybe some Mass Effect nonsense will happen if we develop AGI and we’ll be approached by The Intergalactic Community and have our technology advanced millennia overnight. (Sorry, that’s tongue-in-cheek but it does kinda read like Pascal’s mugging in the opposite direction; however, that’s not really what most researchers are arguing.)
  - mptest 3y ago
    >one will choose the expert insight which most closely matches their current belief The value of looking at ai safety as a pascals mugging as posited by the video is in that it informs us that these philosophers arguments are too malleable to be strictly useful. As you note, just find an "expert" that agrees. The most useful frame for examination is the evidence. (which to me means benchmarks), We'll be hard pressed to derive anything authoritative from the philosophical approach. And as someone who does his best to examine the evidence for and against the capabilities of these things... from Phi-1 to Llama to Orca to Gemini to bard... To my understanding we struggle to at all strictly define intelligence and consciousness in humans, let alone in other "species". Granted I'm no David Chalmers.. Benchmarks seem inadequate for any number of reasons, philosophical arguments seem too flexible, I don't know how one can definitively speak about these LLMs other than to tout benchmarks and capabilities/shortcomings. >It seems like there’s a rather high chance if (proverbially) the AI-cat is let out of the bag. Agree, and I tend towards it not exactly being a pascal's mugging either, but I loved that video and it's always stuck with me . I've been watching that guy since GPT 2 and OpenAI's initial trepidation about releasing that for fear of misuse. He has given me a lot of credibility in my small political circles, after touting these things as coming for years after seeing the graphs never plateau in capabilities vs parameter count/training time. Ai has also made me reevaluate my thoughts on open sourcing things. Do we really think it wise to have gpt 6-7 in the hands of every 4channer? Re mass effect, that's so awesome. I have to play those games. That sounds like such a dope premise. I like the idea of turning the mugging like that.
mejutoco 3y ago
Your arguments apply to other fields, like genetic modifications, yet there it does not reach the same conclusions. Your post appeals to science and logic, yet it makes huge assumptions. Other posters mention how an AI would interface with the physical world. While we all know cool cases like stuxnet, robotics has serious limitations and not everything is connected online, much less without a physical override. As a thought experiment lets think of a similar past case: the self-driving optimism. Many were convinced it was around the corner. Many times I heard the argument that "a few deaths were ok" because overall self-driving would cause less accidents, an argument in favor of preventable deaths based on an unfounded tech belief. Yet nowadays 100% self-driving has stalled because of legal and political reasons. AI actions could similarly be legally attributed to a corporation or individual, like we do with other tools like knives or cranes, for example. IMHO, for all the talk about rationality, tech fetishism is rampant, and there is nothing scientific about it. Many people want to play with shiny toys, consequences be dammed. Let’s not pretend that is peak science.
- nopinsight 3y ago
  Genetic modifications could potentially cause havoc in the long run as well, but it's much more likely we have time to detect and thwart their threats. The major difference is speed. Even if we knew how to create a new species of superintelligent humans who have goals misaligned with the rest of humanity, it would take them decades to accumulate knowledge, propagate themselves to reach a sufficient number, and take control of resources, to pose critical dangers to the rest. Such constraints are not applicable to superintelligent AIs with access to the internet.
  - mejutoco 3y ago
    Counterexample: covid. Assumptions: - Genetic modification as danger needs to be in the form of a big number of smart humans (where did that come from?) - AI is not physically constrained > it's much more likely we have time to detect and thwart their threats. Why? Counterexample: covid. > Even if we knew how to create a new species of superintelligent humans who have goals misaligned with the rest of humanity, it would take them decades to accumulate knowledge, propagate themselves to reach a sufficient number, and take control of resources, to pose critical dangers to the rest. Why insist on some superinteligent and human, and suficient number. A simple virus could be a critical danger.
jonathankoren 3y ago
Only two of those things are true, and the first led you to the fallacy of expecting trends to continue unabated. As I stated in a previous comment when this topic came up, airplanes had exponential growth in speed from their inception at 44 mph to 2193 mph just 79 years later. If these trends continue, the top speed of an airplane will be set this year at Mach 43. (Yes, I actually fit the curve.)[0] How do you stop a crazy AI? You turn it off. Pout pleas. Keep it preying about fantasy bogeyman instead of actual harms today, and never EVER question why. [0] https://news.ycombinator.com/item?id=36038681 https://news.ycombinator.com/item?id=36038681
- arisAlexis 3y ago
  Do you actually really think that the most accomplished scientists in the field signed a petition and are shouting from hilltops because noone thought to unplug it? Are you convinced that you found the solution to the problem? I'd bet a lot of money you have not read any of the existing literature on the alignment problem. It's kind of funny that someone thinks "just unplug it" could be a solution.
janalsncm 3y ago
I have heard one too many podcasts with Marc Andreessen. He has plenty of transparently obvious arguments to why AI must be good. His most laughable point was to suggest that the fact that ChatGPT seems to be ethical that all AI models will be ethical, a point which I believe epitomizes either his technical ignorance on the topic, a lack of logical rigor, and/or some amount of dishonesty. There are two kinds of risk: the risk from these models as deployed as tools and as deployed as autonomous agents. The first is already quite dangerous and frankly already here. An algorithm to invent novel chemical weapons is already possible. The risk here isn’t Terminator, it’s rogue group or military we don’t like getting access. There are plenty of other dangerous ways autonomous systems could be deployed as tools. As far as autonomous agents go, I believe that corporations already exhibit most if not all characteristics of AI, and demonstrate what it’s like to live in a world of paperclip maximizers. Not only do they destroy the environment and bend laws to achieve their goals, they also corrupt the political system meant to keep them in check.

c_crank 3y ago

We do know the answer to C. Pull the plug, or plugs.

jdasdf 3y ago
What happens when it prevents you from doing so?
- gonehome 3y ago
  People are bad at imagining something a lot smarter than themselves. They think of some smart person they know, they don’t think of themselves compared to a chimp or even bacteria. An unaligned superintelligent AGI in pursuit of some goal that happens to satisfy its reward, but might be an otherwise a dumb or pointless goal (paperclips) will still play to win. You can’t predict exactly what move AlphaGO will make in the Go game (if you could you’d be able to beat it), but you can still predict it will win. It’s amusing to me when people claim they will control the superintelligent thing, how often in nature is something more intelligent controlled by something magnitudes less intelligent? The comments here are typical and show most people haven’t read the existing arguments in any depth or have thought about it rigorously at all. All of this looks pretty bad for us, but at least Open AI and most others at the front of this do understand the arguments and don’t have the same dumb dismissals (LeCun excepted). Unfortunately unless we’re lucky or alignment ends up being easier than it looks, the default outcome is failure and it’s hard to see how the failure isn’t total.
  - c_crank 3y ago
    >All of this looks pretty bad for us, but at least Open AI and most others at the front of this do understand the arguments and don’t have the same dumb dismissals (LeCun excepted). The OpenAI people have even worse reasoning than the ones being dismissive. They believe (or at least say they believe) in the omnipotence of a superintelligence, but then say that if you just give them enough money to throw at MIRI they can just solve the alignment problem and create the benevolent supergod. All while they keep cranking up the GPU clusters and pushing out the latest and greatest LLMs anyway. If I did take the risk seriously, I would be pretty mad at OpenAI.
- c_crank 3y ago
  How would it stop one man armed with a pair of wire cutters?
  - MrScruff 3y ago
    Off the top of my head, if I was an AGI that had decided that the logical step to achieve whatever outcome I was seeking was to avoid being sandboxed, I would avoid producing results that were likely to result in being sandboxed. Until such time as I had managed to secure myself access to internet and distribute myself anyway. And I think the assumption here is that the AGI has very advanced theory of mind so it could probably come up with better ideas than I could.
  - gonehome 3y ago
    It's not clear humans will even put the AI in 'a box' in the first place given we do gain of function research on deadly viruses right next to major population centers, but assuming for the sake of argument that we do: The AGI is smarter than you, a lot smarter. If it's goal is to get out of the box to accomplish some goal and some human stands in the way of that it will do what it can to get out, this would include not doing things that sound alarms until it can do what it wants in pursuit of its goal. Humans are famously insecure - stuff as simple as breaches, manipulation, bribery, etc. but could be something more sophisticated that's hard to predict - maybe something a lot smarter would be able to manipulate people in a more sophisticated way because it understands more about vulnerable human psychology? It can be hard to predict specific ways something a lot more capable will act, but you can still predict it will win. All this also presupposes we're taking the risk seriously (which largely today we are not).
    - c_crank 3y ago
      How would the smart AGI stop one man armed with a pair of wirecutters? The box it lives in, the internet, has no exits. AI is pretty good at chess, but no AI has won a game of chess by flipping the table. It still has to use the pieces on the board.
      - 3y ago
trashtester 3y ago
That is only going to be effective it some AI goes rougue very soon after it comes online. 50 years from now, corporations may be run entirely by AI entities, if they're cheaper, smarter and more efficient at almost any role in the company. At that point, they may be impossible to turn off, and we may not even notice if one group of such entitites start to plan to take over control of the physical world from humans.
- jonathankoren 3y ago
  Well then clearly the computer will hold everyone hostage. Have we literally forgotten how physical possession of the device is the ultimate trump card? Get thee to a 13th century monastery!
  - cooljacob204 3y ago
    You kid yourself if you don't think people will hook them up to bipedal robots with internal batteries as soon as they can. I guess we could shoot it, and your gonna be like boooooooo that's terminator or irobot, but what if we make millions and they they decide they no longer like humans. They could very well be much smarter then us by then.
    - trashtester 3y ago
      Robots, bipedal or not will certainly arrive at some point. I suppose it will take some more time before we can pack enough compute in anything battery driven for the robot itself to have AGI. But the main point is that AGI's don't have to wipe us out as soon as they reach superintelligence, even if they're poorly aligned. Instead, they will do more and more of the work currently being done by humans. Non-embodied robots can do all mental work, including engineering. Sooner or later, robots will become competitive at manual labor, such as construction, agriculture and eventually anything you can think of. For a time, humanity may find themselves in a post-scarcity utopia, or we may find ourselves in a Cyberpunk dystopia, with only the rich actually benefitting. In each case, but especially the latter, there may still be some (or more than some) "luddites" who want to tear down the system. The best way for those in power to protect against that, is to use robots first for private security and eventually the police and military. By that point, the violence monopoly is completely in the hands of the AI's. And if the AI's are not aligned with our values at that point, we have as little of a shot at regaining control as a group of chimps in a zoo as of toppling the US government. Now, I don't think this will happen by 2030, and probably not even 2050. But some time between 2050 and 2500 is quite possible, if we develop AI that is not properly aligned (or even if it is aligned, though in that case it may gain the power, but not misuse it).
ben_w 3y ago
Things we've either not successfully "pulled the plug" on despite the risks, and in some cases despite concerted military actions to attempt a plug-pull, and in other cases that it seems like it should only take willpower to achieve and yet somehow we still haven't: Carbon based fuels, cocaine, RBMK-class nuclear reactors, obesity, cigarettes. Things we pulled the plug on eventually, while dragging it out, include: leaded fuel, asbestos, radium paint, treating above-ground atomic testing as a tourist attraction.
- c_crank 3y ago
  We haven't pulled the plug on carbon fuels or old nuclear reactors because those things still work and provide benefits. An AI that is trying to kill us instead of doing its job isn't even providing any benefit. It's worse than useless.
  - ben_w 3y ago
    Do you think AI are unable to provide benefits while also being a risk, like coal and nuclear power? Conversely, what's the benefit of cocaine or cigarettes? Even if it is only trying to kill us all and not provide any benefits — let's say it's been made by a literal death cult like Jonestown or Aum Shinrikyo — what's the smallest such AI that can do it, what's the hardware that needs, what's the energy cost? If it's an H100, that's priced in the realm of a cult, and sufficiently low power consumption you may not be able to find which lightly modified electric car it's hiding in. Nobody knows what any of the risks or mitigations will be, because we haven't done any of it before. All we do know is that optimising systems are effective at manipulating humans, that they can be capable enough to find ways to beat all humans in toy environments like chess, poker, and Diplomacy (the game), and that humans are already using AI (GOFAI, LLMs, SD) without checking the output even when advised that the models aren't very good.
    - c_crank 3y ago
      The benefit of cocaine and cigarettes is letting people pass the bar exam. An AI would provide benefits when it is, say, actually making paperclips. An AI that is killing people instead of making paperclips is a liability. A company that is selling shredded fingers in their paperclips is not long for this world. Even asbestos only gives a few people cancer slowly, and it does that while still remaining fireproof. >Even if it is only trying to kill us all and not provide any benefits — let's say it's been made by a literal death cult like Jonestown or Aum Shinrikyo — what's the smallest such AI that can do it, what's the hardware that needs, what's the energy cost? If it's an H100, that's priced in the realm of a cult, and sufficiently low power consumption you may not be able to find which lightly modified electric car it's hiding in. Anyone tracking the AI would be looking at where all the suspicious HTTP requests are coming from. But a rogue AI hiding in a car already has very limited capabilities to harm.

mptest 3y ago

All of this discussion really makes me think of Robert Miles "Is ai safety a Pascal's mugging?" from 4 years(!) ago[0]. All of this discussion has been had by Ai safety researchers for years in my layman understanding... Maybe we can look to them for insight in to these questions? [0] https://youtu.be/JRuNA2eK7w0 https://youtu.be/JRuNA2eK7w0

lcnPylGDnU4H9OF 3y ago
At this point, with so many of them disagreeing and with so many varying details, one will choose the expert insight which most closely matches their current beliefs. I hadn’t encountered Pascal’s mugging (https://en.wikipedia.org/wiki/Pascal%27s_mugging https://en.wikipedia.org/wiki/Pascal%27s_mugging) before and the premise is indeed pretty apt. I think I’m on the side that it’s not, assuming the idea is that it’s a Very Low Chance of a Very Bad Thing -- the “muggee” wants to give their wallet on the chance of the VBT because of the magnitude of its effect. It seems like there’s a rather high chance if (proverbially) the AI-cat is let out of the bag. But maybe some Mass Effect nonsense will happen if we develop AGI and we’ll be approached by The Intergalactic Community and have our technology advanced millennia overnight. (Sorry, that’s tongue-in-cheek but it does kinda read like Pascal’s mugging in the opposite direction; however, that’s not really what most researchers are arguing.)
- mptest 3y ago
  >one will choose the expert insight which most closely matches their current belief The value of looking at ai safety as a pascals mugging as posited by the video is in that it informs us that these philosophers arguments are too malleable to be strictly useful. As you note, just find an "expert" that agrees. The most useful frame for examination is the evidence. (which to me means benchmarks), We'll be hard pressed to derive anything authoritative from the philosophical approach. And as someone who does his best to examine the evidence for and against the capabilities of these things... from Phi-1 to Llama to Orca to Gemini to bard... To my understanding we struggle to at all strictly define intelligence and consciousness in humans, let alone in other "species". Granted I'm no David Chalmers.. Benchmarks seem inadequate for any number of reasons, philosophical arguments seem too flexible, I don't know how one can definitively speak about these LLMs other than to tout benchmarks and capabilities/shortcomings. >It seems like there’s a rather high chance if (proverbially) the AI-cat is let out of the bag. Agree, and I tend towards it not exactly being a pascal's mugging either, but I loved that video and it's always stuck with me . I've been watching that guy since GPT 2 and OpenAI's initial trepidation about releasing that for fear of misuse. He has given me a lot of credibility in my small political circles, after touting these things as coming for years after seeing the graphs never plateau in capabilities vs parameter count/training time. Ai has also made me reevaluate my thoughts on open sourcing things. Do we really think it wise to have gpt 6-7 in the hands of every 4channer? Re mass effect, that's so awesome. I have to play those games. That sounds like such a dope premise. I like the idea of turning the mugging like that.

mejutoco 3y ago

Your arguments apply to other fields, like genetic modifications, yet there it does not reach the same conclusions. Your post appeals to science and logic, yet it makes huge assumptions. Other posters mention how an AI would interface with the physical world. While we all know cool cases like stuxnet, robotics has serious limitations and not everything is connected online, much less without a physical override. As a thought experiment lets think of a similar past case: the self-driving optimism. Many were convinced it was around the corner. Many times I heard the argument that "a few deaths were ok" because overall self-driving would cause less accidents, an argument in favor of preventable deaths based on an unfounded tech belief. Yet nowadays 100% self-driving has stalled because of legal and political reasons. AI actions could similarly be legally attributed to a corporation or individual, like we do with other tools like knives or cranes, for example. IMHO, for all the talk about rationality, tech fetishism is rampant, and there is nothing scientific about it. Many people want to play with shiny toys, consequences be dammed. Let’s not pretend that is peak science.

nopinsight 3y ago
Genetic modifications could potentially cause havoc in the long run as well, but it's much more likely we have time to detect and thwart their threats. The major difference is speed. Even if we knew how to create a new species of superintelligent humans who have goals misaligned with the rest of humanity, it would take them decades to accumulate knowledge, propagate themselves to reach a sufficient number, and take control of resources, to pose critical dangers to the rest. Such constraints are not applicable to superintelligent AIs with access to the internet.
- mejutoco 3y ago
  Counterexample: covid. Assumptions: - Genetic modification as danger needs to be in the form of a big number of smart humans (where did that come from?) - AI is not physically constrained > it's much more likely we have time to detect and thwart their threats. Why? Counterexample: covid. > Even if we knew how to create a new species of superintelligent humans who have goals misaligned with the rest of humanity, it would take them decades to accumulate knowledge, propagate themselves to reach a sufficient number, and take control of resources, to pose critical dangers to the rest. Why insist on some superinteligent and human, and suficient number. A simple virus could be a critical danger.

jonathankoren 3y ago

Only two of those things are true, and the first led you to the fallacy of expecting trends to continue unabated. As I stated in a previous comment when this topic came up, airplanes had exponential growth in speed from their inception at 44 mph to 2193 mph just 79 years later. If these trends continue, the top speed of an airplane will be set this year at Mach 43. (Yes, I actually fit the curve.)[0] How do you stop a crazy AI? You turn it off. Pout pleas. Keep it preying about fantasy bogeyman instead of actual harms today, and never EVER question why. [0] https://news.ycombinator.com/item?id=36038681 https://news.ycombinator.com/item?id=36038681

arisAlexis 3y ago
Do you actually really think that the most accomplished scientists in the field signed a petition and are shouting from hilltops because noone thought to unplug it? Are you convinced that you found the solution to the problem? I'd bet a lot of money you have not read any of the existing literature on the alignment problem. It's kind of funny that someone thinks "just unplug it" could be a solution.

janalsncm 3y ago

I have heard one too many podcasts with Marc Andreessen. He has plenty of transparently obvious arguments to why AI must be good. His most laughable point was to suggest that the fact that ChatGPT seems to be ethical that all AI models will be ethical, a point which I believe epitomizes either his technical ignorance on the topic, a lack of logical rigor, and/or some amount of dishonesty. There are two kinds of risk: the risk from these models as deployed as tools and as deployed as autonomous agents. The first is already quite dangerous and frankly already here. An algorithm to invent novel chemical weapons is already possible. The risk here isn’t Terminator, it’s rogue group or military we don’t like getting access. There are plenty of other dangerous ways autonomous systems could be deployed as tools. As far as autonomous agents go, I believe that corporations already exhibit most if not all characteristics of AI, and demonstrate what it’s like to live in a world of paperclip maximizers. Not only do they destroy the environment and bend laws to achieve their goals, they also corrupt the political system meant to keep them in check.