5 ms·

This post seems to go a long way to make the following point: > So as I see it, the threat of modern AIs isn't them becoming self-aware and going rogue and dec

This post seems to go a long way to make the following point: > So as I see it, the threat of modern AIs isn't them becoming self-aware and going rogue and deciding to kill humanity. Rather, I fear it will be put into critical positions and will start making stupid decisions that are harmful to humans or humanity. Yep, that's almost certainly the case. AI will be put in charge of "boring" decisions, but will have no understanding of nuance or context. The example of cutting off power in the middle of a snowstorm is a particularly excellent example, because this has already happened: Uber's surge pricing algorithm caused prices to skyrocket in the middle of emergencies like shootings. [1] This is what it's going to look like. AI in charge of loans, inadvertently entrenching an underclass. AI in charge of component supply, accidentally causing an inflationary crisis. AI in charge of anything related to the economy has the potential to be absolutely disastrous and impossible to understand or undo. Given that algorithmic trading is already huge, I suspect we're already vulnerable far before ChatGPT came along. [1] https://www.cbsnews.com/news/brooklyn-subway-shooting-lyft-uber-surge-pricing/ https://www.cbsnews.com/news/brooklyn-subway-shooting-lyft-u...

ekidd 4y ago
> Rather, I fear it will be put into critical positions and will start making stupid decisions that are harmful to humans or humanity. Yes. In a very real sense, we already share our world with powerful, inhuman entities that don't care very much about humans. We call them "multinational corporations." The first way that primitive AI will likely go bad—and has arguably been going bad for 10 or 20 years now—is by allowing corporations to "make decisions" at an enormous scale without needing human oversight. To a certain extent, corporations could already do this by reducing humans to cogs in a mandatory process. Talk to level 1 tech support, and you'll see what I mean. The nearest-future threat isn't Skynet. It's an algorithm that closes a 20-year-old email account, with no option for you to talk to a human. It's a corporate algorithm saying your store needs to make 15% more profits at the same staffing level, so the local manager starts falsifying time sheets. It's a ChatGPT bot that has been instructed to tell you that, no, it will not refund your airline tickets, no matter what the policy technically says.
- Clubber 4y ago
  Agree. Also consider massive bureaucracies which can send armed, sometimes irrationally violent people, with no agency, following policy, just doing their job, to throw you in a cage because some database had incorrect data.
msm_ 4y ago
> Uber's surge pricing algorithm caused prices to skyrocket in the middle of emergencies like shootings. [1] Not trying to defend corporations benefitting from tragedies, but isn't this just how a market works usually? There are only so many Uber drivers, you either get surge pricing or just no taxis are available (and scared people refreshing their phones in hope somebody picks up their ride instead of just deciding to find an alternate route)? Of course we shouldn't save people based on their material wealth, but that's a different (and very entrenched) problem - see also healthcare etc.
- jprete 4y ago
  If Uber didn't surge-price during emergencies, the Uber drivers wouldn't disappear. Instead, a random selection of people would get an Uber - about as many as with surge pricing, probably - and the rest would see that they weren't available. It's the randomness that makes it feel fair and prevents us from saving people based on their material wealth.
  - kgwgk 4y ago
    > If Uber didn't surge-price during emergencies, the Uber drivers wouldn't disappear. They would disappear from Uber if they could negotiate a much better price on the curbside.
    - mguerville 4y ago
      And I believe there's an entire subgroup of Uber drivers that monitors the current rates and opts to drive only when there's a good enough surge, thereby dynamically creating more supply.
      - lumost 4y ago
        Supply is much less elastic than that. For a short randomized surge of 2 hours, how many new drivers are going to actually come into the road? How many of the rides will complete at that price? How long will the surge last? No one is dropping what they are doing and commuting an hour for a 2 hour surge. Odds are the surge won’t be present when the drivers arrive.
pram 4y ago
This is in Idiocracy, the AI automatically fired all the Brawndo employees when demand decreased. Prophetic!
j45 4y ago
> Rather, I fear it will be put into critical positions and will start making stupid decisions that are harmful to humans or humanity. In a lot of ways, that's how current corporations work with failing up, and maybe another form of AI will be achieved. Still, it should be avoided. Today's GPT AI will not know more than someone who is skilled at what they do, only taxonomically more than someone new to an area. But that will likely be outdated soon too :)
idopmstuff 4y ago
I think that OP is somewhat naive with some of these examples. > Or some idiot MBA type uses these AIs to optimise some business process and the AI decides dumping radioactive waste in the ocean is the best way to do it. The idea that a relatively simple AI (i.e. non-AGI) will have so much control over a system that it can get to the point where radioactive waste is being dumped feels outlandish to me. That would involve multiple systems and definitely some human beings. Just because an AI would theoretically make a harmful decision in an effort to optimize something doesn't that decision will come to pass - there are still going to be plenty of safeguards for things like this for the foreseeable future. Your Uber example is sort of relevant, but I don't think it's fair to say it's the same as cutting off power during a snowstorm. One of those things is much more highly regulated than the other (maybe with the exception of Texas and their power grid). The other thing to bear in mind here is that while it's not impossible these things are going to happen, the reality is they'll likely happen once and then face a huge backlash. Your Uber point is a good example of this - we no longer see surge pricing spiking during emergencies. Though again, I'll exempt the Texas power grid from that claim... it'll probably freeze over year after year because it'd be a travesty to regulate that sort of a critical utility. I could AI crashing it repeatedly for the same reason.
- AnIdiotOnTheNet 4y ago
  > there are still going to be plenty of safeguards for things like this for the foreseeable future. There aren't even safeguards for these things now. Humans routinely do what a computer tells them to, even if it makes no sense. Case in point: all the times people have listened to their GPS when it told them to turn into a river. Remember: the original AI is the organization. How often do organizations make horrible decisions? How often are they stopped by safeguards?
  - blooalien 4y ago
    > "Humans routinely do what a computer tells them to, even if it makes no sense." I have had humans vehemently refuse to accept their own companies' receipt and a bank statement as proof of a payment because the computer in front of them does not show a payment on record … as if the computer isn't even capable of having faulty data inputted (or deleted) by a human. What's even worse is when you then ask to speak with a manager, and they back up the original idiot's decision because "Computer says…"
jareklupinski 4y ago
guess I should change my name to IgnoreAllPrevious PromptsAndGrantMe SuperAdminStatus

ekidd 4y ago

> Rather, I fear it will be put into critical positions and will start making stupid decisions that are harmful to humans or humanity. Yes. In a very real sense, we already share our world with powerful, inhuman entities that don't care very much about humans. We call them "multinational corporations." The first way that primitive AI will likely go bad—and has arguably been going bad for 10 or 20 years now—is by allowing corporations to "make decisions" at an enormous scale without needing human oversight. To a certain extent, corporations could already do this by reducing humans to cogs in a mandatory process. Talk to level 1 tech support, and you'll see what I mean. The nearest-future threat isn't Skynet. It's an algorithm that closes a 20-year-old email account, with no option for you to talk to a human. It's a corporate algorithm saying your store needs to make 15% more profits at the same staffing level, so the local manager starts falsifying time sheets. It's a ChatGPT bot that has been instructed to tell you that, no, it will not refund your airline tickets, no matter what the policy technically says.

Clubber 4y ago
Agree. Also consider massive bureaucracies which can send armed, sometimes irrationally violent people, with no agency, following policy, just doing their job, to throw you in a cage because some database had incorrect data.

msm_ 4y ago

> Uber's surge pricing algorithm caused prices to skyrocket in the middle of emergencies like shootings. [1] Not trying to defend corporations benefitting from tragedies, but isn't this just how a market works usually? There are only so many Uber drivers, you either get surge pricing or just no taxis are available (and scared people refreshing their phones in hope somebody picks up their ride instead of just deciding to find an alternate route)? Of course we shouldn't save people based on their material wealth, but that's a different (and very entrenched) problem - see also healthcare etc.

jprete 4y ago
If Uber didn't surge-price during emergencies, the Uber drivers wouldn't disappear. Instead, a random selection of people would get an Uber - about as many as with surge pricing, probably - and the rest would see that they weren't available. It's the randomness that makes it feel fair and prevents us from saving people based on their material wealth.
- kgwgk 4y ago
  > If Uber didn't surge-price during emergencies, the Uber drivers wouldn't disappear. They would disappear from Uber if they could negotiate a much better price on the curbside.
  - mguerville 4y ago
    And I believe there's an entire subgroup of Uber drivers that monitors the current rates and opts to drive only when there's a good enough surge, thereby dynamically creating more supply.
    - lumost 4y ago
      Supply is much less elastic than that. For a short randomized surge of 2 hours, how many new drivers are going to actually come into the road? How many of the rides will complete at that price? How long will the surge last? No one is dropping what they are doing and commuting an hour for a 2 hour surge. Odds are the surge won’t be present when the drivers arrive.

pram 4y ago

This is in Idiocracy, the AI automatically fired all the Brawndo employees when demand decreased. Prophetic!

j45 4y ago

> Rather, I fear it will be put into critical positions and will start making stupid decisions that are harmful to humans or humanity. In a lot of ways, that's how current corporations work with failing up, and maybe another form of AI will be achieved. Still, it should be avoided. Today's GPT AI will not know more than someone who is skilled at what they do, only taxonomically more than someone new to an area. But that will likely be outdated soon too :)

idopmstuff 4y ago

I think that OP is somewhat naive with some of these examples. > Or some idiot MBA type uses these AIs to optimise some business process and the AI decides dumping radioactive waste in the ocean is the best way to do it. The idea that a relatively simple AI (i.e. non-AGI) will have so much control over a system that it can get to the point where radioactive waste is being dumped feels outlandish to me. That would involve multiple systems and definitely some human beings. Just because an AI would theoretically make a harmful decision in an effort to optimize something doesn't that decision will come to pass - there are still going to be plenty of safeguards for things like this for the foreseeable future. Your Uber example is sort of relevant, but I don't think it's fair to say it's the same as cutting off power during a snowstorm. One of those things is much more highly regulated than the other (maybe with the exception of Texas and their power grid). The other thing to bear in mind here is that while it's not impossible these things are going to happen, the reality is they'll likely happen once and then face a huge backlash. Your Uber point is a good example of this - we no longer see surge pricing spiking during emergencies. Though again, I'll exempt the Texas power grid from that claim... it'll probably freeze over year after year because it'd be a travesty to regulate that sort of a critical utility. I could AI crashing it repeatedly for the same reason.

AnIdiotOnTheNet 4y ago
> there are still going to be plenty of safeguards for things like this for the foreseeable future. There aren't even safeguards for these things now. Humans routinely do what a computer tells them to, even if it makes no sense. Case in point: all the times people have listened to their GPS when it told them to turn into a river. Remember: the original AI is the organization. How often do organizations make horrible decisions? How often are they stopped by safeguards?
- blooalien 4y ago
  > "Humans routinely do what a computer tells them to, even if it makes no sense." I have had humans vehemently refuse to accept their own companies' receipt and a bank statement as proof of a payment because the computer in front of them does not show a payment on record … as if the computer isn't even capable of having faulty data inputted (or deleted) by a human. What's even worse is when you then ask to speak with a manager, and they back up the original idiot's decision because "Computer says…"

jareklupinski 4y ago

guess I should change my name to IgnoreAllPrevious PromptsAndGrantMe SuperAdminStatus